Skip to content

Add test for checking used glibc symbols #135164

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jan 23, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/doc/rustc-dev-guide/src/tests/directives.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,8 @@ Some examples of `X` in `ignore-X` or `only-X`:
`compare-mode-split-dwarf`, `compare-mode-split-dwarf-single`
- The two different test modes used by coverage tests:
`ignore-coverage-map`, `ignore-coverage-run`
- When testing a dist toolchain: `dist`
- This needs to be enabled with `COMPILETEST_ENABLE_DIST_TESTS=1`

The following directives will check rustc build settings and target
settings:
Expand Down
1 change: 1 addition & 0 deletions src/tools/compiletest/src/directive-list.rs
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,7 @@ const KNOWN_DIRECTIVE_NAMES: &[&str] = &[
"only-beta",
"only-bpf",
"only-cdb",
"only-dist",
"only-gnu",
"only-i686-pc-windows-gnu",
"only-i686-pc-windows-msvc",
Expand Down
6 changes: 6 additions & 0 deletions src/tools/compiletest/src/header/cfg.rs
Original file line number Diff line number Diff line change
Expand Up @@ -235,6 +235,12 @@ fn parse_cfg_name_directive<'a>(
message: "when the test mode is {name}",
}

condition! {
name: "dist",
condition: std::env::var("COMPILETEST_ENABLE_DIST_TESTS") == Ok("1".to_string()),
message: "when performing tests on dist toolchain"
}

if prefix == "ignore" && outcome == MatchOutcome::Invalid {
// Don't error out for ignore-tidy-* diretives, as those are not handled by compiletest.
if name.starts_with("tidy-") {
Expand Down
9 changes: 8 additions & 1 deletion src/tools/opt-dist/src/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ change-id = 115898

[rust]
channel = "{channel}"
verbose-tests = true

[build]
rustc = "{rustc}"
Expand Down Expand Up @@ -102,13 +103,19 @@ llvm-config = "{llvm_config}"
"tests/incremental",
"tests/mir-opt",
"tests/pretty",
"tests/run-make/glibc-symbols-x86_64-unknown-linux-gnu",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally agree that having this guard is beneficial!

However, it may cause a problem to opt-dist users outside rust-lang CI who don't care if their rustc artifacts use newer glibc symbols.
Just FYI, it is a bit too minor that I don't think it is worth of a separate issue.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, opt-dist is primarily used for rust-lang's CI. Any support of it for external users is best-effort. And supporting also the tests is completely out of scope. Tests should now be opt-in after #139962, so it should be fine.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I knew that, and that was why I left a comment here only. No need to do anything on your side and sorry for the noise 😞.

"tests/ui",
"tests/crashes",
];
for test_path in env.skipped_tests() {
args.extend(["--skip", test_path]);
}
cmd(&args).env("COMPILETEST_FORCE_STAGE0", "1").run().context("Cannot execute tests")
cmd(&args)
.env("COMPILETEST_FORCE_STAGE0", "1")
// Also run dist-only tests
.env("COMPILETEST_ENABLE_DIST_TESTS", "1")
.run()
.context("Cannot execute tests")
}

/// Tries to find the version of the dist artifacts (either nightly, beta, or 1.XY.Z).
Expand Down
112 changes: 112 additions & 0 deletions tests/run-make/glibc-symbols-x86_64-unknown-linux-gnu/rmake.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
// Check that the compiler toolchain (rustc) that we distribute is not using newer glibc
// symbols than a specified minimum.
// This test should only be executed on an extracted dist archive or in a dist-* CI job.

//@ only-dist
//@ only-x86_64-unknown-linux-gnu
//@ ignore-cross-compile

use std::path::{Path, PathBuf};

use run_make_support::{cmd, llvm_objdump, regex, rustc_path};

fn main() {
// This is the maximum glibc version that we are *permitted* to use for the
// x86_64-unknown-linux-gnu target.
// All glibc symbols used in the compiler must be lower or equal than this version.
// So that if a given machine only has glibc 2.17, it is able to run the compiler.
let max_supported = (2, 17, 99);

let rustc = PathBuf::from(rustc_path());
// Check symbols directly in rustc
check_symbols(&rustc, max_supported);

// Find dynamic libraries referenced by rustc that come from our lib directory
let lib_path = rustc.parent().unwrap().parent().unwrap().join("lib");
let dynamic_libs = find_dynamic_libs(&rustc)
.into_iter()
.filter_map(|path| path.canonicalize().ok())
.filter(|lib| lib.starts_with(&lib_path))
.collect::<Vec<_>>();
for lib in dynamic_libs {
check_symbols(&lib, max_supported);
}
}

#[derive(Debug, Ord, PartialOrd, Eq, PartialEq)]
struct GlibcSymbol {
name: String,
version: (u32, u32, u32),
}

fn find_dynamic_libs(path: &Path) -> Vec<PathBuf> {
cmd("ldd")
.arg(path)
.run()
.stdout_utf8()
.lines()
.filter_map(|line| {
let line = line.trim();
let Some((_, line)) = line.split_once(" => ") else {
return None;
};
line.split_ascii_whitespace().next().map(|path| PathBuf::from(path))
})
.collect()
}

fn check_symbols(file: &Path, max_supported: (u32, u32, u32)) {
println!("Checking {}", file.display());
let mut invalid: Vec<GlibcSymbol> = get_glibc_symbols(file)
.into_iter()
.filter(|symbol| symbol.version > max_supported)
.collect();
if !invalid.is_empty() {
invalid.sort();
panic!(
"Found invalid glibc symbols in {}:\n{}",
file.display(),
invalid
.into_iter()
.map(|symbol| format!(
"{} ({:?} higher than max allowed {:?})",
symbol.name, symbol.version, max_supported
))
.collect::<Vec<_>>()
.join("\n")
)
}
}

fn get_glibc_symbols(file: &Path) -> Vec<GlibcSymbol> {
let regex = regex::Regex::new(r#"GLIBC_(\d)+\.(\d+)(:?\.(\d+))?"#).unwrap();

// FIXME(kobzol): llvm-objdump currently chokes on the BOLTed librustc_driver.so file.
// Use objdump instead, since it seems to work, and we only run this test in a specific
// CI environment anyway.
cmd("objdump")
.arg("--dynamic-syms")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, I find readelf -V (--version-info) easier to deal with, especially looking just at the latter .gnu.version_r section of the output. It looks like llvm-readelf supports this too, without choking on the current library. You can't see the name the relevant symbols this way though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted the test to print the offending symbols, to give us a notion of what symbols cause the issue.

.arg(file)
.run()
.stdout_utf8()
.lines()
.filter_map(|line| {
// Example line
// 0000000000000000 DF *UND* 0000000000000000 (GLIBC_2.2.5) sbrk
let mut parts = line.split(" ").collect::<Vec<_>>().into_iter().rev();
let Some(name) = parts.next() else {
return None;
};
let Some(lib) = parts.next() else {
return None;
};
let Some(version) = regex.captures(lib) else {
return None;
};
let major = version.get(1).and_then(|m| m.as_str().parse().ok()).unwrap_or(0);
let minor = version.get(2).and_then(|m| m.as_str().parse().ok()).unwrap_or(0);
let patch = version.get(3).and_then(|m| m.as_str().parse().ok()).unwrap_or(0);
Some(GlibcSymbol { version: (major, minor, patch), name: name.to_string() })
})
.collect()
}
Loading