-
-
Notifications
You must be signed in to change notification settings - Fork 14.2k
cstyle-export-rules: export global symbols from upstream static libraries
#150335
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
|
The job Click to see the possible cause of the failure (guessed by this bot) |
|
|
||
| let mut symbols = Vec::new(); | ||
| let export_threshold = symbol_export::crates_export_threshold(&[crate_type]); | ||
| for_each_exported_symbols_include_dep(tcx, crate_type, |symbol, info, cnum| { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This lists all exported symbols, while you need all non-exported symbols too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So this is the correct position to iterate through all exported symbols and invert the selection rules of the original exported_symbols?
| let mut symbols = Vec::new(); | ||
| let export_threshold = symbol_export::crates_export_threshold(&[crate_type]); | ||
| for_each_exported_symbols_include_dep(tcx, crate_type, |symbol, info, cnum| { | ||
| if !info.level.is_below_threshold(export_threshold) && !tcx.is_compiler_builtins(cnum) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
|| is_compiler_builtins
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I know that the original criterion for selecting symbols into exported_symbols was info.level.is_below_threshold(export_threshold) && !tcx.is_compiler_builtins(cnum), but here we cannot perform a negation operation, i.e., info.level.is_below_threshold(export_threshold) || tcx.is_compiler_builtins(cnum), as this will result in the following error:
exit status: 1
|
= note: "cc" "-Wl,--version-script=/tmp/rustcenvqxB/list" "-Wl,--no-undefined-version" "-m64" "/tmp/rustcenvqxB/symbols.o" "<2 object files omitted>" "-Wl,--as-needed" "-Wl,-Bstatic" "/root/cezarbbb/include-libs-test3/downstream-cdylib/target/debug/deps/libupstream-8102da413a3d22f9.rlib" "<sysroot>/lib/rustlib/x86_64-unknown-linux-gnu/lib/{libstd-*,libpanic_unwind-*,libobject-*,libmemchr-*,libaddr2line-*,libgimli-*,librustc_demangle-*,libstd_detect-*,libhashbrown-*,librustc_std_workspace_alloc-*,libminiz_oxide-*,libadler2-*,libunwind-*,libcfg_if-*,liblibc-*,librustc_std_workspace_core-*,liballoc-*,libcore-*,libcompiler_builtins-*}.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-L" "/tmp/rustcenvqxB/raw-dylibs" "-B<sysroot>/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld" "-fuse-ld=lld" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "<sysroot>/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/root/cezarbbb/include-libs-test3/downstream-cdylib/target/debug/deps/libdownstream_cdylib.so" "-Wl,--gc-sections" "-shared" "-Wl,-z,relro,-z,now" "-nodefaultlibs"
= note: some arguments are omitted. use `--verbose` to show all linker arguments
= note: rust-lld: error: version script assignment of 'local' to symbol '__rustc_proc_macro_decls_908feb11bfb2de80__' failed: symbol not defined
rust-lld: error: version script assignment of 'local' to symbol 'rust_metadata_downstream_cdylib_908feb11bfb2de80' failed: symbol not defined
rust-lld: error: version script assignment of 'local' to symbol '__addtf3' failed: symbol not defined
rust-lld: error: version script assignment of 'local' to symbol '__ashlti3' failed: symbol not defined
rust-lld: error: version script assignment of 'local' to symbol '__ashrti3' failed: symbol not defined
rust-lld: error: version script assignment of 'local' to symbol '__bswapdi2' failed: symbol not defined
rust-lld: error: version script assignment of 'local' to symbol '__bswapsi2' failed: symbol not defined
rust-lld: error: version script assignment of 'local' to symbol '__clzdi2' failed: symbol not defined
rust-lld: error: version script assignment of 'local' to symbol '__clzti2' failed: symbol not defined
rust-lld: error: version script assignment of 'local' to symbol '__ctzdi2' failed: symbol not defined
rust-lld: error: version script assignment of 'local' to symbol '__ctzti2' failed: symbol not defined
rust-lld: error: version script assignment of 'local' to symbol '__divmodti4' failed: symbol not defined
rust-lld: error: version script assignment of 'local' to symbol '__divtf3' failed: symbol not defined
rust-lld: error: version script assignment of 'local' to symbol '__divti3' failed: symbol not defined
rust-lld: error: version script assignment of 'local' to symbol '__eqtf2' failed: symbol not defined
rust-lld: error: version script assignment of 'local' to symbol '__extenddftf2' failed: symbol not defined
rust-lld: error: version script assignment of 'local' to symbol '__extendhfdf2' failed: symbol not defined
rust-lld: error: version script assignment of 'local' to symbol '__extendhftf2' failed: symbol not defined
rust-lld: error: version script assignment of 'local' to symbol '__extendsfdf2' failed: symbol not defined
rust-lld: error: version script assignment of 'local' to symbol '__extendsftf2' failed: symbol not defined
rust-lld: error: too many errors emitted, stopping now (use --error-limit=0 to see all errors) collect2: error: ld returned 1 exit statusIt seems linker is not selecting some of certain compilerbuiltin functions, resulting in undefined symbols?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right. All compiler-builtins symbols have hidden visibility, so I guess they shouldn't be listed in the version script at all.
| } | ||
| } | ||
|
|
||
| pub(crate) fn exported_symbols_cstyle( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For proc-macros we should keep using an allowlist containing just two symbols.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The same situation occurs when adding proc_macro, as can be seen in the error message above(__rustc_proc_macro_decls_908feb11bfb2de80__ and rust_metadata_downstream_cdylib_908feb11bfb2de80).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These symbols must be exported from proc-macros. If they aren't, rustc will fail to load the proc-macro. In any case it looks like you put those symbols on the denylist rather than the allowlist.
This is a demo I wrote based on an idea from this Zulip thread, and it's currently in a very early stage (it doesn't yet limit unsupported platforms or consider LTO).
I wrote a test to check the effect of enabling this option. First, I wrote an upstream C static library and compiled it into an object file.
Then I created a
cdylibproject downstream.Cargo.tomlsrc/lib.rsThen I tested by enabling and disabling
cstyle-export-rulesin the following two toolchains, and checked their symbol tables and the size of the generated .so files.default:+c-e-rHere are the results:
RUSTFLAGS="-Z cstyle-export-rules" cargo +c-e-r rustc --release -- -C link-arg=c_add.osize: 451KBU abort@GLIBC_2.2.5 U bcmp@GLIBC_2.2.5 00000000000520ad T c_add U calloc@GLIBC_2.2.5 U close@GLIBC_2.2.5 00000000000520c5 T c_sub w __cxa_finalize@GLIBC_2.2.5 w __cxa_thread_atexit_impl@GLIBC_2.18 U dl_iterate_phdr@GLIBC_2.2.5 0000000000015160 T downstream_add U __errno_location@GLIBC_2.2.5 U free@GLIBC_2.2.5 U fstat64@GLIBC_2.33 U getcwd@GLIBC_2.2.5 U getenv@GLIBC_2.2.5 w __gmon_start__ w _ITM_deregisterTMCloneTable w _ITM_registerTMCloneTable U lseek64@GLIBC_2.2.5 U malloc@GLIBC_2.2.5 U memcpy@GLIBC_2.14 U memmove@GLIBC_2.2.5 U memset@GLIBC_2.2.5 U mmap64@GLIBC_2.2.5 U munmap@GLIBC_2.2.5 U open64@GLIBC_2.2.5 U posix_memalign@GLIBC_2.2.5 U pthread_key_create@GLIBC_2.34 U pthread_key_delete@GLIBC_2.34 U pthread_setspecific@GLIBC_2.34 U read@GLIBC_2.2.5 U readlink@GLIBC_2.2.5 U realloc@GLIBC_2.2.5 U realpath@GLIBC_2.3 0000000000052074 T _RNvCs6OckbHhR1MK_7___rustc17___rust_probestack U stat64@GLIBC_2.33 w statx@GLIBC_2.28 U strlen@GLIBC_2.2.5 U syscall@GLIBC_2.2.5 U __tls_get_addr@GLIBC_2.3 U _Unwind_Backtrace@GCC_3.3 U _Unwind_DeleteException@GCC_3.0 U _Unwind_GetDataRelBase@GCC_3.0 U _Unwind_GetIP@GCC_3.0 U _Unwind_GetIPInfo@GCC_4.2.0 U _Unwind_GetLanguageSpecificData@GCC_3.0 U _Unwind_GetRegionStart@GCC_3.0 U _Unwind_GetTextRelBase@GCC_3.0 U _Unwind_RaiseException@GCC_3.0 U _Unwind_Resume@GCC_3.0 U _Unwind_SetGR@GCC_3.0 U _Unwind_SetIP@GCC_3.0 U write@GLIBC_2.2.5 U writev@GLIBC_2.2.5 0000000000044bcc T _ZN119_$LT$core..iter..adapters..copied..Copied$LT$I$GT$$u20$as$u20$core..iter..traits..double_ended..DoubleEndedIterator$GT$9try_rfold17h00588b6766da7b19E 000000000001a800 T _ZN3std3sys9backtrace26__rust_end_short_backtrace17h9a45a5fa40018bf0E 00000000000414a0 T _ZN4core5slice4sort8unstable7ipnsort17hd24504d702ce9a58E 0000000000041590 T _ZN4core5slice4sort8unstable8heapsort8heapsort17h5983f0d9c6b77eb4Ecargo +c-e-r rustc --release -- -C link-arg=c_add.osize: 450KBU abort@GLIBC_2.2.5 U bcmp@GLIBC_2.2.5 U calloc@GLIBC_2.2.5 U close@GLIBC_2.2.5 w __cxa_finalize@GLIBC_2.2.5 w __cxa_thread_atexit_impl@GLIBC_2.18 U dl_iterate_phdr@GLIBC_2.2.5 0000000000014ea0 T downstream_add U __errno_location@GLIBC_2.2.5 U free@GLIBC_2.2.5 U fstat64@GLIBC_2.33 U getcwd@GLIBC_2.2.5 U getenv@GLIBC_2.2.5 w __gmon_start__ w _ITM_deregisterTMCloneTable w _ITM_registerTMCloneTable U lseek64@GLIBC_2.2.5 U malloc@GLIBC_2.2.5 U memcpy@GLIBC_2.14 U memmove@GLIBC_2.2.5 U memset@GLIBC_2.2.5 U mmap64@GLIBC_2.2.5 U munmap@GLIBC_2.2.5 U open64@GLIBC_2.2.5 U posix_memalign@GLIBC_2.2.5 U pthread_key_create@GLIBC_2.34 U pthread_key_delete@GLIBC_2.34 U pthread_setspecific@GLIBC_2.34 U read@GLIBC_2.2.5 U readlink@GLIBC_2.2.5 U realloc@GLIBC_2.2.5 U realpath@GLIBC_2.3 U stat64@GLIBC_2.33 w statx@GLIBC_2.28 U strlen@GLIBC_2.2.5 U syscall@GLIBC_2.2.5 U __tls_get_addr@GLIBC_2.3 U _Unwind_Backtrace@GCC_3.3 U _Unwind_DeleteException@GCC_3.0 U _Unwind_GetDataRelBase@GCC_3.0 U _Unwind_GetIP@GCC_3.0 U _Unwind_GetIPInfo@GCC_4.2.0 U _Unwind_GetLanguageSpecificData@GCC_3.0 U _Unwind_GetRegionStart@GCC_3.0 U _Unwind_GetTextRelBase@GCC_3.0 U _Unwind_RaiseException@GCC_3.0 U _Unwind_Resume@GCC_3.0 U _Unwind_SetGR@GCC_3.0 U _Unwind_SetIP@GCC_3.0 U write@GLIBC_2.2.5 U writev@GLIBC_2.2.5cargo rustc --release -- -C link-arg=c_add.osize: 16KBw __cxa_finalize@GLIBC_2.2.5 0000000000001100 T downstream_add w __gmon_start__ w _ITM_deregisterTMCloneTable w _ITM_registerTMCloneTableIt appears that the upstream
c_addandc_subwere successfully exported, but many more symbols were exported, and the file size increased significantly. I'm not sure why this is happening.r? @bjorn3