Version 2.33 of the GNU C library is out. Changes this time include anumber of dynamic linker improvements, 32-bit RISC-V support, and a numberof security fixes.
Using the Library. This section describes some of the practical issues involved in using the GNU C library. Header Files: How to include the header files in your programs. Macro Definitions: Some functions in the library may really be implemented as macros. Aug 02, 2021 The GNU C Library version 2.34 is now available. Item posted by Carlos O'Donell on Mon 02 Aug 2021 03:57:01 AM UTC. The GNU C Library The GNU C Library version 2.34 is now available. The GNU C Library is used as the C library in the GNU system and in GNU/Linux systems, as well as many other systems that use Linux as the kernel. C Technical Specifications Implementation Status 1.7. C 2020 Implementation Status 1.8. C TR1 Implementation Status 1.9. C TR 24733 Implementation Status 1.10. C Special Functions Implementation Status 3.1. C Command Options 3.2. C 1998 Library Headers 3.3. C 1998 Library Headers for C Library Facilities 3.4. C 1998 Deprecated. The GNU C Library version 2.34 is now available. Software Release. Posted by 5 minutes ago. The GNU C Library version 2.34 is now available.
| From: | Adhemerval Zanella <adhemerval.zanella-AT-linaro.org> |
| To: | GNU C Library <libc-alpha-AT-sourceware.org>, libc-announce-AT-sourceware.org, info-gnu-AT-gnu.org |
| Subject: | The GNU C Library version 2.33 is now available |
| Date: | Mon, 01 Feb 2021 16:33:47 -0300 |
| Message-ID: | <7d6c163d-1d37-a7fe-e52a-bf4422261b57__25427.8764057503$1612215152$gmane$org@linaro.org> |
GNU C library 2.33 released
Posted Feb 2, 2021 16:52 UTC (Tue) by josh (subscriber, #17465) [Link]
This is the headline feature:> The dynamic linker loads optimized implementations of shared objects from subdirectories under the glibc-hwcaps directory on the library search path if the system's capabilities meet the requirements for that subdirectory. Initially supported subdirectories include 'power9' and 'power10' for the powerpc64le-linux-gnu architecture, 'z13', 'z14', 'z15' for s390x-linux-gnu, and 'x86-64-v2', 'x86-64-v3', 'x86-64-v4' for x86_64-linux-gnu. In the x86_64-linux-gnu case, the subdirectory names correspond to the vendor-independent x86-64 microarchitecture levels defined in the x86-64 psABI supplement.
This is a replacement for the old hardware capabilities mechanism, which caused a combinatorial explosion of checks for each capability. This new mechanism groups capabilities together into logical architecture levels that are increasing supersets, so it can just look for one of them at a time.
GNU C library 2.33 released
Posted Feb 4, 2021 6:39 UTC (Thu) by dvdeug (subscriber, #10998) [Link]
Except that looking at https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#... :For various AVX-512 features--see Wikipedia for the descriptions--
Knights Landing supports ER.
Knights Mill supports ER, 4FMAPS and VPOPCNTDQ.
Skylake-SP supports VL (and none of the above).
Cannon Lake supports VL and IFMA.
Cascade Lake-SP supports VL and VNNI.
Ice Lake supports VL, IFMA, VBMI2, VPOPCNTDQ and VNNI.
That is, Knights Mill is a superset of Knights Landing; Skylake is a subset of Cannon Lake and Cascade Lake (neither of which are a subset of each other), and Ice Lake is a superset of Cannon Lake and Cascade Lake (but not Knights Landing).
Likewise, FMA4 can never be supported in this scheme, since more recent processors don't support it.
Basically AMD64 is a mismash of supported features, across all chips and even within just Intel chips. Even if you're not concerned about getting optimal performance out of older chips, nobody could have known when FMA4 came out if and how it was going to be supported in the future. Looking at the full AVX-512 table, I don't know what features in Ice Lake are going to be standard in the future; it seems like either they're going to be running far behind the leading edge, or end up defining the top 'logical architecture level' as something that it turns out no new processors are supporting, and thus being forced to break the architecture levels.
I've got 'System V Application Binary Interface: AMD64 Architecture Processor Supplement (With LP64 and ILP32 Programming Models) Version 1.0', but I don't see any details about the microarchitecture levels; if that's the right document, the newer versions only come as TeX files.
GNU C library 2.33 released
Posted Feb 4, 2021 9:45 UTC (Thu) by anton (subscriber, #25547) [Link]
This mess is probably the reason for the new scheme. So yes, glibc may not use all the features of the CPU at hand, but then, does it make a difference? And if it makes a difference, I guess the shared object for, say, x86-64-v4, can still do a CPUID check for the additional feature and then put the appropriate function pointer in the dispatch table, but at least the code can rely on all the v4 features and does not have to worry about whether these are available.GNU C library 2.33 released
Posted Feb 5, 2021 19:14 UTC (Fri) by nix (subscriber, #2304) [Link]
GNU C library 2.33 released
Posted Feb 5, 2021 20:17 UTC (Fri) by joib (subscriber, #8541) [Link]
GNU C library 2.33 released
Posted Feb 6, 2021 6:08 UTC (Sat) by dvdeug (subscriber, #10998) [Link]
Looking at the PDFs joib so helpfully linked to, the latest level is AVX-512, which AMD has apparently issued no public plans to support. It seems not impossible that AMD will never support AVX-512, and will add its own extensions that GCC wants to support. H. J. Lu, the lead author on the PDF, works for Intel, and none of the authors list amd.com emails; it seems likely they're more tied into Intel's plans than AMD's plans.From another angle, https://www.extremetech.com/computing/312673-linus-torval... mentions that Atom CPUs don't even support AVX; what happens when Atom CPUs come out with new instructions specialized for low-power ChromeBooks? Or even some specific instructions, like AES enhancements, make their way all the way down the line to Atom CPUs?
Either way, AMD adding its own instructions without supporting AVX-512 or newer instructions making their way down to cheap CPUs with limited AVX support, it seems entirely out of the control of the GNU toolchain people that there are no new instructions they need to support on CPUs that don't support AVX-512.
GNU C library 2.33 released
Posted Feb 6, 2021 10:20 UTC (Sat) by anton (subscriber, #25547) [Link]
It is unlikely that AMD will add their own extensions, at least as long as they don't dominate the market for AMD64 architectures. They tried that with 3DNow!, it did not become popular, and since about 2011 AMD CPUs have dropped 3DNow! (but they had to support it for quite a while, in K8 and K10). FMA4 is a similar story.But if AMD should choose to not implement AVX-512 (I expect they will support it at some time), AMD users run the code compiled for v3; I don't see a problem here. However, looking at Bug 23249, one can get the impression that Intel succeeds in hampering AMD CPUs not just in code compiled with the Intel compiler and using Intel-supplied libraries, but also in free software; and if they succeeded in the hwcaps case, why should they fail in the instruction-set-level case?
As for Intel CPUs not supporting instruction set extensions, it's not just the Atom product line. Lots of Intel CPUs disable this instruction set feature or that, even on silicon where it is implemented (e.g., Skylake Pentiums have AVX disabled); apparently Intel's marketing thinks that they can perform market segmentation in that way.
The only thing it achieves is that almost nobody uses the more advanced features. This instruction-set level idea has a certain potential at improving this situation, but I wonder: Even if it's as easy as invoking gcc with some architecture-level option, I guess that most upstream developers will set up their build infrastructure to just compile for the base level by default (and the builder configure to have it compiled with specific options). Will distributions compile every packet for four different levels and install the one for the particular machine?
GNU C library 2.33 released
Posted Feb 6, 2021 12:23 UTC (Sat) by joib (subscriber, #8541) [Link]
> However, looking at Bug 23249, one can get the impression that Intel succeeds in hampering AMD CPUs not just in code compiled with the Intel compiler and using Intel-supplied libraries, but also in free software; and if they succeeded in the hwcaps case, why should they fail in the instruction-set-level case?Well, it seems that story got a happy ending, in that the -hwcaps is the ostensibly vendor-neutral solution to the same problem the old vendor-specific 'haswell' subdirectory tried to solve?
> Lots of Intel CPUs disable this instruction set feature or that, even on silicon where it is implemented (e.g., Skylake Pentiums have AVX disabled); apparently Intel's marketing thinks that they can perform market segmentation in that way.
Stupid indeed. One can understand not wanting to spend chip area on large vector units, but fusing out functional units for product segmentation reason? Argh.
> Will distributions compile every packet for four different levels and install the one for the particular machine?
I would guess that the binary package for libfoo would contain the .so's for all 4 microarchitecture levels (or if the developers or distributors do benchmarking, only those levels where it makes a difference). Then again, I recall Debian/Ubuntu used to have libc6-i686 which was glibc compiled for the i686 target rather than the default i386 or i486, so maybe they will do separate packages. Sounds tedious, but oh well.
GNU C library 2.33 released
Posted Feb 6, 2021 16:54 UTC (Sat) by Wol (subscriber, #4433) [Link]
> Stupid indeed. One can understand not wanting to spend chip area on large vector units, but fusing out functional units for product segmentation reason? Argh.Or is this a bit of complex real-estate that has a habit of not working? Fusing out stuff that fails QA and segmenting the product has some logic behind it.
Cheers,
Wol
GNU C library 2.33 released
Posted Feb 6, 2021 17:49 UTC (Sat) by anton (subscriber, #25547) [Link]
At least Skylake (and its variants up to Comet Lake) can do AVX/AVX2 work with the upper 128 bits of the unit powered down, at lower performance (and it works that way until the upper 128 bits have been powered up). So a better way to deal with both market segmentation and partially failing SIMD units would be to support AVX but on the Pentiums run it without the upper 128 bits of the functional unit. If you wonder about non-working lower 128 bits: then SSE won't work and they cannot sell the die even with disabled AVX.GNU C library 2.33 released
Posted Feb 6, 2021 17:06 UTC (Sat) by anton (subscriber, #25547) [Link]
Well, it seems that story got a happy ending, in that the -hwcaps is the ostensibly vendor-neutral solution to the same problem the old vendor-specific 'haswell' subdirectory tried to solve?Sorry for mentioning hwcaps, which played no role in this. As the bug report explains, 'haswell' is documented as vendor-neutral (despite its name), with the same meaning as 'x86-64-v3', but the implementation worked only on Intel CPUs as documented, by explicitly checking for Intel CPUs (like Intel does in the MKL) and using the instruction set extensions only for them. The bug was not fixed for two years, and it's not clear that it is fixed now; and the reason given does not sound convincing at all. So, can we expect that 'x86-64-v3' will fare any better than 'haswell'?
Concerning chip area, AMD has demonstrated that AVX can be implemented with 128-bit wide functional units, including the low-power low-cost Jaguar (competition for Intel's Atom family).
Concerning packaging, at least with Debian's multiarch support, treating them as separate architectures and compiling all libraries for every level might be less work. But distributions that only have lib32/lib64 to deal with multiple architectures cannot go there.
GNU C library 2.33 released
Posted Feb 4, 2021 20:53 UTC (Thu) by joib (subscriber, #8541) [Link]
> Basically AMD64 is a mismash of supported features, across all chips and even within just Intel chips.AFAIU the glibc-hwcaps and the microarchitecture levels are meant to provide somewhat decent baselines that apply to many CPU's out there and are expected to be 'standard' features in x86 CPU's going forwards. It's up to the psABI developers to not be too aggressive in adding stuff there that might disappear from the market or are of interest only to a small subset of the market (both of these applying to the Xeon Phi's you mentioned).
> I've got 'System V Application Binary Interface: AMD64 Architecture Processor Supplement (With LP64 and ILP32 Programming Models) Version 1.0', but I don't see any details about the microarchitecture levels; if that's the right document, the newer versions only come as TeX files.
For some reason the PDF's are excruciatingly hard to find. If you click on the 'Wiki' section you get a placeholder page saying that the project has no wiki pages. However, the PDF's are there, the URL is
https://gitlab.com/x86-psABIs/x86-64-ABI/-/wikis/x86-64-p...
(The microarchitecture levels are in Table 3.1)
GNU C library 2.33 released
Posted Feb 2, 2021 22:32 UTC (Tue) by arekm (subscriber, #4846) [Link]
GNU C library 2.33 released
Posted Feb 4, 2021 10:00 UTC (Thu) by anton (subscriber, #25547) [Link]
glibc has alternative implementations for some functions already before (e.g., glibc-2.28 contains names such as __memmove_ssse3, __memmove_avx512_unaligned, __memmove_avx512_no_vzeroupper and more). So this seems to be a mechanism to help select between such versions.Gnu C Library For Windows
GNU C library 2.33 released
Posted Feb 4, 2021 10:03 UTC (Thu) by arekm (subscriber, #4846) [Link]
GNU C library 2.33 released
Posted Feb 4, 2021 12:24 UTC (Thu) by fweimer (guest, #111581) [Link]
GNU C library 2.33 released
Posted Feb 4, 2021 12:54 UTC (Thu) by arekm (subscriber, #4846) [Link]
GNU C library 2.33 released
Posted Feb 4, 2021 21:03 UTC (Thu) by joib (subscriber, #8541) [Link]
But there is no interaction between the glibc-hwcaps and function multiversioning, no?
So the x86-64-v4 libc.so will still contain __memmove_ssse3 etc. though it will never be used?
GNU C library 2.33 released
Posted Feb 6, 2021 10:40 UTC (Sat) by anton (subscriber, #25547) [Link]
That could be the case, unless somebody adjust the source code to not compile it unless the level is v2.Anyway, reading this message makes the whole idea much clearer to me: The architecture levels are something that you set the compiler to compile for and for the dynamic linking mechanism to use in searching for libraries (not sure how it will work out for finding executables; do we have to set the PATH depending on which machine we are working on?). So the idea is that people will compile a library for, say, the v3 level, and if the auto-vectorizer finds something to vectorize, it will use AVX and AVX2 rather than, at most, SSE2.
So this will be more automatic and require less development effort to make use of instruction set extensions than what glibc does for its memmove variants.
I wonder whether the benefits are enough to make this a success. It reminds me of the x32 ABI; it had some demonstrated performance advantages over the i386 and x86-64 ABIs, but in the end it was not enough to make it fly.
GNU C library 2.33 released
Posted Feb 6, 2021 12:04 UTC (Sat) by joib (subscriber, #8541) [Link]
> The architecture levels are something that you set the compiler to compile for and for the dynamic linking mechanism to use in searching for libraries (not sure how it will work out for finding executables; do we have to set the PATH depending on which machine we are working on?). So the idea is that people will compile a library for, say, the v3 level, and if the auto-vectorizer finds something to vectorize, it will use AVX and AVX2 rather than, at most, SSE2.Yes, that is my understanding as well. I don't think there is an equivalent mechanism for executables (like you say it could be done with $PATH, but, ugh).
> So this will be more automatic and require less development effort to make use of instruction set extensions than what glibc does for its memmove variants.
Yes. Though they are separate, there is a (minor) bloat reduction potential in not generating function versions 'below' the microarchitecture level. But I guess the implementation could be somewhat hairy, and it's just a little bit of extra space on disk as it won't be loaded into memory, so maybe not worth worrying about.
> I wonder whether the benefits are enough to make this a success. It reminds me of the x32 ABI; it had some demonstrated performance advantages over the i386 and x86-64 ABIs, but in the end it was not enough to make it fly.
I think the big difference here is that it's not a new ABI, requiring the recompilation of everything. Rather this can be opt-in on a per-library basis, with the levels used chosen by the developers or distro packagers depending on what performance impact it might have. Say, glibc could provide all of x86-64{,-v2,-v3,-v4}, libfoo could provide x86-64{,-v3} and libbar could provide only the base x86-64, and depending on what hardware you run on the dynamic loader would use the best available variant of each library, mixing them freely.
GNU C library 2.33 released
Posted Feb 5, 2021 7:42 UTC (Fri) by atai (subscriber, #10977) [Link]
GNU C library 2.33 released
Posted Feb 7, 2021 17:27 UTC (Sun) by nix (subscriber, #2304) [Link]
Well upgrading my firewall was fun! We have more x86-32 rot! (This is *after* recovery, with all /lib links pointed back to 2.32.)nix@fold 4 /home/nix% /lib/ld-2.33.so --preload /lib/libc-2.33.so /bin/ls
/lib/libc-2.33.so: CPU ISA level is lower than required
Looks like -march=geode does not cause binutils to guess right, at least as of binutils 2.35 (yes, I know this is old, but it still shouldn't cause glibc to fail like this):
Displaying notes found in: .note.gnu.property
Owner Data size Description
GNU 0x0000000c NT_GNU_PROPERTY_TYPE_0
Properties: x86 ISA needed: x86-64-v2
(will report to the glibc list.)
Looks like the only way to turn off this stuff right now is to pass libc_cv_include_x86_isa_level=no to configure, which gentoo and others have just started doing.
Gnu C Library On Pc
GNU C library 2.33 released
Posted Feb 7, 2021 17:54 UTC (Sun) by nix (subscriber, #2304) [Link]
GNU C library 2.33 released
Posted Feb 8, 2021 13:49 UTC (Mon) by nix (subscriber, #2304) [Link]
Nope, this is real. march=... does not (reliably? at all) affect the ISA stamping done by sysdeps/x86/isa-level.c, leading to e.g. 32-bit glibcs being built on fast machines for execution on old slow machines being stamped as requiring whatever fast x86-64 box you did your builds on, even if it was built with -march=i586 or something. (ld.so validates this using the CPUID and has no way to override the validation, so everything seems to work when you test it on your build host, even if you test in a VM, and then fails disastrously when you install it on the target machine, with every binary failing to run, even though they would all run just fine if only ld.so let them).See this thread: https://sourceware.org/pipermail/libc-alpha/2021-February... (HJ's initial patch is definitely not sufficient).
For now, it's safest to turn off ISA stamping if you're building on one machine for execution on any machine which might be older (or otherwise of a lower ISA level). Of course there is no configure flag for this: you have to override the cache variable by adding libc_cv_include_x86_isa_level=no to your configure line.
GNU C library 2.33 released
Posted Feb 8, 2021 15:55 UTC (Mon) by nix (subscriber, #2304) [Link]
Other fallout: openssh fails on 32-bit platforms due to needing some 64-bit time syscalls added to the seccomp filter list. (Patch submitted.)Also some obscure --enable-stack-protector=all failures, but I suspect that use of that particular configure flag is quite rare (patches in preparation but they're all in the testsuite anyway and do not affect glibc itself).