This test was problematic, and also unnecessary. It's not really
a test of the libc functionality or ABI. That's already covered
by the LlvmLibcStackChkFail.Death test. The Smash test was in
fact just testing that the compiler produces the call in the
expected situation. That's a compiler test, not a libc test.
It's not really feasible to make a test like this both reliable
and safe. Since it's not something libc needs to test, it's not
worth trying.
Previously this test was entirely disabled under asan, but not
hwasan. Instead of disabling the test, make the test compatible
with both asan and hwasan by disabling sanitizers only on the
subroutine that does the stack-smashing.
Currently the nan* functions use nullptr dereferencing to crash with
SIGSEGV if the input is nullptr. Both `nan(nullptr)` and `nullptr`
dereferencing are undefined behaviors according to the C standard.
Employing `nullptr` dereference in the `nan` function implementation is
ok if users only linked against the pre-built library, but it might be
completely removed by the compilers' optimizations if it is built from
source together with the users' code.
See for instance: https://godbolt.org/z/fd8KcM9bx
This PR uses volatile load to prevent the undefined behavior if libc is
built without sanitizers, and leave the current undefined behavior if
libc is built with sanitizers, so that the undefined behavior can be
caught for users' codes.
Use a size smaller than the smallest supported page size so that we
don't
clobber over any guard pages, which may result in a segfault before
__stack_chk_fail can be called.
Also, move __stack_chk_fail outside of our namespace.
__stack_chk_fail should be provided by libc.a, not startup files.
Add __stack_chk_fail to existing linux and arm entrypoints. On Windows
(when
not targeting MinGW), it seems that the corresponding function
identifier is
__security_check_cookie, so no entrypoint is added for Windows.
Baremetal
targets also ought to be compileable with `-fstack-protector*`
There is no common header for this prototype, since calls to
__stack_chk_fail
are meant to be inserted by the compiler upon function return when
compiled
`-fstack-protector*`.