2288 Commits

Author SHA1 Message Date
Rahul Joshi
6bed79b3f0
[Support] Add scaling support in indent (#109478)
Scaled indent is useful when indentation is always in steps of a fixed
number (the Scale) and still allow using the +/- operators to adjust
indentation.
2024-09-23 18:41:40 -07:00
Jay Foad
eb6e7e8f89
[unittests] Use {} instead of std::nullopt to initialize empty ArrayRef (#109388)
Follow up to #109133.
2024-09-21 10:59:50 +01:00
Fangrui Song
b84d773fd0 [Parallel] Revert sequential task changes
https://reviews.llvm.org/D148728 introduced `bool Sequential` to unify
`execute` and the old `spawn` without argument. However, sequential
tasks might be executed by any worker thread (non-deterministic),
leading to non-determinism output for ld.lld -z nocombreloc (see
https://reviews.llvm.org/D133003).

In addition, the extra member variables have overhead.
This sequential task has only been used for lld parallel relocation
scanning.

This patch restores the behavior before https://reviews.llvm.org/D148728 .

Fix #105958

Pull Request: https://github.com/llvm/llvm-project/pull/109084
2024-09-20 21:15:42 -07:00
Abhina Sreeskantharajan
efdb3ae232 Revert "[SystemZ][z/OS] Propagate IsText parameter to open text files as text (#107906)"
This reverts commit edf3b277a5f2ebe144827ed47463c22743cac5f9.
2024-09-20 08:18:16 -04:00
Nikita Popov
37e5319a12 [UnitTests] Fix APInt signed flags (NFC)
This makes unit tests compatible with the assertion added in
https://github.com/llvm/llvm-project/pull/106524, by setting the
isSigned flag to the correct value or changing how the value is
constructed.
2024-09-20 12:13:33 +02:00
Abhina Sree
edf3b277a5
[SystemZ][z/OS] Propagate IsText parameter to open text files as text (#107906)
This patch adds an IsText parameter to the following functions
openFileForRead, getBufferForFile, getBufferForFileImpl and determines
whether a file is text by querying the file tag on z/OS. The default is
set to OF_Text instead of OF_None, this change in value does not affect
any other platforms other than z/OS.
2024-09-19 14:30:10 -04:00
Rahul Joshi
2731be7ac5
[Support] Add helper struct indent for adding indentation (#108966)
Add helper struct indent() for adding indentation to raw_ostream.
2024-09-18 08:54:11 -07:00
Rahul Joshi
2f7ffbaad3
[Support] Fix bugs in formatv automatic index assignment (#108384)
Fix bugs found when actually trying to use formatv() automatic index
assignment in IntrinsicEmitter.cpp:
- Assign automatic index only for `ReplacementType::Format`.
- Make the check for all replacement indices being either automatic orexplicit more accurate.
  The existing check fails for formatv("{}{0}{}", 0, 1) (added as a unit test). Explicitly track if we
  have seen any explicit and any automatic index instead.
2024-09-16 21:48:05 -07:00
Jake Egan
53d60398ef
[CMake] Use old DynamicLibrary symbol behavior on AIX for now (#108692)
New behavior broke the AIX bot, so fall back to the old behavior on AIX
for now to give time to investigate
2024-09-16 10:57:06 -04:00
JOE1994
459a82e689 [llvm][unittests] Don't call raw_string_ostream::flush() (NFC)
raw_string_ostream::flush() is essentially a no-op (also specified in docs).
Don't call it in tests that aren't meant to test 'raw_string_ostream' itself.

p.s. remove a few redundant calls to raw_string_ostream::str()
2024-09-13 19:55:44 -04:00
JOE1994
77bab2a6f3 [llvm][unittests] Strip unneeded use of raw_string_ostream::str() (NFC)
Avoid unneeded layer of indirection.
2024-09-13 18:33:41 -04:00
Rahul Joshi
d5d6b44784
[Support] Add automatic index assignment in formatv (#107459)
Make index in replacement field optional. It will be automatically
assigned in incremental order by formatv.

Make mixed use of automatic and explicit indices an error that will fail
validation.
Adopt uses of formatv() within FormatVariadic to use automatic index.
2024-09-12 04:38:36 -07:00
Lang Hames
2e18f636fd Fix some unit tests for LLVM_ENABLE_EXPORTED_SYMBOLS_IN_EXECUTABLES=Off.
Building with -DLLVM_ENABLE_EXPORTED_SYMBOLS_IN_EXECUTABLES=Off should not
prevent the PluginsTests and DynamicLibraryTests unit tests from working.

This fix uses the approach implemented in
https://github.com/llvm/llvm-project/pull/101741.

rdar://135849875
2024-09-12 15:18:53 +10:00
Jan Svoboda
70fcdb3d52
[llvm][support] Implement tracing virtual file system (#88326)
LLVM-based tools often use the `llvm::vfs::FileSystem` instrastructure
to access the file system. This patch adds new kind of a VFS that
performs lightweight tracing of file system operations on an underlying
VFS. This is supposed to aid in investigating file system traffic
without resorting to instrumentation on the operating system level.
There will be follow-up patches that integrate this into Clang and its
dependency scanner.
2024-09-06 14:14:44 -07:00
Rahul Joshi
c2018fa40f
[NFC][Support] Refactor FormatVariadic code. (#106610)
- Rename `Align` field in ReplacementItem/FmtAlign to `Width` to 
  accurately reflect its use.
- Change both `Width` and `Index` in ReplacementItem to 32-bit int 
  instead of size_t (as 64-bits seems excessive in this context).
- Eliminate the use of `Empty` ReplacementType, and use the
  existing std::optional<> instead to indicate that.
- Eliminate some boilerplate type code in formatv().
- Eliminate the loop in `splitLiteralAndReplacement`. The existing
  code will never loop back.
- Directly use constructor instead of std::make_pair.
2024-09-05 09:01:19 -07:00
Rahul Joshi
ad30a050ca
[NFC][Support] Add FormatVariadic sub-test for validation (#106578)
- Add validation subtest that tests assert failures in assert enabled
  builds, and that validation is disabled in assert disabled builds.
2024-09-02 06:27:12 -07:00
Daniil Fukalov
89e6a28867
[NFC] Add explicit #include llvm-config.h where its macros are used. (#106621)
Without these explicit includes, removing other headers, who implicitly
include llvm-config.h, may have non-trivial side effects.
2024-08-30 09:35:06 +02:00
Rahul Joshi
9ce4af5cad
Revert "Revert "[Support] Validate number of arguments passed to formatv()"" (#106592)
Reverts llvm/llvm-project#106589
The fix for bot failures caused by the reverted commit was committed
already, so this revert is not needed.
2024-08-29 10:39:40 -07:00
Mehdi Amini
ed37b5f6c3
Revert "[Support] Validate number of arguments passed to formatv()" (#106589)
Reverts llvm/llvm-project#105745

Some bots are broken apparently.
2024-08-29 10:30:11 -07:00
Rahul Joshi
115b87636b
[NFC][Support] Eliminate ',' at end of MemoryEffects print (#106545)
- Eliminate comma at end of a MemoryEffects print.
- Added basic unit test to validate that.
2024-08-29 09:28:45 -07:00
Rahul Joshi
5048fabb05
[Support] Delete FormatVariadicTest Validate sub-test (#106570)
- The subtest, if enabled correctly, will fail with assert in Debug
  builds and validation is disabled in Release builds.
- Hence deleting the test to fix test failures in CI.
2024-08-29 08:40:15 -07:00
Rahul Joshi
fc110202df
[Support] Validate number of arguments passed to formatv() (#105745)
Change formatv() to validate that the number of arguments passed matches
number of replacement fields in the format string, and that the replacement
indices do not contain holes.

To support cases where this cannot be guaranteed, introduce a formatv()
overload that allows disabling validation with a bool flag as its first argument.
2024-08-29 08:00:25 -07:00
Kazu Hirata
22e55ba329
[llvm] Prefer StringRef::substr to StringRef::slice (NFC) (#106330)
S.substr(N) is simpler than S.slice(N, StringRef::npos). Also, substr
is probably better recognizable than slice thanks to
std::string_view::substr.
2024-08-28 01:13:46 -07:00
Fangrui Song
7c06786479
[CMake] Remove HAVE_LINK_H
We can remove the variable from https://reviews.llvm.org/D5610 since
link.h is available on Linux (glibc/musl/Bionic), FreeBSD, and NetBSD.
Use `__has_include(<link.h>)` before including it.

Pull Request: https://github.com/llvm/llvm-project/pull/104893
2024-08-20 08:50:24 -07:00
Ramkumar Ramachandra
7dd6340bda
MathExtras: template'ize alignToPowerOf2 (#97814)
Follow up on 5627794 (MathExtras: avoid unnecessarily widening types) to
change the overflow behavior of alignToPowerOf2 to only overflow if the
result is not representable in the return type. This allows us to
template'ize it, and avoid unnecessarily widening the types of
arguments.
2024-08-19 16:03:42 +01:00
Steven Wu
2596464dcd [CMake] Re-land #102671 after the flaky test is disabled
Reland #102671 after the other attempted fixes are all flawed in some
build configurations. The previous try for the same commit was breaking
a flaky tests under ASAN which is not marked as unsupported. This fix
should be good to go now.

Fixes AIX bot for DynamicLibraryTests failure.
2024-08-13 15:18:09 -07:00
Steven Wu
29314b4b27 Revert "[CMake] Fix DynamicLibraryTests exports symbol when plugins are enabled (#102941)"
This reverts commit a9604cd3212beb6cdb41a79f248723140ab9939d.

The export list in unit-tests will break sanitizer build since that
requires more symbols than listed in the export file.
2024-08-13 13:59:12 -07:00
Steven Wu
a9604cd321
[CMake] Fix DynamicLibraryTests exports symbol when plugins are enabled (#102941)
Exporting symbols from tools directory never worked with
`LLVM_EXPORT_SYMBOLS_FOR_PLUGINS=On`. When that options is on, only
symbols from the static library is linked is exported due to the export
symbols computation script. DynamicLibraryTests needs to export a symbol
from the tools/executable directory, so update it to use export list
instead.
2024-08-13 09:56:02 -07:00
Simon Pilgrim
11ba72e651
[KnownBits] Add KnownBits::add and KnownBits::sub helper wrappers. (#99468) 2024-08-12 10:21:28 +01:00
Steven Wu
435654b3d2 Revert "[CMake] Followup to #102396 and restore old DynamicLibrary symbols behavior (#102671)"
This reverts commit 32973b08d8cb02c213d96df453ff323470304645. This fix
doesn't fix the build failure as expected and making few other
configuration broken too.
2024-08-11 22:35:01 -07:00
Steven Wu
32973b08d8
[CMake] Followup to #102396 and restore old DynamicLibrary symbols behavior (#102671)
Followup to #102138 and #102396, restore more old behavior to fix
ppc64-aix bot.
2024-08-11 12:58:44 -07:00
Steven Wu
01b488faab
Reapply "[CMake] Fold export_executable_symbols_* into function args. (#101741)" (#102138)
Fix the builds with LLVM_TOOL_LLVM_DRIVER_BUILD enabled.

LLVM_ENABLE_EXPORTED_SYMBOLS_IN_EXECUTABLES is not completely
compatible with export_executable_symbols as the later will be ignored
if the previous is set to NO.

Fix the issue by passing if symbols need to be exported to
llvm_add_exectuable so the link flag can be determined directly
without calling export_executable_symbols_* later.
2024-08-07 09:12:15 -07:00
Steven Wu
f9b69a378c Revert "[CMake] Fold export_executable_symbols_* into function args. (#101741)"
This reverts commit 5c56b46a32a8856a022a54291bc9294068f7ddbd. This break
lld build when using GENERATE_DRIVER.
2024-08-06 06:08:16 -07:00
Alexis Engelke
c2f92fa3ab
[Support] Store dominator tree nodes in a vector (#101705)
Use basic block numbers to store dominator tree nodes in a vector. This
avoids frequent map lookups. Use block number epochs to validate that no
renumbering occured since the tree was created; after a renumbering, the
dominator tree can be updated with updateBlockNumbers().

Block numbers, block number epoch, and max block number are fetched via
newly added GraphTraits methods. Graphs which do not implement
getNumber() for blocks will use a DenseMap for an ad-hoc numbering.
2024-08-06 09:15:22 +02:00
Steven Wu
5c56b46a32
[CMake] Fold export_executable_symbols_* into function args. (#101741)
`LLVM_ENABLE_EXPORTED_SYMBOLS_IN_EXECUTABLES` is not completely
compatible with `export_executable_symbols` as the later will be ignored
if the previous is set to NO.

 Fix the issue by passing if symbols need to be exported to
`llvm_add_exectuable` so the link flag can be determined directly
without calling `export_executable_symbols_*` later.
2024-08-05 19:08:27 -07:00
Kazu Hirata
7df9da7d78
[llvm] Construct SmallVector with ArrayRef (NFC) (#101872) 2024-08-04 08:54:23 -07:00
David Blaikie
45ef0d492f Add llvm::Error C API, LLVMCantFail
It's barely testable - the test does exercise the code, but wouldn't
fail on an empty implementation. It would cause a memory leak though
(because the error handle wouldn't be unwrapped/reowned) which could be
detected by asan and other leak detectors.
2024-07-31 16:40:26 +00:00
Alex Langford
4aa4ee9090
[Support] Add SpecificBumpPtrAllocator::identifyObject (#100475)
This already exists in BumpPtrAllocatorImpl, but I would like to use it
from SpecificBumpPtrAllocator.

I also noticed there was no test for identifyObject so I added one.
2024-07-25 14:05:00 -07:00
Connor Sughrue
76321b9f08
[llvm][Support] Implement raw_socket_stream::read with optional timeout (#92308)
This PR implements `raw_socket_stream::read`, which overloads the base
class `raw_fd_stream::read`. `raw_socket_stream::read` provides a way to
timeout the underlying `::read`. The timeout functionality was not added
to `raw_fd_stream::read` to avoid needlessly increasing compile times
and allow for convenient code reuse with `raw_socket_stream::accept`,
which also requires timeout functionality. This PR supports the module
build daemon and will help guarantee it never becomes a zombie process.
2024-07-21 23:50:28 -04:00
Ramkumar Ramachandra
f1eed011b4
MathExtras: add overflow query for signed-div (#97901)
5221634 (Do not trigger UB during AffineExpr parsing) noticed that
divideCeilSigned and divideFloorSigned would overflow when Numerator =
INT_MIN, and Denominator = -1. This observation has already been made by
DynamicAPInt, and it has code to check this. To avoid checks in multiple
callers, centralize this query in MathExtras, and change
divideCeilSigned/divideFloorSigned to assert on overflow.
2024-07-09 09:33:46 +01:00
Alexis Engelke
d8c07342c0
[Support] Move raw_ostream::tie to raw_fd_ostream (#97396)
Originally, tie was introduced by D81156 to flush stdout before writing
to stderr. 030897523 reverted this due to race conditions. Nonetheless,
it does cost performance, causing an extra check in the "cold" path,
which is actually the hot path for raw_svector_ostream. Given that this
feature is only used for errs(), move it to raw_fd_ostream so that it no
longer affects performance of other stream classes.
2024-07-03 11:15:02 +02:00
Kazu Hirata
ddaa93b095
[llvm] Use std::make_unique (NFC) (#97165)
This patch is based on clang-tidy's modernize-make-unique but limited
to those cases where type names are mentioned twice like
std::unique_ptr<Type>(new Type()), which is a bit mouthful.
2024-06-29 11:50:41 -07:00
Ramkumar Ramachandra
56277948d5
MathExtras: avoid unnecessarily widening types (#95426)
Several multi-argument functions unnecessarily widen types beyond the
argument types. Template'ize the functions, and use std::common_type_t
to avoid this, hence optimizing the functions. A requirement of this
patch is to change the overflow behavior of alignTo to only overflow
when the result isn't representable in the return type.
2024-06-29 12:16:47 +01:00
Brendan Duke
f991ebbb46
[Support] Add llvm::xxh3_128bits (#95863)
Add a 128-bit xxhash function, following the existing
`llvm::xxh3_64bits` and `llvm::xxHash` implementations. Previously,
48e93f57f1ee914ca29aa31bf2ccd916565a3610 added support for
`llvm::xxh3_64bits`, which closely follows the upstream implementation
at https://github.com/Cyan4973/xxHash, with simplifications from Devin
Hussey's xxhash-clean.
However, it is desirable to have a larger 128-bit hash key for use cases
such as filesystem checksums where chance of collision needs to be
negligible.

So to that end this also ports over the 128-bit xxh3_128bits as
`llvm::xxh3_128bits`.

Testing:
- Add a test based on xsum_sanity_check.c in upstream xxhash.
2024-06-19 15:24:54 -04:00
Xuan Zhang
d9a00ed366
[MachineOutliner] Leaf Descendants (#90275)
This PR  depends on https://github.com/llvm/llvm-project/pull/90264

In the current implementation, only leaf children of each internal node
in the suffix tree are included as candidates for outlining. But all
leaf descendants are outlining candidates, which we include in the new
implementation. This is enabled on a flag `outliner-leaf-descendants`
which is default to be true.

The reason for _enabling this on a flag_ is because machine outliner is
not the only pass that uses suffix tree.

The reason for _having this default to be true_ is because including all
leaf descendants show consistent size win.
* For Clang/LLD, it shows around 3% reduction in text segment size when
compared to the baseline `-Oz` linker binary.
 * For selected benchmark tests in LLVM test suite 
 
| run (CTMark/) | only leaf children | all leaf descendants | reduction
% |

|------------------|--------------------|----------------------|-------------|
| lencod | 349624 | 348564 | -0.2004% |
| SPASS | 219672 | 218440 | -0.4738% |
| kc | 271956 | 250068 | -0.4506% |
| sqlite3 | 223920 | 222484 | -0.5471% |
| 7zip-benchmark | 405364 | 401244 | -0.3428% |
| bullet | 139820 | 138340 | -0.8315% |
| consumer-typeset | 295684 | 286628 | -1.2295% |
| pairlocalalign | 72236 | 71936 | -0.2164% |
| tramp3d-v4 | 189572 | 183676 | -2.9668% |

This is part of an enhanced version of machine outliner -- see
[RFC](https://discourse.llvm.org/t/rfc-enhanced-machine-outliner-part-1-fulllto-part-2-thinlto-nolto-to-come/78732).
2024-06-18 07:13:05 -07:00
Ramkumar Ramachandra
bfd95a0031
MathExtras: rewrite some methods to never overflow (#95556)
Rewrite divideCeil, divideNearest, divideFloorSigned, and
divideCeilSigned to never overflow.
2024-06-15 10:22:53 +01:00
Ahmed Bougacha
61069bd5a3
[Support] Add SipHash-based 16-bit ptrauth ABI-stable hash. (#93902)
This finally wraps the now-lightly-modified SipHash C reference
implementation, for the main interface we need (16-bit ptrauth
discriminators).

The exact algorithm is the little-endian interpretation of the
non-doubled (i.e. 64-bit) result of applying a SipHash-2-4 using the
constant seed `b5d4c9eb79104a796fec8b1b428781d4` (big-endian), with the
result reduced by modulo to the range of non-zero discriminators (i.e.
`(rawHash % 65535) + 1`).

By "stable" we mean that the result of this hash algorithm will the same
across different compiler versions and target platforms.

The 16-bit hashes are used extensively for the AArch64 ptrauth ABI,
because AArch64 can efficiently load a 16-bit immediate into the high
bits of a register without disturbing the remainder of the value, which
serves as a nice blend operation.

16 bits is also sufficiently compact to not inflate a loader relocation.
We disallow zero to guarantee a different discriminator from the places
in the ABI that use a constant zero.

Co-authored-by: John McCall <rjmccall@apple.com>
2024-06-14 17:57:12 -07:00
Ahmed Bougacha
577c3f114b
[Support] Integrate SipHash.cpp into libSupport. (#94394)
Start building it as part of the library, with some minor
tweaks compared to the reference implementation:
- clang-format to match libSupport
- remove tracing support
- add file header
- templatize cROUNDS/dROUNDS, as well as 8B/16B result length
- replace assert with static_assert
- use LLVM_FALLTHROUGH

This also exports interfaces for SipHash-2-4-64/-128, and
tests them using the reference test vectors.
2024-06-14 17:07:41 -07:00
Ahmed Bougacha
cfbed2c0e6
[Support] Import SipHash c reference implementation. (#94393)
This brings the unmodified SipHash reference implementation:
  https://github.com/veorq/SipHash
which has been very graciously licensed under our llvm license
(Apache-2.0 WITH LLVM-exception) by Jean-Philippe Aumasson.

SipHash is a lightweight hash function optimized for speed on short
messages. We use it as part of the AArch64 ptrauth ABI (in arm64e and
ELF PAuth) to generate discriminators based on language identifiers and
mangled names.

This commit brings the unmodified reference implementation and tests
as of f26d35e, specifically siphash.c and vectors.h, as SipHash.cpp and
SipHashTest.cpp.

Next, we will integrate it properly into libSupport, with a wrapping API
suited for the ptrauth use-case.
2024-06-14 16:56:04 -07:00
Nikita Popov
44df1167f8
[Error] Add non-consuming toString (#95375)
There are some places that want to convert an Error to string, but still
retain the original Error object, for example to emit a non-fatal
warning.

This currently isn't possible, because the entire Error infra is
move-based. And what people end up doing in this case is to move the
Error... twice.

This patch introduces a toStringWithoutConsuming() function to
accommodate this use case. This also requires some infrastructure that
allows visiting Errors without consuming them.
2024-06-14 11:35:27 +02:00