Switches the following headers to hdrgen-produced ones by referencing
some macro from C standard and the file containing the declarations in
corresponding YAML files:
* limits.h (referenced _WIDTH / _MAX / _MIN families).
* locale.h (referenced LC_ family).
* time.h (referenced CLOCKS_PER_SEC).
* wchar.h (referenced WEOF).
The in-process ThinLTO backend typically generates object files in
memory and adds them directly to the link, except when the ThinLTO cache
is in use. DTLTO is unusual in that it adds files to the link from disk
in all cases.
When the ThinLTO cache is not in use, ThinLTO adds files via an
`AddStreamFn` callback provided by the linker, which ultimately appends
to a `SmallVector` in LLD. When the cache is in use, the linker supplies
an `AddBufferFn` callback that adds files more efficiently (by moving
`MemoryBuffer` ownership).
This patch adds a mandatory `AddBufferFn` to the DTLTO ThinLTO backend.
The backend uses this to add files to the link more efficiently.
Additionally:
- Move AddStream from CGThinBackend to InProcessThinBackend, for reader
clarity.
- Modify linker comments that implied the AddBuffer path is
cache-specific.
For a Clang link (Debug build with sanitizers and instrumentation) using
an optimized toolchain (PGO non-LTO, llvmorg-22.1.0), measuring the mean
`Add DTLTO files to the link` time trace scope duration:
- On Windows (Windows 11 Pro Build 26200, AMD Family 25 @ ~4.5 GHz, 16
cores/32 threads, 64 GB RAM), this patch reduces the mean from
2799.148 ms to 157.972 ms.
- On Linux (Ubuntu 24.04.3 LTS Kernel 6.14, Ryzen 9 5950X, 16
cores/32 threads, boost up to 5.09 GHz, 64 GB RAM), this patch reduces
the mean from 255.291 ms to 41.630 ms.
Based on work by @romanova-ekaterina and @kbelochapka.
This reverts commit 11b439c5c5a07c95d30ce25abd6adf7f5fbb7105.
timeTraceProfilerCleanup() can be called before the temporary file
deletion has completed in LLD. This causes memory leaks that were
flagged up by sanitizer builds, e.g.:
https://lab.llvm.org/buildbot/#/builders/24/builds/18840/steps/11/logs/stdio
Added a brief description of the libc-shared-tests target to the
Building and Testing page.
This target allows running tests for shared standalone components like
math primitives without the full libc runtime.
`loopUnrollByFactor` used `getConstantIntValue()` to read loop bounds,
which sign-extends the constant to `int64_t`. For unsigned `scf.for`
loops with narrow integer types (e.g. i1, i2, i3), this produces wrong
results: a bound such as `1 : i1` has `getSExtValue() == -1` but should
be treated as `1` (unsigned).
Two bugs were introduced by this:
1. **Wrong epilogue detection**: the comparison `upperBoundUnrolledCst <
ubCst` used signed int64, so e.g. `0 < -1` (where ubCst is the
sign-extended i1 value 1) evaluated to false, suppressing the epilogue
that should execute the remaining iterations.
2. **Zero step after overflow**: when `tripCountEvenMultiple == 0` (all
iterations go to the epilogue), `stepUnrolledCst = stepCst *
unrollFactor` can overflow the bound type's bitwidth and wrap to 0. A
zero step causes `constantTripCount` to return `nullopt`, preventing the
zero-trip main loop from being elided.
Fix:
- Use zero-extension (`getZExtValue`) instead of sign-extension when
reading bounds for unsigned loops.
- When `tripCountEvenMultiple == 0`, keep the original step for the main
loop to avoid the zero-step issue (the step value is irrelevant for a
zero-trip loop anyway).
Fixes#163743
Assisted-by: Claude Code
This PR adds scalar type to convert_layout op's result and operand. It
also enhance convert_layout pattern in wg-to-sg, unrolling, and
sg-to-lane distribution.
It is to support reduction to scalar, whether currently the layout
propagation doesn't support scalar to carry any layout. The design
choice to insert convert_layout op after reduction-to-scalar op to
record the layout information permanently across the passes.
Fix broken POSIX basedefs links for nested headers in llvm-libc docs.
The docgen script currently emits paths like `sys/wait.h.html`, but the
Open Group uses `sys_wait.h.html`, for example:
-
https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/sys_wait.h.html
This updates nested-header link generation while leaving flat headers
unchanged.
This is to highlight places where we (probably unintentionally)
construct an `Address` object from an already resolved address, making
it unresolved again.
See the changes in `DynamicLoaderDarwin.cpp` for a quick example.
Also, use this constructor instead of `Address(lldb::addr_t file_addr,
const SectionList *section_list)` when `section_list` is `nullptr`.
The revision clears a long-due TODO, which supports the lowering when
transfer_read/write ops have mask via inserting a vector.shape_cast op
for the masked value.
---------
Signed-off-by: hanhanW <hanhan0912@gmail.com>
Refactor to proceed #185964.
Much of this is a refactor to address this issues. Instead of iterating over one chain at a time, attempting all VFs for that given change, we now iterate over VFs, trying each chain for the current VF.
Includes fix for use after free bug.
Summary:
Currently, building through runtimes will yield this warning:
```
CMake Warning at compiler-rt/cmake/Modules/CompilerRTUtils.cmake:335 (message):
LLVMTestingSupport not found in LLVM_AVAILABLE_LIBS
Call Stack (most recent call first)
```
This is due to the fact that the builtins target does not go through the
s tandard runtimes patch and sets them as BUILDTREE_ONLY so they do not
show up. These are not used in this case, so just guard the condition to
suppress the warning.
[This
case](bc35715696/llvm/lib/Target/SPIRV/SPIRVEmitIntrinsics.cpp (L2815-L2818))
is not covered by any existing test (checked via code coverage and
inserting an `abort`). New test proposed that covers this line, as
demonstrated by test failure when an `abort` is present in that line.
The Microsoft mangle implementation does not produce back-references for
anonymous namespaces, which results in nonsensical output from both
`undname` and `llvm-undname`. Consider the following example:
```
namespace {
struct X {};
X foo(X, X);
}
int main() {
foo({}, {});
}
```
Clang 22.1.0
```
?foo@?A0xC9C482F4@@YA?AUX@?A0xC9C482F4@@U1?A0xC9C482F4@@0@Z
undname:
struct `anonymous namespace'::X __cdecl `anonymous namespace'::foo(struct `anonymous namespace'::A0xC9C482F4,struct `anonymous namespace'::A0xC9C482F4)
llvm-undname:
struct `anonymous namespace'::X __cdecl `anonymous namespace'::foo(struct `anonymous namespace'::0xC9C482F4, struct `anonymous namespace'::0xC9C482F4)
```
MSVC 19.50
```
?foo@?A0xa6a4f20e@@YA?AUX@1@U21@0@Z
undname:
struct A0xa6a4f20e::X __cdecl `anonymous namespace'::foo(struct A0xa6a4f20e::X,struct A0xa6a4f20e::X)
llvm-undname:
struct 0xa6a4f20e::X __cdecl `anonymous namespace'::foo(struct 0xa6a4f20e::X, struct 0xa6a4f20e::X)
```
As seen in the undname output for Clang's mangling, not recording a name
back-reference for the anonymous namespace when mangling results in
references to the anonymous namespace's hash in-place of the struct's
name.
When recompiling the example with the changes, Clang yields
`?foo@?A0x69FA3AC2@@YA?AUX@1@U21@0@Z`. Aside from the meaningless
discrepancy with the hash, this is identical to what is produced by
MSVC.
Resolves#37999
Correct the colors used in the OpenMP support tables so they
consistently match their status text:
- :good: (green) is for 'done' only
- :part: (yellow) is for in-progress states ('partial', 'worked on', 'in
progress', 'prototyped', etc.)
- :none: (red) is for 'unclaimed' only
Assisted with copilot
Previously there was a single rule for stdbit, this PR adds generated
header targets for the rest of the linux headers. It also adds a
cc_library
for all of the public headers which also includes the types and macros
headers.
LLVMLoadStoreToOCLPattern::matchAndRewrite was calling op->removeAttr()
and op->setOperand() directly without going through the rewriter API.
This caused MLIR_ENABLE_EXPENSIVE_PATTERN_API_CHECKS to report "expected
pattern to replace the root operation or modify it in place".
Fix: wrap the direct mutations in rewriter.modifyOpInPlace().
Assisted-by: Claude Code
Fix a failure present with MLIR_ENABLE_EXPENSIVE_PATTERN_API_CHECKS=ON.
After PR #150846 changed MCFragment to use fixed 16KB blocks, large
sections like .debug_names produce thousands of fragments. The
DW_IDX_parent emission called emitLabelDifference(ParentSymbol,
EntryPool, 4) per entry, which fell through to
attemptToFoldSymbolOffsetDifference's O(n) fragment walk when symbols
were in different fragments. This caused O(entries * fragments)
quadratic behavior, regressing dsymutil on large binaries like clang.
Fix by pre-computing byte offsets from EntryPool for each entry in a
first pass, then emitting emitInt32(offset) directly for DW_IDX_parent
instead of symbol differences. This also eliminates per-entry MCSymbol
allocation that was only needed for parent references.
Fix#188376
Issue #152743 reports that DefaultValuedProp is printed even when the
property value equals the default, unlike DefaultValuedAttr which is not
printed in that case.
The fix for this was already present in the codebase since commit
8955e285e1ac ("[mlir] Add property combinators, initial ODS support"),
which added elision of default-valued properties in the
genPropDictPrinter
function in OpFormatGen.cpp.
This commit adds:
- Documentation in Operations.md clarifying that DefaultValuedProp is
also elided from prop-dict output when the value equals the default,
consistent with the existing documentation for DefaultValuedAttr.
- An explicit test in properties.mlir verifying that DefaultValuedProp
with value equal to default is elided from prop-dict output, and that
DefaultValuedProp with a non-default value is still printed.
Fixes#152743
Assisted-by: Claude Code
Fix two issues in `MatchingSubsets::populateSubsetOpsAtIterArg`:
1. The `collectHoistableOps` parameter was declared but never used when
inserting subset ops via `insert(subsetOp)`. As a result, when recursing
into nested loops with `collectHoistableOps=false`, the nested loop's
subset ops were incorrectly added to the hoistable extraction/insertion
pairs of the parent loop. This caused spurious failures in the
`allDisjoint` check, preventing valid hoisting when nested loop ops
overlapped with outer loop ops. Fix by passing the parameter:
`insert(subsetOp, collectHoistableOps)`.
2. In the nested loop handling branch, there was no guard to detect when
a value has multiple nested loop uses (i.e., is used as an init arg in
more than one nested loop). Without the guard, `nextValue` would be
silently overwritten, leading to an incorrect use-def chain traversal.
Add `if (nextValue) return failure()` before setting `nextValue` for the
nested loop case, mirroring the existing guard for insertion ops.
Fixes#147096
Assisted-by: Claude Code
SwitchOp::getEntrySuccessorRegions and getRegionInvocationBounds called
IntegerAttr::getInt() to retrieve the constant switch argument, but
getInt() asserts that the attribute type must be a signless integer or
index. For unsigned integer types (e.g. ui32), this assertion fired and
crashed the process.
Fix by selecting the appropriate accessor based on the attribute type:
getInt() for signless/index, getSInt() for signed, and getUInt() (cast
to int64_t) for unsigned integer types. Unknown types fall back to the
conservative "all regions possible" path.
The same fix is applied to getRegionInvocationBounds, which had an
identical call to getInt().
Fixes#187973
Assisted-by: Claude Code
The RemoveDeadValuesPass previously emitted an error and skipped
optimization when the IR contained non-function symbol ops, non-call
symbol user ops, or branch ops. This restriction was later removed, but
the comments in RemoveDeadValues.cpp and Passes.td still described the
pass as operating "iff the IR doesn't have any non-function symbol ops,
non-call symbol user ops and branch ops."
Remove the stale restriction text from both the .cpp file comment and
the Passes.td description. Also add a test that verifies dead function
arguments are correctly removed inside a module that defines a symbol
(has a sym_name attribute), which was the original failure case reported
in issue #98700.
Fixes#98700
Assisted-by: Claude Code
Calling attribute setters and MutableOperandRange::assign() without
going through the PatternRewriter, bypassing the rewriter's
change-tracking triggered "operation finger print changed" after the
pattern returned success under
MLIR_ENABLE_EXPENSIVE_PATTERN_API_CHECKS.
Assisted-by: Claude Code
LLVMLoadStoreToOCLPattern::matchAndRewrite was calling op->removeAttr()
and op->setOperand() directly without going through the rewriter API.
This caused MLIR_ENABLE_EXPENSIVE_PATTERN_API_CHECKS to report "expected
pattern to replace the root operation or modify it in place".
Fix: wrap the direct mutations in rewriter.modifyOpInPlace().
Assisted-by: Claude Code
Fix a failure present with MLIR_ENABLE_EXPENSIVE_PATTERN_API_CHECKS=ON.
Add explicit tests for condition propagation in scf.if then and else
branches, including the void-return case. These tests serve as
regression
tests for the bug reported in #159165 where the
SCFIfConditionPropagationPass
(since reverted) had a visited-set that was never populated, causing the
pass
to not propagate conditions into nested scf.if statements.
The current ConditionPropagation canonicalization pattern in SCF.cpp
correctly handles both nested ifs and direct condition uses within
branches
using the getParentType() ancestor check.
Fixes#159165
Assisted-by: Claude Code
LegalizeLaunchFuncOpPattern previously rejected gpu.launch_func ops with
more than one async dependency. This change removes that limitation by
synchronizing additional dependencies onto the primary stream using
CUDA/HIP events, following the same approach already used in
ConvertWaitAsyncOpToGpuRuntimeCallPattern for gpu.wait async.
For each additional async dependency beyond the first:
- If it is a stream (produced by mgpuStreamCreate), create an event,
record it on that stream, wait for it on the primary stream, then
destroy the event.
- If it is already an event, wait for it directly on the primary stream
and destroy it.
Fixes#156984
Assisted-by: Claude Code
The DebugValueManager constructor scans the basic block to find all DEBUG_VALUE
uses of a def. This can lead to quadratic behavior as reported in #168326.
Instead, use MRI.use_instructions to find only the def's uses to limit the
search.
We still scan the BB so that we get the uses in order, and so we can stop
when a new def is found.
This is flakey on the buildbot.
https://lab.llvm.org/buildbot/#/builders/141/builds/16719 as reported on
#188751.
```
FAIL: test_step_over_with_python_api_dwarf (TestInlineStepping.TestInlineStepping.test_step_over_with_python_api_dwarf)
Test stepping over and into inlined functions.
---------------------------------------------------------------------- Traceback (most recent call last):
File "C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\lldb\packages\Python\lldbsuite\test\lldbtest.py", line 2019, in test_method
return attrvalue(self)
^^^^^^^^^^^^^^^
File "C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\lldb\test\API\functionalities\inline-stepping\TestInlineStepping.py", line 25, in test_step_over_with_python_api
self.inline_stepping_step_over()
File "C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\lldb\test\API\functionalities\inline-stepping\TestInlineStepping.py", line 317, in inline_stepping_step_over
self.run_step_sequence(step_sequence)
File "C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\lldb\test\API\functionalities\inline-stepping\TestInlineStepping.py", line 152, in run_step_sequence
self.do_step(step_pattern[1], target_line_entry, test_stack_depth)
File "C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\lldb\test\API\functionalities\inline-stepping\TestInlineStepping.py", line 83, in do_step
self.fail(
AssertionError: Failed to stop due to step into operation stepping to: C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\lldb\test\API\functionalities\inline-stepping\calling.cpp:46
```
[This
body](73768f88fb/llvm/lib/Target/SPIRV/SPIRVEmitIntrinsics.cpp (L857-L860))
is not covered by any existing test (checked via code coverage and
inserting an abort). New test proposed that covers this line, as
demonstrated by test failure when an abort is present in that line.
Motivated by the need from IREE side to support ArgMax/ArgMin-like
operation using dpp and ballot operation (refer to
https://github.com/iree-org/iree/discussions/23609#discussioncomment-16311655
for more details), this PR adds `gpu.ballot` operation to the MLIR GPU
dialect with ROCDL, NVVM, and SPIR-V lowering support.
Assisted-by: [Claude Code](https://claude.ai/code)
---------
Signed-off-by: Bangtian Liu <liubangtian@gmail.com>
Fix the documented support set for AArch64 `-ffixed-xN` options to match
the current implementation.
This updates the generated option help text so that only the registers
actually supported by AArch64 are listed as `AArch64/RISC-V only`, and
marks `x8`, `x16`, `x17`, `x19`, and `x29`-`x31` as `RISC-V only`.
It also adds a driver test to cover the representative unsupported
AArch64 case for `-ffixed-x8`.
Fixes#186620.
Tests: `build-cir/bin/llvm-lit -sv
clang/test/Driver/aarch64-fixed-x-register.c`
74e4694 moved ObjCARCContract from running before the codegen pipeline
into addISelPrepare(), which runs after PreISelIntrinsicLowering.
This broke ObjCARCContract's retainRV-to-claimRV optimization because
ObjCARCContract identifies ARC calls via intrinsics, not their lowered
counterparts.
This patch restores the pre-74e4694 ordering by moving ObjCARCContract
to addISelPasses.
The IntrinsicInst.cpp change looks extraneous but is required here:
ObjCARCContract may now rewrite the bundle operand from retainRV to
claimRV. When PreISelIntrinsicLowering then encounters this new
intrinsic use, lowerObjCCall asserts mayLowerToFunctionCall.
Assisted-by: claude
rdar://137997453
Added a "Building and Testing with an Emulator" section to
full_cross_build.rst using riscv64 and qemu-riscv64 as the example.
Outlined necessary CMake flags for cross-compiling with Clang, including
CMAKE_C_COMPILER_TARGET, CMAKE_CXX_COMPILER_TARGET, and
LLVM_ENABLE_LLD=ON. Switched from CMAKE_SYSROOT to LIBC_KERNEL_HEADERS
and added the gcc-riscv64-linux-gnu package dependency to ensure sysroot
issues on Debian-based systems are avoided while retaining access to
cross-compiler runtime objects.
Explained the self-hosted libc-hermetic-tests target as the required
target for executing tests during a standalone cross build, since the
standard check-libc tests are not hermetic.
Refactored existing CMake examples in full_cross_build.rst to use -S and
-B flags instead of cd and mkdir.
Removed prompt characters from code blocks and separated host
environment setup into explicit code blocks for easier copy-pasting.
Also removed the initial 'cd llvm-project' directory changes to reduce
boilerplate.
Added a cross-reference to the new emulator section in
build_and_test.rst.
This is a follow-up fix for commit 0f5e9bee.
Only write effects to thread-local memory should be considered safe to
parallelize in workshare lowering, not reads. When both reads and writes
were safe, the cascading effect in moveToSingle could cause entire
SingleRegions to become fully parallelized, eliminating the omp.single
and its implicit barrier. This removed synchronization points needed to
keep threads coordinated inside sequential loops containing workshared
operations, causing race conditions in forall-workshare patterns.
This was exposed by the Fujitsu Test Suite and made the following tests
regress:
FAIL: test-suite :: Fujitsu/Fortran/0398/Fujitsu-Fortran-0398_0031.test
FAIL: test-suite :: Fujitsu/Fortran/0398/Fujitsu-Fortran-0398_0013.test
FAIL: test-suite :: Fujitsu/Fortran/0398/Fujitsu-Fortran-0398_0030.test
FAIL: test-suite :: Fujitsu/Fortran/0398/Fujitsu-Fortran-0398_0014.test
Updates #143330
This PR adds `global_prefetch` op to prefetch a cache line to high-level
caches using the aligned address of the source `memref` and an offset
provided by the indices of the element containing the cache line. This
provides temporal hints (e.g., regular or high-priority). Note that
out-of-bounds access is allowed in speculative mode. Ensure the source
`memref` is in address space `1`.
---------
Co-authored-by: Krzysztof Drewniak <Krzysztof.Drewniak@amd.com>
The `--int-range-optimizations` pass runs the `DataFlowSolver` once,
then calls `applyPatternsGreedily` with a `DataFlowListener` that erases
solver state when ops are deleted. However, the greedy driver's
`simplifyRegions` step (which calls `runRegionDCE` between pattern
iterations) can remove block arguments without notifying the listener.
This frees the `BlockArgumentImpl` storage, which may be reused by a
subsequent allocation. The solver then finds stale lattice state keyed
at the reused address and incorrectly treats the new block argument as a
known constant, causing a miscompile.
The existing `enableFolding(false)` was added for the same class of bug
(folding can also remove block arguments). This patch extends the fix by
also disabling region simplification, preventing dead-arg elimination
from causing the same address-reuse problem.
Fixes#137281Fixes#126195
Assisted-by: Claude Code
The scf-for-loop-range-folding pass transforms loops of the form
for (i = lb; i < ub; i += step) { use(i * c) }
into
for (j = lb*c; j < ub*c; j += step*c) { use(j) }
This transformation is only valid when c is strictly positive, since
scf.for requires a positive step. When c is zero or negative, the new
step becomes zero or effectively negative (wrapping in unsigned
arithmetic for index type), producing an incorrect loop.
Add a guard that restricts the MulIOp folding to cases where the
loop-invariant multiplier is a statically known positive integer
constant. Non-constant loop-invariant multipliers are also excluded
since their sign cannot be determined at compile time.
Fixes#56235Fixes#116664
Assisted-by: Claude Code
Add support for the half-precision complementary error function
`erfcf16``, using a Sollya generated polynomial implementation with
proper handling of special cases.
Extend the MPFR utilities with erfc support to allow tests.
closes: #180927
