This adds code to set the `builtin` attribute when needed on operator
new and delete calls. This also required setting `nobuiltin` for
replaceable global allocation functions and threading the `builtin`
attribute through the LLVM dialect.
This is used somewhat rarely, but is a pretty simple emission of
pointers, and ends up using infrastructure we already have.
Additionally, this is the first use of `getNaturalTypeAlignment` that
uses the `pointee` argument, so this adds the implementation there,
which includes some alignment work for CXXRecordDecls, so this
implements that as well.
We are currently only emitting Vtables that have an 'immediate' need to
emit. There rest, we are supposed to add to a list and emit at the end
of the translation unit if necessary. This patch implements that
infrastructure.
The test added is from classic-codegen and came in at the same time as
the deferred vtable emission over there, and only works with deferred
vtable emission, and while it does test the deferred emission, tests
quite a bit more than that. AND since it came in with the same
functionality in classic codegen, seemed to make sense to come in here
too.
Our current implementation of string lowering did some work to remove
extra trailing zeros, plus do a 'zero' constant. That is unchanged by
this patch. However, this patch ALSO ensures that we do the 'remove
extra trailing zeros' to remove ALL trailing zeros, which likely has
canonicalization benefits later on.
However, the real benefit of this patch is to make string emission by
default emit a null-terminator, which fixes the virtual table 'name'
field get lowered correctly. We do this by making the builder::getString
function take an argument (true by default) that will ensure we add a
null terminator if necessary.
This reflects the llvm::ConstantDataArray::getString function, which has
the same functionality. However, doing this during lowering seems
incorrect, since the FE is the one that knows whether these null
terminators are necessary. There is not currently an 'opt out' use of
the behavior, but the functionality is left in place to better reflect
the ConstantDataArray::getString function interface.
Note with the tests that this fixes some inconsistencies between LLVM
and OGCG lowering.
This patch adds typeid lowering, which uses a lot of the infrastructure
from dynamic_cast. However, this adds a `get_type_info` operation that
gets the type info out of a vtable pointer as well, which lets the
offset be handled by the ABI specific lowering code.
This change upstreams the CIR implementation of global variable
replacement. When we get a call to get or create a global variable using
a type that does not match the previous type for a variable of the same
name, we need to replace the old definition with the new one. In classic
codegen that was as simple as replaceAllUses+eraseFromParent, but in CIR
because we have typed pointers, we need to visit the uses and update
them with bitcasts to reflect the new type.
This implements vtable thunk handling in CIR based on the incubator
code, but also compared against the latest Clang LLVM IR codegen.
Eventually, we'll want to create CIR abstractions for all of this and
move the CXXABI-specific details into the CXXABI lowering pass. For now,
we just implement it directly in codegen.
These are fairly simple, particularly if they don't need special
cleanups (which is left unimplemented), but this provides init for a
global reference variable.
Similar to what I did for return types in #181052, this patch adds
support for 4 of the function attributes on arguments (noundef, nonnull,
dereferenceable, align). The logic for these is all pretty similar
(though SLIGHTLY different enough from eachother unfortunately), so they
are being submitted together. This handles 'this' and normal arguments.
This patch implements the infrastructure for return attributes on
function/call operations, a little of the common infrastructure for arg
attributes on the same, and 4 return attributes lowering: noundef
nonnull
dereferenceable
align
These 4 common attributes are all pretty reasonable/common, so these
will change a lot of tests.
This patch chooses to just use the LLVM-IR-Dialect variant of these
attributes (as a NamedAttr), which means no changes to the dialect or
lowering are necessary.
This implements the lowering of static local variables with the Itanium C++ ABI
guard variable pattern in LoweringPrepare. This is initial support, errorNYI covering all that hasn't been added just yet.
When a GlobalOp has the static_local attribute and a ctor region, this pass:
1. Creates a guard variable global (mangled name from AST)
2. Inserts the guard check pattern at each GetGlobalOp use site:
- Load guard byte with acquire ordering
- If zero, call __cxa_guard_acquire
- If acquire returns non-zero, inline the ctor region code
- Call __cxa_guard_release
3. Clears the static_local attribute and ctor region from the GlobalOp
Once the new design doc lands I'll add more information over there.
This adds CIRGen infrastructure for C++ function-local static variables
that require guarded initialization (Itanium C++ ABI).
Changes:
- Add ASTVarDeclAttr to carry VarDecl AST through the pipeline
- Add emitGuardedInit() to CIRGenCXXABI for guarded initialization
- Add emitCXXGuardedInit() to CIRGenFunction
- Replace NYI in addInitializerToStaticVarDecl() with ctor region emission
- Set static_local attribute on GlobalOp and GetGlobalOp
This doesn't unwrap the high level constructs just yet - The global's ctor region contains the initialization code, which will be lowered by LoweringPrepare to emit the actual guard variable pattern with __cxa_guard_acquire/__cxa_guard_release calls.
related: #175871
This patch adds foundational infra for device-side CUDA/HIP compilation
by introducing NVPTX target info and implementing the global emission
filtering logic.
NVPTX Target Info to allows us to compile against that triple:
- Add NVPTXABIInfo and NVPTXTargetCIRGenInfo classes
- Wire up nvptx and nvptx64 triples in getTargetCIRGenInfo()
- Add createNVPTXTargetCIRGenInfo() factory function
CUDA/HIP Global Emission Filtering (most of this is boilerplate from the
AST) This basically narrows down to:
- Skip host-only functions (no `__device__` attribute) when
`-fcuda-is-device`
- Skip device-only functions (device without host) on host side
- Always emit ` __global__` kernels and `__host__` `__device__`
functions on both sides
- Add `shouldEmitCUDAGlobalVar()` to handle variable emission
(device/constant/shared variables)
- Handle special cases: implicit host/device templates, lambda call
operators
Part of #175871
This patch adds the initial implementation of the CUDA/NV Runtimes
generating code for the device stub body. tested on CUDA. HIP coverage
to be added in a later PR.
Previous efforts in both the incubator and upstreamed added the 'pure'
and 'const' attribute lowering to calls. This patch completes that
effort by making it also appear on functions with this attribute.
While looking into the work required to add attributes to
functions/arguments/etc, I discovered a lot of overlap with calling
conventions, which I know others are working on. Rather than risk a
later conflict, this patch adds some basic infrastructure that will
hopefully limit the fallout. This patch is NFC as it doesn't modify the
IR at all, just adds a few calls to the 'right place' and adds some
arguments.
Additionally, it adds the calling convention type to the IR, but leaves
it empty, as it is useful to be able to pass around but not particularly
possible to define/make do anything yet.
Part of: #153267
This is a continuation of: #154014
This patch handles output operands for inline assembly, taking the
original patch, but adding some additional tests, plus responding to all
of the original comments, plus doing some of my personal cleanup
(including extracting addVariableConstraints in a separate patch).
---------
Co-authored-by: Iris Shi <0.0@owo.li>
Implement emitDeclInvariant to emit llvm.invariant.start intrinsic for
global variables with constant storage. This enables optimizations by
marking when a global becomes read-only after initialization.
## Changes
- Add emitDeclInvariant and emitInvariantStart functions in
CIRGenCXX.cpp
- Add emitInvariantStart declaration in CIRGenFunction.h
- Update emitCXXGlobalVarDeclInit to call emitDeclInvariant for constant
storage globals after initialization
- Update getOrCreateCIRGlobal to set constant flag on globals with
constant storage
- Add comprehensive test covering positive and negative cases
## Implementation Details
The implementation handles address spaces correctly, dynamically
constructing the intrinsic name (e.g., invariant.start.p0,
invariant.start.p10) based on the pointer's address space. The intrinsic
is only emitted when optimizations are enabled, matching classic codegen
behavior.
## Testing
All tests pass (411/412, 1 unsupported). The test file includes CIR,
LLVM, and OGCG checks for both optimized and non-optimized builds.
This patch adds the basic infrastructure for lowering an OpenMP
directive, which should enable someone to take over the OpenMP lowering
in the future. It adds the lowering entry points to CIR in the same way
as OpenACC.
Note that this does nothing with any of the directives, which will
happen in a followup patch. No infrastructure for clauses is added
either, but that will come in a followup patch as well.
This PR migrates support for wide string literals from the incubator to
upstream.
## Changes
- Implement wide string literal support in
`getConstantArrayFromStringLiteral`
- Handle wchar_t, char16_t, and char32_t string literals
- Collect code units and create constant arrays with IntAttr elements
- Use ZeroAttr for null-filled strings
## Testing
- Copied `wide-string.cpp` test file from incubator
- Expanded test to include wchar_t test cases (incubator only had
char16_t and char32_t)
- All tests pass
---------
Co-authored-by: Andy Kaylor <akaylor@nvidia.com>
The bind clause specifies the name of the function to call on the
device, and takes either a string or identifier(per the standard):
"If the name is specified as an identifier, it is callled as if the name
were specified in the language being compiled. If the name is specified
as a string, the string is used for the procedure name unmodified".
The latter (as a string) is already implemented, this patch implements
the former. Unfortunately, no existing implementation of this in C++
seems to exist. Other languages, the 'name' of a function is sufficient
to identify it (in this case 'bind' can refer to undeclared functions),
so it is possible to figure out what the name should be. In C++ with
overloading (without a discriminator, ala-fortran), a name only names an
infinite overload set.
SO, in order to implement this, I've decided that the 'called as'
(bound) function must have the same signature as the one marked by the
'routine'. This is trivially sensible in non-member functions, however
requires a bit more thought for member(and thus lambda-call-operators)
functions. In this case, we 'promote' the type of the function to a
'free' function by turning the implicit 'this' to an explicit 'this'.
I believe this is the most sensible and reasonable way to implement
this, and really the only way to make something usable.
This commit adds full support for thread-local storage variables in
ClangIR, including code generation, lowering to LLVM IR, and
comprehensive testing.
Changes include:
- Added CIR_TLSModel enum with 4 TLS models (GeneralDynamic,
LocalDynamic, InitialExec, LocalExec) to CIROps.td
- Extended GlobalOp with optional tls_model attribute
- Extended GetGlobalOp with thread_local unit attribute
- Added verification to ensure thread_local GetGlobalOp references
globals with tls_model set
- Implemented GetDefaultCIRTLSModel() and setTLSMode() in CIRGenModule
- Updated getAddrOfGlobalVar() to handle TLS access
- Removed MissingFeatures assertions for TLS operations
- Added lowering of GetGlobalOp with TLS to llvm.threadlocal.address
intrinsic
- Added lowering of GlobalOp with tls_model to LLVM thread_local globals
- Added comprehensive test with CIR, LLVM, and OGCG checks
Known limitations (matching incubator):
- Static local TLS variables not yet implemented
- TLS_Dynamic with wrapper functions not yet implemented
Fixes#153270
This PR upstreams support for the `cir.indirectBr` operation, which is
used to implement GCC’s labels-as-values `indirect goto`.
To ensure correct lowering, we introduce precise bookkeeping to
associate each `block_address` operation with its corresponding `label`
op. This is required because a `block_address` may be emitted before the
`label` it refers to. In such cases, the reference is deferred and later
resolved by `resolveBlockAddresses`, which guarantees that all
`indirectBr` successors are emitted in the correct and fully resolved
order.
The 'routine' construct just adds a acc.routine element to the global
module, which contains all of the information about the directive. it
contains a reference to the function, which also contains a reference to
the acc.routine, which this generates.
This handles both the implicit-func version (where the routine is
spelled without parens, and just applies to the next function) and
the explicit-func version (where the routine is spelled with the func
name in parens).
The AST stores the directive in an OpenACCRoutineDeclAttr in the
implicit case, so we can emit that when we hit the function declaration.
The explicit case is held in an OpenACCRoutineAnnotAttr on the function,
however, when we emit the function we haven't necessarily seen the
construct yet, so we can't depend on that attribute. Instead, we save up
the list in Sema so that we can emit them all at the end.
This results in the tests getting really hard to read (because ordering
is a little awkward based on spelling, with no way to fix it), so we
instead split the tests up based on topic.
One last thing: Flang spends some time determining if the clause lists
of two routines on the same function are identical, and omits the
duplicates. However, it seems to do a poor job on this when the ordering
isn't the same, or references are slightly different. This patch doesn't
bother trying that, and instead emits all, trusting the ACC dialect to
remove duplicates/handle duplicates gracefully.
Note; This doesn't cause emission of functions that would otherwise not
be emitted, but DOES emit routine references based on which function
they are attached to.
Switches to more efficient explicit enum property instead of a wrapped
storage, simplifying the string representation. The attribute is now
placed before the symbol name for consistency with other FuncOp
attributes. FileCheck patterns are also simplified to match only the
attributes under test.
This patch does the lowering for a 'declare' construct that is not a
function-local-scope. It also does the lowering for 'create', which has
an entry-op of create and exit-op of delete.
Global/NS/Struct scope 'declare's emit a single 'acc_ctor' and
'acc_dtor' (except in the case of 'link') per variable referenced. The
ctor is the entry op followed by a declare_enter. The dtor is a
get_device_ptr, followed by a declare_exit, followed by a delete(exit
op). This DOES include any necessary bounds.
This patch implements all of the above. We use a separate 'visitor' for
the clauses here since it is particularly different from the other uses,
AND there are only 4 valid clauses. Additionally, we had to split the
modifier conversion into its own 'helpers' file, which will hopefully
get some additional use in the future.
This PR adds a special member attribute to `cir::FuncOp`. This attribute
is also present in the incubator repo. Additionally, I added a
"is_trivial" flag, to mark trivial members. I think that might be useful
when trying to replace calls to the copy constructor with memcpy for
example, but please let me know your thoughts on this. [Here in the
incubator
repo](823e943d1b/clang/lib/CIR/Dialect/Transforms/LoweringPrepare.cpp (L1537-L1550))
this function is called `LowerTrivialConstructorCall`, but I don't see a
check that ensures the constructor is actually trivial.
We previously added support for marking GlobalOp operations as constant,
but the handling to actually do so was left mostly unimplemented. This
fills in the missing pieces.
related: #160386
Add support for address space conversions in CIR.
- Added `createAddrSpaceCast` methods to `CIRBaseBuilderTy` to handle
address space conversions
- Implemented address space conversion handling in `emitCastLValue` and
`VisitCastExpr`
- Added `performAddrSpaceCast` method to `TargetCIRGenInfo` for
target-specific address space casting
- Added `getLangTempAllocaAddressSpace` to `CIRGenModule` to get the
language-specific address space for temporary allocations
- Added a test file `address-space-conversion.cpp` to verify address
space conversion functionality
This patch adds the minimal support for array cookies needed to enable
ClangIR generation for an array new expression that requires cookies but
does not require an explicit initializer.
This only provides the cookie support for the base Itanium CXXABI.
Different cookie calculations are required for AppleARM64, which will be
added in a subsequent patch.
Ported from ClangIR incubator PR
https://github.com/llvm/clangir/pull/1297.
This is the second PR in a series intended to address
https://github.com/llvm/llvm-project/issues/160383.
---------
Co-authored-by: Andy Kaylor <akaylor@nvidia.com>
This patch implements the handling of inline builtin functions in CIR.
There is a known limitation in CIR where direct calls to shadowed inline
builtin functions are generated instead of the intrinsic. This is
expected to be fixed by the introduction of the nobuiltin attribute in a
future patch.
Unlike the incubator, this adds the inline attribute directly to FuncOp
instead of adding the ExtraFnAttr dict.
This adds three new optional keywords to CIR: inline_always,
inline_never and inline_hint. Just like in OGCG -O0 implies inline_never
on functions withoutt the C++ `inline` keyword and no other
inlining-related attribute.
This patch also adapts all tests that use functions so they account for
LLVM attributes being attached now.
This adds the code needed to emit alloca operations for variable length
array local variables and the necessary calls to stacksave and
stackrestore to adjust the local stack as the array variables go in an
out of scope.
This adds support for adding the `global_ctor` or `global_dtor`
attribute to the CIR representation of functions defined with
`__attribute__((constructor))` or `__attribute__((destructor))` and
adding them to the `@llvm.global_ctors` or `@llvm.global_dtors` list
during lowering to LLVM IR.
This adds support for dynamic cast handling and generating
`cir.dyn_cast` operations and `cir.dyn_cast_info` attributes.
This does not include support for lowering the dynamic cast to LLVM IR,
which will require changes to the LoweringPrepare pass that will be made
in a future change.
This also does not yet handle dynamic cast to void or exact dynamic
casts.
This PR adds support for address spaces in CIR pointer types by:
1. Introducing a `TargetAddressSpaceAttr` to represent target-specific
numeric address spaces (A Lang-specific attribute is to be implemented
in a different PR)
2. Extending the `PointerType` to include an optional address space
parameter
3. Adding helper methods in `CIRBaseBuilder` to create pointers with
address spaces
4. Implementing custom parsers and printers for address space attributes
5. Updating the LLVM lowering to properly handle address spaces when
converting CIR to LLVM IR
The implementation allows for creating pointers with specific address
spaces, which is necessary for supporting language features like Clang's
`__attribute__((address_space(N)))`. Address spaces are preserved
through the CIR representation and correctly lowered to LLVM IR.
