related: #175871
This patch adds foundational infra for device-side CUDA/HIP compilation
by introducing NVPTX target info and implementing the global emission
filtering logic.
NVPTX Target Info to allows us to compile against that triple:
- Add NVPTXABIInfo and NVPTXTargetCIRGenInfo classes
- Wire up nvptx and nvptx64 triples in getTargetCIRGenInfo()
- Add createNVPTXTargetCIRGenInfo() factory function
CUDA/HIP Global Emission Filtering (most of this is boilerplate from the
AST) This basically narrows down to:
- Skip host-only functions (no `__device__` attribute) when
`-fcuda-is-device`
- Skip device-only functions (device without host) on host side
- Always emit ` __global__` kernels and `__host__` `__device__`
functions on both sides
- Add `shouldEmitCUDAGlobalVar()` to handle variable emission
(device/constant/shared variables)
- Handle special cases: implicit host/device templates, lambda call
operators
Part of #175871
This patch adds the initial implementation of the CUDA/NV Runtimes
generating code for the device stub body. tested on CUDA. HIP coverage
to be added in a later PR.
Previous efforts in both the incubator and upstreamed added the 'pure'
and 'const' attribute lowering to calls. This patch completes that
effort by making it also appear on functions with this attribute.
While looking into the work required to add attributes to
functions/arguments/etc, I discovered a lot of overlap with calling
conventions, which I know others are working on. Rather than risk a
later conflict, this patch adds some basic infrastructure that will
hopefully limit the fallout. This patch is NFC as it doesn't modify the
IR at all, just adds a few calls to the 'right place' and adds some
arguments.
Additionally, it adds the calling convention type to the IR, but leaves
it empty, as it is useful to be able to pass around but not particularly
possible to define/make do anything yet.
Part of: #153267
This is a continuation of: #154014
This patch handles output operands for inline assembly, taking the
original patch, but adding some additional tests, plus responding to all
of the original comments, plus doing some of my personal cleanup
(including extracting addVariableConstraints in a separate patch).
---------
Co-authored-by: Iris Shi <0.0@owo.li>
Implement emitDeclInvariant to emit llvm.invariant.start intrinsic for
global variables with constant storage. This enables optimizations by
marking when a global becomes read-only after initialization.
## Changes
- Add emitDeclInvariant and emitInvariantStart functions in
CIRGenCXX.cpp
- Add emitInvariantStart declaration in CIRGenFunction.h
- Update emitCXXGlobalVarDeclInit to call emitDeclInvariant for constant
storage globals after initialization
- Update getOrCreateCIRGlobal to set constant flag on globals with
constant storage
- Add comprehensive test covering positive and negative cases
## Implementation Details
The implementation handles address spaces correctly, dynamically
constructing the intrinsic name (e.g., invariant.start.p0,
invariant.start.p10) based on the pointer's address space. The intrinsic
is only emitted when optimizations are enabled, matching classic codegen
behavior.
## Testing
All tests pass (411/412, 1 unsupported). The test file includes CIR,
LLVM, and OGCG checks for both optimized and non-optimized builds.
This patch adds the basic infrastructure for lowering an OpenMP
directive, which should enable someone to take over the OpenMP lowering
in the future. It adds the lowering entry points to CIR in the same way
as OpenACC.
Note that this does nothing with any of the directives, which will
happen in a followup patch. No infrastructure for clauses is added
either, but that will come in a followup patch as well.
This PR migrates support for wide string literals from the incubator to
upstream.
## Changes
- Implement wide string literal support in
`getConstantArrayFromStringLiteral`
- Handle wchar_t, char16_t, and char32_t string literals
- Collect code units and create constant arrays with IntAttr elements
- Use ZeroAttr for null-filled strings
## Testing
- Copied `wide-string.cpp` test file from incubator
- Expanded test to include wchar_t test cases (incubator only had
char16_t and char32_t)
- All tests pass
---------
Co-authored-by: Andy Kaylor <akaylor@nvidia.com>
The bind clause specifies the name of the function to call on the
device, and takes either a string or identifier(per the standard):
"If the name is specified as an identifier, it is callled as if the name
were specified in the language being compiled. If the name is specified
as a string, the string is used for the procedure name unmodified".
The latter (as a string) is already implemented, this patch implements
the former. Unfortunately, no existing implementation of this in C++
seems to exist. Other languages, the 'name' of a function is sufficient
to identify it (in this case 'bind' can refer to undeclared functions),
so it is possible to figure out what the name should be. In C++ with
overloading (without a discriminator, ala-fortran), a name only names an
infinite overload set.
SO, in order to implement this, I've decided that the 'called as'
(bound) function must have the same signature as the one marked by the
'routine'. This is trivially sensible in non-member functions, however
requires a bit more thought for member(and thus lambda-call-operators)
functions. In this case, we 'promote' the type of the function to a
'free' function by turning the implicit 'this' to an explicit 'this'.
I believe this is the most sensible and reasonable way to implement
this, and really the only way to make something usable.
This commit adds full support for thread-local storage variables in
ClangIR, including code generation, lowering to LLVM IR, and
comprehensive testing.
Changes include:
- Added CIR_TLSModel enum with 4 TLS models (GeneralDynamic,
LocalDynamic, InitialExec, LocalExec) to CIROps.td
- Extended GlobalOp with optional tls_model attribute
- Extended GetGlobalOp with thread_local unit attribute
- Added verification to ensure thread_local GetGlobalOp references
globals with tls_model set
- Implemented GetDefaultCIRTLSModel() and setTLSMode() in CIRGenModule
- Updated getAddrOfGlobalVar() to handle TLS access
- Removed MissingFeatures assertions for TLS operations
- Added lowering of GetGlobalOp with TLS to llvm.threadlocal.address
intrinsic
- Added lowering of GlobalOp with tls_model to LLVM thread_local globals
- Added comprehensive test with CIR, LLVM, and OGCG checks
Known limitations (matching incubator):
- Static local TLS variables not yet implemented
- TLS_Dynamic with wrapper functions not yet implemented
Fixes#153270
This PR upstreams support for the `cir.indirectBr` operation, which is
used to implement GCC’s labels-as-values `indirect goto`.
To ensure correct lowering, we introduce precise bookkeeping to
associate each `block_address` operation with its corresponding `label`
op. This is required because a `block_address` may be emitted before the
`label` it refers to. In such cases, the reference is deferred and later
resolved by `resolveBlockAddresses`, which guarantees that all
`indirectBr` successors are emitted in the correct and fully resolved
order.
The 'routine' construct just adds a acc.routine element to the global
module, which contains all of the information about the directive. it
contains a reference to the function, which also contains a reference to
the acc.routine, which this generates.
This handles both the implicit-func version (where the routine is
spelled without parens, and just applies to the next function) and
the explicit-func version (where the routine is spelled with the func
name in parens).
The AST stores the directive in an OpenACCRoutineDeclAttr in the
implicit case, so we can emit that when we hit the function declaration.
The explicit case is held in an OpenACCRoutineAnnotAttr on the function,
however, when we emit the function we haven't necessarily seen the
construct yet, so we can't depend on that attribute. Instead, we save up
the list in Sema so that we can emit them all at the end.
This results in the tests getting really hard to read (because ordering
is a little awkward based on spelling, with no way to fix it), so we
instead split the tests up based on topic.
One last thing: Flang spends some time determining if the clause lists
of two routines on the same function are identical, and omits the
duplicates. However, it seems to do a poor job on this when the ordering
isn't the same, or references are slightly different. This patch doesn't
bother trying that, and instead emits all, trusting the ACC dialect to
remove duplicates/handle duplicates gracefully.
Note; This doesn't cause emission of functions that would otherwise not
be emitted, but DOES emit routine references based on which function
they are attached to.
Switches to more efficient explicit enum property instead of a wrapped
storage, simplifying the string representation. The attribute is now
placed before the symbol name for consistency with other FuncOp
attributes. FileCheck patterns are also simplified to match only the
attributes under test.
This patch does the lowering for a 'declare' construct that is not a
function-local-scope. It also does the lowering for 'create', which has
an entry-op of create and exit-op of delete.
Global/NS/Struct scope 'declare's emit a single 'acc_ctor' and
'acc_dtor' (except in the case of 'link') per variable referenced. The
ctor is the entry op followed by a declare_enter. The dtor is a
get_device_ptr, followed by a declare_exit, followed by a delete(exit
op). This DOES include any necessary bounds.
This patch implements all of the above. We use a separate 'visitor' for
the clauses here since it is particularly different from the other uses,
AND there are only 4 valid clauses. Additionally, we had to split the
modifier conversion into its own 'helpers' file, which will hopefully
get some additional use in the future.
This PR adds a special member attribute to `cir::FuncOp`. This attribute
is also present in the incubator repo. Additionally, I added a
"is_trivial" flag, to mark trivial members. I think that might be useful
when trying to replace calls to the copy constructor with memcpy for
example, but please let me know your thoughts on this. [Here in the
incubator
repo](823e943d1b/clang/lib/CIR/Dialect/Transforms/LoweringPrepare.cpp (L1537-L1550))
this function is called `LowerTrivialConstructorCall`, but I don't see a
check that ensures the constructor is actually trivial.
We previously added support for marking GlobalOp operations as constant,
but the handling to actually do so was left mostly unimplemented. This
fills in the missing pieces.
related: #160386
Add support for address space conversions in CIR.
- Added `createAddrSpaceCast` methods to `CIRBaseBuilderTy` to handle
address space conversions
- Implemented address space conversion handling in `emitCastLValue` and
`VisitCastExpr`
- Added `performAddrSpaceCast` method to `TargetCIRGenInfo` for
target-specific address space casting
- Added `getLangTempAllocaAddressSpace` to `CIRGenModule` to get the
language-specific address space for temporary allocations
- Added a test file `address-space-conversion.cpp` to verify address
space conversion functionality
This patch adds the minimal support for array cookies needed to enable
ClangIR generation for an array new expression that requires cookies but
does not require an explicit initializer.
This only provides the cookie support for the base Itanium CXXABI.
Different cookie calculations are required for AppleARM64, which will be
added in a subsequent patch.
Ported from ClangIR incubator PR
https://github.com/llvm/clangir/pull/1297.
This is the second PR in a series intended to address
https://github.com/llvm/llvm-project/issues/160383.
---------
Co-authored-by: Andy Kaylor <akaylor@nvidia.com>
This patch implements the handling of inline builtin functions in CIR.
There is a known limitation in CIR where direct calls to shadowed inline
builtin functions are generated instead of the intrinsic. This is
expected to be fixed by the introduction of the nobuiltin attribute in a
future patch.
Unlike the incubator, this adds the inline attribute directly to FuncOp
instead of adding the ExtraFnAttr dict.
This adds three new optional keywords to CIR: inline_always,
inline_never and inline_hint. Just like in OGCG -O0 implies inline_never
on functions withoutt the C++ `inline` keyword and no other
inlining-related attribute.
This patch also adapts all tests that use functions so they account for
LLVM attributes being attached now.
This adds the code needed to emit alloca operations for variable length
array local variables and the necessary calls to stacksave and
stackrestore to adjust the local stack as the array variables go in an
out of scope.
This adds support for adding the `global_ctor` or `global_dtor`
attribute to the CIR representation of functions defined with
`__attribute__((constructor))` or `__attribute__((destructor))` and
adding them to the `@llvm.global_ctors` or `@llvm.global_dtors` list
during lowering to LLVM IR.
This adds support for dynamic cast handling and generating
`cir.dyn_cast` operations and `cir.dyn_cast_info` attributes.
This does not include support for lowering the dynamic cast to LLVM IR,
which will require changes to the LoweringPrepare pass that will be made
in a future change.
This also does not yet handle dynamic cast to void or exact dynamic
casts.
This PR adds support for address spaces in CIR pointer types by:
1. Introducing a `TargetAddressSpaceAttr` to represent target-specific
numeric address spaces (A Lang-specific attribute is to be implemented
in a different PR)
2. Extending the `PointerType` to include an optional address space
parameter
3. Adding helper methods in `CIRBaseBuilder` to create pointers with
address spaces
4. Implementing custom parsers and printers for address space attributes
5. Updating the LLVM lowering to properly handle address spaces when
converting CIR to LLVM IR
The implementation allows for creating pointers with specific address
spaces, which is necessary for supporting language features like Clang's
`__attribute__((address_space(N)))`. Address spaces are preserved
through the CIR representation and correctly lowered to LLVM IR.
This PR adds new `FuncOp` attributes (`coroutine` and `builtin`) and
begins the implementation of the `emitCoroutineBody` function. Feature
markers were also added for guidance in future PRs.
This adds support for emitting pseudo-macro expressions that represent
some form of the name of a function (such as `__func__` or
`__PRETTY_FUNCTION__`) as l-values.
This adds support for handling global variables with non-trivial
constructors. The constructor call is emitted in CIR as a 'ctor' region
associated with the global definition. This form of global definition
cannot be lowered to LLVM IR yet.
A later change will add support in LoweringPrepare to move the ctor code
into a __cxx_global_var_init() function and add that function to the
list of global global ctors, but for now we must stop at the initial CIR
generation.
This sets the MLIR module name to the main filename (according to the
SourceManager), if one is available. The module name gets used when
creating global init functions, so we will need it to be set.
This patch adds basic support for constant record initializer list
expressions. There's a couple of limitations:
* No zero initialized padding bytes in C mode
* No bitfields
* No designated initializer lists
* Record alignments are not calculated, yet
* ILEs of derived records don't work, yet
* The constant attribute is not propagated to the backend, resulting in
non-constants being emitted in the LLVM IR
This changes a bunch of places which use getAs<TagType>, including
derived types, just to obtain the tag definition.
This is preparation for #155028, offloading all the changes that PR used
to introduce which don't depend on any new helpers.
The getCIRSourceLanguage wasn't returning a value if the source language
was anything other than C or C++. This change updates that function to
return a std::optional value and only adds the source language attribute
if one was returned.
This patch adds the constant attribute to cir.global, the appropriate
lowering to LLVM constant and updates the tests.
---------
Co-authored-by: Andy Kaylor <akaylor@nvidia.com>
This change introduces the #cir.global_view attribute and adds support
for using that attribute to handle initializing a global variable with
the address of another global variable.
This does not yet include support for the optional list of indices to
get an offset from the base address. Those will be added in a follow-up
patch.
