The `SourceLocation` of a `RootSignatureToken` is incorrectly set to be
the "offset" into the concatenated string that denotes the
rootsignature. This causes an issue when the `StringLiteral` is a
multi-line expansion macro, since the offset will not account for the
characters between `StringLiteral` tokens.
This pr resolves this by retaining the `SourceLocation` information that
is kept in `StringLiteral` and then converting the offset in the
concatenated string into the proper `SourceLocation` using the
`StringLiteral::getLocationOfByte` interface. To do so, we will need to
adjust the `RootSignatureToken` to only hold its offset into the root
signature string. Then when the parser will use the token, it will need
to compute its actual `SourceLocation`.
See linked issue for more context.
For example:
```
#define DemoRootSignature \
"CBV(b0)," \
"RootConstants(num32BitConstants = 3, b0, invalid)"
expected caret location ---------------^
actual caret location ------------^
```
The caret points 5 characters early because the current offset did not
account for the characters:
```
'"' ' ' '\' ' ' '"'
1 2 3 4 5
```
- Updates `RootSignatureParser` to retain `SourceLocation` information
by retaining the `StringLiteral` and passing the underlying `StringRef`
to the `Lexer`
- Updates `RootSignatureLexer` so that the constructed tokens only
reflect an offset into the `StringRef`
- Updates `RootSignatureParser` to directly construct its used `Lexer`
so that the `StringLiteral` is directly tied with the string used in the
`RootSignatureLexer`
- Updates `RootSignatureParser` to use
`StringLiteral::getLocationOfByte` to get the actual token location for
diagnostics
- Updates `ParseHLSLRootSignatureTest` to construct a phony
`AST`/`StringLiteral` for the test cases
- Adds a test to `RootSignature-err.hlsl` showing that the
`SourceLocation` is correctly set for diagnostics in a multi-line macro
expansion
Resolves: https://github.com/llvm/llvm-project/issues/146967
- this takes care to add support to match the [behaviour of
DXC](34b6d0f91e/tools/clang/lib/Parse/HLSLRootSignature.cpp (L74))
acceptable floating point integers
Namely:
- Allow for specifying the decimal '.'
- Allow for specifying exponents with 'e' or 'E' and allow for 'f' to
denote an otherwise interpreted integer as a float
This pr is simply responsible of creating a token that could be
interpeted as a floating point integer by `NumericLiteralParser`. As
such, we are not required to validate that the special characters only
occur once and that 'f' is only at the end of the string. These will be
validated when invoking `NumericLiteralParser` during parsing.
Resolves#126565
- when developing the RootSignatureLexer library, we are creating new
files so we should set the standard to adhere to the coding conventions
for function naming
- this was missed in the initial review but caught in the review of the
parser pr
[here](https://github.com/llvm/llvm-project/pull/133302#discussion_r2017632092)
Co-authored-by: Finn Plummer <finnplummer@microsoft.com>
This pr relands https://github.com/llvm/llvm-project/pull/133302.
It resolves two issues:
- Linking error during build,
[here](https://github.com/llvm/llvm-project/pull/133302#issuecomment-2767259848).
There was a missing dependency for `clangLex` for the
`ParseHLSLRootSignatureTest.cpp` unit testing. This library was added to
the dependencies to resolve the error. It wasn't caught previously as
the library was transitively linked in most build environments
- Warning of unused declaration,
[here](https://github.com/llvm/llvm-project/pull/133302#issuecomment-2767091368).
There was a usability line in `LexHLSLRootSignature.h` of the form
`using TokenKind = enum RootSignatureToken::Kind` which causes this
error. The declaration is removed from the header file to be used
locally in the `.cpp` files that use it.
Notably, the original pr would also exposed `clang::hlsl::TokenKind` to
everywhere it was included, which had a name clash with
`tok::TokenKind`. This is another motivation to change to the proposed
resolution.
---------
Co-authored-by: Finn Plummer <finnplummer@microsoft.com>
From the corrections to the Root Signature specification here:
https://github.com/llvm/wg-hlsl/issues/192. It was denoted that keywords
are also case-insensitive in DXC.
This pr adjusts the lexer to adhere to the updated spec.
We also have a NFC to add a missing license to a file while in the area.
---------
Co-authored-by: Finn Plummer <finnplummer@microsoft.com>
For the sake of scope, we will let the lexing of floating literals be
deferred until needed for Static Samplers. Other than that this pr
should allow us to simply define new enumerations/keywords in
`RootSignatureTokenKinds.def` for when they are used in the parser. We
could have defined all of these keywords here, but for the sake of
correctness in review we will let them be split up.
- Define `RootSignatureLexer` and provide a public `LexToken` method for
external use
- Define the file `RootSignatureTokenKinds` to define required tokens
and allow for future custom keywords/enums
- Implement the internal methods required to parse the different types
of tokens (integers, flag enums, puncuators...)
- Add test harness for unit testing and the respective unit tests for
lexing the tokens
Resolves#126563
---------
Co-authored-by: Chris B <beanz@abolishcrlf.org>
