This patch adds a Clang-compatible --save-stats option to opt, to
provide an easy to use way to save LLVM statistics files when working
with opt on the middle end.
This is a follow up on the addition to `llc`:
https://github.com/llvm/llvm-project/pull/163967
Like on Clang, one can specify --save-stats, --save-stats=cwd, and
--save-stats=obj with the same semantics and JSON format. The
pre-existing --stats option is not affected.
The implementation extracts the flag and its methods into the common
`CodeGen/CommandFlags` as `LLVM_ABI`, using a new registration class to
conservatively enable opt-in rather than let all tools take it. Its only
needed for llc and opt for now. Then it refactors llc and adds support
for opt.
This patch adds a Clang-compatible `--save-stats` option, to provide an
easy to use way to save LLVM statistics files when working with llc on
the backend.
Like on Clang, one can specify `--save-stats`, `--save-stats=cwd`, and
`--save-stats=obj` with the same semantics and JSON format.
The implementation uses 2 methods `MaybeEnableStats` and
`MaybeSaveStats` called before and after `compileModule` respectively
that externally own the statistics related logic, while `compileModule`
is now required to return the resolved output filename via an output
param.
Note: like on Clang, the pre-existing `--stats` option is not affected.
This patch adds a new option `--child-tags` (`-t` for short), which
makes dwarfdump only dump children whose DWARF tag is in the list of
tags specified by the user.
Motivating examples are:
* dumping all global variables in a CU
* dumping all non-static data members of a structure
* dumping all module import declarations of a CU
* etc.
For tags not known to dwarfdump, we pretend that the tag wasn't
specified.
Note, this flag only takes effect when `--show-children` is set (either
explicitly or implicitly). We error out when trying to use the flag
without dumping children.
Example:
```
$ builds/release/bin/llvm-dwarfdump -t DW_TAG_structure_type a.out.dSYM
...
0x0000000c: DW_TAG_compile_unit
DW_AT_producer ("clang version 22.0.0git (git@github.com:Michael137/llvm-project.git 737da3347c2fb01dd403420cf83e9b8fbea32618)")
DW_AT_language (DW_LANG_C11)
...
0x0000002a: DW_TAG_structure_type
DW_AT_APPLE_block (true)
DW_AT_byte_size (0x20)
0x00000067: DW_TAG_structure_type
DW_AT_APPLE_block (true)
DW_AT_name ("__block_descriptor")
DW_AT_byte_size (0x10)
...
```
```
$ builds/release/bin/llvm-dwarfdump -t DW_TAG_structure_type -t DW_TAG_member a.out.dSYM
...
0x0000000c: DW_TAG_compile_unit
DW_AT_producer ("clang version 22.0.0git (git@github.com:Michael137/llvm-project.git 737da3347c2fb01dd403420cf83e9b8fbea32618)")
DW_AT_language (DW_LANG_C11)
DW_AT_name ("macro.c")
...
0x0000002a: DW_TAG_structure_type
DW_AT_APPLE_block (true)
DW_AT_byte_size (0x20)
0x0000002c: DW_TAG_member
DW_AT_name ("__isa")
DW_AT_type (0x00000051 "void *")
DW_AT_data_member_location (0x00)
0x00000033: DW_TAG_member
DW_AT_name ("__flags")
DW_AT_type (0x00000052 "int")
DW_AT_data_member_location (0x08)
0x0000003a: DW_TAG_member
DW_AT_name ("__reserved")
DW_AT_type (0x00000052 "int")
DW_AT_data_member_location (0x0c)
0x00000041: DW_TAG_member
DW_AT_name ("__FuncPtr")
DW_AT_type (0x00000056 "void (*)(int)")
DW_AT_data_member_location (0x10)
0x00000048: DW_TAG_member
DW_AT_name ("__descriptor")
DW_AT_type (0x00000062 "__block_descriptor *")
DW_AT_alignment (8)
DW_AT_data_member_location (0x18)
0x00000067: DW_TAG_structure_type
DW_AT_APPLE_block (true)
DW_AT_name ("__block_descriptor")
DW_AT_byte_size (0x10)
0x0000006a: DW_TAG_member
DW_AT_name ("reserved")
DW_AT_type (0x00000079 "unsigned long")
DW_AT_data_member_location (0x00)
0x00000071: DW_TAG_member
DW_AT_name ("Size")
DW_AT_type (0x00000079 "unsigned long")
DW_AT_data_member_location (0x08)
...
```
The default behavior is to _not_ copy such swiftmodules into the dSYM,
as perviously implemented in 96f95c9d89d8a1784d3865fa941fb1c510f4e2d7.
This patch adds the option to override the behavior, so that such
swiftmodules can be copied into the dSYM.
This is useful when the dSYM will be used on a machine which has a
different Xcode/SDK than where the swiftmodules were built. Without
this, when LLDB is asked to "p/po" a Swift variable, the underlying
Swift compiler code would rebuild the dependent `.swiftmodule` files of
the Swift stdlibs, which takes ~1 minute in some cases.
See PR for tests.
It looks like the documentation for `llvm-cxxfilt`'s
`--[no-]strip-underscore` options weren't updated when
https://github.com/llvm/llvm-project/pull/106233 was made.
CC @Michael137 (I don't have merge rights myself).
%t is currently documented as:
temporary file name unique to the test
https://llvm.org/docs/CommandGuide/lit.html#substitutions
Which I take to mean if the path is a/b/c/tempfile, then %t would be
tempfile. It is not, it's the whole path.
(which is hinted at by %basename_t, but why would you read that if you
didn't need to use it)
As seen in #164396 this can create confusion when people use it as if it
were just the file name.
Make it clear in the docs that this is a unique path, which can be used
to make files or folders.
Add support for Machine IR (MIR) triplet and entity generation in llvm-ir2vec.
This change extends llvm-ir2vec to support Machine IR (MIR) in addition to LLVM IR, enabling the generation of training data for MIR2Vec embeddings. MIR2Vec provides machine-level code embeddings that capture target-specific instruction semantics, complementing the target-independent IR2Vec embeddings.
- Extended llvm-ir2vec to support triplet and entity generation for Machine IR (MIR)
- Added `--mode=mir` option to specify MIR mode (vs LLVM IR mode)
- Implemented MIR triplet generation with Next and Arg relationships
- Added entity mapping generation for MIR vocabulary
- Updated documentation to explain MIR-specific features and usage
(Partially addresses #162200 ; Tracking issue - #141817)
Add MIR2Vec support to the llvm-ir2vec tool, enabling embedding generation for Machine IR alongside the existing LLVM IR functionality.
(This is an initial integration; Other entity/triplet gen for vocab generation would follow as separate patches)
Summary:
This tool is pretty much a generic interface into creating and managing
the offloading binary format. The binary format itself is just a fat
binary block used to create heterogeneous objects. This should be made
more general than just `clang` since it's likely going to be used for
larger offloading projects and is the expected way to extract
heterogeneous objects from offloading code.
Relatively straightforward rename, a few tweaks and documentation
changes. Kept in `clang-offload-packager` for legacy compatibility as we
looked this tool up by name in places, will probably delete it next
release.
Utilize new extensions to LLVM Offloading API to
handle offloading fatbin Bundles.
The tool will output a list of available offload bundles
using URI syntax.
---------
Co-authored-by: dsalinas_amdeng <david.salinas@amd.com>
Do not include the ``__PAGEZERO`` segment when calculating size information
for Mach-O files when `--exclude-pagezero` is used. The ``__PAGEZERO``
segment is a virtual memory region used for memory protection that does not
contribute to actual size, and excluding can provide a better representation of
actual size.
Fixes#86644
This patch removes support for %T from llvm-lit. For now we mark the
test unresolved and add an error message noting the substitution is
deprecated. This is exactly the same as the error handling for other
substitution failures. We intend to remove support for the nice error
message once 22 branches as users should have moved over by the they are
upgrading to v23.
Reviewers: petrhosek, jh7370, ilovepi, pogo59, cmtice
Reviewed By: cmtice, jh7370, ilovepi
Pull Request: https://github.com/llvm/llvm-project/pull/160028
Currently MCA takes instruction properties from scheduling model.
However, some instructions may execute differently depending on external
factors - for example, latency of memory instructions may vary
differently depending on whether the load comes from L1 cache, L2 or
DRAM. While MCA as a static analysis tool cannot model such differences
(and currently takes some static decision, e.g. all memory ops are
treated as L1 accesses), it makes sense to allow manual modification of
instruction properties to model different behavior (e.g. sensitivity of
code performance to cache misses in particular load instruction). This
patch addresses this need.
The library modification is intentionally generic - arbitrary
modifications to InstrDesc are allowed. The tool support is currently
limited to changing instruction latencies (single number applies to all
output arguments and MaxLatency) via coments in the input assembler
code; the format is the like this:
add (%eax), eax // LLVM-MCA-LATENCY:100
Users of MCA library can already make additional customizations; command
line tool can be extended in the future.
Note that InstructionView currently shows per-instruction information
according to scheduling model and is not affected by this change.
See https://github.com/llvm/llvm-project/issues/133429 for additional
clarifications (including explanation why existing customization
mechanisms do not provide required functionality)
---------
Co-authored-by: Min-Yih Hsu <min@myhsu.dev>
This patch adds a new %{readfile:<file name>} substitution to lit. This
is needed for porting a couple of tests to lit's internal shell. These
tests are all using subshells to pass some option to a command are not
feasible to run within the internal shell without this functionality.
Reviewers: petrhosek, jh7370, ilovepi, cmtice
Reviewed By: jh7370, cmtice
Pull Request: https://github.com/llvm/llvm-project/pull/158441
- The output for `--output-sort=id` matches `--output-sort=offset` for
the available readers. Tests were updated accordingly.
- For `--output-sort=none`, and per `LVReader::sortScopes()`,
`LVScope::sort()` is called on the root scope.
`LVScope::sort()` has no effect if `getSortFunction() == nullptr`, and
thus the elements are currently traversed in the order in which they
were initially added. This should change, however, after
`LVScope::Children` is removed.
This pr adds the `extract-section` option to `llvm-objcopy` as a common
option. It differs from `dump-section` as it will produce a standalone
object with just one section, as opposed to just the section contents.
For more context as to other options considered, see
https://github.com/llvm/llvm-project/pull/153265#issuecomment-3195696003.
This difference in behaviour is used for DXC compatibility with
`extract-rootsignature` and `/Frs`.
This pr then implements this functionality for `DXContainer` objects.
This is the second step of
https://github.com/llvm/llvm-project/issues/150277 to implement as a
compiler action that invokes `llvm-objcopy` for functionality.
This also completes the implementation of `extract-rootsignature` as
described in https://github.com/llvm/llvm-project/issues/149560.
Refactor llvm-ir2vec to use subcommands instead of a mode flag for better CLI usability.
- Converted the `--mode` flag to three distinct subcommands: `triplets`, `entities`, and `embeddings`
- Updated documentation, tests, and python script
This patch hyphenates words that are used as adjecives, such as:
- architecture specific
- human readable
- implementation defined
- language independent
- language specific
- machine readable
- machine specific
- target independent
- target specific
See the related issue. We want to set up a build bot where `opt` runs with `-enable-profcheck`, which inserts `MD_prof` before running the rest of the pipeline requested from `opt`, and then validates resulting profile information (more info in the RFC linked by the aforementioned issue)
In that setup, we will also ignore `FileCheck`: while the profile info inserted is, currently, equivalent to the profile info a pass would observe via `BranchProbabilityInfo`/`BlockFrequencyInfo`, (1) we may want to change that, and (2) some tests are quite sensitive to the output IR, and break if, for instance, extra metadata is present (which it would be due to `-enable-profcheck`). Since we're just interested in profile consistency on the upcoming bot, ignoring `FileCheck` is simpler and sufficient. However, this has the effect of passing XFAIL tests. Rather than listing them all, the alternative is to just exclude XFAIL tests.
This PR adds support for that by introducing a `--exclude-xfail` option to `llvm-lit`.
Issue #147390
Added a Python utility script for generating IR2Vec triplets and updated documentation to reference it.
The script generates triplets in a form suitable for training the vocabulary.
(Tracking issues - #141817, #141834; closes - #141834)
Add entity mapping mode to llvm-ir2vec and improve triplet generation format for knowledge graph embedding training.
This change streamlines the workflow for training the vocabulary embeddings with IR2Vec by:
1. Directly generating numeric IDs instead of requiring string-to-ID preprocessing
2. Providing entity mappings in standard knowledge graph embedding format
3. Structuring triplet output in train2id format compatible with knowledge graph embedding frameworks
4. Adding metadata headers to simplify post-processing and training setup
These improvements make IR2Vec more compatible with standard knowledge graph embedding training pipelines and reduce the preprocessing steps needed before training.
See #149215 for more details on how it is used.
(Tracking issues - #141817, #141834)
This patch adds the support for displaying inlined functions into
llvm-objdump.
1) It extends the source variable display
support for inlined functions both for ascii and unicode formats.
2) It also introduces a new format called limits-only that only prints a
line for the start and end of an inlined function without line-drawing
characters.
Similar to the existing implementations for X86 and PPC, support
symbolizing branch targets for AArch64. Do not omit the address for ADRP
as the target is typically not at an intended location.
Pull Request: https://github.com/llvm/llvm-project/pull/145009
Background: The yaml-strtab format looks just like the yaml format,
except that the values in the key/value pairs of the remarks are
deduplicated and replaced by indices into a string table (see removed
test cases for examples). The motivation behind this format was to
reduce size of the remarks files. However, it was quickly superseded by
the bitstream format.
Therefore, remove the yaml-strtab format, as it doesn't have a good
usecase anymore:
- It isn't particularly efficient
- It isn't human-readable
- It isn't straightforward to parse in external tools that can't use the
remarks library. We don't even support it in opt-viewer.
llvm-remarkutil is also missing options to parse/convert yaml-strtab, so
the chance that anyone is actually using this format is low.
Some data members are only part of a class definition in a Debug build,
e.g. `LVObject::ID`. If `debuginfologicalview` is used as a library,
`NDEBUG` cannot be used for this purpose, as this PP macro may have a
different definition in a downstream project, which in turn triggers an
ODR violation. Fix it by
- Making `LVObject::ID` an unconditional data member.
- Making `LVObject::dump()` non-virtual. Rationale: `virtual` is not
needed (and it calls `print()`, which is virtual anyway).
Fixes#139098.
When no profile is provided, but the new --empty-profile option is
specified, the export/report/show commands now emit coverage data
equivalent to that obtained from a profile with all zero counters
("baseline coverage").
This is useful for build systems (e.g. Bazel) that can track coverage
information for each build target, even those that are never linked into
tests and thus don't have runtime coverage data recorded. By merging in
baseline coverage, lines in files that aren't linked into tests are
correctly reported as uncovered.
Reland with fixes to `CoverageMappingTest.cpp`.
Reverts llvm/llvm-project#144121
Reverts llvm/llvm-project#117910
```
/home/buildbots/llvm-external-buildbots/workers/ppc64le-lld-multistage-test/ppc64le-lld-multistage-test/llvm-project/llvm/unittests/ProfileData/CoverageMappingTest.cpp
/home/buildbots/llvm-external-buildbots/workers/ppc64le-lld-multistage-test/ppc64le-lld-multistage-test/llvm-project/llvm/unittests/ProfileData/CoverageMappingTest.cpp:281:28: error: 'std::reference_wrapper' may not intend to support class template argument deduction [-Werror,-Wctad-maybe-unsupported]
281 | std::make_optional(std::reference_wrapper(*ProfileReader));
| ^
/usr/lib/gcc/ppc64le-redhat-linux/8/../../../../include/c++/8/bits/refwrap.h:289:11: note: add a deduction guide to suppress this warning
289 | class reference_wrapper
| ^
```
When no profile is provided, but the new --empty-profile option is
specifed, the export/report/show commands now emit coverage data
equivalent to that obtained from a profile with all zero counters
("baseline coverage").
This is useful for build systems (e.g. Bazel) that can track coverage
information for each build target, even those that are never linked into
tests and thus don't have runtime coverage data recorded. By merging in
baseline coverage, lines in files that aren't linked into tests are
correctly reported as uncovered.
This is a cli tool to that tests the conformance of LLVM's mustache
implementation against the public Mustache spec, hosted at
https://github.com/mustache/spec. This is a revised version of the
patches in #111487.
Co-authored-by: Peter Chou <peter.chou@mail.utoronto.ca>
This pull request adds support for parsing the source language in both
DWARF and CodeView. Specifically,
- The `LVSourceLanguage` class is introduced to represent any supported
language by any of the debug info representations.
- Update `LVDWARFReader.cpp` and `LVCodeViewVisitor.cpp` to parse the
source language where it applies. Added a new `=Language` attribute;
`getAttributeLanguage()` is internally used to control whether this
information is being printed.
If a test took more than one attempt to complete, show the number of attempts and the maximum allowed attempts as `2 of 4 attempts` inside the `<progress info>` (see [TEST RUN OUTPUT FORMAT](https://llvm.org/docs/CommandGuide/lit.html#test-run-output-format)).
NOTE: Additionally this is a fixup for #141851 where the tests were not quite right. `max-retries-per-test/allow-retries-test_retry_attempts/test.py` was added but never used there. Now we're calling it. To correlate better between the test output and the test script I've used higher numbers of max allowed retries.
This is a fixup for #141851 and removes `=` from all
options with additional arguments.
Before 14 out of 22 options with arguments used "=" and 7 didn't.
When packaging LLVM we've seen arbitrary tests fail.
It happened sporadically and most of the times the test
do work if they are run a second time on the next day.
The tests themselves were always different and we didn't
know ahead of time which ones we wanted to re-run.
That's we filter-out a lot of `libomp` and `libarcher` tests [1].
This change allows us to set
`LIT_OPTS="--max-retries-per-test=12"`
when running any "check-XXX" build target. Then any lit test
will at most be re-run 12 times, unless there's an `ALLOW_RETRIES:`
in one of the test scripts that's specifying a different value
than `12`. `12` is just an example here, any positive integer
will work.
Please note, that this only adds the possibility to re-run
lit tests. It does not actually do it until the caller specifies
`--max-retries-per-test=<POSITIVE_INT>` either on a call to `lit` or
in `LIT_OPTS`.
Also note, that one can still use `ALLOW_RETRIES:` in test scripts
and it will always rule over `--max-retries-per-test`. When
`--max-retries-per-test` is set too low, but the
`config.test_retry_attempts`
is high enough, it works as well.
Any option in the list below overrules its predecessor:
* `--max-retries-per-test`
* `config.test_retry_attempts`
* `ALLOW_RETRIES` keyword
From the above options to re-run tests, `--max-retries-per-test` is the
only one that doesn't require a change in the test scripts or the test
config.
[1]:
https://src.fedoraproject.org/rpms/llvm/blob/rawhide/f/llvm.spec#_2326
Downstream PR to make use of the `--max-retries-per-test` option:
https://src.fedoraproject.org/rpms/llvm/pull-request/434
Downstream ticket: https://issues.redhat.com/browse/LLVM-145
Utilize the new extensions to the LLVM Offloading API to extend to
llvm-objdump to handle dumping fatbin offload bundles generated by HIP.
This extension to llvm-objdump adds the option --offload-fatbin.
Specifying this option will take the input object/executable and extract
all offload fatbin bundle entries into distinct code object files with
names reflecting the source file name combined with the Bundle Entry ID.
Users can also use the --arch-name option to filter offload fatbin
bundle entries by their target triple.
---------
Co-authored-by: dsalinas <dsalinas@MKM-L1-DSALINAS.amd.com>
This PR was split from https://github.com/llvm/llvm-project/pull/137228
(which introduced support for `DW_TAG_module` and `DW_AT_byte_size`).
This PR improves `LVDWARFReader` by introducing handling of
`DW_AT_byte_size`. Most DWARF emitters include this attribute for types
to specify the size of an entity of the given type.
