2738828c0e
This is an attempt to reland #151660 by including a missing STL header found by a buildbot failure. The stable function map could be huge for a large application. Fully loading it is slow and consumes a significant amount of memory, which is unnecessary and drastically slows down compilation especially for non-LTO and distributed-ThinLTO setups. This patch introduces an opt-in lazy loading support for the stable function map. The detailed changes are: - `StableFunctionMap` - The map now stores entries in an `EntryStorage` struct, which includes offsets for serialized entries and a `std::once_flag` for thread-safe lazy loading. - The underlying map type is changed from `DenseMap` to `std::unordered_map` for compatibility with `std::once_flag`. - `contains()`, `size()` and `at()` are implemented to only load requested entries on demand. - Lazy Loading Mechanism - When reading indexed codegen data, if the newly-introduced `-indexed-codegen-data-lazy-loading` flag is set, the stable function map is not fully deserialized up front. The binary format for the stable function map now includes offsets and sizes to support lazy loading. - The safety of lazy loading is guarded by the once flag per function hash. This guarantees that even in a multi-threaded environment, the deserialization for a given function hash will happen exactly once. The first thread to request it performs the load, and subsequent threads will wait for it to complete before using the data. For single-threaded builds, the overhead is negligible (a single check on the once flag). For multi-threaded scenarios, users can omit the flag to retain the previous eager-loading behavior.
38 lines
1.6 KiB
Plaintext
38 lines
1.6 KiB
Plaintext
# Test no input file
|
|
RUN: not llvm-cgdata --convert --output - 2>&1 | FileCheck %s --check-prefix=NOFILE --ignore-case
|
|
NOFILE: error: No input file is specified.
|
|
|
|
# Test for empty cgdata file, which is invalid.
|
|
RUN: touch %t_emptyfile.cgtext
|
|
RUN: not llvm-cgdata --convert %t_emptyfile.cgtext --format text 2>&1 | FileCheck %s --check-prefix=EMPTY
|
|
EMPTY: {{.}}emptyfile.cgtext: empty codegen data
|
|
|
|
# Test for empty header in the text format. It can be converted to a valid binary file.
|
|
RUN: printf '#' > %t_emptyheader.cgtext
|
|
RUN: llvm-cgdata --convert %t_emptyheader.cgtext --format binary -o %t_emptyheader.cgdata
|
|
|
|
# Without any cgdata other than the header, no data shows by default.
|
|
RUN: llvm-cgdata --show %t_emptyheader.cgdata | count 0
|
|
|
|
# The version number appears when asked, as it's in the header
|
|
RUN: llvm-cgdata --show --cgdata-version %t_emptyheader.cgdata | FileCheck %s --check-prefix=VERSION
|
|
VERSION: Version: 4
|
|
|
|
# When converting a binary file (w/ the header only) to a text file, it's an empty file as the text format does not have an explicit header.
|
|
RUN: llvm-cgdata --convert %t_emptyheader.cgdata --format text | count 0
|
|
|
|
# Synthesize a header only cgdata.
|
|
# struct Header {
|
|
# uint64_t Magic;
|
|
# uint32_t Version;
|
|
# uint32_t DataKind;
|
|
# uint64_t OutlinedHashTreeOffset;
|
|
# uint64_t StableFunctionMapOffset;
|
|
# }
|
|
RUN: printf '\xffcgdata\x81' > %t_header.cgdata
|
|
RUN: printf '\x04\x00\x00\x00' >> %t_header.cgdata
|
|
RUN: printf '\x00\x00\x00\x00' >> %t_header.cgdata
|
|
RUN: printf '\x20\x00\x00\x00\x00\x00\x00\x00' >> %t_header.cgdata
|
|
RUN: printf '\x20\x00\x00\x00\x00\x00\x00\x00' >> %t_header.cgdata
|
|
RUN: diff %t_header.cgdata %t_emptyheader.cgdata
|