Relocate HTTPClient and HTTPServer from the Debuginfod library to
llvm/Support/HTTP so they can be reused by other components.
---------
Relanding with fixes in CMakeLists.txt to account for dependency to new LLVMSupportHTTP in tools.
Relanding with one more fix in libSupportHTTP that adds it as a component in libLLVM.so
---------
Co-authored-by: Alexandre Ganea <aganea@havenstudios.com>
Co-authored-by: Jonas Devlieghere <jonas@devlieghere.com>
Relocate HTTPClient and HTTPServer from the Debuginfod library to
llvm/Support/HTTP so they can be reused by other components.
---------
Relanding with fixes in CMakeLists.txt to account for dependency to new
LLVMSupportHTTP in tools.
---------
Co-authored-by: Alexandre Ganea <aganea@havenstudios.com>
Co-authored-by: Jonas Devlieghere <jonas@devlieghere.com>
Relocate HTTPClient and HTTPServer from the Debuginfod library to
llvm/Support/HTTP so they can be reused by other components.
---------
Co-authored-by: Alexandre Ganea <aganea@havenstudios.com>
Co-authored-by: Jonas Devlieghere <jonas@devlieghere.com>
This patch reverts 44e791c6ff1a982de9651aad7d1c83d1ad96da8a,
3cc1031a827d319c6cb48df1c3aafc9ba7e96d72 and
adbd43250ade1d5357542d8bd7c3dfed212ddec0. Which breaks debuginfod build
and tests when httplib is used.
This is a follow up of adbd43250ade1d5357542d8bd7c3dfed212ddec0. The
problem is, HTTPServer class will lost its implicit move ctor if httplib
is used. This patch adds the move ctor explicitly to solve this issue.
The default move ctor is not added in the header due to a limitation
that "httplib::Server" is a forward declaration and it is incomplete.
This patch fixes:
llvm/lib/Debuginfod/Debuginfod.cpp:592:12: error: moving a local
object in a return statement prevents copy elision
[-Werror,-Wpessimizing-move]
llvm/lib/Debuginfod/Debuginfod.cpp:613:12: error: moving a local
object in a return statement prevents copy elision
[-Werror,-Wpessimizing-move]
These are identified by misc-include-cleaner. I've filtered out those
that break builds. Also, I'm staying away from llvm-config.h,
config.h, and Compiler.h, which likely cause platform- or
compiler-specific build failures.
…Stream.
CachedFileStream has previously performed the commit step in its
destructor, but this means its only recourse for error handling is
report_fatal_error. Modify this to add an explicit commit() method, and
call this in the appropriate places with appropriate error handling for
the location.
Currently the destructor of CacheStream gives an assert failure in Debug
builds if commit() was not called. This will help track down any
remaining uses of the API that assume the old destructior behaviour. In
Release builds we fall back to the previous behaviour and call
report_fatal_error if the commit fails.
This is version 2 of this PR, superseding reverted PR
https://github.com/llvm/llvm-project/pull/115331 . I have incorporated a
change to the testcase to make it more reliable on Windows, as well as
two follow-up changes
(df79000896
and
b0baa1d8bd)
that were also reverted when 115331 was reverted.
---------
Co-authored-by: Augie Fackler <augie@google.com>
Co-authored-by: Vitaly Buka <vitalybuka@google.com>
This reverts commit df79000896101acc9b8d7435e59f767b36c00ac8.
Reverting follow-up commit to ce9e1d3c15ed6290f1cb07b482939976fa8115cd since the original commit test is flaky.
…Stream.
CachedFileStream has previously performed the commit step in its
destructor, but this means its only recourse for error handling is
report_fatal_error. Modify this to add an explicit commit() method, and
call this in the appropriate places with appropriate error handling for
the location.
Currently the destructor of CacheStream gives an assert failure in Debug
builds if commit() was not called. This will help track down any
remaining uses of the API that assume the old destructior behaviour. In
Release builds we fall back to the previous behaviour and call
report_fatal_error if the commit fails.
This reverts commit 7b808e73aa0193c8a42eae8f2420a803f424bee1.
Previous commit which change default debuginfod cache path and pruning
policy settings is problematic. It broke multiple tests across lldb and
llvm. Reverting for now.
Co-authored-by: George Hu <georgehuyubo@gmail.com>
This PR include two changes:
1. Change debuginfod cache file name to include origin file name, the
new file name would be something like:
llvmcache-13267c5f5d2e3df472c133c8efa45fb3331ef1ea-liblzma.so.5.2.2.debuginfo.dwp
So it will provide more information in image list instead of a plain
llvmcache-123
2. Switch debuginfod cache to use lldb index cache settings. Currently
we don't have proper settings for setting the cache path or the cache
expiration time for debuginfod cache. We want to use the lldb index
cache settings, as they make sense to be in the same place and have the
same TTL.
---------
Co-authored-by: George Hu <georgehuyubo@gmail.com>
I've been working on more/better configuration for improving DEBUGINFOD
support. This is the first (and easiest) slice of the work.
I've added `timeout` and `cache-path` settings that can override the
DEBUGINFOD library defaults (and environment variables.) I also renamed
the `plugin.symbol-locator.debuginfod.server_urls` setting to
`server-urls` to be more consistent with the rest of LLDB's settings
(the underscore switch is switched to a hyphen)
I've got a few tests that validate the cache-path setting (as a
side-effect), but they've exposed a few bugs that I'll be putting up a
separate PR for (which will include the tests).
---------
Co-authored-by: Kevin Frei <freik@meta.com>
@nico pointed out that my usage of `std::shared_mutex` broke builds on
older macOS devices. Switching to `llvm::sys::RWMutex` is the solution
that they provided.
Tracked in issue #74382
Co-authored-by: Kevin Frei <freik@meta.com>
I've plumbed the LLVM DebugInfoD client into LLDB, and added automatic
downloading of DWP files to the SymbolFileDWARF.cpp plugin. If you have
DEBUGINFOD_URLS set to a space delimited set of web servers, LLDB will
try to use them as a last resort when searching for DWP files. If you do
*not* have that environment variable set, nothing should be changed.
There's also a setting, per @clayborg 's suggestion, that will override
the environment variable, or can be used instead of the environment
variable. The setting is why I also needed to add an API to the
llvm-debuginfod library
### Test Plan:
Suggestions are welcome here. I should probably have some positive and
negative tests, but I wanted to get the diff up for people who have a
clue what they're doing to rip it to pieces before spending too much
time validating the initial implementation.
---------
Co-authored-by: Kevin Frei <freik@meta.com>
Co-authored-by: Alex Langford <nirvashtzero@gmail.com>
All encodings that the underlying libcurl supports are automatically
detected and accepted. libcurl will then decompress on the fly. This
improves effective network transfer rate and prevents server-side
decompression for files stored compressed.
Fixes#63534
Many sources show that xxh3 is much better than xxh64. This particular
instance may or may not have noticeable difference, but this change
moves us toward removing xxHash64.
This change will invalid cache.
Reviewed By: mysterymath
Differential Revision: https://reviews.llvm.org/D155993
This will result in a tool that's not usable, but it'll still link
correctly rather than failing with a CMake error. This matches what
we do for other tools that have dependencies such as llvm-mt.
Differential Revision: https://reviews.llvm.org/D155323
This makes parsing for build IDs in the markup filter slightly more
permissive, in line with fromHex.
It also removes the distinction between missing build ID and empty build
ID; empty build IDs aren't a useful concept, since their purpose is to
uniquely identify a binary. This removes a layer of indirection wherever
build IDs are obtained.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D147485
Previously, the size of the debuginfod cache would grow without bound.
This change prunes the cache after a successful debuginfod lookup, as is
done in libdebuginfod.
The cache pruning behavior is configured by a new
DEBUGINFOD_CACHE_POLICY environment variable. The semantics of this are
the same as --thinlto_cache_policy.
Reviewed By: gulfem
Differential Revision: https://reviews.llvm.org/D145618
std::optional::value() has undesired exception checking semantics and is
unavailable in some older Xcode. The call sites block std::optional migration.
value() has undesired exception checking semantics and calls
__throw_bad_optional_access in libc++. Moreover, the API is unavailable without
_LIBCPP_NO_EXCEPTIONS on older Mach-O platforms (see
_LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS).
This fixes check-llvm.
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.
This is part of an effort to migrate from llvm::Optional to
std::optional:
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
Breaks build of LLVMgold here:
```
/repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1108:19: error: no matching function for call to 'localCache'
Cache = check(localCache("ThinLTO", "Thin", options::cache_dir, AddBuffer));
^~~~~~~~~~
/repositories/llvm-project/llvm/include/llvm/Support/Caching.h:72:21: note: candidate function not viable: no known conversion from '(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1102:20)' to 'llvm::AddBufferFn' (aka 'function<void (unsigned int, const llvm::Twine &, std::unique_ptr<MemoryBuffer>)>') for 4th argument
Expected<FileCache> localCache(
^
/repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1110:18: error: no viable conversion from '(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20)' to 'llvm::AddStreamFn' (aka 'function<Expected<std::unique_ptr<CachedFileStream>> (unsigned int, const llvm::Twine &)>')
check(Lto->run(AddStream, Cache));
^~~~~~~~~
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/std_function.h:375:7: note: candidate constructor not viable: no known conversion from '(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20)' to 'std::nullptr_t' for 1st argument
function(nullptr_t) noexcept
^
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/std_function.h:386:7: note: candidate constructor not viable: no known conversion from '(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20)' to 'const std::function<llvm::Expected<std::unique_ptr<llvm::CachedFileStream>> (unsigned int, const llvm::Twine &)> &' for 1st argument
function(const function& __x)
^
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/std_function.h:404:7: note: candidate constructor not viable: no known conversion from '(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20)' to 'std::function<llvm::Expected<std::unique_ptr<llvm::CachedFileStream>> (unsigned int, const llvm::Twine &)> &&' for 1st argument
function(function&& __x) noexcept
^
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/std_function.h:435:2: note: candidate template ignored: requirement '_Callable<(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20) &, (lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20), std::__invoke_result<(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20) &, unsigned int, const llvm::Twine &>>::value' was not satisfied [with _Functor = (lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20) &]
function(_Functor&& __f)
^
/repositories/llvm-project/llvm/include/llvm/LTO/LTO.h:278:25: note: passing argument to parameter 'AddStream' here
Error run(AddStreamFn AddStream, FileCache Cache = nullptr);
^
```
This reverts commit 387620aa8cea33174b6c1fb80c1af713fee732ac.
Currently the lto native object files have names like main.exe.lto.1.obj. In
PDB, those names are used as names for each compiland. Microsoft’s tool
SizeBench uses those names to present to users the size of each object files.
So, names like main.exe.lto.1.obj is not user friendly.
This patch makes the lto native object file names more readable by using
the bitcode file names as part of the file names. For example, if the input
bitcode file has path like "path/to/foo.obj", its corresponding lto native
object file path would be "path/to/main.exe.lto.foo.obj". Since the lto native
object file name only bothers PDB, this patch only changes the lld-linker's
behavior.
Reviewed By: tejohnson, MaskRay, #lld-macho
Differential Revision: https://reviews.llvm.org/D137217
This change adds a DEBUGINFOD_HEADERS_FILE environment variable provides
a file containing HTTP headers to attach to outgoing HTTP requests, one
per line. This allows a file permissioned with OS access control
mechanisms to supply bearer credentials for Debuginfod requests.
This matches the mechanism recently added to elfutils' libdebuginfod.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D136303
This creates a library for fetching debug info by build ID, whether
locally or remotely via debuginfod. The functionality was refactored
out of existing code in the Symboliize library. Existing utilities
were refactored to use this library.
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D132504
One of our silent bots is currently failing:
https://lab.llvm.org/staging/#/builders/171/builds/169
With:
<...>/Debuginfod.cpp:298:23: error: no type named 'sleep_for' in namespace 'std::this_thread'
std::this_thread::sleep_for(Interval);
~~~~~~~~~~~~~~~~~~^
Add missing thread include to that file,
which is what all the other users of sleep_for do.
I think we are seeing this now because we disabled
llvm threading for this builder. Maybe debuginfod should account
for that but that's for another time.
