96 Commits

Author SHA1 Message Date
Takuya Shimizu
01b88dd66d [NFC] Remove unused variables declared in conditions
D152495 makes clang warn on unused variables that are declared in conditions like `if (int var = init) {}`
This patch is an NFC fix to suppress the new warning in llvm,clang,lld builds to pass CI in the above patch.

Differential Revision: https://reviews.llvm.org/D158016
2023-08-30 10:05:06 +09:00
Takuya Shimizu
5aded521ea Revert "Reland "[Clang][SemaCXX] Add unused warning for variables declared in condition expressions""
This causes a lot of warning in sanitizer build:
https://lab.llvm.org/buildbot/#/builders/258/builds/5424
https://lab.llvm.org/buildbot/#/builders/36/builds/36560

This reverts commit 8e329caa944c377c51ef567d5aa67cfac9ffd0fa.
2023-08-15 19:54:51 +09:00
Takuya Shimizu
8e329caa94 Reland "[Clang][SemaCXX] Add unused warning for variables declared in condition expressions"
This patch marks the declarations with initializations in condition expressions such as
if (int var = init) as unreferenced so that -Wunused can warn on them.

Fixes https://github.com/llvm/llvm-project/issues/61681

Reviewed By: cor3ntin
Differential Revision: https://reviews.llvm.org/D152495
2023-08-15 17:24:13 +09:00
Fangrui Song
c51c607dfc [VirtualFileSystem] Use std::map::emplace 2023-07-22 00:00:57 -07:00
Fangrui Song
b55ac6e9cb [VirtualFileSystem] Make gcc<7.5 happy after 75d71800aa384ee58663d892c325572f5588df2a
There is a libstdc++ stl_map.h bug that is only back ported to 7.5.
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=78595#c13

```
/tmp/opt/gcc-7.3.0/include/c++/7.3.0/bits/stl_tree.h:2091:28: error: no matching function for call to ‘std::_Rb_tree<std::__cxx11::basic_string<char>, std::pair<const std::__cxx11::basic_string<char>, std::unique_ptr<llvm::vfs::detail::InMemoryNode> >, std::_Select1st<std::pair<const std::__cxx11::basic_string<char>, std::unique_ptr<llvm::vfs::detail::InMemoryNode> > >, std::less<std::__cxx11::basic_string<char> >, std::allocator<std::pair<const std::__cxx11::basic_string<char>, std::unique_ptr<llvm::vfs::detail::InMemoryNode> > > >::_M_get_insert_unique_pos(std::pair<llvm::StringRef, std::unique_ptr<llvm::vfs::detail::InMemoryNode> >::first_type&)’
  = _M_get_insert_unique_pos(_KeyOfValue()(__v));
    ~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
```

Just construct a std::string from StringRef to work around it.
2023-07-21 23:23:32 -07:00
Fangrui Song
75d71800aa [VirtualFileSystem] Use map to stabilize iteration order
StringMap iteration order is not guaranteed to be deterministic
(https://llvm.org/docs/ProgrammersManual.html#llvm-adt-stringmap-h).

Tested by `TEST_F(InMemoryFileSystemTest, DirectoryIteration)`.
2023-07-19 22:25:40 -07:00
Jan Svoboda
d77588df45 [llvm][vfs] For virtual directories, use the virtual path as the real path
A follow-up to D135841. This patch returns the virtual path for directories from `RedirectingFileSystem`. This ensures the contents of `Path` are the same as the contents of `FS->getRealPath(Path)`. This also means we can drop the workaround in Clang's module map canonicalization, where we couldn't use the real path for a directory if it resolved to a different `DirectoryEntry`. In addition to that, we can also avoid introducing new workaround for a bug triggered by the newly introduced test case.

Reviewed By: benlangmuir

Differential Revision: https://reviews.llvm.org/D135849
2023-07-10 10:41:15 -07:00
Ben Langmuir
5437a4c5e9 [llvm][vfs] Avoid silent fallback to process-wide working directory
In createPhysicalFileSystem, preserve the per-instance working
directory, even after the first call to getcwd fails.

rdar://108213753

Differential Revision: https://reviews.llvm.org/D149173
2023-05-02 09:39:07 -07:00
Kazu Hirata
da6642a1c8 [llvm] Include <optional> instead of "llvm/ADT/Optional.h" (NFC)
This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2023-01-14 15:09:00 -08:00
Kazu Hirata
77c90c8ce0 [llvm] Use std::optional instead of Optional
This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-12-20 15:42:32 -08:00
Fangrui Song
2fa744e631 std::optional::value => operator*/operator->
value() has undesired exception checking semantics and calls
__throw_bad_optional_access in libc++. Moreover, the API is unavailable without
_LIBCPP_NO_EXCEPTIONS on older Mach-O platforms (see
_LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS).

This commit fixes LLVMAnalysis and its dependencies.
2022-12-16 22:44:08 +00:00
Haowei Wu
a903ecb4a2 [vfs] Allow root paths relative to the vfsoverlay YAML file
This change adds 'root-relative' option in vfsoverlay YAML file format
so the root patchs can be relative to the YAML file directory instead of
the current working directory.

Differential Revision: https://reviews.llvm.org/D137473
2022-12-16 11:45:36 -08:00
Fangrui Song
b1df3a2c0b [Support] llvm::Optional => std::optional
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-12-16 08:49:10 +00:00
Kazu Hirata
f7dffc28b3 Don't include None.h (NFC)
I've converted all known uses of None to std::nullopt, so we no longer
need to include None.h.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-12-10 11:24:26 -08:00
Gregory Alfonso
7f86bb0a71 [llvm] Call reserve before push_back in a loop
It is generally good practice, if you know how big the vector is going to be in the end, to reserve before continually calling "push_back" or "emplace_back"

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D139483
2022-12-08 08:42:50 +00:00
Kazu Hirata
aadaaface2 [llvm] Use std::nullopt instead of None (NFC)
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated.  The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-12-02 21:11:44 -08:00
Kazu Hirata
0af53df883 [Support] Use std::optional in VirtualFileSystem.cpp (NFC)
This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-11-25 21:53:00 -08:00
Fangrui Song
de9d80c1c5 [llvm] LLVM_FALLTHROUGH => [[fallthrough]]. NFC
With C++17 there is no Clang pedantic warning or MSVC C5051.
2022-08-08 11:24:15 -07:00
Kazu Hirata
611ffcf4e4 [llvm] Use value instead of getValue (NFC) 2022-07-13 23:11:56 -07:00
Kazu Hirata
a7938c74f1 [llvm] Don't use Optional::hasValue (NFC)
This patch replaces Optional::hasValue with the implicit cast to bool
in conditionals only.
2022-06-25 21:42:52 -07:00
Kazu Hirata
3b7c3a654c Revert "Don't use Optional::hasValue (NFC)"
This reverts commit aa8feeefd3ac6c78ee8f67bf033976fc7d68bc6d.
2022-06-25 11:56:50 -07:00
Kazu Hirata
aa8feeefd3 Don't use Optional::hasValue (NFC) 2022-06-25 11:55:57 -07:00
Jan Svoboda
a44c6453fe [llvm][vfs] Implement in-memory symlinks
This patch implements symlinks for the in-memory VFS. Original author: @erik.pilkington.

Depends on D117648 & D117649.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D117650
2022-06-21 16:29:54 +02:00
Jan Svoboda
b439a08dfc [llvm][vfs] NFC: Promote InMemoryDirIterator to nested class 2022-06-21 16:29:54 +02:00
Jan Svoboda
9e0398da8d [llvm][vfs] NFC: Promote lookupInMemoryNode() to member function 2022-06-21 16:29:53 +02:00
Jan Svoboda
1ff5330ea3 [llvm][vfs] NFC: Rename InMemoryFileSystem::addHardLink() arguments 2022-06-21 16:29:53 +02:00
Kazu Hirata
129b531c9c [llvm] Use value_or instead of getValueOr (NFC) 2022-06-18 23:07:11 -07:00
Ben Barham
fe2478d44e [VFS] RedirectingFileSystem only replace path if not already mapped
If the `ExternalFS` has already remapped to an external path then
`RedirectingFileSystem` should not change it to the originally provided
path. This fixes the original path always being used if multiple VFS
overlays were provided and the path wasn't found in the highest (ie.
first in the chain).

For now this is accomplished through the use of a new
`ExposesExternalVFSPath` field on `vfs::Status`. This flag is true when
the `Status` has an external path that's different from its virtual
path, ie. the contained path is the external path. See the plan in
`FileManager::getFileRef` for where this is going - eventually we won't
need `IsVFSMapped` any more and all returned paths should be virtual.

Resolves rdar://90578880 and llvm-project#53306.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D123398
2022-04-11 14:52:48 -07:00
Ben Barham
f65b0b5dcf Revert "[VFS] RedirectingFileSystem only replace path if not already mapped"
This reverts commit 3fda0edc51fd68192a30e302d45db081bb02d7f9, which
breaks crash reproducers in very specific circumstances. Specifically,
since crash reproducers have `UseExternalNames` set to false, the
`File->getFileEntry().getDir()->getName()` call in `DoFrameworkLookup`
would use the *cached* directory name instead of the directory of the
looked-up file.

The plan is to re-commit this patch but to *add*
`ExposesExternalVFSPath` rather than replace `IsVFSMapped`.

Differential Revision: https://reviews.llvm.org/D123103
2022-04-05 14:24:40 -07:00
Ben Barham
3fda0edc51 [VFS] RedirectingFileSystem only replace path if not already mapped
If the `ExternalFS` has already remapped a path then the
`RedirectingFileSystem` should not change it to the originally provided
path. This fixes the original path always being used if multiple VFS
overlays were provided and the path wasn't found in the highest (ie.
first in the chain).

This also renames `IsVFSMapped` to `ExposesExternalVFSPath` and only
sets it if `UseExternalName` is true. This flag then represents that the
`Status` has an external path that's different from its virtual path.
Right now the contained path is still the external path, but further PRs
will change this to *always* be the virtual path. Clients that need the
external can then request it specifically.

Note that even though `ExposesExternalVFSPath` isn't set for all
VFS-mapped paths, `IsVFSMapped` was only being used by a hack in
`FileManager` that was specific to module searching. In that case
`UseExternalNames` is always `true` and so that hack still applies.

Resolves rdar://90578880 and llvm-project#53306.

Differential Revision: https://reviews.llvm.org/D122549
2022-03-30 11:52:41 -07:00
Ben Barham
4125524112 [VFS] Add print/dump to the whole FileSystem hierarchy
For now most are implemented by printing out the name of the filesystem,
but this can be expanded in the future. Only `OverlayFileSystem` and
`RedirectingFileSystem` are properly implemented in this patch.
  - `OverlayFileSystem`: Prints each filesystem in the order that any
    operations are actually run on them. Optionally prints recursively.
  - `RedirectingFileSystem`: Prints out all mappings, as well as the
    `ExternalFS`. Most of this was already implemented other than the
    handling for the `DirectoryRemap` case and to actually print out the
    mapping.

Each FS should implement `printImpl` rather than `print`, where the
latter just fowards to the former. This is to avoid spreading the
default arguments through to the subclasses (where we may miss updating
in the future).

Differential Revision: https://reviews.llvm.org/D121421
2022-03-17 13:02:40 -07:00
Ben Barham
cc63ae42d7 [VFS] Rename RedirectingFileSystem::dump to print
The rest of LLVM uses `print` for the method taking the `raw_ostream`
and `dump` only for the method with no parameters. Use the same for
`RedirectingFileSystem`.

Differential Revision: https://reviews.llvm.org/D121494
2022-03-14 11:44:07 -07:00
Ben Barham
ed4f0cb878 [VFS] Use generic_category for errors generated from the VFS
Errors are generally checked in clients by comparing to the portable
error condition in `std::errc`, which will have the `generic_category`
(eg. `std::errc::no_such_file_or_directory`). While in practice these
are usually equivalent for the standard errno's, they are not in *all*
implementations. One such example is CentOS 7.

Differential Revision: https://reviews.llvm.org/D120299
2022-02-22 15:05:58 -08:00
Ben Barham
a5cff6af1d [VFS] Add back setFallthrough for downstream users
This fixes lldb's build. We can remove this in the future if we want but
for now this will be nicer to existing consumers.
2022-02-03 13:46:18 -08:00
Ben Barham
502f14d6f2 [VFS] Add a "redirecting-with" field to overlays
Extend "fallthrough" to allow a third option: "fallback". Fallthrough
allows the original path to used if the redirected (or mapped) path
fails. Fallback is the reverse of this, ie. use the original path and
fallback to the mapped path otherwise.

While this result *can* be achieved today using multiple overlays, this
adds a much more intuitive option. As an example, take two directories
"A" and "B". We would like files from "A" to be used, unless they don't
exist, in which case the VFS should fallback to those in "B".

With the current fallthrough option this is possible by adding two
overlays: one mapping from A -> B and another mapping from B -> A. Since
the frontend *nests* the two RedirectingFileSystems, the result will
be that "A" is mapped to "B" and back to "A", unless it isn't in "A" in
which case it fallsthrough to "B" (or fails if it exists in neither).

Using "fallback" semantics allows a single overlay instead: one mapping
from "A" to "B" but only using that mapping if the operation in "A"
fails first.

"redirect-only" is used to represent the current "fallthrough: false"
case.

Differential Revision: https://reviews.llvm.org/D117937
2022-02-03 13:10:23 -08:00
serge-sans-paille
75e164f61d [llvm] Cleanup header dependencies in ADT and Support
The cleanup was manual, but assisted by "include-what-you-use". It consists in

1. Removing unused forward declaration. No impact expected.
2. Removing unused headers in .cpp files. No impact expected.
3. Removing unused headers in .h files. This removes implicit dependencies and
   is generally considered a good thing, but this may break downstream builds.
   I've updated llvm, clang, lld, lldb and mlir deps, and included a list of the
   modification in the second part of the commit.
4. Replacing header inclusion by forward declaration. This has the same impact
   as 3.

Notable changes:

- llvm/Support/TargetParser.h no longer includes llvm/Support/AArch64TargetParser.h nor llvm/Support/ARMTargetParser.h
- llvm/Support/TypeSize.h no longer includes llvm/Support/WithColor.h
- llvm/Support/YAMLTraits.h no longer includes llvm/Support/Regex.h
- llvm/ADT/SmallVector.h no longer includes llvm/Support/MemAlloc.h nor llvm/Support/ErrorHandling.h

You may need to add some of these headers in your compilation units, if needs be.

As an hint to the impact of the cleanup, running

clang++ -E  -Iinclude -I../llvm/include ../llvm/lib/Support/*.cpp -std=c++14 -fno-rtti -fno-exceptions | wc -l

before: 8000919 lines
after:  7917500 lines

Reduced dependencies also helps incremental rebuilds and is more ccache
friendly, something not shown by the above metric :-)

Discourse thread on the topic: https://llvm.discourse.group/t/include-what-you-use-include-cleanup/5831
2022-01-21 13:54:49 +01:00
Jan Svoboda
9011903e36 [llvm][vfs] Abstract in-memory node creation
The creation of in-memory VFS nodes happens in a single function that deduces what kind of node to create from the arguments. This leads to complicated if-then-else logic that's difficult to cleanly extend.

This patch abstracts away in-memory node creation via a type-erased factory function that's passed instead.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D117648
2022-01-20 15:48:02 +01:00
Jan Svoboda
9e24d14ac8 [llvm][vfs] NFC: Virtualize in-memory getStatus
This patch virtualizes the `getStatus` function on `InMemoryNode` in LLVM VFS. Currently, this is implemented via top-level function `getNodeStatus` that tries to cast `InMemoryNode *` into each subtype. Virtual functions seem to be the simpler solution here.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D117649
2022-01-20 15:48:02 +01:00
Richard Howell
4f61749e16 [clang] support relative roots to vfs overlays
This diff adds support for relative roots to VFS overlays. The directory root
will be made absolute from the current working directory and will be used to
determine the path style to use. This supports the use of VFS overlays with
remote build systems that might use a different working directory for each
compilation.

Reviewed By: benlangmuir

Differential Revision: https://reviews.llvm.org/D116174
2022-01-19 10:13:06 -08:00
Jan Svoboda
f66803457e [clang][deps] Squash caches for original and minimized files
The minimizing and caching filesystem used by the dependency scanner keeps minimized and original files in separate caches.

This setup is not well suited for dealing with files that are sometimes minimized and sometimes not. Such files are being stat-ed and read twice, which is wasteful and also means the two versions of the file can get "out of sync".

This patch squashes the two caches together. When a file is stat-ed or read, its original contents are populated. If a file needs to be minimized, we give the minimizer the already loaded contents instead of reading the file again.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D115346
2021-12-16 09:57:21 +01:00
Keith Smiley
86e2af8043 reland: [VFS] Use original path when falling back to external FS
This reverts commit f0cf544d6f6fe6cbca4c07772998272d6bb433d8.

Just a small change to fix:

```
/home/buildbot/as-builder-4/llvm-clang-x86_64-expensive-checks-ubuntu/llvm-project/llvm/lib/Support/VirtualFileSystem.cpp: In static member function ‘static llvm::ErrorOr<std::unique_ptr<llvm::vfs::File> > llvm::vfs::File::getWithPath(llvm::ErrorOr<std::unique_ptr<llvm::vfs::File> >, const llvm::Twine&)’:
/home/buildbot/as-builder-4/llvm-clang-x86_64-expensive-checks-ubuntu/llvm-project/llvm/lib/Support/VirtualFileSystem.cpp:2084:10: error: could not convert ‘F’ from ‘std::unique_ptr<llvm::vfs::File>’ to ‘llvm::ErrorOr<std::unique_ptr<llvm::vfs::File> >’
   return F;
          ^
```

Differential Revision: https://reviews.llvm.org/D113832
2021-11-13 12:14:34 -08:00
Keith Smiley
f0cf544d6f Revert "[VFS] Use original path when falling back to external FS"
```
/work/omp-vega20-0/openmp-offload-amdgpu-runtime/llvm.src/llvm/lib/Support/VirtualFileSystem.cpp: In static member function 'static llvm::ErrorOr<std::unique_ptr<llvm::vfs::File> > llvm::vfs::File::getWithPath(llvm::ErrorOr<std::unique_ptr<llvm::vfs::File> >, const llvm::Twine&)':
/work/omp-vega20-0/openmp-offload-amdgpu-runtime/llvm.src/llvm/lib/Support/VirtualFileSystem.cpp:2084:10: error: could not convert 'F' from 'std::unique_ptr<llvm::vfs::File>' to 'llvm::ErrorOr<std::unique_ptr<llvm::vfs::File> >'
   return F;
          ^
```

This reverts commit c972175649f4bb50d40d911659a04d5620ce6fe0.
2021-11-13 10:11:51 -08:00
Keith Smiley
c972175649 [VFS] Use original path when falling back to external FS
This is a follow up to 0be9ca7c0f9a733f846bb6bc4e8e36d46b518162 to make
paths in the case of falling back to the external file system use the
original format, preserving relative paths, and allow the external
filesystem to canonicalize them if needed.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D109128
2021-11-13 09:34:44 -08:00
Martin Storsjö
46ec93a457 [Support] [VirtualFileSystem] Detect the windows_slash path style
This fixes the following clang VFS tests, if `windows_slash` is the
default style:

  Clang :: VFS/implicit-include.c
  Clang :: VFS/relative-path.c
  Clang-Unit :: Frontend/./FrontendTests.exe/CompilerInstance.DefaultVFSOverlayFromInvocation

Also clarify a couple references to `Style::windows` into
`Style::windows_backslash`, to make it clearer that each of them are
opinionated in different directions (even if it doesn't matter for
calls to e.g. `is_absolute`).

Differential Revision: https://reviews.llvm.org/D113272
2021-11-08 22:21:29 +02:00
Benjamin Kramer
9b8b16457c Put implementation details into anonymous namespaces. NFCI. 2021-11-07 15:18:30 +01:00
Sam McCall
22555bafe9 [VFS] InMemoryFilesystem's UniqueIDs are a function of path and content.
This ensures that re-creating "the same" FS results in the same UIDs for files.
In turn, this means that creating a clang module (preamble) using one in-memory
filesystem and consuming it using another doesn't create duplicate FileEntrys
for files that are the same in both FSes.

It's tempting to give the creator control over the UIDs instead. However that
requires fiddly API changes, e.g. what should the UIDs of intermediate
directories be?
This change is more "magic" but seems safe given:
 - InMemoryFilesystem is used in testing more than production
 - comparing UIDs across filesystems is unusual
 - files with the same path and content are usually logically equivalent

(The usual reason for re-creating virtual filesystems rather than reusing them
is that typical use involves mutating their CWD and so is not threadsafe).

Differential Revision: https://reviews.llvm.org/D110711
2021-09-29 23:24:18 +02:00
Kazu Hirata
3322354bfc [Support] Qualify auto (NFC)
Identified with readability-qualified-auto.
2021-09-06 09:10:07 -07:00
David Blaikie
1def2579e1 PR51018: Remove explicit conversions from SmallString to StringRef to future-proof against C++23
C++23 will make these conversions ambiguous - so fix them to make the
codebase forward-compatible with C++23 (& a follow-up change I've made
will make this ambiguous/invalid even in <C++23 so we don't regress
this & it generally improves the code anyway)
2021-07-08 13:37:57 -07:00
Martin Storsjö
42f74e8249 [llvm] Rename StringRef _lower() method calls to _insensitive()
This is a mechanical change. This actually also renames the
similarly named methods in the SmallString class, however these
methods don't seem to be used outside of the llvm subproject, so
this doesn't break building of the rest of the monorepo.
2021-06-25 00:22:01 +03:00
Alex Lorenz
2509f9fbad [clang] Don't crash when loading invalid VFS for the module dep collector
The VFS is null when it's invalid so return early in collectVFSFromYAML.
2021-04-26 17:05:22 -07:00