This switches to `makeIntrusiveRefCnt<FileSystem>` where creating a new
object, and to passing/returning by `IntrusiveRefCntPtr<FileSystem>`
instead of `FileSystem*` or `FileSystem&`, when dealing with existing
objects.
Part of cleanup #151026.
We noticed that when a directory's content changes, its size reported by
`status` can change as well. We do not want to include such "false
positive" cases. This PR revises the computation so that only regular
file size changes are considered out-of-date.
rdar://152247357
`DependencyScanningFileSystemSharedCache` can currently report
out-of-date negatively stat cached paths. This PR enhances the reporting
with two modifications.
1. The reported path are now null terminated char arrays instead of
`StringRef`s. This way the API's user can avoid copying `StringRef`s to
other containers because the char arrays can be used directly.
2. The API now reports out-of-date cache entry due to file size changes.
Specifically, we check each file's cached size against the size of the
same file on the underlying FS. If the sizes are different, diagnostics
will be reported.
rdar://152247357
While the source code isn't supposed to change during a build, in some
environments it does. This adds an option that disables caching of stat
failures, meaning that source files can be added to the build during
scanning.
This adds a `-no-cache-negative-stats` option to clang-scan-deps to
enable this behavior. There are no tests for clang-scan-deps as there's
no reliable way to do so from it. A unit test has been added that
modifies the filesystem between scans to test it.
This PR makes it so that `CompilerInvocation` needs to be provided to
`CompilerInstance` on construction. There are a couple of benefits in my
view:
* Making it impossible to mis-use some `CompilerInstance` APIs. For
example there are cases, where `createDiagnostics()` was called before
`setInvocation()`, causing the `DiagnosticEngine` to use the
default-constructed `DiagnosticOptions` instead of the intended ones.
* This shrinks `CompilerInstance`'s state space.
* This makes it possible to access **the** invocation in
`CompilerInstance`'s constructor (to be used in a follow-up).
We have had numerous situations where the negatively stat cached paths
are invalidated during the build, and such invalidations lead to build
errors.
This PR adds an API to diagnose such cases.
`DependencyScanningFilesystemSharedCache::diagnoseNegativeStatCachedPaths`
allows users of the cache to ask the cache to examine all negatively
stat cached paths, and re-stat the paths using the passed-in file
system. If any re-stat succeeds, the API emits diagnostics.
rdar://149147920
When using the clang dependency scanner with an arbitrary
DiagnosticConsumer, it is important that we always call finish().
Previously, if there was an error preventing us from reaching the
scanning action, or if the command line contained no scannable actions
we would fail to finish(), which would break some consumers (e.g.
serialized diag consumer).
Starting with 41e3919ded78d8870f7c95e9181c7f7e29aa3cc4 DiagnosticsEngine
creation might perform IO. It was implicitly defaulting to
getRealFileSystem. This patch makes it explicit by pushing the decision
making to callers.
It uses ambient VFS if one is available, and keeps using
`getRealFileSystem` if there aren't any VFS.
This patch adds an IsText parameter to the following functions
openFileForRead, getBufferForFile, getBufferForFileImpl and determines
whether a file is text by querying the file tag on z/OS. The default is
set to OF_Text instead of OF_None, this change in value does not affect
any other platforms other than z/OS.
LLVM-based tools often use the `llvm::vfs::FileSystem` instrastructure
to access the file system. This patch adds new kind of a VFS that
performs lightweight tracing of file system operations on an underlying
VFS. This is supposed to aid in investigating file system traffic
without resorting to instrumentation on the operating system level.
There will be follow-up patches that integrate this into Clang and its
dependency scanner.
The scanning VFS doesn't cache stat failures of paths with no extension.
This was originally implemented to avoid caching the non-existence of
the modules cache directory that the modular scanner will eventually
create if it does not exist.
However, this prevents caching of the non-existence of all directories
and notably also header files from the standard C++ library, which can
lead to sub-par performance.
This patch adds an API to the scanning VFS that allows clients to
configure path prefix for which to bypass the scanning VFS and use the
underlying VFS directly.
As-is, calls to `exists()` fallback on the implementation in
`ProxyFileSystem::exists` which explicitly calls out to the underlying
`FS`, which for the `DependencyScanningFilesystem` (overlay) is the real
underlying filesystem.
Instead, directly overloading `exists` allows us to have it rely on the
cached `status` behavior used elsewhere by the
`DependencyScanningFilesystem`.
This PR starts caching calls to
`DependencyScanningWorkerFilesystem::getRealPath()` that we use whenever
we canonicalize module map path. In the case of the real VFS, this
functions performs an expensive syscall that we'd like to do as rarely
as possible.
This PR keeps the real path out of `CachedFileSystemEntry`, since that's
**immutable**; populating the real path on creation of this data
structure (every stat/open) would be expensive.
