This reapplies 5ffd9bdb50b57 (#133545) with fixes.
The BUILD_SHARED_LIBS=ON build was fixed by adding missing LLVM
dependencies to the InterpTests binary in
unittests/AST/ByteCode/CMakeLists.txt .
Pass all the dependencies into add_clang_unittest. This is consistent
with how it is done for LLDB. I borrowed the same named argument list
structure from add_lldb_unittest. This is a necessary step towards
consolidating unit tests into fewer binaries, but seems like a good
refactoring in its own right.
std::optional::value() has undesired exception checking semantics and is
unavailable in older Xcode (see _LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS). The
call sites block std::optional migration.
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.
This is part of an effort to migrate from llvm::Optional to
std::optional:
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
This reverts commit fb32de9e97af0921242a021e30020ffacf7aa6e2.
Remove the secondary synchronization point as noted by Adrian. This is
technically only to make the builders happier about tests and should not
be needed. This also pushes the condition variable setting to after the
watch is actually established (which was the source of the original race
condition, but would normally succeed as the thread shouldn't get put to
sleep immediately on the trigger of the condition variable).
This also was pretested on the chromium builders:
https://ci.chromium.org/ui/p/chromium/builders/try/win_upload_clang/1612/overview.
This reverts commit 76f1baa7875acd88bdd4b431eed6e2d2decfc0fe.
Also reverts 2 follow-ups:
1. Revert "DirectoryWatcher: also wait for the notifier thread"
This reverts commit 527a1821e6f8e115db3335a3341c7ac491725a0d.
2. Revert "DirectoryWatcher: close a possible window of race on Windows"
This reverts commit a6948da86ad7e78d66b26263c2681ef6385cc234.
Makes tests hang, see comments on https://reviews.llvm.org/D88666
This reverts commit 0ec1cf13f2a4e31aa2c5ccc665c5fbdcd3a94577.
Restore the implementation with some minor tweaks:
- Use std::unique_ptr for the path instead of std::vector
* Stylistic improvement as the buffer is already heap allocated, this
just makes it clearer.
- Correct the notification buffer allocation size
* Memory usage fix: we were allocating 4x the computed size
- Correct the passing of the buffer size to RDC
* Memory usage fix: we were reporting 1/4th of the size
- Convert the operation event to auto-reset
* Bug Fix: we never reset the event
- Remove `FILE_NOTIFY_CHANGE_LAST_ACCESS` from RDC events
* Memory usage fix: we never needed this notification
- Fold events for the notification action
* Stylistic improvement to be clear how the events map
- Update comment
* Stylistic improvement to be clear what the RAII controls
- Fix the race condition that was uncovered previously
* We would return from the construction before the watcher thread
began execution. The test would then proceed to begin execution,
and we would miss the initial notifications. We now ensure that the
watcher thread is initialized before we return. This ensures that
we do not miss the initial notifications.
Running the test on a SSD was able to uncover the access pattern. This
now seems to pass reliably where it was previously flaky locally.
We've observed this test being significantly flaky on our Mac CI
machines when we're running the full check-clang suite. It fails because
the wait_for condition isn't met within 3 seconds. We believe it's
because our CI machines are somewhat underpowered and pretty heavily
loaded when we're running the full check-clang suite.
I ran some experiments on increasing the timeout. I ran the full
check-clang suite 100 times with each timeout value and recorded how
many flaky failures we encountered in these tests. The results are:
3 second timeout (baseline): 20 failures
10 second timeout: 14 failures
20 second timeout: 4 failures
30 second timeout: 2 failures
40 second timeout: 1 failure
50 second timeout: 0 failures
60 second timeout: 0 failures
I ran another set of 100 tests for the 50 second timeout and observed
one flaky failure. By contrast, I ended up running check-clang 500 times
for the 60 second timeout and didn't observe a single flaky failure.
That's how the 60 second timeout value used in this patch was derived.
While a 60 second timeout might seem high, keep in mind that:
- This is a timeout, not a sleep; the test should require much less time
the vast majority of instances, especially on more powerful machines.
- The long timeout is most likely to occur when other tests are also
running at the same time, so the latency of the timeout will also be
masked by the latency of the other tests.
See https://reviews.llvm.org/D58418?id=200123#inline-554211 for where
this timeout was originally introduced and the possibility of raising it
if it wasn't enough was discussed.
Reviewed By: plotfi
Differential Revision: https://reviews.llvm.org/D97878
After D88666, which implemented DirectoryWatcher on Windows, we're
seeing test failures on Chromium's Windows bots.
Try raising the timeout in case the test is failing due to high load on
the machine.
This implements the directory watcher on Windows. It does the most
naive thing for simplicity. ReadDirectoryChangesW is used to monitor
the changes. However, in order to support interruption, we must use
overlapped IO, which allows us to use the blocking, synchronous
mechanism. We create a thread to post the notification to the consumer
to allow the monitoring to continue. The two threads communicate via a
locked queue.
Differential Revision: https://reviews.llvm.org/D88666
Reviewed By: Adrian McCarthy
I observed two bugs in the DirectoryWatcher on macOS
1. We were calling FSEventStreamStop and FSEventStreamInvalidate before
we called FSEventStreamStart and FSEventStreamSetDispatchQueue, if the
DirectoryWatcher was destroyed before the initial async work was done.
This violates the requirements of the FSEvents API.
2. Calls to Receiver could race between the initial work and the
invalidation during destruction.
The second issue is easier to see when using TSan.
Differential Revision: https://reviews.llvm.org/D74371
rdar://59215667
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.
This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.
This doesn't actually modify StringRef yet, I'll do that in a follow-up.
I also have replaced all the instances of
"auto DW = DirectoryWatcher::create" with
llvm::Expected<std::unique_ptr<DirectoryWatcher>> DW = DirectoryWatcher::create
to make it more clear that DirectoryWatcher::create is returning an Expected.
I've also allowed for logAllUnhandledErrors to consume errors in the case were
DirectoryWatcher::create produces them.
Differential Revision: https://reviews.llvm.org/D65829
llvm-svn: 368108
Prior to this patch Unix style errno error reporting from the inotify layer was
used by DirectoryWatcher::create to simply return a nullptr on error. This
would generally be ok, except that in LLVM we have much more robust error
reporting through the facilities of llvm::Expected.
The other critical thing I stumbled across was that the unit tests for
DirectoryWatcher were not failing abruptly when inotify_init() was reporting an
error, but would continue with the testing and eventually hit a deadlock in a
pathological machine state (ie in the unit test, the return nullptr on ::create
was ignored).
Generally this pathological state never happens on any build bot, so it is
totally understandable that it was overlooked, but on a Linux desktop running
a dubious desktop environment (which I will not name) there is a chance that
said desktop environment could use up enough inotify instances to exceed the
user's limit. These are the conditions that led me to hit the deadlock I am
addressing in this patch with more robust error handling.
With the new llvm::Expected error handling when your system runs out of inotify
instances for your user, the unit test will be forced to handle the error or
crash and report the issue to the user instead of weirdly deadlocking on a
condition variable wait.
Differential Revision: https://reviews.llvm.org/D65704
llvm-svn: 367979
This should not affect actual behavior, but should pessimize the threading less
by avoiding the situation where:
* mutex is still locked
* T1 notifies on condition variable
* T2 wakes to check mutex
* T2 sees mutex is still locked
* T2 waits
* T1 unlocks mutex
* T2 tries again, acquires mutex.
Differential Revision: https://reviews.llvm.org/D65708
llvm-svn: 367968
Workaround for FSEvents sometimes sending notifications for events that happened
before DirectoryWatcher was created.
This caused tests to be flaky on green dragon.
llvm-svn: 366138
This reverts commit f561227d133224d2d6a5a016abe4be051fa75501.
- DirectoryWatcher
- Fix the build for platforms that don't have DW implementated.
- Fix the threading dependencies (thanks to compnerd).
llvm-svn: 365954
Asynchronously monitors specified directory for changes and passes notifications to provided callback.
Dependency for index-while-building.
Differential Revision: https://reviews.llvm.org/D58418
llvm-svn: 365574
