Julian Brown 2fdf191e24 [OpenMP] Fix crash with task stealing and task dependencies (#126049)
This patch series demonstrates and fixes a bug that causes crashes with
OpenMP 'taskwait' directives in heavily multi-threaded scenarios.

TLDR: The early return from __kmpc_omp_taskwait_deps_51 missed the
synchronization mechanism in place for the late return.

Additional debug assertions check for the implied invariants of the code.

@jpeyton52 found the timing hole as this sequence of events:
>
> 1. THREAD 1: A regular task with dependences is created, call it T1
> 2. THREAD 1: Call into `__kmpc_omp_taskwait_deps_51()` and create a stack
based depnode (`NULL` task), call it T2 (stack)
> 3. THREAD 2: Steals task T1 and executes it getting to
`__kmp_release_deps()` region.
> 4. THREAD 1: During processing of dependences for T2 (stack) (within
`__kmp_check_deps()` region),  a link is created T1 -> T2. This increases
T2's (stack) `nrefs` count.
> 5. THREAD 2: Iterates through the successors list: decrement the T2's
(stack) npredecessor count. BUT HASN'T YET `__kmp_node_deref()`-ed it.
> 6. THREAD 1: Now when finished with `__kmp_check_deps()`, it returns false
because npredecessor count is 0, but T2's (stack) `nrefs`  count is 2 because
THREAD 2 still references it!
> 7. THREAD 1: Because `__kmp_check_deps()` returns false, early exit.
>    _Now the stack based depnode is invalid, but THREAD 2 still references it._
>
> We've reached improper stack referencing behavior. Varied results/crashes/
asserts can occur if THREAD 1 comes back and recreates the exact same depnode
in the exact same stack address during the same time THREAD 2 calls
`__kmp_node_deref()`.
2025-02-14 10:55:59 +01:00
..
2022-09-19 22:49:15 +00:00

               README for the LLVM* OpenMP* Runtime Library
               ============================================

How to Build Documentation
==========================

The main documentation is in Doxygen* format, and this distribution
should come with pre-built PDF documentation in doc/Reference.pdf.
However, an HTML version can be built by executing:

% doxygen doc/doxygen/config

in the runtime directory.

That will produce HTML documentation in the doc/doxygen/generated
directory, which can be accessed by pointing a web browser at the
index.html file there.

If you don't have Doxygen installed, you can download it from
www.doxygen.org.


How to Build the LLVM* OpenMP* Runtime Library
==============================================
In-tree build:

$ cd where-you-want-to-live
Check out openmp into llvm/projects
$ cd where-you-want-to-build
$ mkdir build && cd build
$ cmake path/to/llvm -DCMAKE_C_COMPILER=<C compiler> -DCMAKE_CXX_COMPILER=<C++ compiler>
$ make omp

Out-of-tree build:

$ cd where-you-want-to-live
Check out openmp
$ cd where-you-want-to-live/openmp/runtime
$ mkdir build && cd build
$ cmake path/to/openmp -DCMAKE_C_COMPILER=<C compiler> -DCMAKE_CXX_COMPILER=<C++ compiler>
$ make

For details about building, please look at README.rst in the parent directory.

Architectures Supported
=======================
* IA-32 architecture
* Intel(R) 64 architecture
* Intel(R) Many Integrated Core Architecture
* ARM* architecture
* Aarch64 (64-bit ARM) architecture
* IBM(R) Power architecture (big endian)
* IBM(R) Power architecture (little endian)
* MIPS and MIPS64 architecture
* RISCV64 architecture
* LoongArch64 architecture

Supported RTL Build Configurations
==================================

Supported Architectures: IA-32 architecture, Intel(R) 64, and
Intel(R) Many Integrated Core Architecture

              ----------------------------------------------
              |   icc/icl     |    gcc      |   clang      |
--------------|---------------|----------------------------|
| Linux* OS   |   Yes(1,5)    |  Yes(2,4)   | Yes(4,6,7)   |
| FreeBSD*    |   No          |  No         | Yes(4,6,7,8) |
| OS X*       |   Yes(1,3,4)  |  No         | Yes(4,6,7)   |
| Windows* OS |   Yes(1,4)    |  No         | No           |
------------------------------------------------------------

(1) On IA-32 architecture and Intel(R) 64, icc/icl versions 12.x are
    supported (12.1 is recommended).
(2) GCC* version 4.7 is supported.
(3) For icc on OS X*, OS X* version 10.5.8 is supported.
(4) Intel(R) Many Integrated Core Architecture not supported.
(5) On Intel(R) Many Integrated Core Architecture, icc/icl versions 13.0
    or later are required.
(6) Clang* version 3.3 is supported.
(7) Clang* currently does not offer a software-implemented 128 bit extended
    precision type.  Thus, all entry points reliant on this type are removed
    from the library and cannot be called in the user program.  The following
    functions are not available:
    __kmpc_atomic_cmplx16_*
    __kmpc_atomic_float16_*
    __kmpc_atomic_*_fp
(8) Community contribution provided AS IS, not tested by Intel.

Supported Architectures: IBM(R) Power 7 and Power 8

              -----------------------------
              |   gcc      |   clang      |
--------------|------------|--------------|
| Linux* OS   |  Yes(1,2)  | Yes(3,4)     |
-------------------------------------------

(1) On Power 7, gcc version 4.8.2 is supported.
(2) On Power 8, gcc version 4.8.2 is supported.
(3) On Power 7, clang version 3.7 is supported.
(4) On Power 8, clang version 3.7 is supported.


Front-end Compilers that work with this RTL
===========================================

The following compilers are known to do compatible code generation for
this RTL: clang (from the OpenMP development branch at
http://clang-omp.github.io/ ), Intel compilers, GCC.  See the documentation
for more details.

-----------------------------------------------------------------------

Notices
=======

*Other names and brands may be claimed as the property of others.