Linking `libomp.so` on 32-bit SPARC `FAIL`s with
```
ld: fatal: file projects/openmp/runtime/src/CMakeFiles/omp.dir/z_Linux_asm.S.o: wrong ELF class: ELFCLASS64
```
This was a 1-stage build with a 64-bit-default `gcc`. Unlike the C++
sources, which were compiled as 32-bit objects due to the use of
`-DCMAKE_CXX_FLAGS=-m32`, the assembler sources were not.
This patch simplifies passing `-m32`: instead of doing it per
architecture, `-m32` is now always passed when the target uses 32-bit
pointers and supports the option.
Tested on `sparc-sun-solaris2.11`, `sparcv9-sun-solaris2.11`,
`sparc-unknown-linux-gnu`, `sparc64-unknown-linux-gnu`,
`i386-pc-solaris2.11`, `amd64-pc-solaris2.11`, `i686-pc-linux-gnu`, and
`x86_64-pc-linux-gnu`.
`openmp` currently doesn't compile on 32-bit Solaris:
```
FAILED: projects/openmp/runtime/src/CMakeFiles/omp.dir/z_Linux_util.cpp.o
[...]
In file included from openmp/runtime/src/z_Linux_util.cpp:78:
In file included from /usr/include/libproc.h:25:
In file included from /usr/include/gelf.h:10:
/usr/include/libelf.h:22:2: error: "large files are not supported by libelf"
22 | #error "large files are not supported by libelf"
| ^
In file included from openmp/runtime/src/z_Linux_util.cpp:78:
/usr/include/libproc.h:42:2: error: "Cannot use libproc in the large file compilation environment"
42 | #error "Cannot use libproc in the large file compilation environment"
| ^
```
Looking closer, there's no point in using `Pgrab` (the only interface
from `<libproc.h>`) at all: the resulting `ps_prochandle_t *` isn't used
in the remainder of the code and the original PR #82930 gives no
indication why it is deemed necessary/useful.
While at it, this patch switches to use `/proc/self/xmap`, matching
`compiler-rt`'s `sanitizer_procmaps_solaris.cpp`, and makes some minor
formatting fixes.
Tested on `sparc-sun-solaris2.11`, `sparcv9-sun-solaris2.11`,
`i386-pc-solaris2.11`, and `amd64-pc-solaris2.11`.
Parts of the `openmp` testsuite currently don't build on SPARC due to
the lack of a `print_possible_return_addresses` definition.
This patch provides one. With it, the vast majority of tests `PASS` on
Solaris/sparcv9 and, with an additional patch, on Linux/sparc64.
The current definition was obtained empirically.
Tested on `sparcv9-sun-solaris2.11`, `sparc64-unknown-linux-gnu`,
`amd64-pc-solaris2.11`, and `x86_64-pc-linux-gnu`.
`libomp` uses `__builtin_return_address` in two places. However, on some
targets those calls need to wrapped in `___builtin_extract_return_addr`
to get at the actual return address. SPARC is among those targets and
the only one where `clang` actually implements this, cf. [[clang][Sparc]
Fix __builtin_extract_return_addr
etc.](https://reviews.llvm.org/D91607). `compiler-rt` needed the same
adjustment, cf. [[sanitizer_common][test] Enable tests on
SPARC](https://reviews.llvm.org/D91608). On other targets, this is a
no-op. However, there are more targets that have the same issue and
`gcc`, unlike `clang`, correctly implements it, so there might be issues
when building `libomp` with `gcc`.
This patch adds the necessary calls.
Tested on `sparcv9-sun-solaris2.11`, `sparc64-unknown-linux-gnu`,
`amd64-pc-solaris2.11`, and `x86_64-pc-linux-gnu`.
Only 2 `openmp` testsuite failures remain on Solaris/amd64. They are due
the same issue: the tests in question assume `NULL` pointers to be
printed as `(nil)` while the rest of the testsuite uses `[[NULL]]` for
that.
This patch changes them to follow suit.
Tested on `amd64-pc-solaris2.11` and `x86_64-pc-linux-gnu`.
After PR #138514, only 3 testsuite failures remain on Solaris/amd64. One
of them is
```
libomp :: ompt/loadtool/tool_available_search/tool_available_search.c
```
The issue is that the expected message is that emitted by Linux/glibc,
while the Solaris message differs:
On Linux/x86_64, I get
```
Opening projects/openmp/runtime/test/ompt/loadtool/tool_available_search/Output/non_existing_file.so... Failed: projects/openmp/runtime/test/ompt/loadtool/tool_available_search/Output/non_existing_file.so: cannot open shared object file: No such file or directory
```
while Solaris/amd64 emits
```
Opening projects/openmp/runtime/test/ompt/loadtool/tool_available_search/Output/non_existing_file.so... Failed: ld.so.1: tool_available_search.c.tmp: projects/openmp/runtime/test/ompt/loadtool/tool_available_search/Output/non_existing_file.so: open failed: No such file or directory
```
Since the exact wording is obviously an implementation detail, this
patch allows for both forms.
Tested on `amd64-pc-solaris2.11` and `x86_64-pc-linux-gnu`.
Testing `openmp` on Solaris/amd64 shows a large number of failures, all
due to the same issue:
```
# .---command stderr------------
# | openmp/runtime/test/ompt/misc/interoperability.cpp:67:16: error: CHECK-SAME: expected string not found in input
# | // CHECK-SAME: parent_task_frame.reenter={{0x[0-f]+}}
# | ^
# | <stdin>:5:101: note: scanning from here
# | 281474976710658: ompt_event_parallel_begin: parent_task_id=281474976710659, parent_task_frame.exit=0, parent_task_frame.reenter=7fffbedffe90, parallel_id=281474976710661, requested_team_size=2, codeptr_ra=408b8e, invoker=2
```
The testsuite expects pointers to be printed with a `0x` prefix when
using the `%p` format, while Solaris `libc` just prints them in hex
without a prefix.
However, this difference is completely benign. ISO C (up to C23,
7.23.6.1) states
```
p The argument shall be a pointer to void or a pointer to a character
type. The value of the pointer is converted to a sequence of printing
characters, in an implementation-defined manner.
```
I saw two ways around this:
- replace every instance of `%p` with a macro (`KMP_PTR_FMT`, defined as
`"%p"` or `"0x%p" as appropriate), or
- adjust the testsuite to make the `0x` prefix optional
The second route seemed less intrusive and more readable, so that's what
this patch does. While large, it's also completely mechanical.
Tested on `amd64-pc-solaris2.11` and `x86_64-pc-linux-gnu`.
PR #138517 broke the Android LLVM builders: ARM doesn't understand the
`@object` form. As it turns out, one can use `%object` instead, which
does assemble on all targets currently supported by `z_Linux_asm.S`.
Tested by rebuilding `libomp.so` on `sparcv9-sun-solaris2.11`.
`libomp` doesn't currently build on Linux/sparc64 due to lack of
`__NR_sched_setaffinity` and `__NR_sched_getaffinity` definitions.
This patch provides those.
Tested on `sparcv9-sun-solaris2.11`, `sparc64-unknown-linux-gnu`,
`amd64-pc-solaris2.11`, and `x86_64-pc-linux-gnu`.
`libomp.so` currently fails to link on SPARC, both Solaris/sparcv9 and
Linux/sparc64:
```
Undefined first referenced
symbol in file
__kmp_unnamed_critical_addr projects/openmp/runtime/src/CMakeFiles/omp.dir/kmp_gsupport.cpp.o
ld: fatal: symbol referencing errors
```
This patch provides the necessary definition. While at it, I noticed
that on non-x86 targets the symbol wasn't marked as `@object`, which
this patch corrects, too.
Tested on `sparcv9-sun-solaris2.11`, `sparc64-unknown-linux-gnu`,
`amd64-pc-solaris2.11`, and `x86_64-pc-linux-gnu`.
Building `openmp` on Solaris/amd64, I get
```
In file included from openmp/runtime/src/kmp_utility.cpp:16:
openmp/runtime/src/kmp_wrapper_getpid.h:47:2: warning: No gettid found, use getpid instead [-W#warnings]
47 | #warning No gettid found, use getpid instead
| ^
```
There's no reason to do this: Solaris can use `pthread_self` just as AIX
does.
Tested on `amd64-pc-solaris2.11` and `x86_64-pc-linux-gnu`.
When running the `openmp` testsuite on Solaris/amd64, many tests `FAIL`
like
```
# | OMP: Error #11: Stack overflow detected for OpenMP thread #1
```
In a `Debug` build, I also get
```
# | Assertion failure at kmp_runtime.cpp(203): __kmp_gtid_get_specific() < 0 || __kmp_gtid_get_specific() == i.
```
Further investigation shows that just setting `__kmp_gtid_mode` to 3
massively reduces the number of failures.
Tested on `amd64-pc-solaris2.11` and `x86_64-pc-linux-gnu`.
When building `openmp` on Linux/sparc64, I get
```
In file included fromopenmp/runtime/src/kmp_utility.cpp:16:
openmp/runtime/src/kmp_wrapper_getpid.h:47:2: warning: No gettid found, use getpid instead [-W#warnings]
47 | #warning No gettid found, use getpid instead
| ^
```
This is highly confusing since `<sys/syscall.h>` **does** define
`SYS_gettid` and the header is supposed to be included:
```
#if !defined(KMP_OS_AIX) && !defined(KMP_OS_HAIKU)
#include <sys/syscall.h>
#endif
```
However, this actually is **not** the case for two reasons:
- `KMP_OS_HAIKU` is always defined, either as 1 on Haiku or as 0
otherwise.
- `KMP_OS_AIX` is even worse: it is only defined as 1 on on AIX, but
undefined otherwise.
All those `KMP_OS_*` macros are supposed to always be defined as 1/0 as
appropriate, and to be checked with `#if`, not `#ifdef`. AIX is
violating this, causing the problem above.
Other targets probably get `<sys/syscall.h>` indirectly otherwise, but
Linux/sparc64 does not.
This patch fixes this by also defining `KMP_OS_AIX` as 0 on other OSes
and changing the checks to `#if` as necessary.
Tested on `sparc64-unknown-linux-gnu`, `sparcv9-sun-solaris2.11`,
`amd64-pc-solaris2.11`, and `x86_64-pc-linux-gnu`.
Despite our attempt (build-docs.sh)
to build the documentation with SVG,
it still uses PNG https://llvm.org/doxygen/classllvm_1_1StringRef.html,
and that renders terribly on any high dpi display.
SVG leads to smasller installation and works fine
on all browser (that has been true for _a while_
https://caniuse.com/svg), so this patch just unconditionally build all
dot graphs as SVG in all subprojects and remove the option.
This commit resolves multiple issues in the OpenMP taskgraph implementation:
- Fix a potential use of uninitialized is_taskgraph and tdg fields when a task is created outside of a taskgraph construct.
- Fix use of task ID field when accessing the taskgraph’s record_map.
- Fix resizing and copying of the successors array when its capacity is exceeded.
Fixes memory management flaws, invalid memory accesses, and uninitialized data risks in taskgraph operations.
TR11 introduced changes to support target memory management in a unified
way by defining a series of API routines and additional traits. Host
runtime is oblivious to how actual memory resources are mapped when
using the new API routines, so it can only support how the composed
memory space is maintained, and the offload backend must handle which
memory resources are actually used to allocate memory from the memory
space.
Here is summary of the implementation.
* Implemented 12 API routines to get/mainpulate memory space/allocator.
* Memory space composed with a list of devices has a state with resource
description, and runtime is responsible for maintaining the allocated
memory space objects.
* Defined interface with offload runtime to access memory resource list,
and to redirect calls to omp_alloc/omp_free since it requires
backend-specific information.
* Value of omp_default_mem_space changed from 0 to 99, and
omp_null_mem_space took the value 0 as defined in the language.
* New allocator traits were introduced, but how to use them is up to the
offload backend.
* Added basic tests for the new API routines.
This patch adds support for memory allocation using hwloc. To enable
memory allocation using hwloc, env KMP_TOPOLOGY_METHOD=hwloc needs to be
used. If hwloc is not supported/available, allocation will fallback to
default path.
This patch attempts to provide a fix for an issue that appears when the
`__kmp_dist_for_static_init` function is called from a serialized team.
This is triggered by code generated by flang for `distribute parallel
do` constructs whenever an `if` clause for the `parallel` leaf construct
is present. This results in the introduction of a call to
`__kmpc_fork_call_if` in place of `__kmpc_fork_call`. When it evaluates
to `false`, it defers execution to `__kmp_serialized_parallel`, which
creates a new serial team that is picked up by
`__kmp_dist_for_static_init`, resulting in an incorrect `team` pointer
that causes the `nteams == (kmp_uint32)team->t.t_parent->t.t_nproc`
assertion to fail.
The sequence of calls replicating this issue can be summarized as:
- `__kmpc_fork_teams`
- `__kmpc_fork_call_if`
- `__kmpc_dist_for_static_init_*`
Since I am not familiar with the implementation of the OpenMP runtime,
it is possible that the above sequence of calls is incorrect, or that
the bug can be better fixed in another way, so I am open to discussing
this.
The following Fortran program can be compiled with flang to show the
issue:
```f90
! Compile and run: flang -fopenmp test.f90 -o test && ./test
! Check LLVM IR: flang -fc1 -emit-llvm -fopenmp test.f90 -o -
program main
implicit none
integer, parameter :: n = 10
integer :: i, idx(n)
!$omp teams
!$omp distribute parallel do if(.false.)
do i=1,n
idx(i) = i
end do
!$omp end teams
print *, idx
end program
```
Updating OpenMP runtime taskgraph support(record/replay mechanism):
- Adds a `graph_reset` bit in `kmp_taskgraph_flags_t` to discard
existing TDG records.
- Switches from a strict index-based TDG ID/IDX to a more flexible
integer-based, which can be any integer (e.g. hashed).
- Adds helper functions like `__kmp_find_tdg`, `__kmp_alloc_tdg`, and
`__kmp_free_tdg` to manage TDGs by their IDs.
These changes pave the way for the integration of OpenMP taskgraph (spec
6.0). Taskgraphs are still recorded in an array with a lookup efficiency
reduced to O(n), where n ≤ `__kmp_max_tdgs`. This can be optimized by
moving the TDGs to a hashtable, making lookups more efficient. The
provided helper routines facilitate easier future optimizations.
This PR creates a new member for task data, which is used to identify
the task in its taskgraph (when ompx taskgraph is enabled).
It aims to remove the overloading of the td_task_id member, which was
used both by the debugger and the taskgraph. This resulted in the
identifier's non-unicity in the case of multiple taskgraphs.
Co-authored-by: Rémy Neveu <rem2007@free.fr>
This PR fixes warning which occurs if one compiles OpenMP runtime with
GCC:
```
warning: comparison of integer expressions of different signedness: 'kmp_intptr_t' {aka 'long int'} and 'long unsigned int' [-Wsign-compare]
```
In several cases the flags entries in ompt_frame_t are not initialized.
According to @jdelsign the address provided as reenter and exit address
is the canonical frame address (cfa) rather than a "framepointer". This
patch makes sure that the flags entry is always initialized and changes
the value from ompt_frame_framepointer to ompt_frame_cfa.
The assertion in the tests makes sure that the flags are always set,
when a tool (callback.h in this case) looks at the value.
Fixes#89058
Switch to using __tsan_init rather than RunningOnValgrind as the means
for detecting TSan instumented binaries. RunningOnValgrind is present in
other libraries (such as Google perftools tcmalloc). An exe that links
with a tcmalloc static library and exports symbols with -rdynamic will
appear to be TSan instrumented even when it is not resulting in "Unable
to fint TSan function ..." messages.
Fixes issue #122319.
This patch series demonstrates and fixes a bug that causes crashes with
OpenMP 'taskwait' directives in heavily multi-threaded scenarios.
TLDR: The early return from __kmpc_omp_taskwait_deps_51 missed the
synchronization mechanism in place for the late return.
Additional debug assertions check for the implied invariants of the code.
@jpeyton52 found the timing hole as this sequence of events:
>
> 1. THREAD 1: A regular task with dependences is created, call it T1
> 2. THREAD 1: Call into `__kmpc_omp_taskwait_deps_51()` and create a stack
based depnode (`NULL` task), call it T2 (stack)
> 3. THREAD 2: Steals task T1 and executes it getting to
`__kmp_release_deps()` region.
> 4. THREAD 1: During processing of dependences for T2 (stack) (within
`__kmp_check_deps()` region), a link is created T1 -> T2. This increases
T2's (stack) `nrefs` count.
> 5. THREAD 2: Iterates through the successors list: decrement the T2's
(stack) npredecessor count. BUT HASN'T YET `__kmp_node_deref()`-ed it.
> 6. THREAD 1: Now when finished with `__kmp_check_deps()`, it returns false
because npredecessor count is 0, but T2's (stack) `nrefs` count is 2 because
THREAD 2 still references it!
> 7. THREAD 1: Because `__kmp_check_deps()` returns false, early exit.
> _Now the stack based depnode is invalid, but THREAD 2 still references it._
>
> We've reached improper stack referencing behavior. Varied results/crashes/
asserts can occur if THREAD 1 comes back and recreates the exact same depnode
in the exact same stack address during the same time THREAD 2 calls
`__kmp_node_deref()`.
A proposed fix for the issue #95611, [OpenMP][SIMD] ordered has no
effect in a loop SIMD region as of LLVM 18.1.0
Changes:
- Implement new lowering behavior: Conservatively serialize "omp simd"
loops that have `omp simd ordered` directive to prevent incorrect
vectorization (which results in incorrect execution behavior of the
miscompiled program).
Implementation outline:
- We start with the optimistic default initial value of
`LoopStack.setParallel(/Enable=/true);` in
`CodeGenFunction::EmitOMPSimdInit(const OMPLoopDirective &D)`.
- We only disable the loop parallel memory access assumption with `if
(HasOrderedDirective) LoopStack.setParallel(/Enable=/false);` using the
`HasOrderedDirective` (which tests for the presence of an
`OMPOrderedDirective`).
- This results in no longer incorrectly vectorizing the loop when the
`omp simd ordered` directive is present.
Motivation: We'd like to prevent incorrect vectorization of the loops
marked with the `#pragma omp ordered simd` directive which has
previously resulted in miscompiled code.
At the same time, we'd like the usage outside of the `#pragma omp
ordered simd` context to remain unaffected: Note that in the test
"clang/test/OpenMP/ordered_codegen.cpp" we only "lose" the
`!llvm.access.group` metadata in `foo_simd` alone.
This is conservative, in that it's possible some of the loops would be
possible to vectorize, but we prefer to avoid miscompilation of the
loops that are currently illegal to vectorize.
A concrete example follows:
```cpp
// "test.c"
#include <float.h>
#include <math.h>
#include <omp.h>
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
int compare_float(float x1, float x2, float scalar) {
const float diff = fabsf(x1 - x2);
x1 = fabsf(x1);
x2 = fabsf(x2);
const float l = (x2 > x1) ? x2 : x1;
if (diff <= l * scalar * FLT_EPSILON)
return 1;
else
return 0;
}
#define ARRAY_SIZE 256
__attribute__((noinline)) void initialization_loop(
float X[ARRAY_SIZE][ARRAY_SIZE], float Y[ARRAY_SIZE][ARRAY_SIZE]) {
const float max = 1000.0;
srand(time(NULL));
for (int r = 0; r < ARRAY_SIZE; r++) {
for (int c = 0; c < ARRAY_SIZE; c++) {
X[r][c] = ((float)rand() / (float)(RAND_MAX)) * max;
Y[r][c] = X[r][c];
}
}
}
__attribute__((noinline)) void omp_simd_loop(float X[ARRAY_SIZE][ARRAY_SIZE]) {
for (int r = 1; r < ARRAY_SIZE; ++r) {
for (int c = 1; c < ARRAY_SIZE; ++c) {
#pragma omp simd
for (int k = 2; k < ARRAY_SIZE; ++k) {
#pragma omp ordered simd
X[r][k] = X[r][k - 2] + sinf((float)(r / c));
}
}
}
}
__attribute__((noinline)) int comparison_loop(float X[ARRAY_SIZE][ARRAY_SIZE],
float Y[ARRAY_SIZE][ARRAY_SIZE]) {
int totalErrors_simd = 0;
const float scalar = 1.0;
for (int r = 1; r < ARRAY_SIZE; ++r) {
for (int c = 1; c < ARRAY_SIZE; ++c) {
for (int k = 2; k < ARRAY_SIZE; ++k) {
Y[r][k] = Y[r][k - 2] + sinf((float)(r / c));
}
}
// check row for simd update
for (int k = 0; k < ARRAY_SIZE; ++k) {
if (!compare_float(X[r][k], Y[r][k], scalar)) {
++totalErrors_simd;
}
}
}
return totalErrors_simd;
}
int main(void) {
float X[ARRAY_SIZE][ARRAY_SIZE];
float Y[ARRAY_SIZE][ARRAY_SIZE];
initialization_loop(X, Y);
omp_simd_loop(X);
const int totalErrors_simd = comparison_loop(X, Y);
if (totalErrors_simd) {
fprintf(stdout, "totalErrors_simd: %d \n", totalErrors_simd);
fprintf(stdout, "%s : %d - FAIL: error in ordered simd computation.\n",
__FILE__, __LINE__);
} else {
fprintf(stdout, "Success!\n");
}
return totalErrors_simd;
}
```
Before:
```
$ clang -fopenmp-simd -O3 -ffast-math -lm test.c -o test && ./test
totalErrors_simd: 15408
test.c : 76 - FAIL: error in ordered simd computation.
```
clang 19.1.0: https://godbolt.org/z/6EvhxqEhe
After:
```
$ clang -fopenmp-simd -O3 -ffast-math test.c -o test && ./test
Success!
```
Co-authored-by: Matt P. Dziubinski <matt-p.dziubinski@hpe.com>
This pull request is the second part of an ongoing effort to extends PGO
instrumentation to GPU device code and depends on #76587. This PR makes
the following changes:
- Introduces `__llvm_write_custom_profile` to PGO compiler-rt library.
This is an external function that can be used to write profiles with
custom data to target-specific files.
- Adds `__llvm_write_custom_profile` as weak symbol to libomptarget so
that it can write the collected data to a profraw file.
- Adds `PGODump` debug flag and only displays dump when the
aforementioned flag is set
Summary:
Freestanding builds have `stddef.h` and `stdint.h` but not `stdlib.h`.
We don't actually use any `stdlib.h` definitions in the OpenMP headers,
and some definitions from this header are usable without the OpenMP
runtime (allocators) so we should be able to do this. This ignores the
include if possible, removing the implicit include would possibly break
some applications so it stays here.
Summary:
The previous offloading entry type did not fit the current use-cases
very well. This widens it and adds a version to prevent further
annoyances. It also includes the kind to better sort who's using it.
The first 64-bytes are reserved as zero so the OpenMP runtime can detect
the old format for binary compatibilitry.
Summary:
We previously built this for every single architecture to deal with
incompatibility. This patch updates it to use the 'generic' IR that
`libc` and other projects use. Who knows if this will have any
side-effects, probably worth testing more but it passes the tests I
expect to pass on my side.
Instead respect what the toolchain default is (or what the user sets via
CMAKE_CXX_FLAGS).
This fixes builds with libcxx, with mingw toolchains targeting
msvcrt.dll, after 5d8be4c036aa5ce4a94f1f37a9155d5c877e23db; after that
commit, the libcxx public headers reference symbols such as iswspace_l,
which are unavailable when targeting msvcrt.dll on older versions of
Windows (it's only available in msvcrt.dll since Windows Vista).
