7c1d2467f1
Reland of https://github.com/llvm/llvm-project/pull/147381 Added changes to fix observed BuildBot failures: * CMake version (reduced minimum to `3.20`, was: `3.22`) * GoogleTest linking (missing `./build/lib/libllvm_gtest.a`) * Related header issue (missing `#include "llvm/Support/raw_os_ostream.h"`) Original message Description =========== OpenMP Tooling Interface Testing Library (ompTest) ompTest is a unit testing framework for testing OpenMP implementations. It offers a simple-to-use framework that allows a tester to check for OMPT events in addition to regular unit testing code, supported by linking against GoogleTest by default. It also facilitates writing concise tests while bridging the semantic gap between the unit under test and the OMPT-event testing. Background ========== This library has been developed to provide the means of testing OMPT implementations with reasonable effort. Especially, asynchronous or unordered events are supported and can be verified with ease, which may prove to be challenging with LIT-based tests. Additionally, since the assertions are part of the code being tested, ompTest can reference all corresponding variables during assertion. Basic Usage =========== OMPT event assertions are placed before the code, which shall be tested. These assertion can either be provided as one block or interleaved with the test code. There are two types of asserters: (1) sequenced "order-sensitive" and (2) set "unordered" assserters. Once the test is being run, the corresponding events are triggered by the OpenMP runtime and can be observed. Each of these observed events notifies asserters, which then determine if the test should pass or fail. Example (partial, interleaved) ============================== ```c++ int N = 100000; int a[N]; int b[N]; OMPT_ASSERT_SEQUENCE(Target, TARGET, BEGIN, 0); OMPT_ASSERT_SEQUENCE(TargetDataOp, ALLOC, N * sizeof(int)); // a ? OMPT_ASSERT_SEQUENCE(TargetDataOp, H2D, N * sizeof(int), &a); OMPT_ASSERT_SEQUENCE(TargetDataOp, ALLOC, N * sizeof(int)); // b ? OMPT_ASSERT_SEQUENCE(TargetDataOp, H2D, N * sizeof(int), &b); OMPT_ASSERT_SEQUENCE(TargetSubmit, 1); OMPT_ASSERT_SEQUENCE(TargetDataOp, D2H, N * sizeof(int), nullptr, &b); OMPT_ASSERT_SEQUENCE(TargetDataOp, D2H, N * sizeof(int), nullptr, &a); OMPT_ASSERT_SEQUENCE(TargetDataOp, DELETE); OMPT_ASSERT_SEQUENCE(TargetDataOp, DELETE); OMPT_ASSERT_SEQUENCE(Target, TARGET, END, 0); #pragma omp target parallel for { for (int j = 0; j < N; j++) a[j] = b[j]; } ``` References ========== This work has been presented at SC'24 workshops, see: https://ieeexplore.ieee.org/document/10820689 Current State and Future Work ============================= ompTest's development was mostly device-centric and aimed at OMPT device callbacks and device-side tracing. Consequentially, a substantial part of host-related events or features may not be supported in its current state. However, we are confident that the related functionality can be added and ompTest provides a general foundation for future OpenMP and especially OMPT testing. This PR will allow us to upstream the corresponding features, like OMPT device-side tracing in the future with significantly reduced risk of introducing regressions in the process. Build ===== ompTest is linked against LLVM's GoogleTest by default, but can also be built 'standalone'. Additionally, it comes with a set of unit tests, which in turn require GoogleTest (overriding a standalone build). The unit tests are added to the `check-openmp` target. Use the following parameters to perform the corresponding build: `LIBOMPTEST_BUILD_STANDALONE` (Default: ${OPENMP_STANDALONE_BUILD}) `LIBOMPTEST_BUILD_UNITTESTS` (Default: OFF) --------- Co-authored-by: Jan-Patrick Lehr <JanPatrick.Lehr@amd.com> Co-authored-by: Joachim <protze@rz.rwth-aachen.de> Co-authored-by: Joachim Jenke <jenke@itc.rwth-aachen.de>
139 lines
7.7 KiB
C++
139 lines
7.7 KiB
C++
//===- AssertMacros.h - Macro aliases for ease-of-use -----------*- C++ -*-===//
|
|
//
|
|
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
|
|
// See https://llvm.org/LICENSE.txt for license information.
|
|
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
|
|
//
|
|
//===----------------------------------------------------------------------===//
|
|
///
|
|
/// \file
|
|
/// Provides macros to be used in unit tests for OMPT events.
|
|
///
|
|
//===----------------------------------------------------------------------===//
|
|
|
|
#ifndef OPENMP_TOOLS_OMPTEST_INCLUDE_ASSERTMACROS_H
|
|
#define OPENMP_TOOLS_OMPTEST_INCLUDE_ASSERTMACROS_H
|
|
|
|
#define OMPTEST_EXCLUDED_EVENT omptest::ObserveState::Never
|
|
#define OMPTEST_REQUIRED_EVENT omptest::ObserveState::Always
|
|
|
|
/// ASSERT MACROS TO BE USED BY THE USER
|
|
|
|
#define OMPT_GENERATE_EVENTS(NumberOfCopies, EventMacro) \
|
|
for (size_t i = 0; i < NumberOfCopies; ++i) { \
|
|
EventMacro \
|
|
}
|
|
|
|
// Handle a minimum unordered set of events
|
|
// Required events
|
|
#define OMPT_ASSERT_SET_EVENT(Name, Group, EventTy, ...) \
|
|
SetAsserter->insert(OmptAssertEvent::EventTy( \
|
|
Name, Group, OMPTEST_REQUIRED_EVENT, __VA_ARGS__));
|
|
#define OMPT_ASSERT_SET(EventTy, ...) \
|
|
OMPT_ASSERT_SET_EVENT("", "", EventTy, __VA_ARGS__)
|
|
#define OMPT_ASSERT_SET_GROUPED(Group, EventTy, ...) \
|
|
OMPT_ASSERT_SET_EVENT("", Group, EventTy, __VA_ARGS__)
|
|
#define OMPT_ASSERT_SET_NAMED(Name, EventTy, ...) \
|
|
OMPT_ASSERT_SET_EVENT(Name, "", EventTy, __VA_ARGS__)
|
|
// Excluded ("NOT") events
|
|
#define OMPT_ASSERT_SET_EVENT_NOT(Name, Group, EventTy, ...) \
|
|
SetAsserter->insert(OmptAssertEvent::EventTy( \
|
|
Name, Group, OMPTEST_EXCLUDED_EVENT, __VA_ARGS__));
|
|
#define OMPT_ASSERT_SET_NOT(EventTy, ...) \
|
|
OMPT_ASSERT_SET_EVENT_NOT("", "", EventTy, __VA_ARGS__)
|
|
#define OMPT_ASSERT_SET_GROUPED_NOT(Group, EventTy, ...) \
|
|
OMPT_ASSERT_SET_EVENT_NOT("", Group, EventTy, __VA_ARGS__)
|
|
#define OMPT_ASSERT_SET_NAMED_NOT(Name, EventTy, ...) \
|
|
OMPT_ASSERT_SET_EVENT_NOT(Name, "", EventTy, __VA_ARGS__)
|
|
|
|
// Handle an exact sequence of events
|
|
// Required events
|
|
#define OMPT_ASSERT_SEQUENCE_EVENT(Name, Group, EventTy, ...) \
|
|
SequenceAsserter->insert(OmptAssertEvent::EventTy( \
|
|
Name, Group, OMPTEST_REQUIRED_EVENT, __VA_ARGS__));
|
|
#define OMPT_ASSERT_SEQUENCE(EventTy, ...) \
|
|
OMPT_ASSERT_SEQUENCE_EVENT("", "", EventTy, __VA_ARGS__)
|
|
#define OMPT_ASSERT_SEQUENCE_GROUPED(Group, EventTy, ...) \
|
|
OMPT_ASSERT_SEQUENCE_EVENT("", Group, EventTy, __VA_ARGS__)
|
|
#define OMPT_ASSERT_SEQUENCE_NAMED(Name, EventTy, ...) \
|
|
OMPT_ASSERT_SEQUENCE_EVENT(Name, "", EventTy, __VA_ARGS__)
|
|
// Excluded ("NOT") events
|
|
#define OMPT_ASSERT_SEQUENCE_EVENT_NOT(Name, Group, EventTy, ...) \
|
|
SequenceAsserter->insert(OmptAssertEvent::EventTy( \
|
|
Name, Group, OMPTEST_EXCLUDED_EVENT, __VA_ARGS__));
|
|
#define OMPT_ASSERT_SEQUENCE_NOT(EventTy, ...) \
|
|
OMPT_ASSERT_SEQUENCE_EVENT_NOT("", "", EventTy, __VA_ARGS__)
|
|
#define OMPT_ASSERT_SEQUENCE_GROUPED_NOT(Group, EventTy, ...) \
|
|
OMPT_ASSERT_SEQUENCE_EVENT_NOT("", Group, EventTy, __VA_ARGS__)
|
|
#define OMPT_ASSERT_SEQUENCE_NAMED_NOT(Name, EventTy, ...) \
|
|
OMPT_ASSERT_SEQUENCE_EVENT_NOT(Name, "", EventTy, __VA_ARGS__)
|
|
// Special command: suspend active assertion
|
|
// The created event is not correlated to any observed event
|
|
#define OMPT_ASSERT_SEQUENCE_SUSPEND() \
|
|
SequenceAsserter->insert( \
|
|
OmptAssertEvent::AssertionSuspend("", "", OMPTEST_EXCLUDED_EVENT));
|
|
#define OMPT_ASSERT_SEQUENCE_ONLY(EventTy, ...) \
|
|
OMPT_ASSERT_SEQUENCE_SUSPEND() \
|
|
OMPT_ASSERT_SEQUENCE_EVENT("", "", EventTy, __VA_ARGS__) \
|
|
OMPT_ASSERT_SEQUENCE_SUSPEND()
|
|
#define OMPT_ASSERT_SEQUENCE_GROUPED_ONLY(Group, EventTy, ...) \
|
|
OMPT_ASSERT_SEQUENCE_SUSPEND() \
|
|
OMPT_ASSERT_SEQUENCE_EVENT("", Group, EventTy, __VA_ARGS__) \
|
|
OMPT_ASSERT_SEQUENCE_SUSPEND()
|
|
#define OMPT_ASSERT_SEQUENCE_NAMED_ONLY(Name, EventTy, ...) \
|
|
OMPT_ASSERT_SEQUENCE_SUSPEND() \
|
|
OMPT_ASSERT_SEQUENCE_EVENT(Name, "", EventTy, __VA_ARGS__) \
|
|
OMPT_ASSERT_SEQUENCE_SUSPEND()
|
|
|
|
#define OMPT_ASSERTER_MODE_STRICT(Asserter) \
|
|
Asserter->setOperationMode(AssertMode::Strict);
|
|
#define OMPT_ASSERTER_MODE_RELAXED(Asserter) \
|
|
Asserter->setOperationMode(AssertMode::Relaxed);
|
|
#define OMPT_ASSERT_SEQUENCE_MODE_STRICT() \
|
|
OMPT_ASSERTER_MODE_STRICT(SequenceAsserter)
|
|
#define OMPT_ASSERT_SEQUENCE_MODE_RELAXED() \
|
|
OMPT_ASSERTER_MODE_RELAXED(SequenceAsserter)
|
|
#define OMPT_ASSERT_SET_MODE_STRICT() OMPT_ASSERTER_MODE_STRICT(SetAsserter)
|
|
#define OMPT_ASSERT_SET_MODE_RELAXED() OMPT_ASSERTER_MODE_RELAXED(SetAsserter)
|
|
|
|
// Enable / disable asserters entirely
|
|
#define OMPT_ASSERTER_DISABLE(Asserter) Asserter->setActive(false);
|
|
#define OMPT_ASSERTER_ENABLE(Asserter) Asserter->setActive(true);
|
|
#define OMPT_ASSERT_SET_DISABLE() OMPT_ASSERTER_DISABLE(SetAsserter)
|
|
#define OMPT_ASSERT_SET_ENABLE() OMPT_ASSERTER_ENABLE(SetAsserter)
|
|
#define OMPT_ASSERT_SEQUENCE_DISABLE() OMPT_ASSERTER_DISABLE(SequenceAsserter)
|
|
#define OMPT_ASSERT_SEQUENCE_ENABLE() OMPT_ASSERTER_ENABLE(SequenceAsserter)
|
|
#define OMPT_REPORT_EVENT_DISABLE() OMPT_ASSERTER_DISABLE(EventReporter)
|
|
#define OMPT_REPORT_EVENT_ENABLE() OMPT_ASSERTER_ENABLE(EventReporter)
|
|
|
|
// Enable / disable certain event types for asserters
|
|
#define OMPT_ASSERTER_PERMIT_EVENT(Asserter, EventTy) \
|
|
Asserter->permitEvent(EventTy);
|
|
#define OMPT_ASSERTER_SUPPRESS_EVENT(Asserter, EventTy) \
|
|
Asserter->suppressEvent(EventTy);
|
|
#define OMPT_PERMIT_EVENT(EventTy) \
|
|
OMPT_ASSERTER_PERMIT_EVENT(SetAsserter, EventTy); \
|
|
OMPT_ASSERTER_PERMIT_EVENT(EventReporter, EventTy); \
|
|
OMPT_ASSERTER_PERMIT_EVENT(SequenceAsserter, EventTy);
|
|
#define OMPT_SUPPRESS_EVENT(EventTy) \
|
|
OMPT_ASSERTER_SUPPRESS_EVENT(SetAsserter, EventTy); \
|
|
OMPT_ASSERTER_SUPPRESS_EVENT(EventReporter, EventTy); \
|
|
OMPT_ASSERTER_SUPPRESS_EVENT(SequenceAsserter, EventTy);
|
|
|
|
// Set logging level for asserters
|
|
// Note: Logger is a singleton, hence this will affect all asserter instances
|
|
#define OMPT_ASSERTER_LOG_LEVEL(Asserter, LogLevel) \
|
|
Asserter->getLog()->setLoggingLevel(LogLevel);
|
|
|
|
// Set log formatting (esp. coloring) for asserters
|
|
// Note: Logger is a singleton, hence this will affect all asserter instances
|
|
#define OMPT_ASSERTER_LOG_FORMATTED(Asserter, FormatLog) \
|
|
Asserter->getLog()->setFormatOutput(FormatLog);
|
|
|
|
// SyncPoint handling
|
|
#define OMPT_ASSERT_SYNC_POINT(SyncPointName) \
|
|
flush_traced_devices(); \
|
|
OmptCallbackHandler::get().handleAssertionSyncPoint(SyncPointName);
|
|
|
|
#endif
|