462170 Commits

Author SHA1 Message Date
Hristo Hristov
cc2d061266 [libc++][spaceship][NFC] Improved consistency - removed inline specifier for operator<=>
Removed `inline` specifier for consistency as discussed in D148416 previously.

Reviewed By: #libc, Mordante

Differential Revision: https://reviews.llvm.org/D151248
2023-05-25 10:20:18 +03:00
Martin Braenne
246626a8cf [clang][dataflow] Add a ControlFlowContext::build() overload taking a FunctionDecl.
This is the most common use case, so it makes sense to have a specific overload for it.

Reviewed By: xazax.hun

Differential Revision: https://reviews.llvm.org/D151183
2023-05-25 07:19:12 +00:00
Matthias Springer
ca7167d5a0 [mlir][Transforms][NFC] GreedyPatternRewriteDriver: Add worklist class
Encapsulate all worklist-related functionality in a separate `Worklist` class. This makes the remaining code more readable and allows for custom worklist implementations (e.g., a randomized worklist for fuzzing pattern application: D142447).

Differential Revision: https://reviews.llvm.org/D151345
2023-05-25 09:16:13 +02:00
Petr Hosek
811cbfc262 [lld][ELF] Implement –print-memory-usage
This option was introduced in GNU ld in
https://sourceware.org/legacy-ml/binutils/2015-06/msg00086.html and is
often used in embedded development. This change implements this option
in LLD matching the GNU ld output verbatim.

Differential Revision: https://reviews.llvm.org/D150644
2023-05-25 07:14:18 +00:00
Matthias Springer
3d90c8126f [mlir][sparse] Fix incorrect API usage in RewritePatterns
Incorrect API usage was detected by
`MLIR_ENABLE_EXPENSIVE_PATTERN_API_CHECKS`.

Differential Revision: https://reviews.llvm.org/D151302
2023-05-25 08:52:47 +02:00
eopXD
e369577cd0 [Clang][RISCV] Update all RVV intrinsic auto-generated test case with new script. NFC
This commit updates all intrinsics under
`clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated` because the
new script of `update_llc_test_checks.py` is generating many new lines
differently.

This NFC commit updates the test cases in a whole batch.

Signed-off by: eop Chen <eop.chen@sifive.com>
2023-05-24 23:50:00 -07:00
Siva Chandra Reddy
aaf758c44a [libc][bazel] Add targets for fmod* and scalbn*.
Reviewed By: lntue

Differential Revision: https://reviews.llvm.org/D151354
2023-05-25 06:29:04 +00:00
Siva Chandra Reddy
99a493e3e3 [libc] Make hermetic test depend on the unit test if it exists.
We want to do this so that build system like ninja don't end up running
the hermetic and unit tests in parallel. Running in parallel can cause
problems for tests which read/write disk files as the hermetic and unit
tests can end up stepping on each other.

Reviewed By: jhuber6

Differential Revision: https://reviews.llvm.org/D151291
2023-05-25 06:27:00 +00:00
Shao-Ce SUN
8b90f8e04b [RISCV][CodeGen] Support Zdinx on RV32 codegen
This patch was split from D122918 .

Co-Author: @StephenFan @liaolucy @realqhc

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D149743
2023-05-25 14:13:37 +08:00
Sergei Barannikov
ee1d5f6372 [MC] Check if register is non-null before calling isSubRegisterEq (NFCI)
D151036 adds an assertions that prohibits iterating over sub- and
super-registers of a null register. This is already the case when
iterating over register units of a null register, and worked by
accident for sub- and super-registers.

Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D151285
2023-05-25 08:53:15 +03:00
Siva Chandra Reddy
25174976e1 [libc] Rearrange error and signal tables.
This is largely a cosmetic change done with a few goals:
1. Reduce the conditionals in picking the correct set of tables for the
   platform.
2. Avoid exposing, for example Linux errors, when building for non-Linux
   platforms. This also prevents build failures when Linux errors are not
   defined on the target non-Linux platform.
3. Some "_table" suffixes have been removed to avoid repeated
   occurance of "table" like "tables/linux_error_table.h".

Reviewed By: michaelrj

Differential Revision: https://reviews.llvm.org/D151367
2023-05-25 05:51:32 +00:00
Aart Bik
bcb698bfdc [mlir][sparse][gpu] various cuSparse refinements
(1) keep all cuSparse ops on single stream without wait() in right order
(2) use more type precise memref types for COO
(3) use ToTensor on resulting memref (even though it folds away again)

Reviewed By: K-Wu

Differential Revision: https://reviews.llvm.org/D151404
2023-05-24 22:32:52 -07:00
Md Abdullah Shahneous Bari
edfd360d78 [mlir][spirv] Add OpConvertPtrToU and OpConvertUToPtr ops
Add the definitions and necessary verifications of these Ops.

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D151074
2023-05-24 21:35:30 -07:00
Thurston Dang
9289b64bc4 sanitizer-common: fix buildbot by temporarily disabling release_to_os test for tsan
This has been causing substantial redness in the buildbots e.g.,
https://lab.llvm.org/buildbot/#/builders/239/builds/2405
which already contained a commit to deflake the test
(3afd1b2c35).
Temporarily disabling the test to prevent this error from
hiding other potential problems.
2023-05-25 04:09:35 +00:00
Sergei Barannikov
4374026d6d [MIPS] Check if register is non-null before calling isSuperOrSubRegisterEq (NFCI)
D151036 adds an assertions that prohibits iterating over sub- and
super-registers of a null register. This is already the case when
iterating over register units of a null register, and worked by
accident for sub- and super-registers.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D151288
2023-05-25 06:33:41 +03:00
Sergei Barannikov
0f3efb8d7f [CodeGen] Skip null physical register in AntiDepBreaker (NFCI)
D151036 adds an assertions that prohibits iterating over sub- and
super-registers of a null register. This is already the case when
iterating over register units of a null register, and worked by
accident for sub- and super-registers.
The only place where the assertion is currently triggering is in
CriticalAntiDepBreaker::ScanInstruction. Other places are changed
in case new assertions are added and should be harmless otherwise.
2023-05-25 05:47:59 +03:00
Sergei Barannikov
dbbcfef64f Revert "[CodeGen] Skip null physical register in AntiDepBreaker (NFCI)"
This reverts commit d41f6cff030369d99ad62bbdba732552c08d48a6.

The commit contained wrong phabricator link.
2023-05-25 05:47:38 +03:00
Manna, Soumi
ad571e0d84 [NFC][CLANG] Fix issue with dereference null return value found by Coverity
Reported by Static Analyzer Tool, Coverity:

Inside "SemaDeclCXX.cpp" file, in clang::Sema::CheckExplicitlyDefaultedSpecialMember(clang::CXXMethodDecl *, clang::Sema::CXXSpecialMember, clang::SourceLocation): Return value of function which returns null is dereferenced without checking.

  //returned_null: getAs returns nullptr (checked 117 out of 143 times).
  // var_assigned: Assigning: Type = nullptr return value from getAs.
  const FunctionProtoType *Type = MD->getType()->getAs<FunctionProtoType>();

  //Dereference null return value (NULL_RETURNS)
  //dereference: Dereferencing a pointer that might be nullptr Type when calling getReturnType.
  ReturnType = Type->getReturnType();

  //Dereference null return value (NULL_RETURNS)
  //dereference: Dereferencing a pointer that might be nullptr Type when calling getParamType.
  QualType ArgType = ExpectedParams ? Type->getParamType(0) : QualType();

This patch uses castAs instead of getAs which will assert if the type doesn't match.

Reviewed By: erichkeane

Differential Revision: https://reviews.llvm.org/D151281
2023-05-24 19:17:03 -07:00
Rahul Kayaith
d0d26ee78c [mlir][python] Hook up PyRegionList.__iter__ to PyRegionIterator
This fixes a -Wunused-member-function warning, at the moment
`PyRegionIterator` is never constructed by anything (the only use was
removed in D111697), and iterating over region lists is just falling
back to a generic python iterator object.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D150244
2023-05-24 22:16:58 -04:00
Enna1
693a1b7024 [gcov] Add nosanitize metadata to memory access instructions inserted by emitProfileNotes()
This patch adds nosantize metadata to memory access instructions inserted by gcov emitProfileNotes(), making sanitizers skip these instructions when gcov and sanitizer are used together.

Reviewed By: nickdesaulniers

Differential Revision: https://reviews.llvm.org/D150460
2023-05-25 10:13:52 +08:00
Rahul Kayaith
514dddbeba [mlir][python] Allow specifying block arg locations
Currently blocks are always created with UnknownLoc's for their arguments. This
adds an `arg_locs` argument to all block creation APIs, which takes an optional
sequence of locations to use, one per block argument. If no locations are
supplied, the current Location context is used.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D150084
2023-05-24 21:55:51 -04:00
Vitaly Buka
07edc1c16f [NFC][sanitizer] Rename *ThreadRegistry functions
Reviewed By: thurston

Differential Revision: https://reviews.llvm.org/D150407
2023-05-24 18:34:14 -07:00
Hanhan Wang
3669d07987 [mlir][linalg] Only apply masking on xfer_write when needed.
If the input vector sizes are as same as tensor.pad result shape, the
masking is not needed. Otherwise, the masking is needed and the masking
operands should be as same as tensor.empty op.

Reviewed By: dcaballe

Differential Revision: https://reviews.llvm.org/D151391
2023-05-24 18:24:19 -07:00
Vitaly Buka
d9b574c312 [hwasan] Fix allocator_interface implementation
__sanitizer_get_current_allocated_bytes had as body, but allocator
caches were not registered to collect stats. It's done by
SizeClassAllocator64LocalCache::Init().

Reviewed By: thurston

Differential Revision: https://reviews.llvm.org/D151389
2023-05-24 18:05:42 -07:00
Aart Bik
76b7dca47d [mlir][sparse][gpu] fixed typo in CUDA test
Test was printing same result twice

Reviewed By: K-Wu

Differential Revision: https://reviews.llvm.org/D151370
2023-05-24 18:00:23 -07:00
Kai Sasaki
be2f384f9e
[mlir][linalg] Treat quant dialect type as unsupported in named conversion
Since the tosa-to-linalg conversion does not support the quant dialect type, we can treat it as unsupported instead of crash.  Issue was reported https://github.com/llvm/llvm-project/issues/62367

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D151296
2023-05-25 09:35:23 +09:00
Craig Topper
2f3e17566b [RISCV] Add a special caes to performVFMADD_VLCombine to support the multiplicand being the same value.
The one use check will fail if there are two uses in the same
instruction. Add a special case for this.
2023-05-24 17:25:33 -07:00
Thurston Dang
5be5746b6e hwasan: refactor order of macros in hwasan_platform_interceptors.h [NFC]
Currently, the header file contains all the undefs, followed by all the
define X 0. This will be inconvenient for re-enabling interceptors,
because we would need to comment out (or delete) the corresponding macros
in two different places.

This patch groups together the macros for each function.

Additionally, it adds the suggestion that interceptors should be
re-enabled by commenting out (not deleting) the macros.

Differential Revision: https://reviews.llvm.org/D151371
2023-05-25 00:12:44 +00:00
Vitaly Buka
8826cd5782 [AST] Construct Capture objects before use
Msan reports https://reviews.llvm.org/P8308
The reason is if PointerIntPair is not properly
constructed, setPointer uses Info::updatePointer
on uninitialized value.

Reviewed By: #clang, rsmith

Differential Revision: https://reviews.llvm.org/D150504
2023-05-24 17:09:45 -07:00
Md Abdullah Shahneous Bari
6d578669f3 [mlir][spirv] Support import/export Functions and GlobalVariables
"LinkageAttributes" decoration allow a SPIR-V module to import
external functions and global variables, or export functions or
global variables for other SPIR-V modules to link against and use.

Import/export capability is extremely important when using outside
libraries (e.g., intrinsic libraries).

Added decorations:
- LinkageAttributes

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D148749
2023-05-24 17:08:46 -07:00
eopXD
5f84641901 [Clang][RISCV] Update vaadd.c test case with new script. NFC 2023-05-24 17:03:39 -07:00
Vitaly Buka
3afd1b2c35 [sanitizer] Deflake test 2023-05-24 16:48:23 -07:00
Craig Topper
40a274c6d2 [RISCV] Remove some unneeded vmacc isel patterns.
The patterns are for a vpmerge with an all 1s mask, but we are
able to handle that with a post-isel peephole recently.
2023-05-24 16:37:16 -07:00
Slava Zakharin
ca2ec4b720 [flang][hlfir][NFC] Make BOZ lowering a TODO.
This change just turns the unhandled BOZ fatal error into TODO
like in non-HLFIR path.
2023-05-24 16:34:56 -07:00
Vitaly Buka
f5808fc197 [sanitizer] Use atomic_fetch_add instead of load/store 2023-05-24 16:22:36 -07:00
Vitaly Buka
ae072da552 [NFC][sanitizer] Remove unused method 2023-05-24 16:22:36 -07:00
Rashmi Mudduluru
ee6b08e993 [-Wunsafe-buffer-usage] Group variables associated by pointer assignments
Differential Revision: https://reviews.llvm.org/D145739
2023-05-24 16:20:55 -07:00
Vitaly Buka
096348b179 [sanitizer] Lazy initialize AllocatorGlobalStats
This allow to have no InitLinkerInitialized and let AllocatorGlobalStats
accept registration before allocator initialization.
2023-05-24 16:10:45 -07:00
Lei Zhang
25a4a9360e [mlir][capi] Drop mlirShapedTypeGetTypeID
ShapedType is a virtual type rather than a concrete one.
We don't have an implmentation for this API too.

Reviewed By: makslevental

Differential Revision: https://reviews.llvm.org/D151376
2023-05-24 17:51:25 -05:00
Peng Sun
a383a481ee [mlir][tosa] Update the limit of 62 on shift.
Aligns the shift requirement with the TOSA specification.

Reviewed By: eric-k256

Differential Revision: https://reviews.llvm.org/D151113
2023-05-24 15:35:50 -07:00
Nikolas Klauser
e837f4b7db [libc++][PSTL] Add a simple std::thread backend
This is just to test that the PSTL works with parallelization. This is not supposed to be a production-ready backend.

Reviewed By: ldionne, #libc

Spies: EricWF, arichardson, libcxx-commits

Differential Revision: https://reviews.llvm.org/D150284
2023-05-24 15:33:58 -07:00
Nishant Patel
266b5bc1f6 [mlir][spirv] Add a missing pattern to MathToSPIRV Conversion pass
The MathToSPIRV conversion pass missed out a pattern for converting
math::AbsIOP to spirv::CLSAbsOp

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D151378
2023-05-24 15:30:54 -07:00
Sterling Augustine
4d1cd1d8ca remove useless visibility spec from bazel build. 2023-05-24 15:11:31 -07:00
Amara Emerson
6e88276411 Pre-commit a min/max idiom breakage test for D149918 2023-05-24 15:02:40 -07:00
Tai Ly
e0537d1ad4 [TOSA] Refactor TosaMakeBroadcastable pass
This refactors and exposes EqualizeRanks utility function
from within TosaMakeBroadcastable pass so it may be used to
reshape operator inputs to equal ranks.

Signed-off-by: Tai Ly <tai.ly@arm.com>

Differential Revision: https://reviews.llvm.org/D150283
2023-05-24 14:43:33 -07:00
Advenam Tacet
605b9c76e0 [2a/3][ASan][libcxx] std::deque annotations
This revision is a part of a series of patches extending AddressSanitizer C++ container overflow detection capabilities by adding annotations, similar to those existing in std::vector, to std::string and `std::deque` collections. These changes allow ASan to detect cases when the instrumented program accesses memory which is internally allocated by the collection but is still not in-use (accesses before or after the stored elements for `std::deque`, or between the size and capacity bounds for `std::string`).

The motivation for the research and those changes was a bug, found by Trail of Bits, in a real code where an out-of-bounds read could happen as two strings were compared via a std::equals function that took `iter1_begin`, `iter1_end`, `iter2_begin` iterators (with a custom comparison function). When object `iter1` was longer than `iter2`, read out-of-bounds on `iter2` could happen. Container sanitization would detect it.

This revision introduces annotations for `std::deque`. Each chunk of the container can now be annotated using the `__sanitizer_annotate_double_ended_contiguous_container` function, which was added in the rG1c5ad6d2c01294a0decde43a88e9c27d7437d157. Any attempt to access poisoned memory will trigger an ASan error. Although false negatives are rare, they are possible due to limitations in the ASan API, where a few (usually up to 7) bytes before the container may remain unpoisoned. There are no false positives in the same way as with `std::vector` annotations.

This patch only supports objects (deques) that use the standard allocator. However, it can be easily extended to support all allocators, as suggested in the D146815 revision.

Furthermore, the patch includes the addition of the `is_double_ended_contiguous_container_asan_correct` function to libcxx/test/support/asan_testing.h. This function can be used to verify whether a `std::deque` object has been correctly annotated.

Finally, the patch extends the unit tests to verify ASan annotations (added LIBCPP_ASSERTs).
If a program is compiled without ASan, all helper functions will be no-ops. In binaries with ASan, there is a negligible performance impact since the code from the change is only executed when the deque container changes in size and it’s proportional to the change. It is important to note that regardless of whether or not these changes are in use, every access to the container's memory is instrumented.

Reviewed By: #libc, philnik

Spies: vitalybuka, hans, mikhail.ramalho, Enna1, #sanitizers, philnik, libcxx-commits

Differential Revision: https://reviews.llvm.org/D132092
2023-05-24 14:26:57 -07:00
Valentin Clement
b4b9ed75d3
[flang][openacc][NFC] Remove unused function
Now that operands have moved to the new data operands lowering, this
function is not used anymore.

Reviewed By: vzakhari

Differential Revision: https://reviews.llvm.org/D151357
2023-05-24 14:00:46 -07:00
Matt Arsenault
9ef1333bf4 AMDGPU: Replace certain llvm.amdgcn.class uses with llvm.is.fpclass
Most transforms should now be performed on llvm.is.fpclass. Unlike the
generic intrinsic, this supports variable test masks.
2023-05-24 21:49:52 +01:00
Mehdi Amini
5f649130eb Fix MLIR bytecode reading of i0 IntegerAttr
The move of the bytecode serialization to be tablegen driven in
https://reviews.llvm.org/D144820 added a new condition in the reading
path that forbid 0-sized integer, even though we still produce them.

Fix #62920

Differential Revision: https://reviews.llvm.org/D151372
2023-05-24 13:45:58 -07:00
AdityaK
63a2b206fa [libc++, std::vector] call the optimized version of __uninitialized_allocator_copy for trivial types
See: https://github.com/llvm/llvm-project/issues/61987

Fix suggested by: @philnik and @var-const

Reviewers: philnik, ldionne, EricWF, var-const

Differential Revision: https://reviews.llvm.org/D147741

Testing:
ninja check-cxx check-clang check-llvm

Benchmark Testcases (BM_CopyConstruct, and BM_Assignment) added.

performance improvement:

Run on (8 X 4800 MHz CPU s)
CPU Caches:
  L1 Data 48 KiB (x4)
  L1 Instruction 32 KiB (x4)
  L2 Unified 1280 KiB (x4)
  L3 Unified 12288 KiB (x1)
Load Average: 1.66, 3.02, 2.43

Comparing build-runtimes-base/libcxx/benchmarks/vector_operations.libcxx.out to build-runtimes/libcxx/benchmarks/vector_operations.libcxx.out
Benchmark                                                   Time             CPU      Time Old      Time New       CPU Old       CPU New
----------------------------------------------------------------------------------------------------------------------------------------
BM_ConstructSize/vector_byte/5140480                     +0.0362         +0.0362        116906        121132        116902        121131
BM_CopyConstruct/vector_int/5140480                      -0.4563         -0.4577       1755224        954241       1755330        951987
BM_Assignment/vector_int/5140480                         -0.0222         -0.0220        990045        968095        989917        968125
BM_ConstructSizeValue/vector_byte/5140480                +0.0308         +0.0307        116970        120567        116977        120573
BM_ConstructIterIter/vector_char/1024                    -0.0831         -0.0831            19            17            19            17
BM_ConstructIterIter/vector_size_t/1024                  +0.0129         +0.0131            88            89            88            89
BM_ConstructIterIter/vector_string/1024                  -0.0064         -0.0018         54455         54109         54208         54112
OVERALL_GEOMEAN                                          -0.0845         -0.0842             0             0             0             0

FYI, the perf improvements for BM_CopyConstruct due to this patch is mostly subsumed by the https://reviews.llvm.org/D149826. However this patch still adds value by converting copy to memmove (the second testcase).

Before the patch:

```
define linkonce_odr dso_local void @_ZNSt3__16vectorIiNS_9allocatorIiEEE18__construct_at_endIPiS5_EEvT_T0_m(ptr noundef nonnull align 8 dereferenceable(24) %0, ptr noundef %1, ptr noundef %2, i64 noundef %3) local_unnamed_addr #4 comdat align 2 {
  %5 = getelementptr inbounds %"class.std::__1::vector", ptr %0, i64 0, i32 1
  %6 = load ptr, ptr %5, align 8, !tbaa !12
  %7 = icmp eq ptr %1, %2
  br i1 %7, label %16, label %8

8:                                                ; preds = %4, %8
  %9 = phi ptr [ %13, %8 ], [ %1, %4 ]
  %10 = phi ptr [ %14, %8 ], [ %6, %4 ]
  %11 = icmp ne ptr %10, null
  tail call void @llvm.assume(i1 %11)
  %12 = load i32, ptr %9, align 4, !tbaa !14
  store i32 %12, ptr %10, align 4, !tbaa !14
  %13 = getelementptr inbounds i32, ptr %9, i64 1
  %14 = getelementptr inbounds i32, ptr %10, i64 1
  %15 = icmp eq ptr %13, %2
  br i1 %15, label %16, label %8, !llvm.loop !16

16:                                               ; preds = %8, %4
  %17 = phi ptr [ %6, %4 ], [ %14, %8 ]
  store ptr %17, ptr %5, align 8, !tbaa !12
  ret void
}
```

After the patch:
```
define linkonce_odr dso_local void @_ZNSt3__16vectorIiNS_9allocatorIiEEE18__construct_at_endIPiS5_EEvT_T0_m(ptr noundef nonnull align 8 dereferenceable(24) %0, ptr noundef %1, ptr noundef %2, i64 noundef %3) local_unnamed_addr #4 comdat align 2 {
  %5 = getelementptr inbounds %"class.std::__1::vector", ptr %0, i64 0, i32 1
  %6 = load ptr, ptr %5, align 8, !tbaa !12
  %7 = ptrtoint ptr %2 to i64
  %8 = ptrtoint ptr %1 to i64
  %9 = sub i64 %7, %8
  %10 = ashr exact i64 %9, 2
  tail call void @llvm.memmove.p0.p0.i64(ptr align 4 %6, ptr align 4 %1, i64 %9, i1 false)
  %11 = getelementptr inbounds i32, ptr %6, i64 %10
  store ptr %11, ptr %5, align 8, !tbaa !12
  ret void
}
```

This is due to the optimized version of uninitialized_allocator_copy function.
2023-05-24 13:40:53 -07:00