Move non-common files from FortranCommon to FortranSupport (analogous to
LLVMSupport) such that
* declarations and definitions that are only used by the Flang compiler,
but not by the runtime, are moved to FortranSupport
* declarations and definitions that are used by both ("common"), the
compiler and the runtime, remain in FortranCommon
* generic STL-like/ADT/utility classes and algorithms remain in
FortranCommon
This allows a for cleaner separation between compiler and runtime
components, which are compiled differently. For instance, runtime
sources must not use STL's `<optional>` which causes problems with CUDA
support. Instead, the surrogate header `flang/Common/optional.h` must be
used. This PR fixes this for `fast-int-sel.h`.
Declarations in include/Runtime are also used by both, but are
header-only. `ISO_Fortran_binding_wrapper.h`, a header used by compiler
and runtime, is also moved into FortranCommon.
Introduce a new `MLIR_LIBS` argument to `add_flang_library`, that uses
`mlir_target_link_libraries` to link the MLIR dylib alterantively to the
component libraries. Use it, along with a few inline
`mlir_target_link_libraries` in tools, to support linking Flang to MLIR
dylib rather than the static libraries.
With these changes, the vast majority of Flang can be linked
dynamically. The only parts still using static libraries are these
requiring MLIR test libraries, that are not included in the dylib.
This commit fixes some but not all memory leaks in Flang. There are
still 91 tests that fail with ASAN.
- Use `mlir::OwningOpRef` instead of `std::unique_ptr`. The latter does
not free allocations of nested blocks.
- Pass `ModuleOp` as value instead of reference.
- Add few missing deallocations in test cases and other places.
As specified in the docs,
1) raw_string_ostream is always unbuffered and
2) the underlying buffer may be used directly
( 65b13610a5226b84889b923bae884ba395ad084d for further reference )
Avoid unneeded calls to raw_string_ostream::str(), to avoid excess indirection.
…ne continuation
Allow preprocessing directives to appear between a source line and its
continuation, including conditional compilation directives (#if, #ifdef,
&c.).
Fixes https://github.com/llvm/llvm-project/issues/95476.
When` SmallVector<char>` is used as a backing storage, it can't be
assumed to end with a \x0. When creating a `StringRef` from it, pass the
length explicitly.
This was detected by address sanitizer.
If `generateLLVMIR()` fails, we still continue using the module we
failed to generate which causes a seg fault if LLVM code-gen failed for
some reason or another. This commit fixes this issue.
Re-applies PR #78269 and adds LLVM and MLIR dependencies that were
missed in the PR. The missing libs were: `LLVMCore` & `MLIRIR`.
This reverts commit 4fc75062745eb5232ea60c37b9ffe61177efa12a.
This reverts commit 99cae9a44fca4cfbd6ee82f196051cbdf6571fa1.
Temporarily until I reproduce and fix a linker issue:
```
FAILED: tools/flang/unittests/Frontend/FlangFrontendTests
...
/usr/bin/ld: tools/flang/unittests/Frontend/CMakeFiles/FlangFrontendTests.dir/CodeGenActionTest.cpp.o: undefined reference to symbol '_ZN4llvm11LLVMContextC1Ev'
/usr/bin/ld: /work1/omp-nightly/build/git/trunk18.0/build/llvm-project/lib/libLLVMCore.so.18git: error adding symbols: DSO missing from command line
```
This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.
I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.
Preliminary patch to change lowering/code generation to use
llvm::DataLayout information instead of generating "sizeof" GEP (see
https://github.com/llvm/llvm-project/issues/71507).
Fortran Semantic analysis needs to know about the target type size and
alignment to deal with common blocks, and intrinsics like
C_SIZEOF/TRANSFER. This information should be obtained from the
llvm::DataLayout so that it is consistent during the whole compilation
flow.
This change is changing flang-new and bbc drivers to:
1. Create the llvm::TargetMachine so that the data layout of the target
can be obtained before semantics.
2. Sharing bbc/flang-new set-up of the
SemanticConstext.targetCharateristics from the llvm::TargetMachine. For
now, the actual part that set-up the Fortran type size and alignment
from the llvm::DataLayout is left TODO so that this change is mostly an
NFC impacting the drivers.
3. Let the lowering bridge set-up the mlir::Module datalayout attributes
since it is doing it for the target attribute, and that allows the llvm
data layout information to be available during lowering.
For flang-new, the changes are code shuffling: the `llvm::TargetMachine`
instance is moved to `CompilerInvocation` class so that it can be used
to set-up the semantic contexts. `setMLIRDataLayout` is moved to
`flang/Optimizer/Support/DataLayout.h` (it will need to be used from
codegen pass for fir-opt target independent testing.)), and the code
setting-up semantics targetCharacteristics is moved to
`Tools/TargetSetup.h` so that it can be shared with bbc.
As a consequence, LLVM targets must be registered when running
semantics, and it is not possible to run semantics for a target that is
not registered with the -triple option (hence the power pc specific
modules can only be built if the PowerPC target is available.
The forwarding header is left in place because of its use in
`polly/lib/External/isl/interface/extract_interface.cc`, but I have
added a GCC warning about the fact it is deprecated, because it is used
in `isl` from where it is included by Polly.
This is a fairly large changeset, but it can be broken into a few
pieces:
- `llvm/Support/*TargetParser*` are all moved from the LLVM Support
component into a new LLVM Component called "TargetParser". This
potentially enables using tablegen to maintain this information, as
is shown in https://reviews.llvm.org/D137517. This cannot currently
be done, as llvm-tblgen relies on LLVM's Support component.
- This also moves two files from Support which use and depend on
information in the TargetParser:
- `llvm/Support/Host.{h,cpp}` which contains functions for inspecting
the current Host machine for info about it, primarily to support
getting the host triple, but also for `-mcpu=native` support in e.g.
Clang. This is fairly tightly intertwined with the information in
`X86TargetParser.h`, so keeping them in the same component makes
sense.
- `llvm/ADT/Triple.h` and `llvm/Support/Triple.cpp`, which contains
the target triple parser and representation. This is very intertwined
with the Arm target parser, because the arm architecture version
appears in canonical triples on arm platforms.
- I moved the relevant unittests to their own directory.
And so, we end up with a single component that has all the information
about the following, which to me seems like a unified component:
- Triples that LLVM Knows about
- Architecture names and CPUs that LLVM knows about
- CPU detection logic for LLVM
Given this, I have also moved `RISCVISAInfo.h` into this component, as
it seems to me to be part of that same set of functionality.
If you get link errors in your components after this patch, you likely
need to add TargetParser into LLVM_LINK_COMPONENTS in CMake.
Differential Revision: https://reviews.llvm.org/D137838
This patch re-factors the driver code in LLVM Flang (frontend +
compiler) to use the MLIR style. For more context, please see:
https://discourse.llvm.org/t/rfc-coding-style-in-the-driver/
Most changes here are rather self-explanatory. Accessors are renamed to
be more consistent with the rest of LLVM (e.g. allSource -->
getAllSources). Additionally, MLIR clang-tidy files are added in the
affected directories.
clang-tidy and clang-format files were copied from MLIR. Small
additional changes are made to silence clang-tidy/clang-format
warnings.
[1] https://mlir.llvm.org/getting_started/DeveloperGuide/
Differential Revision: https://reviews.llvm.org/D125007
*SUMMARY*
Currently, the frontend driver assumes that a target triple is either:
* provided by the frontend itself (e.g. when lowering and generating
code),
* specified through the `-triple/-target` command line flags.
If `-triple/-target` is not used, the frontend will simply use the host
triple.
This is going to be insufficient when e.g. consuming an LLVM IR file
that has no triple specified (reading LLVM files is WIP, see D124667).
We shouldn't require the triple to be specified via the command line in
such situation. Instead, the frontend driver should contain a good
default, e.g. the host triple.
This patch updates Flang's `CompilerInvocation` to do just that, i.e.
defines its default target triple. Similarly to Clang:
* the default `CompilerInvocation` triple is set as the host triple,
* the value specified with `-triple` takes precedence over the frontend
driver default and the current module triple,
* the frontend driver default takes precedence over the module triple.
*TESTS*
This change requires 2 unit tests to be updated. That's because relevant
frontend actions are updated to assume that there's always a valid
triple available in the current `CompilerInvocation`. This update is
required because the unit tests bypass the regular `CompilerInvocation`
set-up (in particular, they don't call
`CompilerInvocation::CreateFromArgs`). I've also taken the liberty to
disable the pre-precossor formatting in the affected unit tests as well
(it is not required).
No new tests are added. As `flang-new -fc1` does not support consuming
LLVM IR files just yet, it is not possible to compile an LLVM IR file
without a triple. More specifically, atm all LLVM IR files are generated
and stored internally and the driver makes sure that these contain a
valid target triple. This is about to change in D124667 (which adds
support for reading LLVM IR/BC files) and that's where tests for
exercising the default frontend driver triple will be added.
*WHAT DOES CLANG DO?*
For reference, the default target triple for Clang's
`CompilerInvocation` is set through option marshalling infra [1] in
Options.td. Please check the definition of the `-triple` flag:
```
def triple : Separate<["-"], "triple">,
HelpText<"Specify target triple (e.g. i686-apple-darwin9)">,
MarshallingInfoString<TargetOpts<"Triple">, "llvm::Triple::normalize(llvm::sys::getDefaultTargetTriple())">,
AlwaysEmit, Normalizer<"normalizeTriple">;
```
Ideally, we should re-use the marshalling infra in Flang.
[1] https://clang.llvm.org/docs/InternalsManual.html#option-marshalling-infrastructure
Differential Revision: https://reviews.llvm.org/D124664
This patch adds support for:
* `-S` in Flang's compiler and frontend drivers,
and implements:
* `-emit-obj` in Flang's frontend driver and `-c` in Flang's compiler
driver (this is consistent with Clang).
(these options were already available before, but only as placeholders).
The semantics of these options in Clang and Flang are identical.
The `EmitObjAction` frontend action is renamed as `BackendAction`. This
new name more accurately reflects the fact that this action will
primarily run the code-gen/backend pipeline in LLVM. It also makes more
sense as an action implementing both `-emit-obj` and `-S` (originally,
it was just `-emit-obj`).
`tripleName` from FirContext.cpp is deleted and, when a target triple is
required, `mlir::LLVM::LLVMDialect::getTargetTripleAttrName()` is used
instead. In practice, this means that `fir.triple` is replaced with
`llvm.target_triple`. The former was effectively ignored. The latter is
used when lowering from the LLVM dialect in MLIR to LLVM IR (i.e. it's
embedded in the generated LLVM IR module). The driver can then re-use
it when configuring the backend. With this change, the LLVM IR files
generated by e.g. `tco` will from now on contain the correct target
triple.
The code-gen.f90 test is replaced with code-gen-x86.f90 and
code-gen-aarch64.f90. With 2 seperate files we can verify that
`--target` is correctly taken into account. LIT configuration is updated
to enable e.g.:
```
! REQUIRES: aarch64-registered-target
```
Differential Revision: https://reviews.llvm.org/D120568
This patch adds support for the `-emit-llvm` option in the frontend
driver (i.e. `flang-new -fc1`). Similarly to Clang, `flang-new -fc1
-emit-llvm file.f` will generate a textual LLVM IR file.
Depends on D118985
Differential Revision: https://reviews.llvm.org/D119012
Rename the current -E option to "-E -Xflang -fno-reformat".
Add a new Parsing::EmitPreprocessedSource() routine to convert the
cooked character stream output of the prescanner back to something
more closely resembling output from a traditional preprocessor;
call this new routine when -E appears.
The new -E output is suitable for use as fixed form Fortran source to
compilation by (one hopes) any Fortran compiler. If the original
top-level source file had been free form source, the output will be
suitable for use as free form source as well; otherwise there may be
diagnostics about missing spaces if they were indeed absent in the
original fixed form source.
Unless the -P option appears, #line directives are interspersed
with the output (but be advised, f18 will ignore these if presented
with them in a later compilation).
An effort has been made to preserve original alphabetic character case
and source indentation.
Add -P and -fno-reformat to the new drivers.
Tweak test options to avoid confusion with prior -E output; use
-fno-reformat where needed, but prefer to keep -E, sometimes
in concert with -P, on most, updating expected results accordingly.
Differential Revision: https://reviews.llvm.org/D106727
In https://reviews.llvm.org/D103612, a definition of an instance of
`Fortran::parser::AnalyzedObjectsAsFortran` was moved (that object is
used in unparsing). That, in turn, introduced a dependency of the unit
tests on the `FortranEvaluate` library, which defines
`AnalyzedObjectsAsFortran`.
That dependency was missed in D103612 and has caused shared-library
builds to fail. I'm submitting this without a review, as it's rather
straightforward omission.
Add support for the following Fortran dialect options:
- -default*
- -flarge-sizes
It also adds two test cases:
# For checking whether `flang-new` is passing options correctly to `flang-new -fc1`.
# For checking if `fdefault-` arguments are processed properly.
Also moves the Dialect related option parsing to a dedicated function
and adds a member `defaultKinds()` to `CompilerInvocation`
Depends on: D96032
Differential Revision: https://reviews.llvm.org/D96344
Make the #include "file" preprocessing directive begin its
search in the same directory as the file containing the directive,
as other preprocessors and our Fortran INCLUDE statement do.
Avoid current working directory for all source files except the original.
Resolve tests.
Differential Revision: https://reviews.llvm.org/D95481
Move the unit test from InputOutputTest.cpp to FrontendActionTest.cpp
and re-implement it in terms of the FrontendActionTest fixture. This is
just a small code clean-up and a continuation of:
* https://reviews.llvm.org/D93544
Moving forward, we should try be implementing all unit-test cases for
Flang's frontend actions in terms of FrontendActionTest.
Reviewed By: sameeranjoshi
Differential Revision: https://reviews.llvm.org/D94922
Currently the new flang driver always runs in free form mode. This patch
adds support for fixed form mode detection based on the file extensions.
Like `f18`, `flang-new` will treat files ending with ".f", ".F" and
".ff" as fixed form. Additionally, ".for", ".FOR", ".fpp" and ".FPP"
file extensions are recognised as fixed form files. This is consistent
with gfortran [1]. In summary, files with the following extensions are
treated as fixed-form:
* ".f", ".F", ".ff", ".for", ".FOR", ".fpp", ".FPP"
For consistency with flang/test/lit.cfg.py and f18, this patch also adds
support for the following file extensions:
* ".ff", ".FOR", ".for", ".ff90", ".fpp", ".FPP"
This is added in flang/lib/Frontend/FrontendOptions.cpp. Additionally,
the following extensions are included:
* ".f03", ".F03", ".f08", ".F08"
This is for compatibility with gfortran [1] and other popular Fortran
compilers [2].
NOTE: internally Flang will only differentiate between fixed and free
form files. Currently Flang does not support switching between language
standards, so in this regard file extensions are irrelevant. More
specifically, both `file.f03` and `file.f18` are represented with
`Language::Fortran` (as opposed to e.g. `Language::Fortran03`).
Summary of changes:
- Set Fortran::parser::Options::sFixedForm according to the file type
- Add isFixedFormSuffix and isFreeFormSuffix helper functions to
FrontendTool/Utils.h
- Change FrontendOptions::GetInputKindForExtension to support the missing
file extensions that f18 supports and some additional ones
- FrontendActionTest.cpp is updated to make sure that the test input is
treated as free-form
[1] https://gcc.gnu.org/onlinedocs/gfortran/GNU-Fortran-and-GCC.html
[2] https://github.com/llvm/llvm-project/blob/master/flang/docs/OptionComparison.md#notes
Differential Revision: https://reviews.llvm.org/D94228
These patch implements a few non-functional-changes:
* switch to using test fixtures for better code sharing
* rename some variables (e.g. to communicate their purpose a bit better)
This patch doesn't change _what_ is being tested.
Differential Revision: https://reviews.llvm.org/D93544
Using files with identical names leads to unexpected failures when tests
are run in parallel. This is tricky to reproduce, but has been happening
on some buildbots since merging https://reviews.llvm.org/D92854. In that
patch I added a unit test with a non-unique test file. This patch fixes
that.
The behaviour triggered with this flag is consistent with `-fparse-only`
in `flang` (i.e. the throwaway driver). This new spelling is consistent
with Clang and gfortran, and was proposed and agreed on for the new
driver in [1].
This patch also adds some minimal logic to communicate whether the
semantic checks have failed or not. When semantic checks fail, a
frontend driver error is generated. The return code from the frontend
driver is then determined by checking the driver diagnostics - the
presence of driver errors means that the compilation has failed. This
logic is consistent with `clang -cc1`.
[1] http://lists.llvm.org/pipermail/flang-dev/2020-November/000588.html
Differential Revision: https://reviews.llvm.org/D92854
Remove clangFrontend from the list of dependencies. These should have
been removed in: 8d51d37e0628bde3eb5a3200507ba7135dfc2751. See also
https://reviews.llvm.org/D87774.
This patch implements the first frontend action for the Flang parser (i.e.
Fortran::parser). This action runs the preprocessor and is invoked with the
`-E` flag. (i.e. `flang-new -E <input-file>). The generated output is printed
to either stdout or the output file (specified with `-` or `-o <output-file>`).
Note that currently there is no mechanism to map options for the
frontend driver (i.e. Fortran::frontend::FrontendOptions) to options for
the parser (i.e. Fortran::parser::Options). Instead,
Frotran::parser::options are hard-coded to:
```
std::vector<std::string> searchDirectories{"."s};
searchDirectories = searchDirectories;
isFixedForm = false;
_encoding(Fortran::parser::Encoding::UTF_8);
```
These default settings are compatible with the current Flang driver. Further
work is required in order for CompilerInvocation to read and map
clang::driver::options to Fortran::parser::options.
Co-authored-by: Andrzej Warzynski <andrzej.warzynski@arm.com>
Differential Revision: https://reviews.llvm.org/D88381
As per point 3 in [1]:
```
Accessor member functions are named with the non-public data member's
name, less the trailing underscore. Mutator member functions are named
set_...
```
Originally we just followed the LLVM's style, which is incompatible with
Flang. This patch renames the accessors and mutators accordingly.
`getDiagnostics` and `GetDiagnostics` are replaced with one accessor:
`diagnostics`. `SetDiagnostics` was neither implemented nor used, so
it's deleted.
[1] https://github.com/llvm/llvm-project/blob/master/flang/docs/C++style.md#naming
Differential Revision: https://reviews.llvm.org/D90300
This patch introduces the dependencies required to read and manage input files
provided by the command line option. It also adds the infrastructure to create
and write to output files. The output is sent to either stdout or a file
(specified with the `-o` flag).
Separately, in order to be able to test the code for file I/O, it adds
infrastructure to create frontend actions. As a basic testable example, it adds
the `InputOutputTest` FrontendAction. The sole purpose of this action is to
read a file from the command line and print it either to stdout or the output
file. This action is run by using the `-test-io` flag also introduced in this
patch (available for `flang-new` and `flang-new -fc1`). With this patch:
```
flang-new -test-io input-file.f90
```
will read input-file.f90 and print it in the output file.
The `InputOutputTest` frontend action has been introduced primarily to
facilitate testing. It is hidden from users (i.e. it's only displayed with
`--help-hidden`). Currently Clang doesn’t have an equivalent action.
`-test-io` is used to trigger the InputOutputTest action in the Flang frontend
driver. This patch makes sure that “flang-new” forwards it to “flang-new -fc1"
by creating a preprocessor job. However, in Flang.cpp, `-test-io` is passed to
“flang-new -fc1” without `-E`. This way we make sure that the preprocessor is
_not_ run in the frontend driver. This is the desired behaviour: `-test-io`
should only read the input file and print it to the output stream.
co-authored-by: Andrzej Warzynski <andrzej.warzynski@arm.com>
Differential Revision: https://reviews.llvm.org/D87989
As usual, it's difficult to handle all different configuration in the first row,
but this one has been extensively tested
Differential Revision: https://reviews.llvm.org/D89452
Currently Flang uses TextDiagnostic, TextDiagnosticPrinter &
TestDiagnosticBuffer classes from Clang (more specifically, from
libclangFrontend). This patch introduces simplified equivalents of these
classes in Flang (i.e. it removes the dependency on libclangFrontend).
Flang only needs these diagnostics classes for the compiler driver
diagnostics. This is unlike in Clang in which similar diagnostic classes
are used for e.g. Lexing/Parsing/Sema diagnostics. For this reason, the
implementations introduced here are relatively basic. We can extend them
in the future if this is required.
This patch also enhances how the diagnostics are printed. In particular,
this is the diagnostic that you'd get _before_ the changes introduced here
(no text formatting):
```
$ bin/flang-new
error: no input files
```
This is the diagnostic that you get _after_ the changes introduced here
(in terminals that support it, the text is formatted - bold + red):
```
$ bin/flang-new
flang-new: error: no input files
```
Tests are updated accordingly and options related to enabling/disabling
color diagnostics are flagged as supported by Flang.
Reviewed By: sameeranjoshi, CarolineConcatto
Differential Revision: https://reviews.llvm.org/D87774
`FlangFrontendTests` depends on libclangFrontend (it uses
DiagnosticConsumer classes from there). This patch adds the missing
dependency in CMake.
The missing dependency manifests itself only with BUILD_SHARED_LIBS=ON.
This symbol is linked in statically with libflangFrontend when
BUILD_SHARED_LIBS=OFF.
Summary:
This is the first patch implementing the new Flang driver as outlined in [1],
[2] & [3]. It creates Flang driver (`flang-new`) and Flang frontend driver
(`flang-new -fc1`). These will be renamed as `flang` and `flang -fc1` once the
current Flang throwaway driver, `flang`, can be replaced with `flang-new`.
Currently only 2 options are supported: `-help` and `--version`.
`flang-new` is implemented in terms of libclangDriver, defaulting the driver
mode to `FlangMode` (added to libclangDriver in [4]). This ensures that the
driver runs in Flang mode regardless of the name of the binary inferred from
argv[0].
The design of the new Flang compiler and frontend drivers is inspired by it
counterparts in Clang [3]. Currently, the new Flang compiler and frontend
drivers re-use Clang libraries: clangBasic, clangDriver and clangFrontend.
To identify Flang options, this patch adds FlangOption/FC1Option enums.
Driver::printHelp is updated so that `flang-new` prints only Flang options.
The new Flang driver is disabled by default. To enable it, set
`-DBUILD_FLANG_NEW_DRIVER=ON` when configuring CMake and add clang to
`LLVM_ENABLE_PROJECTS` (e.g. -DLLVM_ENABLE_PROJECTS=“clang;flang;mlir”).
[1] “RFC: new Flang driver - next steps”
http://lists.llvm.org/pipermail/flang-dev/2020-July/000470.html
[2] “RFC: Adding a fortran mode to the clang driver for flang”
http://lists.llvm.org/pipermail/cfe-dev/2019-June/062669.html
[3] “RFC: refactoring libclangDriver/libclangFrontend to share with Flang”
http://lists.llvm.org/pipermail/cfe-dev/2020-July/066393.html
[4] https://reviews.llvm.org/rG6bf55804924d5a1d902925ad080b1a2b57c5c75c
co-authored-by: Andrzej Warzynski <andrzej.warzynski@arm.com>
Reviewed By: richard.barton.arm, sameeranjoshi
Differential Revision: https://reviews.llvm.org/D86089
