…dy.py
Currently, run_clang_tidy.py does not correctly display the list of
checks picked up from the top-level .clang-tidy file. The reason for
that is that we are passing an empty string as input file.
However, that's not how we are supposed to use clang-tidy to list
checks. Per
65eccb463d,
we simply should not pass any file at all - the internal code of
clang-tidy will pass a "dummy" file if that's the case and get the
.clang-tidy file from the current working directory.
Fixes#136659
Co-authored-by: Carlos Gálvez <carlos.galvez@zenseact.com>
On my new machine, the script outputs some shutdown noise:
```
Ctrl-C detected, goodbye.
Traceback (most recent call last):
File "/home/nvankempen/llvm-project/./clang-tools-extra/clang-tidy/tool/run-clang-tidy.py", line 626, in <module>
asyncio.run(main())
File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.10/asyncio/base_events.py", line 636, in run_until_complete
self.run_forever()
File "/usr/lib/python3.10/asyncio/base_events.py", line 603, in run_forever
self._run_once()
File "/usr/lib/python3.10/asyncio/base_events.py", line 1871, in _run_once
event_list = self._selector.select(timeout)
File "/usr/lib/python3.10/selectors.py", line 469, in select
fd_event_list = self._selector.poll(timeout, max_ev)
KeyboardInterrupt
```
This fixes it. Also remove an unused typing import.
Relevant documentation:
https://docs.python.org/3/library/asyncio-runner.html#handling-keyboard-interruption
There is no need to support Python 2.7 anymore, Python 3.3+ has
`subprocess.DEVNULL`. This is good practice and also prevents file
handles from
staying open unnecessarily.
Also remove a couple unused or unneeded `__future__` imports.
[There is
work](https://discourse.llvm.org/t/rfc-upgrading-llvms-minimum-required-python-version/67571)
to make Python 3.8 the minimum Python version for LLVM.
I edited this script because I wanted some indicator of progress while
going through files.
It now outputs `[XX/YYY]` with the number of processed and total files
after each completion.
The current version of this script is compatible downto Python 3.6 (this
is PyYAML's minimum version).
It would probably work with older Python 3 versions with an older PyYAML
or when YAML is disabled.
With the updates here, it is compatible downto Python 3.7. Python 3.7
was released June 2018.
https://github.com/llvm/llvm-project/pull/89302 is also touching this
file, I don't mind rebasing on top of that work if needed.
### Summary
- Add type annotations.
- Replace `threading` + `queue` with `asyncio`.
- **Add indicator of processed files over total files**. This is what I
set out to do initially.
- Only print the filename after completion, not the entire Clang-Tidy
invocation command. I find this neater but the behavior can easily be
restored.
When clang-tidy get an empty checks, it will throw "no checks enabled"
error and exit with non-zero return value.
It make clang-tidy's wrapper program confused when in big project some
files don't want to be checked and use `-checks=-*` to disable all
checks.
---------
Co-authored-by: Danny Mösch <danny.moesch@icloud.com>
This is a renewed attempt to land @toddlipcon's D34654. The comments on
that patch indicate a broad desire for some ability to ignore headers.
After considering various options, including migrating to std::regex, I
believe this is the best path forward. It's intuitive to have separate
regexes for including headers versus excluding them, and this approach
has the added benefit of being completely opt-in. No existing configs
will break, regardless of existing HeaderFilterRegex values.
This functionality is useful for improving performance when analyzing a
targeted subset of code, as well as in cases where some collection of
headers cannot be modified (third party source, for example).
I added the option -source-filter to the
`run-clang-tidy.py` script in the clang-tools-extra.
This option allows for handing over a regex, to filter out source files
from the compilation database (not run `clang-tidy` on them).
It contained 'thewarnings' without a space. It also fixes some odd
formatting in the text introduced by
commit dd3c26a045c081620375a878159f536758baba6e
Author: Tobias Hieta <tobias@hieta.se>
Date: Wed May 17 10:56:49 2023 +0200
[NFC][Py Reformat] Reformat python files in clang and clang-tools-extra
Adding an additional parameter to run_clang_tidy.py to accept a
directory where the clang-tidy fixes are saved to. This directory can
then be used to run `clang-apply-replacements`.
Closes#69450
Differential https://reviews.llvm.org/D145477 removed the check for `(yaml and args.export_fixes)` in line 303 to skip looking for the `clang-apply-replacements` binary. However, the `tmpdir` variable was set in this true branch when exporting fixes and therefore is `None` when invoking run-clang-tidy with `run-clang-tidy -p . -export-fixes fixes.yaml`.
Reviewed By: PiotrZSL
Differential Revision: https://reviews.llvm.org/D157773
This is an ongoing series of commits that are reformatting our
Python code.
Reformatting is done with `black`.
If you end up having problems merging this commit because you
have made changes to a python file, the best way to handle that
is to run git checkout --ours <yourfile> and then reformat it
with black.
If you run into any problems, post to discourse about it and
we will try to help.
RFC Thread below:
https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style
Reviewed By: MatzeB
Differential Revision: https://reviews.llvm.org/D150761
run-clang-tidy.py should only search for the clang-apply-replacements if really needed.
Reviewed By: carlosgalvezp
Differential Revision: https://reviews.llvm.org/D145477
The current way to specify CheckOptions is pretty verbose and unintuitive.
Given that the options are a dictionary it makes much more sense to treat them as such in the config files.
Example:
```
CheckOptions: {SomeCheck.Option: true, SomeCheck.OtherOption: 'ignore'}
# Or
CheckOptions:
SomeCheck.Option: true
SomeCheck.OtherOption: 'ignore'
```
This change will still handle the old syntax with no issue, ensuring we don't screw up users current config files.
The only observable differences are support for the new syntax and `-dump=config` will emit using the new syntax.
Reviewed By: aaron.ballman
Differential Revision: https://reviews.llvm.org/D128337
If two different texts are inserted at the same offset, clang-apply-replacements prints the conflict error and discards all fixes. This patch adds support for adjusting conflict offset and keeps running to continually fix them.
https://godbolt.org/z/P938EGoxj doesn't have any fixes when I run run-clang-tidy.py to generate a YAML file with clang-tidy and fix them with clang-apply-replacements. The YAML file has two different header texts insertions at the same offset, unlike clang-tidy with '-fix', clang-apply-replacements does not adjust for this conflict.
Reviewed By: aaron.ballman
Differential Revision: https://reviews.llvm.org/D123924
Support for loading shared objects as plugins into clang-tidy was added
in http://reviews.llvm.org/D111100. Unfortunately, the utility scripts
`clang-tidy-diff.py` and `run-clang-tidy.py` did not receive
corresponding arguments to forward such plugins to clang-tidy.
This diff adds a `-load=plugin` option to both scripts.
Differential Revision: http://reviews.llvm.org/D12306
Reviewed By: aaron.ballman
`find_compilation_database` checked only for "/" as exit point, but on Windows, this root is impossible.
Fixes#53642
Authored By: Febbe
Reviewed By: JonasToth
Differential Revision: https://reviews.llvm.org/D119481
The new -config-file option introduced by 9e1f4f1 was accidentally
referenced as args.config_path on the python side. This patch renames
args.config_path to args.config_file.
To avoid confusion with python file objects, the input argument for
get_tidy_invocation has been renamed from config_path to
config_file_path.
See GitHub issue #54728 for a discussion.
Link to the GitHub Issue: https://github.com/llvm/llvm-project/issues/53745
Added config_path variable within the python script which makes the
required call to the clang-tidy binary with --config-file option.
If the config_path is None then config will be used. No error is raised
if both are given but silently chooses config_path over config
This changes the clang-tidy script to discover binaries you've built
locally without having to pass them.
Differential Revision: https://reviews.llvm.org/D100692
D90110 modified the behavior of `run-clang-tidy` to always pass the
`--use-color` option to clang-tidy, which enabled colored diagnostics
output regardless of TTY status or .clang-tidy settings. This left the
user with no option to disable the colored output.
This presents an issue when trying to parse the output of run-clang-tidy
programmaticall, as the output is polluted with ANSI escape characters.
This PR fixes this issue in two ways:
1. It restores the default behavior of `run-clang-tidy` to let
`clang-tidy` decide whether to color output. This allows the user to
configure color via the `UseColor` option in a .clang-tidy file.
2. It adds mutually exclusive, optional `-use-color` and `-no-use-color`
argument flags that let the user explicitly set the color option via
the invocation.
After this change the default behavior of `run-clang-tidy` when no
.clang-tidy file is available is now to show no color, presumably
because `clang-tidy` detects that the output is being piped and defaults
to not showing colored output. This seems like an acceptable tradeoff
to respect .clang-tidy configurations, as users can still use the
`-use-color` option to explicitly enable color.
Fixes#49441 (50097 in Bugzilla)
Reviewed By: njames93
Differential Revision: https://reviews.llvm.org/D119562
The test invocation at the start of run-clang-tidy.py (line 257) prints
all enabled checks - meaning either the default set or anything
configured via the -checks option. If any checks were (un-)configured
via the -config option, these are not printed. This is confusing to the
user, since the list of checks that are printed may be different from
the list of checks that are used by the non-testing calls to clang-tidy,
where the -config option is passed correctly.
This patch adds the -config option to the test invocation of clang-tidy
at the start of the script. This means that checks (un-)configured via
the -config option (rather than the -checks option) are applied
correctly, when printing the list of enabled checks.
The files in compile-commands.json can potentially include duplicates.
Change run-clang-tidy.py so that it does not run on the duplicate entries.
Differential Revision: https://reviews.llvm.org/D112926
If a clang-tidy child process exits with a signal then run-clang-tidy will exit
with an error but there is no hint why in the output, since the clang-tidy
doesn't log anything and may not even have had the opportunity to do so
depending on the signal used.
`subprocess.CompletedProcess.returncode` is the negative signal number in this
case.
I hit this in a CI system where the parallelism used exceeded the RAM assigned
to the container causing the OOM killer to SIGKILL clang-tidy processes.
Reviewed By: sylvestre.ledru
Differential Revision: https://reviews.llvm.org/D99081
Now that clang-tidy supports the --use-color command line option, it's
a better user experience to use --use-color in run-clang-tidy.py and
preserving the colored output.
The action='store_true' option of argparse.add_argument implicitly
generates a default value of False if the argument is not specified.
Thus, the allow_enabling_alpha_checkers argument of
get_tidy_invocation is never None.
Summary:
run-clang-tidy.py was enforcing '-header-filter' parameter with an
unfortunate default value when none was given. Thus, leading to
overwritten clang-tidy configuration (e.g. from .clang-tidy).
This change removes the default value for '-header-filter' resulting in
the default behaviour of clang-tidy itself.
Fixes PR#41426
Reviewed By: hintonda
Patch by Torbjörn Klatt!
Tags: #clang-tools-extra, #clang
Differential Revision: https://reviews.llvm.org/D61747
llvm-svn: 361344
Summary: The output of clang-tidy itself already has enough newlines, so the resulting output is more in line with the usual compiler output.
Patch by svenpanne.
Reviewers: alexfh, JonasToth
Reviewed By: JonasToth
Subscribers: JonasToth, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D61850
llvm-svn: 360883
Summary: When run-clang-tidy is given the -quiet flag, do not output the potentially hundreds of enabled check names at the beginning.
Patch by svenpanne.
Reviewers: alexfh, JonasToth
Reviewed By: JonasToth
Subscribers: JonasToth, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D61849
llvm-svn: 360869
Summary:
At Mozilla we are using this tool in order to perform review-time static-analysis, since some patches contain a large number of files we've discovered this issue, where `stderr` gets mixed with `stdout` thus obfuscating our possibility to parse the output.
The patch that we are currently use can be found [here](https://searchfox.org/mozilla-central/source/build/build-clang/clang-tidy-8.patch).
This is just an upstream of the original patch.
Reviewers: JonasToth
Reviewed By: JonasToth
Subscribers: cfe-commits
Tags: #clang, #clang-tools-extra
Differential Revision: https://reviews.llvm.org/D60453
llvm-svn: 357994
The Yaml module is missing on some systems and on many of clang buildbots.
But the test for run-clang-tidy.py doesn't fail due to 'NOT' statement masking a python runtime error.
This patch conditionally imports and enables the yaml module only if it's present in the system.
If not, then '-export-fixes' is disabled.
Differential Revision: https://reviews.llvm.org/D59734
llvm-svn: 357114
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
Exit with a non-zero value in case any of the underlying clang-tidy
invocations exit with a non-zero value.
This is useful in case WarningsAsErrors is enabled for some of the
checks: if any of those checks find something, the exit status now
reflects that.
Also add the ability to use run-clang-tidy.py via lit, and assert that
the exit code is not 0 when modernize-use-auto is triggered
intentionally.
Reviewers: alexfh, aaron.ballman
Reviewed By: aaron.ballman
Differential Revision: https://reviews.llvm.org/D44366
llvm-svn: 327854
Updating the run-clang-tidy.py script to allow specification of the
config argument to the clang-tidy invocation.
Differential Revision: https://reviews.llvm.org/D43538
llvm-svn: 327186
Unfortunately, these python scripts are not tested currently. I did the testing
manually on LLVM by editing the CMake generated compilation database to
contain relative paths for some of the files.
Differential Revision: https://reviews.llvm.org/D39603
llvm-svn: 317468
This patch adds the option to keep the list of proposed fixes even though they
should not be applied.
This allows to detect possible fixes using the parallelised run-clang-tidy.py
and apply them using clang-apply-replacements at a later time.
Essentially the patch causes the individual temporary yaml files by the parallel
clang-tidy instances to be merged into one user-defined file.
Patch by Michael F. Herbst!
Differential revision: https://reviews.llvm.org/D31326
llvm-svn: 308726
This adds a new flag -style which is passed to clang-apply-replacements and
defaults to file meaning it would pick up the closest .clang-format file in tree.
llvm-svn: 305125
Summary:
When running run-clang-tidy.py with -fix it tries to apply found replacements at the end.
If there are errors running clang-apply-replacements, the script currently crashes or displays no error at all.
This patch checks for errors running clang-apply-replacements the same way clang-tidy binary is handled.
Another option would be probably checking for clang-apply-replacements (when -fix is passed) even before running clang-tidy.
Reviewers: Prazek, alexfh, bkramer, mfherbst
Reviewed By: Prazek, alexfh
Subscribers: kimgr, cfe-commits
Tags: #clang-tools-extra
Differential Revision: https://reviews.llvm.org/D32294
llvm-svn: 301365
