This gets us the full list of users with commit access to
llvm/llvm-project rather than the list of people in the llvm-committers
team. This will ensure we are able everyone to track everyone with
commit access in case they receive access some other way.
There is no need to support Python 2.7 anymore, Python 3.3+ has
`subprocess.DEVNULL`. This is good practice and also prevents file
handles from
staying open unnecessarily.
Also remove a couple unused or unneeded `__future__` imports.
Followup to #90109.
In Microsoft, our automated scans are warning that LLVM has vulnerable
dependencies. Specifically:
* [CVE-2024-35195](https://nvd.nist.gov/vuln/detail/CVE-2024-35195) was
fixed in `requests` 2.32.0.
* [CVE-2024-37891](https://nvd.nist.gov/vuln/detail/CVE-2024-37891) was
fixed in `urllib3` 2.2.2.
I've updated LLVM's dependencies by running the following commands in
`llvm/utils/git`:
```
pip-compile --upgrade --generate-hashes --output-file=requirements.txt requirements.txt.in
pip-compile --upgrade --generate-hashes --output-file=requirements_formatting.txt requirements_formatting.txt.in
```
Note that for `requirements_formatting.txt` this adds
`--generate-hashes` (according to my vague understanding, it's highly
desirable and was already used for `requirements.txt`) and was locally
run within `llvm/utils/git` (changing the recorded command, which
apparently was originally run from the repo root - again,
`requirements.txt` was already being regenerated with a locally run
command, so this increases consistency).
I observe that this has updated the relevant components to pick up the
CVE fixes. Note that I am largely clueless in this area, so I hope that
(like #90109) no other changes will be necessary.
This script linkifies (i.e. makes clickable in the terminal) text that appears
to be a pull request or issue reference (e.g. #12345 or PR12345) or a
40-character commit hash (e.g. abc123). You can configure git to automatically
send the output of commands that pipe their output through a pager, such as
`git log` and `git show`, through this script by running this command from
within your LLVM checkout:
git config core.pager 'llvm/utils/git/linkify | pager'
The pager command is run from the root of the repository even if the git
command is run from a subdirectory, so the relative path should always work.
It requires OSC 8 support in the terminal. For a list of compatible terminals,
see https://github.com/Alhadis/OSC8-Adoption
Reviewers: MaskRay
Reviewed By: MaskRay
Pull Request: https://github.com/llvm/llvm-project/pull/103496
This patch ensures that the CI script controls which file extensions are
considered instead of letting git-clang-format apply its own filtering
rules. In particular, this properly handles libc++ extension-less
headers which were passed to git-clang-format, but then dropped by
the tool as having an unrecognized extension.
This is a second attempt to land 7620fe0d2d1e, which was reverted in
9572388 because it caused formatting not to be enforced for several patches.
The problem was that we'd incorrectly pass the extensions with additional
quoting to git-clang-format. The incorrect quoting has been removed in this
version of the patch.
This reverts commit 7620fe0d2d1e0257611c0ab0d96f3bf1bf7a1079.
This patch was causing some significant portion of code formatting jobs
to succeed when they should have failed. More investigation is needed.
Tracked in #97060.
This ensures that the CI script controls which file extensions are
considered instead of letting git-clang-format apply its own filtering
rules. In particular, this properly handles libc++ extension-less
headers which were passed to git-clang-format, but then dropped by that
tool as having an unrecognized extension.
Hi! Here is a patch for #81859 that fix the vulnerabilities found in
gitpython, cryptography, urllib3 and requests.
I have just regenerated the requirements.txt files running pip-compile
again. Fortunately, this was enough to set all the dependencies on safe
versions.
I have also checked if new vulnerabilities were introduced by running
scorecard on my fork, but none has been introduced.
Thanks!
Signed-off-by: Joyce Brum <joycebrum@google.com>
Actions triggered by pull_request_target events have access to all
repository secrets, so it is unsafe to use them when executing untrusted
code. The pr-code-format workflow does not execute any untrusted code,
but it passes untrused input into clang-format. An attacker could use
this to exploit a flaw in clang-format and potentially gain access to
the repository secrets.
By splitting the workflow, we can use the pull_request target which is
more secure and isolate the issue write permissions in a separate job.
The pull_request target also makes it easier to test changes to the
code-format-helepr.py script, because the version of the script from the
pull request will be used rather than the version of the script from
main.
Fixes#77142
The answer to many requests in issues to be assigned to users is often
"just create a pull request". That's in contradiction to the
"introductory issue" instructions posted by the GitHub bot.
This change updates the instructions, mentioning the shortcut of "just
creating a PR". Moreover, it now explains linking PRs to issues in order
to close them automatically upon merge.
This makes it possible to rebase the branch using the Web UI, which
makes it easier to manually merge the PRs. Manual merge is required when
squash merge won't preserve author information of the backport.
Follow-up from PR #83491. `Darker`'s configuration is ignored because of the way we invoke it - with an explicit list of files. We need to filter it in `code-format-helper.py`.
When a backport request is made, the resulting pull request will have a
title like this:
<release branch>: <First line of HEAD commit for the branch>
And a body that says:
Backport <commit0> <commit1> ..
Requested By: <user>
This will allow us to track the state of the backport request in the PR,
rather than in the issue. The state updates for PRs can be automated, so
this will save us some triage work.
This reverts commit bc06cd5cbcfc22dd976f6742d10bc934e1353b8a.
This caused the job to fail for PRs which still had an older version
of code-format-helper.py in their tree.
Actions triggered by pull_request_target events have access to all
repository secrets, so it is unsafe to use them when executing untrusted
code. The pr-code-format workflow does not execute any untrusted code,
but it passes untrused input into clang-format. An attacker could use
this to exploit a flaw in clang-format and potentially gain access to
the repository secrets.
By splitting the workflow, we can use the pull_request target which is
more secure and isolate the issue write permissions in a separate job.
The pull_request target also makes it easier to test changes to the
code-format-helepr.py script, because the version of the script from the
pull request will be used rather than the version of the script from
main.
Fixes#77142
This change adds a comment to the first PR from a new contributor that
is merged, which tells them what to expect post merge from the build
bots.
How they will be notified, where to ask questions, that you're more
likely to be reverted than in other projects, etc. The information
overlaps with, and links to
https://llvm.org/docs/MyFirstTypoFix.html#myfirsttypofix-issues-after-landing-your-pr.
So that users who simply read the email are still aware, and know where
to follow up if they do get reports.
To do this, I have added a hidden HTML comment to the new contributor
greeting comment. This workflow will look for that to tell if the author
of the PR was a new contributor at the time they opened the merge. It
has to be done this way because as soon as the PR is merged, they are by
GitHub's definition no longer a new contributor and I suspect that their
author association will be "contributor" instead.
I cannot 100% confirm that without a whole lot of effort and probably
breaking GitHub's terms of service, but it's fairly cheap to work around
anyway. It seems rare / almost impossible to reopen a PR in llvm at
least, but in case it does happen the buildbot info comment has its own
hidden HTML comment. If we find this we will not post another copy of
the same information.
We used to support a /branch comment to specify a branch with commits to
backport to the release branch. However, now that we can use pull
requests this is not needed.
This also simplifies the process, because now the cherry-pick job can
create the pull request directly instead of having it split across two
separate jobs.
This is currently broken, because the check is performed on the wrong
repository. repo here is llvm/llvm-project, which is not a fork (so this
will always trigger), then we'll push a new branch to
llvmbot/llvm-project, and then again set the wrong owner, so we'll look
for the branch in llvm/llvm-project rather than llvmbot/llvm-project.
Rather than fixing this, I'm removing the code entirely, as it shouldn't
be needed anymore (llvmbot/llvm-project is a fork of llvm/llvm-project).
Before this patch, there was a regression in comment formatting due to
some code formatting in bd3e8eb6e325081bf7cfbe93652aa825de3170e5. This
was fixed in 428660cfb986dd0a59cd2a16972c5f7109080522. Github interprets
a tab before a string as starting code formatting. The message that
indicted the code formatting in a PR had been fixed was refactored to a
python multi-line string, but with a tab in front, causing these
messages to be rendered as code blocks in Github, instead of as
intended. This patch builds upon the original fix to reformat the
strings so that they fit within ~80 character lines and are simpler to
modify in the future, hopefully removing traps like the one that caused
the original issue.
Due to the way the f string was written, the text ended up with 4 spaces
at the start. 4 space indent in Markdown means plain text, which is not
what we intend here.
As part of #73798 there was some discussion about using the format
helper to run from a git-hook. That was not possible for a number of
reasons, but with these changes it can now be installed as a hook and
then run on the local cache in git instead of a diff between revisions.
This also checks for two environment variables DARKER_FORMAT_PATH and
CLANG_FORMAT_PATH where you can specify the path to the program you want
to use.
Using a two dot diff allows changes made in main after the merge base to
show up in the formatting diff. Using a three dot diff fixes this and
ensures that only changes made in the source branch (branch from the PR
author) will get passed along to the formatter.
Without this, issues like #73873 occur.
This includes some commonly needed information like how to add
reviewers.
This is implemented as a job before the labeler, so that on a new PR the
comment is added before there are any subscribers and only the author
gets a nofitication.
The labeler job depends on the greeter having run or having been
skipped. So if the PR wasn't just opened, or it's from a regular
contributor, the labeling still happens.
But we can be sure that when a greeting comment is left, it's the very
first thing we do.
The ignore_format.txt file and the associated checks have been causing a
lot of confusion since we introduced them. Formatting becomes one of the
main hurdle for contributors (especially new contributors), and that is
not great.
The original goal of ignore_format.txt was to enforce clang-format only
in a subset of the files of the project. In practice, we have now
shifted to a model where we have a Github action that checks whether new
code surrounding edits is formatted. In that context, it probably
doesn't make sense to keep having a ignore list for formatting files.
After this patch, the clang-format job will enforce that all new code is
formatted properly, and that all edits to existing files are formatted
properly, regardless of which files the edits are in. This seems
reasonable and I believe will lead to much less confusion than our
current setup.
In the future, we could consider clang-formatting the whole code base
once and for all but this requires a bit of upfront technical work to
put in place a merge driver to help resolve merge conflicts across
formatting changes.
These headers were skipped by the job because they didn't have an
extension. However, such headers are extremely common in libc++.
As a drive-by change, also include `.cppm` and `.inc` extensions since
those are also common in libc++.
Currently, when the code format action fails, it leaves no log of the
diff in the output within the action itself. This has caused confusion
for some users of the action, especially when the comment becomes buried
in a 100+ comment review and someone isn't super familiar with the inner
workings of the CI. This patch prints the diff produced by the code
formatter to stdout so that it is viewable by clicking on the failed
action. This should have very little cost and make things slightly less
confusing for those that run into this situation.
Before this patch, the github automation script would fail when trying
to escape the text of a PR/issue description that was empty as the
Github library returns None instead of an empty string in those
scenarios. This patch special cases this and makes the escape function
return an empty string when given a value of None.
* Consider the darker/clang-format command to have failed if the exit
code is non-zero, regardless of the stdout/stderr output.
* Propagate stderr from the formatter command to the script's
caller (and into the GitHub log).
* On success, dump stdout to the caller, so it ends up in GitHub's log.
I'm not sure what this would ever be, but if it exists, it should be
preserved.
* Just before the script exits, if any formatter failed, print a line
showing which formatters failed.
Fix type errors that mypy reports with code-format-helper.py.
Add a few return type annotations and change `param: [str]` to
`param: list[str]`.
Leave a few required FormatHelper members missing instead of defining a
placeholder:
- FormatHelper.name
- FormatHelper.friendly_name
- FormatHelper.format_run: NotImplementedError() instead of `pass`
Currently the email that gets sent out to people subscribing to a label
that the bot tags on the PR doesn't include any authorship information
which some people are interested in having. This patch adds an author
field to the message with the relevant information.
I landed this format helper, but unfortunately, it didn't work because
of permissions, it could not add comments on a fork's PR. @cor3ntin
informed me there are fixes for this that you had worked on @tstellar -
but I didn't have time to read up on it too much. Can you explain what
changes are needed to get the action to be able to write comments on
fork's PR?
