Add "View Autofixes" feature for variant analysis results by jcogs33 · Pull Request #4065 · github/vscode-codeql · GitHub | Latest TMZ Celebrity News & Gossip | Watch TMZ Live
Skip to content

Add "View Autofixes" feature for variant analysis results #4065

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 38 commits into
base: main
Choose a base branch
from

Conversation

jcogs33
Copy link

@jcogs33 jcogs33 commented Jun 29, 2025

Description

Adds a "View Autofixes" feature for variant analysis results. This is a canary-only feature.
Generates autofixes for selected variant analysis results and opens the resulting autofix outputs in a markdown file.
Activated via the "View Autofixes" button in the variant analysis webview or via right-clicking the variant analysis query history item.

AutofixInMrvaScreenshot

At a high-level, the implementation:
  • Checks for a local checkout of autofix
  • Sets up to run autofix by:
    • Overriding the relevant query help
    • Getting the relevant SARIF(s)
    • Downloading source root(s)
  • Runs autofix locally
  • Outputs autofix results in a markdown file

Commit-by-commit review recommended. Let me know if you want me to split anything onto a separate PR.

Consideration

  • See in-line comments
  • Change from my original implementation: Instead of downloading databases and then extracting the source root from the database, I directly download the source root since it seemed a bit faster. Let me know if you think the database approach is better.
  • I will make a follow-up PR to convert this to support Go autofix instead of cocofix, and I'll update the queryHelpOverrideDirectory to the new go/pkg/autofix/prompt/qhelps directory in that PR.

Testing

  • I've manually tested this feature but have not added unit/view/integration tests since it seems another canary feature did not add tests. Let me know if I should add tests after all.
  • Note: There is a known issue with the E2E CI test failing. It is not caused by changes in this PR.

@jcogs33 jcogs33 force-pushed the jcogs33/view-autofixes branch 3 times, most recently from 8095989 to c5f0e5e Compare June 30, 2025 21:29
@jcogs33 jcogs33 force-pushed the jcogs33/view-autofixes branch from c5f0e5e to 3ff5d64 Compare July 2, 2025 00:44
@jcogs33 jcogs33 marked this pull request as ready for review July 2, 2025 02:21
@jcogs33 jcogs33 requested review from a team as code owners July 2, 2025 02:21
Copy link
Member

@koesie10 koesie10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I've not tested this locally, but the demo you shared looks good!

Comment on lines +650 to +653
env: {
CAPI_DEV_KEY: process.env.CAPI_DEV_KEY,
PATH: process.env.PATH,
},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to allow specifying the environment variables in the config (in something like codeQL.autofix.env)?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I'm understanding how to specify these in the config. i.e. Are you suggesting specifying the API key for CAPI_DEV_KEY directly in the settings.json file (is that safe)? Or specifying the path to a local file where the API key is stored? Or something else?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we want to add the secret as a config setting. There are ways of adding secrets securely, but I think just keeping it as an env var is fine. Much simpler. Just please be sure to document.

There are some complications if you are using a codespace. I'm not sure how it would work. Can you inject env vars into it?

As for the PATH variable. I'm not sure if it makes sense either.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that we probably don't want to add the secret in the config file, but allowing you to set custom environment variables might still be useful, for example an ENVIRONMENT environment variable. We could also just pass through all of process.env to set custom environment variables though.

I'm not sure exactly how VS Code picks up environment variables, but it seems like it's possible to specify secrets when using Codespaces: Using secrets.

`Invalid empty or undefined arguments: ${args.join(" ")}`,
);
}
const p = spawn(bin, args, { stdio: [0, 1, 2], ...options });
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think stdio: [0, 1, 2] will ensure that the autofix CLI logs to the stdout/stderr of VS Code, but I don't think you can view those. I think it would make sense to log those to the extension logs. A good example of this can be found in runCodeQlCliInNewProcess, specifically in handleProcessOutput.

Copy link
Contributor

@aeisenberg aeisenberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I haven't tried this locally.


// Extract the tarball
await new Promise<void>((resolve, reject) => {
const process = spawn("tar", ["-xzf", tarballPath], { cwd: downloadDir });
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're using tar-stream elsewhere for extracting tarballs. It would be better to use something like that instead of spawning a new process.

Just be aware that tar-stream is currently a dev dependency and would need to be moved to a dependency.

@jcogs33
Copy link
Author

jcogs33 commented Jul 3, 2025

Thanks for the reviews @koesie10 and @aeisenberg! 🙇
I've addressed some of your comments. I'll re-request your reviews once I'm finished with them all.

Copy link
Contributor

@aeisenberg aeisenberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's one more comment from @koesie10 that should be addressed, but after that I think it's good to go.

@jcogs33
Copy link
Author

jcogs33 commented Jul 4, 2025

There's one more comment from @koesie10 that should be addressed

@aeisenberg Which comment are you referring to? There were four things I still was planning to address: three comments related to editing parts of downloadPublicCommitSource, and one comment about getting the autofix CLI logs into the extension logs.

@aeisenberg
Copy link
Contributor

Oh...there might be more things that I missed. I'm out tomorrow. I think it's also a holiday in the US, so I can take another look on Monday.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants

TMZ Celebrity News – Breaking Stories, Videos & Gossip

Looking for the latest TMZ celebrity news? You've come to the right place. From shocking Hollywood scandals to exclusive videos, TMZ delivers it all in real time.

Whether it’s a red carpet slip-up, a viral paparazzi moment, or a legal drama involving your favorite stars, TMZ news is always first to break the story. Stay in the loop with daily updates, insider tips, and jaw-dropping photos.

🎥 Watch TMZ Live

TMZ Live brings you daily celebrity news and interviews straight from the TMZ newsroom. Don’t miss a beat—watch now and see what’s trending in Hollywood.