Introduce support for agentic autofix

[mbaluda]

Description

This pull request introduces comprehensive repository-wide instructions for GitHub Copilot, specifically focused on configuring Copilot's agentic autofix feature for CodeQL Coding Standards compliance. It also updates the user manual to reference these new Copilot instructions as part of the certified use cases.

Key changes:

Copilot configuration and guidance:

• Added a new docs/copilot-instructions.md file with detailed guidelines for Copilot's agentic autofix.

Documentation updates:

• Updated docs/user_manual.md to include "Using Agentic Autofix in Code Scanning" and to reference the new copilot-instructions.md file for configuration.

Change request type

• Release or process automation (GitHub workflows, internal scripts)
• Internal documentation
• External documentation
• Query files (.ql, .qll, .qls or unit tests)
• External scripts (analysis report or other code shipped as part of a release)

Rules with added or modified queries

• No rules added
• Queries have been added for the following rules:
  • rule number here
• Queries have been modified for the following rules:
  • rule number here

Release change checklist

A change note (development_handbook.md#change-notes) is required for any pull request which modifies:

• The structure or layout of the release artifacts.
• The evaluation performance (memory, execution time) of an existing query.
• The results of an existing query in any circumstance.

If you are only adding new rule queries, a change note is not required.

Author: Is a change note required?

• Yes
• No

🚨🚨🚨
Reviewer: Confirm that format of shared queries (not the .qll file, the
.ql file that imports it) is valid by running them within VS Code.

• Confirmed

Reviewer: Confirm that either a change note is not required or the change note is required and has been added.

• Confirmed

Query development review checklist

For PRs that add new queries or modify existing queries, the following checklist should be completed by both the author and reviewer:

Author

• Have all the relevant rule package description files been checked in?
• Have you verified that the metadata properties of each new query is set appropriately?
• Do all the unit tests contain both "COMPLIANT" and "NON_COMPLIANT" cases?
• Are the alert messages properly formatted and consistent with the style guide?
• Have you run the queries on OpenPilot and verified that the performance and results are acceptable?
  As a rule of thumb, predicates specific to the query should take no more than 1 minute, and for simple queries be under 10 seconds. If this is not the case, this should be highlighted and agreed in the code review process.
• Does the query have an appropriate level of in-query comments/documentation?
• Have you considered/identified possible edge cases?
• Does the query not reinvent features in the standard library?
• Can the query be simplified further (not golfed!)

Reviewer

• Have all the relevant rule package description files been checked in?
• Have you verified that the metadata properties of each new query is set appropriately?
• Do all the unit tests contain both "COMPLIANT" and "NON_COMPLIANT" cases?
• Are the alert messages properly formatted and consistent with the style guide?
• Have you run the queries on OpenPilot and verified that the performance and results are acceptable?
  As a rule of thumb, predicates specific to the query should take no more than 1 minute, and for simple queries be under 10 seconds. If this is not the case, this should be highlighted and agreed in the code review process.
• Does the query have an appropriate level of in-query comments/documentation?
• Have you considered/identified possible edge cases?
• Does the query not reinvent features in the standard library?
• Can the query be simplified further (not golfed!)

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[MichaelRFairhurst]

Looking great!! A few thoughts!

[MichaelRFairhurst]

I might rework this section a bit.

In my use of deviations on qlx, I've found that many of the rules do really need to be deviated for certain behaviors. For instance, MISRA disallows recursion but qlx has to be recursive since it's a compiler of sorts. So I think copilot should make a best guess at whether the fix should be deviated based on context and existing deviations.

So I might suggest copilot do something more like:

• Locate the deviations file if you can
• Look for existing deviations of that rule, and see if any should apply
• Consider whether an existing code identifier should be used in this case
• Consider whether a file-wide exception should be used in this case
• Consider whether a new code identifier should be used in this case
• If using a code-identifier, look for examples to determine whether to use [[attribute]] form
• If using an [[attribute]] look for project formatting configurations or code examples to determine how to format the new attribute relative to its declaration
• When using deviation comments, consider project formatting, the specific violation in question, and other example deviation comments in the project to determine whether to use same-line, next-line, or begin/end comment deviations
• Project formatting config may be .clang-format, etc

[mbaluda]

sorry this PR went to review too early... I'll take your comments into account as soon as this is testable

[MichaelRFairhurst]

I might emphasize and/or expand this further.

In conversation with Vincent, he expressed a lot of skepticism about autofix. In some of the examples we saw from other similar projects, their autofix often did dangerous things to suppress the warning and/or introduce UB.

So maybe this should be something more like:

• Preserve safe and desired functional behavior: ensure the resulting code handles all reasonable real world scenarios as the code originally intended. This may involve precisely maintaining the existing code behavior, or it may involve fixing subtle or rare bugs, for instance dangerous overflows, that the existing code does not handle and the rule is designed to detect.
• Fix dangerous bugs: If the alert is flagging unsafe or undefined behavior, critically examine how that safety issue in the code should be properly fixed. Add detections and error handling if necessary to make the code safe under all conditions without introducing unnecessary complexity. Follow existing project guidelines on how to handle rare, dangerous, or unexpected scenarios that occur at runtime.
• Thoroughly explain analysis and functional changes: If the alert does not introduce any unwanted behavior and the change is functionally equivalent, explain your thinking, and clearly show that the code before was safe, and that the new code is exactly equivalent in behavior. If there was a dangerous edge case or condition, explain exactly how that scenario would create problems in the code and how the fix will prevent such issues and improves the safety and quality of the codebase.

[MichaelRFairhurst]

We should give instructions here and in the user_manual that tells the user where to put this file.

Additionally, I'm not sure that we want to make this part of instructions. I think we may want to make this an AGENT.md, or a skill.

Typically copilot instructions will be loaded into the context in full for every prompt. The instructions here should probably only be loaded when necessary. We may have to sync with folks internally about whether we can use an agent file, and how to make that the agent used by autofix. That might not be possible. If so we hopefully can use skill effectively.

If we end up making this a skill, it might be worth making a copilot-instructions.md that is very short, that ensures the skill is loaded. "When fixing alerts, use the skill [...]"

[MichaelRFairhurst]

I'd provide more detailed instructions here for how to install into a project.

And we also need to update the user_manual version with author and date etc