Remediate findings
Overview
Section titled “Overview”Remediation turns a supported finding into a proposed response. When the Instruction Hub owns the cause, a remediation agent can prepare a focused pull request. Your team decides whether to merge it under the repository’s review and CI rules. This guide covers reviewing that pull request, publishing the fix, and measuring what it recovered.
For Acme, a missing product-version check might need a small change to the documentation skill. An expired deployment credential belongs to an infrastructure owner, so it stays out of the skill.
Before you start
Section titled “Before you start”You need:
- A high-confidence finding.
- A connected GitHub Instruction Hub. Remediation works only with GitHub repositories. Publishing plugins from GitLab is a separate toolchain capability.
- Propose Remediations enabled in PIG Settings for the finding’s destination repository, which is the instruction repository chosen for its GitHub issue.
- Working model access for the analyzer, and the repository connection Promptless needs to issue remediation credentials.
- Someone authorized to review proposed changes.
How remediation chooses a response
Section titled “How remediation chooses a response”The remediation agent reads the finding and its evidence, then inspects the relevant hub assets in an isolated checkout. It determines whether the hub can address the supported cause.
| Outcome | What happens next |
|---|---|
| A narrow hub bug has one clear correction | The agent prepares the focused change and checks it. |
| The hub can address the problem, but the choice changes policy or capability | The workflow can ask for a decision in Slack before editing. Review the options and their tradeoffs. |
| A tool, permission, credential, or instruction outside the hub owns the cause | The agent updates the finding notes with the external input and responsible owner. It does not invent a hub fix. |
| The cause remains uncertain | The outcome explains the missing evidence and no pull request opens. |
The agent receives a repository-scoped GitHub credential for the remediation task and uses the configured model. Isolation separates the checkout and the task’s execution. The task still uses model credentials and can read session evidence.
Review a proposed change
Section titled “Review a proposed change”For a hub change, the agent works on a dedicated branch and runs relevant checks. It then opens a pull request against main in the finding’s destination repository. The workflow records the pull request against the finding. Follow repeated evidence and retried attempts through that recorded history, as part of the same fix.
Review the pull request in this order:
-
Match the change to the finding. Confirm the proposed behavior addresses the observed failure and stays within the hub’s ownership.
-
Check the scope. Prefer a focused correction, simplification, or removal over an unrelated expansion of instructions.
-
Exercise a representative task. For Acme’s documentation example, try both the affected product version and a version that previously worked.
-
Review validation results. Require the hub’s validation and build checks, plus any repository-specific CI or evaluation you maintain.
-
Approve and merge through your normal controls. CODEOWNERS, required reviews, and branch protection remain yours to configure.
A recorded pull request proves that a proposal exists. It doesn’t prove that the change fixes every affected session. PIG has no automatic behavior evaluation and doesn’t merge the proposal for you. Your team’s review and checks decide whether it merges.
Publish and verify adoption
Section titled “Publish and verify adoption”After merge, use the hub’s publish workflow to release the updated plugin artifacts. A merged source change and a published release are separate checkpoints. Confirm that the release completed before you ask users to update.
Refresh the plugin through the supported agent’s installation or update flow. Confirm the expected plugin version on a pilot host, then run a new representative session. For enrolled hosts, check collection and subsequent analysis as well.
Adoption across the team depends on hosts refreshing their installed plugins. A successful release doesn’t mean every machine has the fix. Preserve the finding and release links so reviewers can relate later evidence to the version users actually ran.
Measure what a fix recovered
Section titled “Measure what a fix recovered”Use the Instruction Hub in the Promptless Dashboard to follow proposed fixes and check whether a merged fix reduced wasted agent time and tokens. Two pages in its sidebar cover this:
- Fixes lists every proposed fix and its pull request, whatever its status.
- Impact lists each fix whose pull request has merged, newest first. Each row shows the pull request, the Instruction Hub, and when it merged. Select a fix’s title to open it on the Fixes page.
PIG compares the 30 days before a fix merged with the 30 days after. When both windows have closed and each contains analyzed sessions, the fix shows the agent time and tokens it recovered per analyzed session. While the post-merge window is open, the fix is marked as collecting data. If either window closed without analyzed sessions, the fix is marked as needing more data.
The windows group sessions by when the analyzer observed them. Sessions from hosts that haven’t installed the updated plugin yet count toward the post-merge window, so slow adoption lowers the measured recovery.
Troubleshooting
Section titled “Troubleshooting”| Situation | Next action |
|---|---|
| The finding is high confidence but has no pull request | Read its outcome and notes. It may need an external owner, a decision, or more ownership evidence. |
| The workflow is waiting for a decision | Have the appropriate owner choose the intended behavior. Approve a policy change only when the owner wants it. |
| Repository access fails | Ask the operator to check the hosted repository connection and remediation credential path. The analyzer’s clone token is a separate credential. |
| CI rejects the proposal | Review the failure and update the branch through the normal repository workflow. Do not bypass required checks. |
| The source change merged but users see old behavior | Confirm publication, the installed plugin version, and whether a conflicting local instruction still applies. |
| The problem recurs after update | Compare new evidence with the release and installed version before concluding the fix failed or opening a duplicate change. |
See also
Section titled “See also”- Understand findings, to read a finding’s evidence and decide who owns it
- Publish and install plugins, to release the merged fix and refresh installed plugins
- Observability and troubleshooting, to monitor continued analysis