Skip to content

For the complete documentation index, see llms.txt.

Understand findings

A finding describes an instruction problem supported by evidence from real agent sessions. It helps your team decide what to change, why the change matters, and who should own it.

For example, Acme’s documentation skill might tell an agent to verify every example but omit how to choose the correct product version. A finding can connect repeated incorrect examples to that missing guidance. A failed API request caused by an expired credential may instead belong to the team that manages the credential.

Findings appear after an operator has verified completed analysis. To work with one, you need access to your connected Instruction Hub. You also need the finding’s GitHub issue or its page in the Promptless Dashboard.

  • A finding is high severity but low confidence, and you must decide whether to act now. Read the evidence before choosing a fix explains what each field measures.
  • A finding waits for a repository before Promptless opens its GitHub issue. Follow the finding into your repository explains how Promptless picks a destination and when an administrator chooses one.
  • A finding points at a failure the hub doesn’t own, such as an expired credential. Decide the next action helps you route it to the right owner.
  • An analysis run completes with zero findings. That is a valid result, and Troubleshooting lists what to confirm.

Start with the summary, impact, and cited occurrences. Check that the described behavior happened and that the instruction failure explains it. A finding carries these fields:

FieldWhat it tells youHow to use it
Severity: low, medium, high, criticalHow serious the observed problem is.Prioritize the effect on users and your systems.
Confidence: low or highHow strongly the evidence supports the finding.Separate a suspected problem from one supported well enough for remediation.
Instruction failureThe gap, conflict, or misleading guidance behind the behavior.Locate the instruction that needs attention, or question whether the hub owns the cause.
EvidenceCited occurrences and a summary of what happened.Compare the claim with the session details.
NotesAdditional context, uncertainty, or an external owner.Check whether action belongs outside the hub.

Severity and confidence answer different questions. A high-severity finding can still have low confidence and need more evidence. Later analysis can add occurrences to an existing finding and promote its confidence when the evidence supports that change.

The analyzer cites evidence from a readable session digest. A digest_range identifies a location in that digest. It is unrelated to the cryptographic hashes that verify stored traces.

PIG analyzes every instruction repository your organization selects in PIG Settings. Each finding records the instruction source it came from: the repository and the exact commit the analyzer read. The finding lists these sources under Instruction sources, each linked to that commit.

The analyzer sends finding updates to Promptless. Promptless creates or updates a GitHub issue for the finding in its destination repository. The issue carries the finding’s title, impact, instruction failure, severity, confidence, and evidence. A destination repository must have Create Repo Issues enabled in PIG Settings.

Promptless creates the issue automatically when the finding’s recorded sources resolve to one selected repository that has Create Repo Issues enabled. A finding needs a destination first in either of these cases:

  • Its sources span more than one selected repository.
  • It records no source, and your organization has more than one repository selected.

Removing an instruction repository in PIG Settings stops future analysis and issue automation for that source. Findings already recorded against it keep their instruction sources, issues, and pull requests.

As the analyzer processes more sessions, recurring evidence can update an existing finding and its issue. Use the finding identity and the linked issue to follow that history. Creating the issue is a separate step from analysis, so a completed run alone doesn’t prove that GitHub access works.

Finding text and evidence summaries can include session details. Make sure the destination repository’s access matches your intended audience, as described in Trust and data model.

An organization administrator sets the destination once per finding, while the finding is open:

  1. On the Findings page of the Promptless Dashboard, open the finding.

  2. Select Instruction sources and fix destination.

  3. Select a repository from the list. The list shows only repositories that have Create Repo Issues enabled.

  4. Select Use repository.

Promptless then creates the issue, and any proposed fix, in that repository.

  1. Confirm the observed problem. Read the impact and evidence, including enough surrounding context to avoid correcting intentional behavior.

  2. Identify the owner. Shared hub instructions, a repository-local instruction, a tool bug, and a missing permission have different owners.

  3. Check for an existing response. Read the finding notes, linked pull request, and current hub instructions before proposing a duplicate fix.

  4. Review the remediation outcome. High-confidence findings can enter the remediation workflow. That workflow may propose a hub change, ask for a policy decision, or identify an external action.

The expected outcome is a clear owner and a justified next action. Not every finding needs a new instruction or a pull request.

SituationWhat to check
The run succeeded, but no finding appearedZero findings is valid. Confirm the run belongs to the session you intended to analyze.
A local finding has no GitHub issueAsk the operator to check hosted synchronization and the repository connection.
The finding asks you to select a repository for its GitHub issueIts recorded sources map to more than one selected repository, or to none. An organization administrator opens the finding and, under Instruction sources, selects a repository that has Create Repo Issues enabled.
Evidence seems inconsistent with the current hubCompare the analyzed repository revision with the current instructions and the session’s installed plugin version.
No remediation pull request openedCheck confidence, ownership notes, pending decisions, repository access, and the remediation outcome.
Similar issues appear to overlapCompare their failure mechanism and evidence before treating them as duplicates; different symptoms can have one cause, and similar symptoms can have different causes.