Skip to content

For the complete documentation index, see llms.txt.

Measure the impact of Promptless

Start with the Impact page in Settings. It charts the suggestions Promptless created and the suggestions your team merged or closed, week by week. Then add numbers from the Published page, your Git history, and your support and analytics tools, and combine them in a report.

If you maintain an open-source project, use the same report for a foundation, a sponsor, or your community.

  1. Open Settings, then select Impact under Account.

  2. Set Range to 4w, 8w, or 12w for the last 4, 8, or 12 weeks.

  3. To report on one doc collection, select it in the Collection filter. The filter appears when your organization has more than one doc collection.

  4. Read the chart. Each week shows the suggestions Created, Merged, and Closed, and the Backlog of open suggestions that remains.

Impact page over 12 weeks in the Waterfall view, where each week's Created, Merged, and Closed bars lead to the Backlog still open, with the priority pills and legend below the chart

Select a priority below the chart to highlight its suggestions. Set View to Stacked to compare the suggestions created each week by priority.

  • Published page. The suggestions merged in the last 30 days, with who merged each one and its doc collection. Select Published in the sidebar.
  • Suggestions list. Every suggestion Promptless created, sorted into Ready, PR Open, and Closed, each with an estimated review time. Filter by Collection or Trigger source.
  • Triggers page. Every event Promptless processed. The Suggestion Activity filter separates events that produced a suggestion from events that didn’t. The From and To fields set the date range, and the summary line shows the number of matching triggers.
  • Git history. Merged docs PRs, with author, reviewers, and timestamps. On GitHub, Promptless opens PRs as promptless[bot]. On GitLab, find the account name on any merge request Promptless opened.

Track only the metrics your leadership asks about.

  • Backlog trend. On the Impact page, compare the backlog in the first and last week of the period. A flat or falling backlog means your team reviews suggestions as fast as Promptless creates them.
  • Doc updates shipped. Count the merged PRs that Promptless authored in the period. For the last 30 days, count the suggestions on the Published page.
  • Share of doc changes automated. Divide the merged Promptless PRs by all merged docs PRs in the same repo and period.
  • Coverage of product changes. On the Triggers page, set From and To. Set Suggestion Activity to Suggestion Activity and record the count in the summary line. Repeat with No Suggestion Activity. The first count is the events that led to a docs change. The second is the events Promptless checked and skipped.
  • Review load. Count the suggestions in Ready, PR Open, and Closed, and add up their estimated review times.
  • Time saved. Multiply doc updates shipped by your estimate of the hours a manual update takes. State the estimate in the report.

Run these commands in your docs repo. Replace the dates with your period.

Count the merged PRs that Promptless opened, with the GitHub CLI:

Terminal window
gh pr list --author "app/promptless" --state merged \
--search "merged:2026-10-01..2026-10-31" \
--limit 1000 --json number --jq length

Count all merged PRs in the same period:

Terminal window
gh pr list --state merged \
--search "merged:2026-10-01..2026-10-31" \
--limit 1000 --json number --jq length

Divide the first count by the second to get the share of doc changes automated.

Without the GitHub CLI, count the commits on your main branch with Git. With squash merges, each commit is one PR:

Terminal window
git log origin/main --author='promptless\[bot\]' \
--since=2026-10-01 --until=2026-11-01 --oneline | wc -l

List the pages Promptless changed most often in the period:

Terminal window
git log origin/main --author='promptless\[bot\]' \
--since=2026-10-01 --until=2026-11-01 --name-only --format= \
| sort | uniq -c | sort -rn | head -20

The metrics above count output. To show the effect on readers and on cost, add metrics from your support, analytics, and docs-platform tools.

MetricWhere to read itHow to use it
Support tickets per topicYour helpdesk, such as Zendesk or Intercom. GitHub Issues or Discussions, if users ask questions there.Compare ticket volume on a topic before and after a Promptless update to that topic.
”Was this helpful?” ratings and commentsYour docs platform, such as Mintlify, GitBook, or ReadMe, or a feedback widget.Compare ratings on a page before and after an update. Respondents are self-selected and skew negative, so use the trend, not the score.
Site searches with no resultsYour site-search analytics, such as Algolia or your docs platform’s search dashboard.Recurring queries show missing content and the words readers use for it. After an update, check that the no-results rate for the topic falls.
Page views and engagementYour web analytics, such as GA4 or Plausible.Confirm that an updated page has readers. Read engagement time with views. A high bounce rate is normal on a page that answers a question quickly.
Task completion rateUsability tests or an instrumented funnel.Measure whether readers can complete the task. This costs the most to collect, so use it for high-stakes topics only.
Content freshness, doc coverage, and organic search trafficYour docs repo and a search console, such as Google Search Console.Use as supporting context. Report any link to growth or conversion as a correlation.

If you have the cost figures, calculate:

  • Deflection value = tickets deflected × fully loaded cost per ticket. Support operations usually has the cost per ticket.
  • Contributor time saved = doc updates shipped × hours saved per update × fully loaded hourly rate.
  • ROI % = (attributed value − program and tool cost) ÷ cost × 100.

Without cost figures, report the before-and-after trend.

State each assumption, such as cost per ticket, hours per update, and deflection percentage. Report before-and-after comparisons and ranges, not single figures.

  1. Set the period and scope. Choose a month or a quarter, and the doc collections to cover. Use the same period for every number.

  2. Collect the output numbers. Read the weekly created, merged, and closed counts from the Impact page. Read merged Promptless PRs from your Git history, suggestion counts and review times from the Suggestions list, and coverage from the Triggers page.

  3. Lead with output. Put doc updates shipped and share of doc changes automated first. Add time saved, with your hours-per-update estimate.

  4. Add outcomes. Include one or two outcome metrics for the same collections and period. Convert them to a dollar value if you have the cost figures.

  5. Cite the source of each number. Name the Impact page, the Suggestions list, the Triggers page, the Published page, or the merged PRs.