Run a pilot

Promptless offers 14-day pilot periods for growth and enterprise customers, so you can see how Promptless works with your own documentation, team workflows, and trigger events.

The 14-day pilot lets you:

See Promptless handle different types of documentation updates
Evaluate suggestion quality across different scenarios
See how well Promptless can integrate into your team’s workflow

Your pilot starts when Promptless creates your first suggestion, not when you sign up. Any time that’s spent getting approvals for data source connections doesn’t count against your pilot.

Before your pilot begins

Before the pilot begins, you’ll need to connect Promptless to your relevant workspaces and share resources with the Promptless team so that the Promptless agent can draft an initial batch of suggestions.

Set up Promptless connectors

Connect source code repositories. Connect GitHub, GitHub Enterprise, GitLab, or Bitbucket
Connect documentation repositories. Connect your docs repository on GitHub, GitLab, or Bitbucket (e.g. if you use Mintlify, Fern, ReadMe, Docusaurus, etc.)
Connect communication platforms. Connect Slack or Microsoft Teams to get notified about new suggestions and to tag Promptless in threads or messages to request doc updates directly
Connect project management tools (optional). Connect Jira or Linear—this is optional, but recommended to give Promptless better context about the customer-facing impact of new features

See the Quickstart for integration instructions.

Set up Slack Connect

When you connect Slack during setup, we’ll create a shared Slack Connect channel and send you an invitation. Accept the invitation to get a direct line to our team for questions, feedback, and support throughout your pilot.

Promptless works best when it has the same sort of information that you’d give to a new technical writer joining your team.

This can include:

Your style guide
Vale configuration (if you use it)
Documentation standards or templates

In-person onboarding workshop

We always recommend kicking off customer pilots with an in-person workshop.

What to expect

Duration

90-minute session. For larger teams, we may do multiple workshops over a few days.

Preparation

Before your session, we run Promptless on historical events (like the last 30 days of PRs) so you can review real suggestions during the workshop.

What We'll Cover

How Promptless works, your configuration, and hands-on review of suggestions—with the goal of publishing your first doc updates together.

Workshop goals

Understand your Promptless setup. Review your configuration and how triggers, context sources, and documentation platforms work for your team
Publish your first updates. Review suggestions together and publish documentation updates during the session
Define success criteria. Agree on what a successful pilot with Promptless would look like for your team

Measure success for your pilot

During the workshop, we define the success criteria for your pilot. These vary based on each team’s needs, but across dozens of onboardings, suggestion volume and suggestion quality are the most reliable indicators that your team gets lasting value from Promptless.

Suggestion volume

Track how many suggestions you’re getting from Promptless. More suggestions isn’t necessarily better—sometimes too many suggestions can feel noisy or overwhelming—but a healthy stream of documentation suggestions is a strong indicator that Promptless is plugged into the right parts of your workflow.

Typical volume benchmarks

Teams with 0-1 people working full-time on docs: 20-40 suggestions per month

Teams with 2-5 technical writers: 30-100 suggestions per month

For larger teams, target volume depends heavily on how your team is organized, and it might be more useful to set target volumes for each product or product vertical your team supports.

If Promptless isn’t able to generate 20 suggestions a month, your team may be too early to see enough value from Promptless.

Suggestion quality

Since everyone has different expectations for Promptless, suggestion quality is simply measured by your rating of Promptless’s suggestions, on a scale from 0 to 10.

Quality target: Promptless should consistently rate at least 8/10 for your team’s needs.

Tip

During your workshop, we’ll discuss what “quality” means for your team—technical accuracy, style guide compliance, or catching edge cases. Understanding your quality criteria helps us make Promptless better for you.

Understand quality ratings

To help you calibrate what different quality levels mean for your team:

“9-10”: Most suggestions from Promptless are relevant, and typically can be accepted with little to no edits.
“7-8”: Most suggestions are accurate and relevant, requiring only moderate editing. You’re refining style, adding specific details, or adjusting tone rather than rewriting entire sections. Promptless meaningfully reduces your documentation workload.
“4-6”: Suggestions occasionally catch relevant changes and can serve as a starting point, but require significant editing. You’re rewriting content, fixing inaccuracies, or filling in missing context. Promptless saves some time, but doesn’t dramatically change your workflow.
“0-3”: Promptless is creating negative value. Most/all suggestions are not good enough to be a useful starting point for doc updates. Even if it’s occasionally bringing up things that are relevant, I’d rather start from scratch myself.

Documentation improvement projects

Sometimes teams need foundational work before piloting—like documenting major missing features, completing a large refactor, or creating documentation from scratch.

Subject to availability, the Promptless team may be able to help with these projects before your pilot begins.

Weekly check-ins

After your onboarding workshop, we schedule 15-30 minute check-ins to track progress and address issues.

What we review:

How many suggestions did Promptless create?
How would you rate suggestion quality this week (0-10)?
What’s the gap between current suggestions and 10/10 quality?
How is Promptless fitting into your workflows? Is it triggering at the right times?

These check-ins help us identify and fix issues, adjust your configuration, and make sure you’re getting value from Promptless.