FirstQA Documentation

Everything you need to know about Ovi — your AI QA engineer.

What is FirstQA?

FirstQA is Ovi AI — your first QA hire. Ovi connects to GitHub and Linear, learns your product, asks the questions on every ticket, creates test cases, tests every PR, runs exploratory browser tests, finds bugs, and delivers a GO/NO-GO decision — all automatically, all in your existing tools.

How It Works

  1. Hire Ovi: Sign up at firstqa.dev and connect your tools — no code changes, no CI config needed.
  2. Onboard Ovi: Connect GitHub and Linear in one click. Ovi indexes your codebase and learns your product.
  3. Ovi gets to work: Every PR and ticket is analyzed automatically. Trigger a full test run anytime with /qa.

Getting Started

GitHub App (PR Analysis + Test Execution)

  1. Install the GitHub App: Install FirstQA on your repositories.
  2. Open a Pull Request: Create or update a PR in an enabled repository.
  3. Get Analysis: Ovi automatically analyzes the PR — bugs, risks, test scenarios, ship score. Or comment /qa to trigger manually.
  4. Run Tests: Comment /qa testrun to execute browser tests. Add -env=URL to target a specific environment.
  5. Get Playwright Code: Grab the generated spec files from the analysis comment.

Chrome Extension (Ticket Analysis)

  1. Install the Extension: Add the FirstQA Chrome Extension from your account dashboard.
  2. Open a Ticket: Navigate to any Linear or Jira ticket in your browser.
  3. Trigger Analysis: Ovi reviews the ticket — surfacing gaps, edge cases, risks, and QA questions before coding starts.

Commands

CommandWhereWhat it does
/qaPR commentRun full QA analysis on the PR
/qa testrunPR commentExecute automated browser tests from the latest analysis
/qa testrun -env=URLPR commentRun tests against a specific environment URL
/qa testrun -context "..."PR commentPass context (e.g. user type, auth state) to the test agent
/shortPR commentRun a shorter, faster analysis

PR Analysis

On every PR open or update, Ovi analyzes the diff and produces:

  • Bug Detection — functional bugs, logic errors, data flow issues, race conditions
  • Risk Assessment — edge cases, state transitions, missing error handling
  • UI/UX Review — user-visible issues, confusing flows, inconsistent behavior
  • Ship Score — Go/No-Go recommendation with blockers and non-blockers separated
  • Test Recipe — prioritized scenarios with exact steps and expected results
  • Playwright Code — ready-to-use spec files extracted from your actual codebase

Ticket Analysis

Ovi reviews Linear and Jira tickets before development starts — catching gaps and edge cases at the cheapest possible moment.

  • QA Questions — clarifying questions a senior QA would ask the PM or designer
  • Edge Cases — scenarios not covered in the ticket description
  • Risk Flags — areas that are likely to cause bugs based on the requirements
  • Pre-dev Test Scenarios — acceptance criteria written as testable scenarios

Trigger via /qa in a Linear or Jira comment, or use the Chrome Extension on any ticket page.

Browser Testing

Comment /qa testrun on a PR and Ovi launches an AI agent in a real cloud Chromium browser to execute every test scenario.

  • Real browser — cloud Chromium via Browserbase, no local setup needed
  • Screenshots — captured at every step for evidence
  • Pass/Fail report — posted back as a PR comment with blocker vs. non-blocker distinction
  • Auth support — cookie injection via -context or configured credentials

Speed: Browser test execution takes 5–15 minutes depending on the number of scenarios.

Playwright Test Code

  • Ready-to-use — downloadable .spec.js files with accurate selectors
  • Based on real code — selectors and flows extracted from your actual codebase, not guessed
  • Drop-in ready — add to your test suite and run locally or in CI

Ship Score

Every PR analysis ends with a Go/No-Go recommendation. Ovi separates blockers (must fix before merging) from non-blockers (can ship, address later) so you never hold up a release for minor issues.

🚀 GO — Ship It

No blockers found. Ready for production.

⚠️ GO — Ship with Caution

Non-blockers only. Safe to ship — monitor after.

🚦 NO-GO — Fix Blockers First

One or more blockers that could significantly impact users. Fix before merging.

Test Authentication

If your app requires login, you have two options:

Option A: Context flag (per run)

/qa testrun -env=https://staging.myapp.com -context "logged in as trial user"

Ovi will use this context to understand the expected auth state and attempt login autonomously.

Option B: Cookie injection

/qa testrun -env=https://staging.myapp.com -context cookie:session_id=abc123;auth_token=xyz

Extract cookies from your browser's DevTools (Application → Cookies) while logged in, then pass them directly.

Option C: Configured credentials (dashboard)

Set TEST_USER_EMAIL and TEST_USER_PASSWORD in your FirstQA dashboard. The agent will attempt deterministic login automatically.

Integrations

IntegrationWhat it doesStatus
GitHubPR webhooks, /qa comments, Check Runs, auto-analysis on PR open/updateLive
LinearTicket analysis via webhooks and Chrome ExtensionLive
JiraTicket analysis via Atlassian ConnectBeta
BrowserbaseCloud Chromium for browser test execution with session replayLive

FAQ

How does Ovi analyze my PR?
Ovi reads the PR diff, changed file contents, and commit history. It uses AI to identify functional bugs, logic errors, edge cases, UI/UX risks, and missing error handling — then generates a prioritized test recipe and Go/No-Go recommendation.
How does browser test execution work?
When you comment /qa testrun, Ovi launches an AI agent in a cloud Chromium browser. The agent autonomously navigates your app, executes each test scenario, captures screenshots, and records the session. Results are posted back as a PR comment with pass/fail status and screenshot links.
How fast are the results?
PR analysis: 30–90 seconds depending on PR size.
Browser test execution: 5–15 minutes depending on the number of scenarios.
Is my code stored or shared?
No. Code is processed in memory and never permanently stored. FirstQA has read-only access to your repositories and only writes PR comments and Check Runs. All data is transmitted over HTTPS/TLS.
Do I need to change my CI/CD setup?
No. Ovi works entirely through GitHub App webhooks and PR comments. There are no CI changes, no new config files, and no pipeline modifications required.
What if we already have unit or integration tests?
Ovi complements your existing tests. It catches what unit and integration tests miss — UX edge cases, requirement gaps, and flows that break in a real browser under real conditions.

Need Help?

Ready to hire Ovi?

Set up in under 10 minutes. No engineers needed.

Install GitHub App Book a Demo