testsprite-cli

TestSprite CLI

Overview

TestSprite CLI is the open-source command-line interface for the TestSprite AI testing platform, and it solves one of the most uncomfortable problems in modern software development: who verifies the code your AI agent just wrote? The repo launched on June 11, 2026, and pulled in nearly 500 GitHub stars in its first week. That kind of velocity usually signals a tool that scratched a real itch.

The platform behind the CLI already serves over 100,000 teams for end-to-end software testing — frontend and backend, running against live products in the cloud, not mocks. This repo takes that testing infrastructure and puts it directly into the hands of coding agents like Claude Code, Codex, Cursor, Cline, and Antigravity. The agent writes code, then immediately verifies it by running tests against a real browser or API. Failures come back as a single self-consistent bundle — screenshots, DOM snapshots, a root-cause hypothesis, and a recommended fix — so the agent can correct its own work before a human ever sees a bug.

The core insight is that AI coding agents are fast at producing code but terrible at verifying it. They ship in minutes, but testing still takes hours. TestSprite closes that gap by creating a verification loop: create test → run → read failure → fix → rerun. Every successful pass gets banked into a durable test suite that compounds over time. The CoderCup leaderboard — an open, public benchmark where frontier agents build the same app under the same rules — showed that with TestSprite as the referee, the cheapest model in the field shipped the most correct app at 89% accuracy, at half the cost of the most expensive one. That’s not a theoretical claim; it’s a published result.

Why it matters

The agentic coding revolution has a verification gap. Tools like Claude Code, Codex, and Cursor can generate entire features in minutes, but the testing story is still mostly “run it locally and hope for the best.” Traditional testing tools were built for human-driven workflows — you write a test, run it in CI, read the report. None of that maps cleanly to an agent that needs programmatic, deterministic feedback it can act on without human interpretation.

TestSprite fills that gap. It’s the first tool I’ve seen that treats the coding agent as a first-class user of the testing system. The CLI returns structured JSON, predictable exit codes, and failure bundles that are self-contained — no dashboard scraping, no screenshot interpretation, no ambiguity. This matters because the alternative is agents shipping code that “looks right” but breaks under real-world conditions. Every fullstack developer using AI agents for React frontends, NestJS APIs, or Go microservices has hit this: the agent generates clean code that passes local checks but fails in integration. TestSprite runs against the live product, the way a QA engineer would, and gives the agent exactly what it needs to fix things.

Key Features

Agent-shaped failure bundles. When a test fails, testsprite test failure get returns one self-consistent package: the failing step, neighboring steps, screenshots, DOM snapshots, the test source code, a root-cause hypothesis, and a recommended fix target. Everything shares a single snapshotId. The CLI explicitly refuses to stitch data from two different runs, which prevents agents from reasoning over contradictory context — a subtle but critical design decision.

Cloud-native test execution. Tests run against real browsers and real APIs in the cloud, not mocks or local simulators. Your agent describes intent and reads results. It never needs to know how the test was driven — only what a real user experienced. This means your React components get tested with actual rendering, your API endpoints get hit with real HTTP requests, and your integration flows get exercised end-to-end.

One-command agent onboarding. Running testsprite agent install claude (or cursor, codex, cline, etc.) drops a ready-made skill file into your repo. From that point forward, your coding agent knows how to drive the verification loop on its own. No manual configuration, no YAML files to maintain, no CI pipeline to set up — the agent installs its own testing capability.

Durable test suite that compounds. Every test that passes gets banked into a persistent suite. Over time, this suite becomes a living specification of every behavior your application has ever gotten right. It’s bigger than any context window, and it catches regressions that slip through when agents modify code that “shouldn’t” affect other features.

Deterministic scripting surface. The CLI exposes stable --output json contracts, predictable exit codes (documented in the repo), and a --dry-run flag that exercises the full code path offline with canned data. This makes it usable in CI pipelines, onboarding scripts, and automated workflows — not just interactive agent sessions.

Multi-agent compatibility. Works with Claude Code, Codex, Cursor, Cline, Antigravity, Kimi, Trae, and other coding agents. The skill file is agent-agnostic — install it once, and any agent in your workflow can drive the verification loop. This matters for teams where different developers use different tools.

Frontend and backend coverage. The --type frontend and --type backend flags let you test UI interactions (clicks, navigation, form submissions) and API behavior (endpoints, data flows, error handling) from the same CLI. Backend tests support dependency metadata via --produces and --needs flags, so the runner knows which tests to replay when upstream data changes.

Use Cases

• AI-assisted feature development — When Claude Code generates a new React component or NestJS endpoint, immediately verify it works against the live app before committing. The agent creates a test, runs it, reads the failure bundle if anything breaks, fixes the code, and reruns — all without human intervention.

• Regression detection in agent-driven refactors — Your coding agent refactors a Go microservice. The durable test suite automatically replays all previously passing tests to confirm nothing broke. If the refactor introduced a subtle API contract change, the failure bundle pinpoints exactly which endpoint and which assertion failed.

• CI/CD integration for agent-authored code — Add testsprite test run --all --project <id> --wait to your CI pipeline. Every PR that an agent touches gets full end-to-end coverage before merge. Exit codes map directly to CI pass/fail gates.

• Rapid prototyping with confidence — Solo developers and small teams using AI agents to build MVPs can ship faster because verification is automated. Instead of manually testing every feature the agent generates, the test suite catches issues automatically.

• Cross-browser compatibility checks — Since tests run in cloud browsers, you get real rendering behavior for your React or Vue frontends — not just the one browser you happen to have open locally.

Pros and Cons

Pros:

• Solves a real, immediate problem for anyone using AI coding agents professionally. The verification gap is not theoretical — every team using Claude Code or Codex has shipped bugs that a human QA would have caught.
• The CoderCup leaderboard result (89% correctness with the cheapest model) is a compelling proof point. It suggests that verification infrastructure matters more than model capability for shipping correct code.
• Apache-2.0 license, TypeScript codebase, active development with CI passing. The onboarding experience is polished — testsprite init handles everything from API key setup to agent skill installation.

Cons:

• Requires a TestSprite API key and cloud account. The free tier exists, but serious usage will hit limits. This is a SaaS-adjacent open-source tool — the CLI is open, but the testing infrastructure runs on their cloud.
• Very early stage (version 0.1.1, repo created June 11). The command surface is still evolving. Expect breaking changes in the CLI interface before a 1.0 release.
• The failure bundle format and agent integration are opinionated. If your workflow doesn’t fit the create → run → failure get → fix → rerun loop, adapting the tool requires work.

Getting Started

# Install the CLI globally
npm install -g @testsprite/testsprite-cli

# Or use without installing
npx @testsprite/testsprite-cli

# Initialize — sets up API key and installs agent skill
testsprite init

# Non-interactive setup for CI or scripting
TESTSPRITE_API_KEY=your-key testsprite init --from-env --yes --agent claude

# Create and run a frontend test
testsprite test create --project your-project-id --type frontend \
  --plan-from ./test-plan.json --run --wait --output json

# When a test fails, get the failure bundle
testsprite test failure get test-id-here --out ./.testsprite/failure

# After fixing, replay the test
testsprite test rerun test-id-here --wait --output json

# Run all tests in a project
testsprite test run --all --project your-project-id --wait

# Install the agent skill for a specific coding agent
testsprite agent install claude
# or: cursor, codex, cline, antigravity

Alternatives

Playwright — Microsoft’s browser automation framework is the industry standard for E2E testing. It’s more mature, has a larger ecosystem, and gives you full control over test authoring. Choose Playwright when you need to write custom test logic, handle complex multi-step scenarios, or when your team writes tests manually. TestSprite is better when your primary workflow is agent-driven and you want the agent to create and manage tests automatically.

Cypress — The go-to for frontend developers who want fast, reliable E2E tests with excellent developer experience and time-travel debugging. Cypress is stronger for teams with dedicated QA engineers who write and maintain test suites by hand. TestSprite wins when the “test author” is an AI agent that needs programmatic, structured feedback rather than a visual test runner.

Browserbase / Stagehand — Cloud browser infrastructure for AI agents that focuses on browser automation and interaction. These tools are more general-purpose — they let agents navigate websites, fill forms, and extract data. TestSprite is narrower but deeper: it’s specifically about verifying that your application works correctly, with a structured loop designed for coding agents to self-correct.

Verdict

TestSprite CLI addresses the most obvious blind spot in the agentic coding stack. Everyone is optimizing for code generation speed — bigger context windows, faster models, better prompts — but almost nobody is building the verification infrastructure that determines whether the generated code actually works. The CoderCup result is hard to argue with: a cheap model with TestSprite verification outperformed expensive models without it. That tells you where the real bottleneck is.

If you’re a fullstack developer using Claude Code, Codex, or Cursor to build React frontends, NestJS APIs, or Go services, this tool belongs in your workflow. It’s early (0.1.1), the SaaS dependency is a real consideration, and the command surface will change. But the core idea — agent-shaped failure bundles, a durable test suite that compounds, and a verification loop that agents can drive autonomously — is exactly right. This is the kind of infrastructure that makes AI-generated code production-ready instead of demo-ready.