Agent reliabilityCheckpointCo-founder & CTO2026Ongoing
Checkpoint
CI/CD for AI agents. Adversarial test suites that run before an agent ever meets a user.
Problem
Teams shipping LLM agents have unit tests for their code and almost nothing for the agent.
The failures that matter are not exceptions. They are a tool called with the wrong argument, a policy boundary quietly crossed, a multi-turn conversation that drifts. None of that shows up in a green build.
Approach
I co-founded Checkpoint with Ayushman Gupta and Aaditya Gaur and led engineering as CTO.
The bet was that agents need a pre-production failure surface, somewhere to break the loop on purpose before a user does it by accident.
How it works
Engineers submit an agent config: prompts, tools, schemas.
Checkpoint does the rest.
Test generation across happy paths, edge cases, adversarial prompts, policy boundaries, and ambiguous inputs.
Synthetic environments with mocked tool calls that hold state across turns, so multi-step failures are reproducible.
LLM-judged scoring against structured rubrics, which catches semantic regressions that assertion-based tests miss.
Impact
YC ranked us in the top 10% and then didn’t interview us.
The product is live in private beta, and the thesis still holds. Agents need somewhere to fail on purpose.
What I’d do differently
Adversarial generation is the easy half.
Deciding what counts as a failure, and making that judgment reproducible, is the actual product.
Stateful tool mocking mattered more than model choice. Most interesting agent bugs only appear on turn three or later.
A strong application signal is not a business. Ranking well told us the problem was legible. It didn’t tell us anyone would pay.
Built with
Next.js · Python · FastAPI · OpenAI API · AWS