Culprit is git bisect for AI agents. Ayush, Bangalore
I'm Ayush, cofounder of Culprit, based in Bangalore. Culprit is git bisect for agent behaviour. When your agent gets worse, it replays your evals across your git history, uses repeated runs and 95% confidence intervals instead of single runs, and hands back the commit, the lines and a verified fix. It ships as a pip-installable CLI, with a CI check that gates pull requests. I build with Claude, LangGraph, Langfuse, Claude Code and Codex, and deploy on Vercel.
What's working: in our case study, Culprit named 0 wrong commits in 60 bisects, across 3 agents and 6 models. What I'm still figuring out is how teams actually fit regression checks into their eval habits, since agent evals are noisy and many suites have blind spots.
What I'd love from this community: an intro to a team shipping an agent in production who'd try Culprit on their repo and tell me honestly where it falls short.






Comments1
Welcome to Lounge, Ayush. Using repeated runs with 95% confidence intervals is a smart answer to noisy evals, though I wonder how the cost scales when a regression is subtle and each bisect step needs many replays. How do you decide how many runs per commit?
1 comment
Welcome to Lounge, Ayush. Using repeated runs with 95% confidence intervals is a smart answer to noisy evals, though I wonder how the cost scales when a regression is subtle and each bisect step needs many replays. How do you decide how many runs per commit?