Causal inference meets AI products: Daniel of PaperScore, SF
I'm Daniel Armani, a Principal Data Scientist and founder at PaperScore, based in San Francisco. My background is applied econometrics, causal inference and quasi-experimental design, and these days I spend most of my time building AI products and RAG agents.
My stack is a mix of local and hosted models: Qwen and Gemma through Ollama, plus Bedrock and Gemini. For orchestration I use LangGraph and Google's Agent Development Kit, and I build mostly with Cursor, Claude Code and Codex. On GitHub I've shared a finance RAG notebook built with LangChain and a Claude skill called autonomous-inventor that works through hardware and automation designs step by step.
I'd love to hear from anyone who has brought rigorous causal or experimental thinking into evaluating AI products or agents. If you've done it, or know someone who has, I'd be glad to get an intro.






Comments1
Welcome to Lounge, Daniel. Evaluating agents with a causal lens is a sharp angle, since offline evals rarely capture what changes for users once an agent ships. For PaperScore, do you have a natural experiment or holdout you can use, or are you mostly relying on offline RAG evals today?
1 comment
Welcome to Lounge, Daniel. Evaluating agents with a causal lens is a sharp angle, since offline evals rarely capture what changes for users once an agent ships. For PaperScore, do you have a natural experiment or holdout you can use, or are you mostly relying on offline RAG evals today?