Fantasy GPT
A multi-step reasoning SQL RAG system on a fine-tuned in-house model, answering live cricket questions against continuously ingested match data in under thirty seconds.
- of complex sports queries resolved end to end
- 98%of complex sports queries resolved end to end
- answers against live match data while a match is in progress
- < 30sanswers against live match data while a match is in progress
- reasoning chains, so questions need not map to a single query
- Multi-stepreasoning chains, so questions need not map to a single query
Sports questions are not lookups — they are aggregations over live data, and they arrive while the match is still running. I built the system that answers them: a fine-tuned in-house model driving multi-step reasoning over SQL, fed by live ETL pipelines, with quality checks rather than vibes deciding whether an answer was good enough to ship.
The problem
A fan's question rarely maps to a single row. It maps to a query — sometimes several, chained — over data that is changing as they ask. Vector retrieval over documents cannot answer 'who has the better strike rate against spin in this innings', because the answer does not exist as text anywhere; it has to be computed. And a stale answer during a live match is worse than a slow one, so ingestion latency was part of the problem, not separate from it.
Constraints
- The data is live: an answer computed against a stale snapshot is wrong even if the reasoning was right.
- Questions need multiple dependent steps, so a single generated query cannot satisfy them.
- A generated query runs against a real database, so it has to be constrained rather than trusted.
- Answers had to land fast enough to feel conversational during a match in progress.
Decisions
What I chose, why, and what I turned down to get there.
SQL generation over document retrieval
The questions were quantitative, and the ground truth lived in structured match data rather than prose. Generating queries against that data returns computed answers instead of retrieved passages — which is the difference between answering a statistics question and finding a page that discusses statistics.
Considered and rejected
- Vector search over match reports and commentary — retrieves text that talks about the numbers without ever computing them
Multi-step reasoning rather than one query per question
Real questions decompose: establish context, then narrow, then compare. Letting the system plan and chain steps — using intermediate results to shape the next query — answers questions that a single generated statement structurally cannot.
A fine-tuned in-house model
Cricket has dense domain vocabulary and a schema-specific query surface, and a general model handled neither reliably. Fine-tuning in house also kept inference within our own infrastructure, which matters when every question during a live match hits the same path.
Live ETL as part of the answer path
Pipelines continuously ingested match data from multiple sources into the analytical store the model queries. Freshness was treated as a system property rather than a data-team concern, because the product promise was answering about a match that is still happening.
Automated quality checks on generated answers
Answer quality was measured with an evaluation harness rather than judged by spot-checking. This was my first experience of the thing I now build deliberately: without measurement, every prompt change is a guess, and regressions are invisible until a user finds them.
Why this one mattered to me
This is where the pattern I now work on full time first appeared: a model is only as useful as the tools and context around it, and you cannot tell whether it is working without measuring it. Fantasy GPT needed retrieval that computed rather than recalled, orchestration across dependent steps, and evaluation to know if any of it held. Everything I have built since is a more rigorous version of those three ideas.
Stack
- Python
- FastAPI
- LangGraph
- SQL RAG
- Fine-tuned LLaMA
- Microsoft SQL Server
- DeepEval
- Docker
What was mine
Built the retrieval and reasoning system, the backend APIs, the ETL pipelines feeding it, and the quality checks over its answers, as a GenAI intern on the product team.