Nikunj Khitha
Applied AI Engineer
I work on the part of AI that starts after the demo: giving agents real tools, real memory, and a way to prove they still work tomorrow.
About
Nikunj
I work on the part of AI that starts after the demo: giving agents real tools, real memory, and a way to prove they still work tomorrow.
- Role
- Applied AI Engineer (SDE 1)ArmorCode · Jan 2025 — present
- Based
- Gurugram, Haryana, India
Open to new roles
ResumeI focus on the engineering work that makes AI useful after the first successful prompt: connecting models to the right tools and context, controlling what those tools can do, preserving useful memory, measuring behavior, and debugging failures across model, protocol, application, and infrastructure boundaries.
Recognition
AI Ninja Award at ArmorCode
First-ever recipient of ArmorCode's AI Ninja Award, presented on the company's global platform, and the youngest person to receive an award at the company.
An OpenTelemetry platform on Langfuse scoring prompts, models, agents, skills, MCP tools, and retrieval against golden datasets — deterministic checks, scikit-learn metrics, Ragas, and LLM-as-a-judge graders validated at Cohen's kappa >= 0.7, enforced as Jenkins CI gates.
Delivered in ArmorCode's shared enterprise tool registry with OAuth2/RBAC, tool-level permission tiers, explicit denial behavior, and audit attribution under multi-tenant AppSec constraints.
Co-built Agentic Office OS, translating ambiguous stakeholder requirements into Slack-accessible agents, skills, MCP integrations, and human-in-the-loop approval workflows — then owning rollout, debugging, and enablement.
ArmorCode's tenant-scoped Knowledge Graph RAG layer in Neo4j and pgvector, with retrieval scored on accuracy, ranking quality, and context precision across five query modes.
Sole maintainer of company-wide model and MCP access on a LiteLLM gateway. Traced 57% of spend across 50,000+ requests to 10+ automations, then migrated models and enabled prompt caching.
Shipped the platform agent memory layer on temporal knowledge graphs, combining session-scoped context with tenant- and person-level long-term recall for multi-step reasoning.
Professional
Experience
I’ve grown from building public-sector software into delivering Applied AI systems, working through stakeholder discovery, agent architecture, governed tool integration, rollout, debugging, and iteration.
And a new adventure ahead
Software Development Intern
Central Electricity Authority, Government of IndiaExperienceMay 2023 - July 2023Built public-sector software that improved data reliability, internal operations, and workflow speed across government systems.
Integrated National Power Portal data into a national renewable energy dashboard serving 150+ power stations and improved reporting accuracy by 30%.
Built a secure Java/PostgreSQL file management system with role-based access control that improved retrieval efficiency by 25% across 5,000+ files.
Developed a MERN conference room booking system that cut booking time by 60% and reduced scheduling errors by 40%.
Software Development Intern (GenAI Specialist)
Xansr Media (Aiko)ExperienceJun 2024 – Dec 2024Shipped full-stack, backend, GenAI, and data systems for AIKO and Fantasy GPT, powering personalized sports experiences, voice AI, and retrieval-backed cricket intelligence.
Built Node.js and FastAPI microservices, improving API performance by 40% and reducing deployment time by 42% with Docker and GitHub Actions.
Engineered Fantasy GPT with RAG, LangGraph, backend APIs, agents, and DeepEval quality checks to resolve 98% of complex sports queries.
Built Python-based ETL pipelines to collect sports data from multiple sources and ingest it into MS SQL for Fantasy GPT SQL RAG workflows.
Worked across AIKO, a voice-based sports companion using Azure Speech SDK for text-to-speech and speech-to-text, user-level personalization, and live AI-generated commentary in 20+ languages.
Built AIKO personalization features for on-the-fly highlight reels, where AI agents stitched sports moments based on each user's profile and interests for a product presented at IBC 2024 in Amsterdam.
Software Development Intern (Applied AI)
ArmorCodeExperienceJan 2025 - Nov 2025Created Anya's agent framework and memory layers, and built backend integrations and AI-assisted scaffolding across ArmorCode's AppSec platform.
Created the initial Java framework for Anya, ArmorCode's platform agent; designed, implemented, and owned its short- and long-term memory layers; and set up Langfuse-backed evaluation for agent accuracy and memory behavior.
Ported 2 of 6 Anya sub-agents from LangChain4j to Spring AI, reimplementing memory and evaluation integrations for the new architecture.
Owned backend integrations for 5+ security tools, including Black Duck, Snyk, and Checkmarx, on an AppSec platform aggregating findings across 130+ connectors.
Created AI-assisted code generation utilities with template engines and AST parsing to automate new integration scaffolding, reducing per-integration boilerplate setup time by 30%.
Applied AI Engineer (SDE 1)
ArmorCodeExperienceDec 2025 – PresentCo-build Agentic Office OS, the internal agent platform serving 20+ business functions, and own the tool, context, governance, and evaluation layers underneath it.
Co-built Agentic Office OS, the internal platform of autonomous, human-triggered, and human-in-the-loop agents reachable org-wide through Slack, translating ambiguous requirements from an external business consultant, executives, and internal teams into agents, skills, MCP integrations, and approval workflows for 20+ business functions, then owning rollout, debugging, and enablement.
Built an OpenTelemetry-based LLM evaluation platform on Langfuse so every prompt, model, agent, skill, and MCP tool change ships on measured accuracy, latency, and cost. It scores 9 AI surfaces against golden datasets using deterministic checks, scikit-learn classification metrics, Ragas RAG scores, and LLM-as-a-judge graders validated at Cohen's kappa >= 0.7, enforced as Jenkins CI gates.
Delivered 10+ production MCP servers in ArmorCode's shared enterprise tool registry under multi-tenant AppSec constraints, implementing OAuth2/RBAC controls, tool-level permission tiers, explicit denial behavior, and audit attribution, with 20 of 20 authorization checks validated across three access tiers.
Govern company-wide model and MCP access as sole maintainer of a LiteLLM gateway, issuing scoped API keys with per-model spend budgets and distributing RBAC-gated MCP servers to employees' Claude Desktop via an .mcpb proxy. Cut recurring LLM spend by tracing 57% of gateway cost across 50,000+ requests to 10+ automations, then migrating models and splitting system/user prompts to enable Bedrock prompt caching, while flagging a deterministic rewrite worth a further ~95% reduction.
Built ArmorCode's tenant-scoped knowledge-graph RAG layer over 1M+ entities of root-cause analyses, test cases, and product documentation in Neo4j and pgvector, giving Office OS agents grounded product knowledge without cross-tenant leakage, with retrieval scored on accuracy, ranking quality, and context precision across five query modes.
Cut CS and support escalations to engineering with a codebase-search MCP service in Go over 8 product repositories, fronting a read-only agent against a daily-reindexed vector index. Restored it after 100+ queries returned zero content by isolating two independent causes in SDK and host internals: a Go output-schema defect that made schema-aware clients discard every answer, and a query deadline sized for a longer client timeout tier. Shipped the fix with a negative-control test and flagged 14 exposed connectors.
Built AI documentation automation that triggers when a Jira ticket is completed: an agent creates or updates documentation in GitHub and opens a pull request, reviewers use Quill (Electron/React/TypeScript) to review and edit the changes, and merged documentation syncs to Zendesk. Built Quill as the workflow's review app with a WYSIWYG editor, an embedded agent terminal, per-branch Git worktrees, and permission-aware GitHub actions.
And a new adventure ahead
Selected Work
Agent systems, retrieval, and platform work I own at ArmorCode, GenAI products shipped at Xansr Media, and the products I have built on my own.
Featured case studies
LLM Evaluation Platform
An OpenTelemetry and Langfuse platform that gates every prompt, model, agent, and tool change on measured accuracy, latency, and cost.
- AI surfaces scored against golden datasets
- 9AI surfaces scored against golden datasets
- Cohen's kappa required before a judge may gate a release
- ≥ 0.7Cohen's kappa required before a judge may gate a release
OpenTelemetry · Langfuse · LiteLLM · Ragas
Read case studyGoverned MCP Tool Registry
Ten-plus production MCP servers behind a shared enterprise registry, with tool-level authorization under multi-tenant AppSec constraints.
- production MCP servers delivered into the shared registry
- 10+production MCP servers delivered into the shared registry
- authorization checks passing across three permission tiers
- 20 / 20authorization checks passing across three permission tiers
Model Context Protocol · Go · OAuth2 · RBAC
Read case studyKnowledge Graph RAG
Tenant-scoped retrieval over a million-plus entity knowledge graph, with a five-layer failure taxonomy so a bad answer can be traced to its cause.
- entities in production across four content sources
- 1M+entities in production across four content sources
- retrieval modes benchmarked into a documented mode-per-tool policy
- 5retrieval modes benchmarked into a documented mode-per-tool policy
Neo4j · pgvector · PostgreSQL · Python
Read case studyBuilt and shipped
CodeNex: AI Builder
An AI-driven code generation SaaS platform for building full React applications from natural-language prompts.
- Role
- SaaS architecture, backend systems, streaming infrastructure, and platform design
- Impact
- Designed for SaaS-scale concurrency with Kubernetes preview pods, token quotas, RBAC, autoscaling, and Stripe subscriptions.
- Java
- Spring Boot
- Spring AI
- React
- TypeScript
- SSE
- Kubernetes
- MinIO
- Stripe
CodeNex AI API Proxy
A unified AI gateway for routing model traffic through one consistent API layer.
- Role
- Gateway architecture, backend implementation, and operational tooling
- Impact
- Demonstrates strong infra instincts around reliability, cost control, abstraction, observability, and multi-model platform design.
- Go
- Gin
- Redis
- PostgreSQL
- React
- OpenAI-compatible APIs
Serenify
A consumer-style AI wellness product with thoughtful UX, not just chat wrapped around a model.
- Role
- Product design, frontend experience, and AI workflow implementation
- Impact
- Shows product empathy, privacy-minded AI interaction design, end-user UX judgment, and the ability to shape AI into a coherent consumer experience users can return to consistently.
- React
- TypeScript
- Supabase
- pgvector
- Gemini AI
- Vercel
Also built
Resume Fit — CodeNex
An AI resume improvement workflow built like a practical product instead of a one-off analyzer.
LLaMa MCP Streamlit
A tool-aware AI interface that pairs LLaMA with MCP for real-time external actions.
Technical
Stack
The technologies I reach for most often when building agent systems, governed MCP tools, evaluation pipelines, and full-stack AI products.
Learning fast.
Applied AI, Agents & Evaluation
22 technologies
The agent, retrieval, and evaluation stack I use to build governed AI systems and prove they still work after a change.
- AI Agents
- Agentic AI
- Multi-Agent Systems
- Model Context Protocol (MCP)
- Tool Calling
- Agent Memory
- LLM Evaluation
- LLM-as-a-Judge
- Golden Datasets
- Langfuse
- Ragas
- OpenTelemetry
- RAG
- GraphRAG
- Knowledge Graph RAG
- Prompt Engineering
- Prompt Caching
- Model Routing
- LangGraph
- Spring AI
- Graphiti
- AWS Bedrock
Languages & Backend
15 technologies
The languages and frameworks I use to build AI products, APIs, and internal platforms end to end.
- Python
- TypeScript
- Go
- Java
- SQL
- FastAPI
- Spring Boot
- Node.js
- Gin
- Next.js
- React
- REST APIs
- Microservices
- OAuth2
- RBAC
Data, Retrieval & Storage
8 technologies
The storage, graph, and vector technologies behind the retrieval and context systems I build.
- Neo4j
- pgvector
- PostgreSQL
- Elasticsearch
- Redis
- MongoDB
- Apache Superset
- S3
Platform, Observability & Delivery
13 technologies
The infrastructure and measurement tooling I use to deploy, route, observe, and gate AI systems in production.
- Docker
- Kubernetes
- AWS
- CI/CD
- Jenkins
- GitHub Actions
- LiteLLM
- pytest
- scikit-learn
- Kafka
- Grafana
- Prometheus
- n8n
Talk to my
AI Twin
Checking whether the live assistant is available right now. You can still open the chat panel while the status loads.
It will also tell you when something is not measured rather than estimating a number \u2014 which is the same discipline the evaluation platform enforces.
Get in Touch
Open to Applied AI, Forward Deployed AI, Agentic AI, AI Product, and LLM/AI Engineer roles where I can own agent systems end to end — governed MCP tool use, retrieval, and the evaluation that keeps them reliable.
Let's Build Something
If you are hiring for Applied AI systems, AI product engineering, agent workflows, or AI-platform reliability, I would welcome a conversation about the team, workflow, or problem you are solving. I'm also open to thoughtful collaborations and open-source conversations.

