ML engineer working on on-device language models, and on the evaluations that show what actually works.

I train and fine-tune small models, run them on phones and laptops, and measure them carefully, including when the answer is "this didn't help." MS CS at Northeastern (May 2027). Apple Swift Student Challenge 2026 winner.

Portfolio · LinkedIn · sandeepvijayarao09@gmail.com

DeviceControlSlm: a small language model for on-device phone and Mac control

• I trained a ~4.1M-parameter transformer from scratch (my own architecture, tokenizer and weights) and compared it with LoRA-tuned Qwen2.5-0.5B on DeviceControlBench, a benchmark I built.
• Result:
  • Familiar phrasing (in-distribution): the scratch model wins, 67.2% vs 51.6% (43 vs 33 of 64 tasks, McNemar p = 0.013).
  • New phrasing (held out): the lead disappears, 32.7% vs 36.4% (18 vs 20 of 55, p = 0.82), a statistical tie.
  • Only the scratch model's drop is significant, which points to pretraining as the source of robustness.
• Method: a held-out phrasing split, a train/test contamination gate, confidence intervals, paired McNemar tests, and a confirmation gate that kept the unsafe-execution rate at 0% for every model.
• PyTorch LoRA MLX Core ML App Intents

RAG From Scratch: ten retrieval techniques, each measured on BEIR SciFact (nDCG@10)

• Result, tested with paired bootstrap CIs (Holm-corrected) across three embedding models:
  • LLM query rewriting (HyDE, step-back, decomposition) made retrieval significantly worse, even with a 12B rewriter.
  • A strong reranker made it significantly better: Qwen3-Reranker-4B, +0.044 on dense and +0.110 on BM25.
  • Chunking, RRF, multi-query, contextual retrieval and late interaction made no detectable difference.
• Method: I checked the baseline (bge-base, 0.759) against a published figure before building anything on it. My first write-up said "seven of ten made it worse" from raw means; the significance tests showed most of that was noise, and I corrected it.
• CRAG-style document grading scored 0.50 balanced accuracy, which is chance level.
• Python BEIR bge Qwen3 cross-encoders · interactive guide

Anu: an on-device agent for iOS, and the Swift Student Challenge 2026 winner

• Gemma 4 E4B runs locally through LiteRT.
• An on-device router (NLEmbedding nearest-centroid) picks the task type, model and tools for each message. It scores 91.7% on a 60-case eval.
• It reaches the cloud only as an optional fallback, with personal information scrubbed first.
• 317 unit tests run in CI.
• Swift 6 LiteRT Gemma ReAct

NeuroRecursion: agents that keep a frozen brain-computer-interface (BCI) decoder usable across days without new labels (Crusoe × Cerebral Valley hackathon, Oct 2026)

• Tested on real recordings from human BCI users (FALCON H1), over 7 held-out days.
• With no labels, the agents beat the frozen decoder on 6 of 7 days: mean velocity R² −0.016 vs −0.062.
• Few-shot recalibration, which does need labels, still does better at +0.106.
• Python Nemotron Crusoe Inference

Dual LLM Router: a cascade router that decides whether each query goes to GPT-4o-mini or GPT-4o

• The router scores 92.6% routing accuracy.
• My follow-up ablation found that a one-line word-count rule gets 89.7%, and the confidence intervals overlap.
• Almost all of the router's gain comes from length features. The cause is a benchmark artifact, documented in FINDINGS.md.
• scikit-learn OpenAI

meeting-bot: a local-first meeting notetaker for macOS

• Whisper transcribes on-device, with speaker attribution, so no audio leaves the Mac.
• Ships as a signed release with a SwiftUI app, a CLI and a Chrome extension.
• Whisper SwiftUI Python

On-device apps

AI tools

Hackathons

• AI Developer, Trominosoft (2025): LLM orchestration and retrieval services on GCP and Kubernetes
• Graduate TA, Northeastern (Computer Vision, Generative AI)
• AI Intern, Infosys Springboard (2024): quantized 7B-model serving with vLLM
• Publication: co-author, Real-Time Human Pose Estimation Using MediaPipe, IEEE AISP 2024

Tools I use most: PyTorch · Hugging Face · LoRA/PEFT · vLLM · MLX · LiteRT · Core ML · Swift · Kotlin · Python