MODEL 03 · Open models for Indian languages
Nandi Carries Every Complaint to the Right Desk: Open-Source AI for Indian Languages
Nandi, an Apache-2.0 model that routes complaints in 13 Indian languages to one of 53 regulator-based categories, and what real posts revealed.
TL;DR
Nandi is a free, local, 560M-parameter model that routes complaints in 13 Indian languages to one of 53 categories drawn from RBI, IRDAI, CPGRAMS and consumer-helpline lists. It hit 92.9% on a synthetic test but 62.5% (81.3% top-3) on real public posts, slightly behind a 35B LLM, and its urgency head ships as experimental.
- 53categories
- 13languages
- 62.5%accuracy, real posts
- 81.3%top-3, real posts
- 560Mparameters
Summary
This article introduces Nandi (arunp0/nandi-grievance-indic), a 560M-parameter, Apache-2.0 model that reads a complaint in Hindi, Hinglish, English or ten more Indian languages and suggests where it should go. It outputs one of 53 categories, its three best guesses with probabilities, and a P1/P2/P3 urgency hint. The categories are borrowed from published complaint lists: RBI for banks, IRDAI for insurers, CPGRAMS for central government services, plus telecom and the National Consumer Helpline. Named after the bull at Shiva's gate who carries wishes onward, it only routes; it never judges a complaint.
Training used no real complaints at all. Gemma 4 and Qwen3.6 wrote and cross-checked 116,199 synthetic messages, keeping only texts where the labellers agreed.
The heart of the piece is a gap. On a synthetic test written by a different model, Nandi reached 92.9% accuracy and beat a 35B LLM. On 443 real public posts gathered after the model was frozen, it fell to 62.5%, with the right answer in its top three 81.3% of the time; Qwen3.6 zero-shot reached 66.4% and handled rare categories far better. Yama, a generalist of similar size, managed 0.366.
Urgency missed a rule fixed in advance: it finds roughly one real urgent post in four, so it ships as experimental. Two retrains that cut false alarms began discarding genuine complaints, so the earlier v5b was kept.
This page is a short summary written for this site. Every figure, table, code sample and reference is in the original article on Towards AI.
Questions this article answers
How do you automatically classify customer complaints written in Hindi or Hinglish?
One option is a fine-tuned encoder such as Nandi, an Apache-2.0 model built on XLM-RoBERTa-large. You pass it the raw message and it returns a category out of 53, the three likeliest categories with probabilities, and a tentative urgency level. It handles Hinglish, Hindi, English and ten other Indian languages in native or Roman script, and it runs locally on a CPU or GPU without calling any API.
Which complaint categories does Nandi use, and where do they come from?
The categories follow lists already published by Indian bodies: 14 banking and payments desks from RBI's ombudsman complaint form, 12 insurance types from IRDAI, 17 central government services from CPGRAMS, 4 telecom groups and 5 consumer sectors largely from the National Consumer Helpline, plus one bucket for messages that are not complaints. The author notes that the mapping is his own, not an official one.
How accurate is Nandi on real complaints compared with an LLM?
On 443 real app reviews and forum posts it never saw, Nandi chose the correct category 62.5% of the time and ranked it in its top three 81.3% of the time. Qwen3.6-35B-A3B used zero-shot scored 66.4% and was much stronger on rare categories, with macro-F1 of 0.62 against 0.43. Nandi's advantage is size and privacy: about 60 times smaller, local, and no text leaves your servers.
Why does a model trained on synthetic data score lower on real text?
Synthetic text shares the definitions, prompt layout and tidy detail of its training data, even when another model writes the test. Nandi scored 92.9% on such a test but 62.5% on genuine posts, a drop of about 30 points. Real complaints tend to be shorter, vaguer and messier, and urgent ones rarely state their timing as clearly as generated scenarios did. The lesson is to budget for a blind real-world test.
Can Nandi detect urgent complaints reliably?
Not yet. The author set a pass bar for urgent recall before collecting the real test, and Nandi fell well short of it, catching about one urgent post in four. Leaving out loan-app harassment, the easiest kind, recall drops to roughly 0.10. The urgency head is therefore published as experimental: treat its output as a hint that a person checks, not as an automatic triage decision.
What are Nandi's main limitations?
Its confidence scores are not calibrated, and it often reports 0.99 on wrong answers, so rely on the top three as suggestions for a human. Around one in five real non-complaints still receives a category, rare categories are weak, and real-text testing covers mostly English, Hindi and Hinglish. All real-text labels were reviewed by language models rather than people, and accuracy on redacted input has not been measured.