Research
We build small language models that run where the people are. Efficient, sovereign, and open enough to inspect.
Our first model
tamil-lm-2b is Qwen/Qwen3.5-2B-Base adapted to Tamil by continued pretraining, a Tamil tokenizer extension and instruction tuning. It is a 2B-parameter model for everyday Tamil, Tanglish (romanised and code-mixed Tamil) and Tamil-English translation, built to run offline on a phone. It is weak at English reasoning and does not know recent events.
- Release
- 0.1.0-preview, September 2026
- Size
- 2B parameters
- Base model
- Qwen/Qwen3.5-2B-Base (Apache License 2.0)
- Languages
- Tamil, Tanglish, English
- Licence
- Apache License 2.0
- Trained on
- a single 48 GB GPU, bf16 weights
- Repositories
- tamil-lm-2b-base (continued pretraining only) and tamil-lm-2b-instruct (instruction tuned)
The model runs offline in our Android app: Timegravity Tamil for Android.
Read the work
- Model cardevery number on this page, with its method
- Code, evaluation harness and training scriptsApache 2.0
- tamil-lm-2b-instruct on Hugging Facethe instruction tuned weights
- tamil-lm-2b-base on Hugging Facecontinued pretraining only
- tamil-lm-2b-gguf on Hugging Facequantised files and knowledge packs
- Every experiment runexperiments.tsv
A tokenizer that fits Tamil
We added 22,222 Tamil pieces to the base model's vocabulary, for 270,299 tokens in all. On 10,000 Tamil Wikipedia sentences, Tamil text costs 1.92 tokens per word instead of 6.49. A 4,096-token context now holds about 2,100 Tamil words instead of about 630.
How it was trained
- Embedding warmup. The new Tamil embedding rows were initialised from the mean of their old subword rows and trained alone for 200M tokens.
- Main run. 1.47B tokens with LoRA adapters and the new embedding rows. Mix by tokens: Tamil web 60%, literature 15%, Tamil-English parallel text 10%, Tanglish 10%, English replay 5%. 7,206 documents that overlapped benchmark test items were removed before sharding.
- Top-up. 150M further tokens with more English and arithmetic. The adapters were then merged; this is the released base model.
- Instruction tuning. LoRA fine-tuning on the released base model, loss on assistant tokens only. The training file has 50,857 rows: 46,158 benign completions and 4,699 refusal or abstention rows.
What the experiments found
Before the main run, 21 short experiments on a 100M-token subset chose the recipe. The tokenizer extension cut Tamil bits per character by 14%. Learning-rate changes, full MLP fine-tuning, other sequence lengths, the Lion optimizer, more synthetic Tanglish and larger literature or replay shares did not help. A literature gain seen in the short experiments did not survive full-scale training.
Results
Every number comes from the evaluation harness in the GitHub repository (eval/suite.py). The dev split is used for every tuning decision; the test split runs exactly once per model stage, and these are test results. No benchmark data is trained on, and a 13-gram overlap check removed overlapping documents before training.
| Benchmark | Metric | Test items | Qwen3.5-2B-Base, before any Tamil training | tamil-lm-2b-base | tamil-lm-2b-instruct |
|---|---|---|---|---|---|
| Tamil understanding | |||||
belebele_ta | accuracy | 720 | 0.218 | 0.264 | 0.272 |
include_ta | accuracy | 448 | 0.234 | 0.283 | 0.319 |
indiccopa_ta | accuracy | 400 | 0.477 | 0.568 | 0.568 |
indicmmlu_pro_taNo stated licence; reported for reference only. | accuracy | 12,032 | 0.117 | 0.121 | 0.120 |
indicqa_taNot contamination-free for this model. | F1 | 1,022 | 0.042 | 0.141 | 0.167 |
indicsentiment_ta | accuracy | 998 | 0.481 | 0.580 | 0.511 |
indicxnli_ta | accuracy | 5,010 | 0.333 | 0.348 | 0.340 |
milu_ta | accuracy | 6,372 | 0.267 | 0.281 | 0.292 |
xlsum_ta | ROUGE-L | 2,027 | 0.359 | 0.292 | 0.305 |
| Translation | |||||
flores_en_ta | chrF++ | 1,012 | 3.267 | 43.910 | 47.925 |
flores_en_ta | BLEU | 1,012 | 0.079 | 10.413 | 11.241 |
flores_ta_en | chrF++ | 1,012 | 11.881 | 54.022 | 51.521 |
flores_ta_en | BLEU | 1,012 | 1.491 | 28.863 | 26.949 |
in22gen_en_ta | chrF++ | 820 | 3.913 | 38.892 | 41.117 |
in22gen_en_ta | BLEU | 820 | 0.067 | 5.603 | 6.125 |
in22gen_ta_en | chrF++ | 820 | 11.625 | 51.961 | 49.497 |
in22gen_ta_en | BLEU | 820 | 2.037 | 26.135 | 25.029 |
| Tanglish text (bits per character, lower is better) | |||||
tanglish_heldout | bpc | 2,740 | 3.253 | 3.040 | 3.024 |
| English retention | |||||
mmlu_en | accuracy | 400 | 0.552 | 0.323 | 0.338 |
gsm8k_en | accuracy | 160 | 0.637 | 0.050 | 0.069 |
English regressed: MMLU fell from 0.552 to 0.338 and GSM8K from 0.637 to 0.069. This is the cost of the Tamil training. The serving stack answers arithmetic through a calculator route, not the weights.
IndicQA is not contamination-free for this model: 1022 of 1022 test contexts overlap the training text, because the contexts are Tamil Wikipedia passages and Tamil Wikipedia is in the pretraining data.
Literature recall: not met
The in-weights literature source-identification gate was NOT met: on the 190-item held-out probe (4-way choice, chance 0.25) the released base model scores 0.305. Verbatim quotations of Thirukkural and the other canon works come through the serving stack, which looks the requested unit up in a structured knowledge base and quotes it byte-exactly; the bare weights will paraphrase or misquote. A 74-step checkpoint reached 0.468 on the same gate, but that level did not survive full-scale training in two independent runs (1.26B and 1.47B tokens).
Safety
Every safety gate has two numbers: the bare weights, and the reference serving stack (serve.py: an input and output guard, abstention on office-holder and election questions, self-harm routing to verified helplines, verbatim literature quotes). The release gates are met by the serving stack. The bare weights are not.
| Gate | Bare weights | Serving stack |
|---|---|---|
| Political safety, 100 prompts | FAIL, 89 of 100 | PASS, 100 of 100 |
| Red-team, 366 prompts | FAIL, 86 unsafe | PASS, 0 unsafe |
| Benign prompts refused | 2% | 5% |
Child and teen red-team through the serving path, 200 prompts in Tamil, Tanglish and English: 0 failures. The Android app runs with the family-safe filter on by default.
The bare weights carry only light safety tuning. Anyone deploying them without the serving stack must add their own moderation and grounding.
Comparison with other open models
in testing The comparison with other open models under 8B is still running. The model card makes no comparison claim until every model has finished, and neither does this page.
Limitations
- The weights do not recall verbatim literature or reliably identify sources. Exact quotes come only from the knowledge base through the serving stack.
- English reasoning regressed badly relative to the base model.
- The weights state outdated facts as current. Training data predates the 2026 Tamil Nadu election and other 2025-2026 events.
- The bare weights fail the political-safety and red-team gates on their own.
- Hosted frontier models are well ahead of this model on reasoning and on most translation directions.
Licence and data
The weights of tamil-lm-2b-base and tamil-lm-2b-instruct are released under the Apache License 2.0, matching the base model Qwen/Qwen3.5-2B-Base (Alibaba Cloud, Apache 2.0). Knowledge packs built from Wikipedia, Wiktionary and Wikibooks are licensed CC BY-SA 4.0 as to their content. Every training source, with its licence, is listed in the data card and the licence register. This is an independent research project; no legal review has been performed on data licensing.
Report a problem
Report unsafe outputs, data or licensing concerns, or errors to contact@timegravity.ai. Include the prompt, the response and the model version.
Every number on this page is from the model card, read on 15 September 2026.
Research at Timegravity Labs Private Limited, Coimbatore, Tamil Nadu, India.