Tag: machine learning

A compact, open-source question generator for 25 languages — and a data factory for building better retrieval systems.

---

Every retrieval system, quiz app, and search engine shares a hidden hunger: it needs questions. Questions to train retrievers, questions to test rerankers, questions to turn static content into interactive learning. For English, that data exists in abundance. For most of the world’s languages, it barely exists at all.

Today we’re releasing mist-qg-1.5b, a compact multilingual question generation model, open-sourced on Hugging Face under Apache 2.0. Give it a passage in any of 25 languages and it returns natural, search-style questions that the passage directly answers — the kind of thing a real user would actually type into a search box.

25 languages, one small model

Coverage spans the languages your users actually speak:

• European: English, French, German, Spanish, Portuguese, Italian, Dutch, Russian, Polish
• Asian & Middle Eastern: Hindi, Japanese, Korean, Vietnamese, Indonesian, Turkish
• African: Yoruba, Igbo, Hausa, Swahili, Amharic, Zulu, Xhosa, Shona, Somali, Afrikaans

At ~1.5 billion parameters (fine-tuned from Qwen2.5-1.5B-Instruct), it runs comfortably on a single modest GPU — no cluster required.

Two jobs, one model

mist-qg-1.5b is deliberately dual-use:

1. A question-generation service. Wrap it behind an endpoint and you have instant question generation for education platforms, quiz builders, FAQ tooling, and content pipelines. The model returns strict JSON ({"questions": [...]}), so outputs slot straight into production systems — and for guaranteed structural validity, serve it with vLLM guided JSON decoding.

2. A data factory. This is the quieter superpower. Every question it generates, paired with its source passage, is a (query, positive) training pair — exactly what you need to fine-tune retrievers and rerankers for RAG. For languages where such data has never existed at scale, this model can mint it.

The interesting part: round-trip verification

Question generation models often produce questions that look plausible but aren’t actually answerable from the source. We attacked this at the data level.

Training data was distilled from the Aya collection using Qwen2.5-32B-Instruct — but every candidate question had to pass a test before entering the training set: embedded alongside its source passage and a pool of distractors, the question had to retrieve its own passage at rank one. Fail the round trip, and the question was discarded. Roughly 150k questions across ~50k passages survived this filter.

We evaluate the same way. On 625 held-out passages never seen during training, the model achieves an overall round-trip keep-rate of 0.955 — meaning 95.5% of generated questions successfully retrieve their source passage from a distractor pool (scored with the larger Qwen3-Embedding-4B, a stricter judge than the one used in training).

Honest numbers, including the weak spots

Six languages — English, Spanish, Portuguese, Turkish, Indonesian, and Afrikaans — scored a perfect 1.000 on the held-out eval, and most high-resource languages cluster between 0.95 and 1.00.

We want to be equally upfront about the other end: Amharic, Somali, and Shona sit at 0.58–0.70, and outputs in those three languages should be treated with lower confidence. Our analysis suggests this gap tracks fine-tuning data volume (~2,000 source passages per language) rather than a hard model-capacity ceiling, which makes it a natural target for the next data-scaling pass. We’re publishing these numbers because per-language transparency is the only way users can make informed decisions — and because we intend to close the gap.

Try it in one minute

pip install -U transformers

import json, torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "olaverse/mist-qg-1.5b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")

passage = "Tides are caused by the gravitational pull of the moon and, to a lesser extent, the sun, acting on Earth's oceans."

messages = [
    {"role": "system", "content": "You write search-style questions that a passage directly answers."},
    {"role": "user", "content": f'''You are given a passage. Write 3 questions that the passage directly answers.

Rules:
- Each question must be answerable using ONLY this passage.
- Vary the type: factual, yes/no, and a comparison or "why/how".
- Natural, like a real user search query. Do NOT write "according to the passage".
- Write the questions in English.

Return ONLY JSON: {{"questions": ["...", "...", "..."]}}

Passage: {passage}'''},
]

input_ids = tokenizer.apply_chat_template(messages, tokenize=True,
    add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(input_ids, max_new_tokens=200, do_sample=False,
    pad_token_id=tokenizer.pad_token_id or tokenizer.eos_token_id)
text = tokenizer.decode(out[0][input_ids.shape[1]:], skip_special_tokens=True)
print(json.loads(text[text.index("{"): text.rindex("}") + 1])["questions"])
# ["What causes ocean tides?", "Does the sun affect tides?",
#  "Which has a bigger effect on tides, the moon or the sun?"]

Or skip the setup entirely and try the live demo Space.

Links

• Model: https://huggingface.co/olaverse/mist-qg-1.5b
• Training dataset: https://huggingface.co/datasets/olaverse/qg-passages-multi
• Demo: https://huggingface.co/spaces/olaverse/Mist_Question_Gen
• GitHub: https://github.com/Olaverse-Labs

If you use mist-qg-1.5b to build something — a quiz app, a retriever for a language nobody else supports, anything — we’d genuinely love to hear about it. Open a discussion on the model page or reach us on Twitter/X.

---

Olaverse Lab builds open-source AI with first-class support for underserved languages. Follow us on Hugging Face and LinkedIn.