A UF Data Science Research Lab and Data Studio Lab collaboration placed second in all five language tracks — earning the highest overall score through consistent performance across Guaraní, Yucatec Maya, Nahuatl, Wixarika, and Bribri.
Winning the Overall Competition
Team Gators, a collaboration between the UF Data Science Research Lab and the UF Data Studio Lab, won the overall AmericasNLP 2026 Shared Task on Cultural Image Captioning for Indigenous Languages, co-located with ACL 2026 in San Diego. The task asked participating systems to generate captions for culturally situated images in five Indigenous languages: Guaraní, Yucatec Maya, Orizaba Nahuatl, Wixarika, and Bribri.
Evaluation used a two-stage process: systems were first ranked by chrF++, a character n-gram metric standard in low-resource translation, and top systems then advanced to human evaluation on a 1–5 scale. The official overall winner was determined by summing rank points across all five languages — rewarding consistency rather than peak performance in a single track. Team Gators placed second in every language track, accumulating 20 points against 14 for NAIST and 13 each for Mila and yaduha.
System Design: A Two-Stage Cascade
Rather than attempting to build a single end-to-end image-to-Indigenous-language model — which would require data that does not exist — Team Gators decomposed the problem into two stages.
Stage 1 uses Qwen2.5-VL-72B-Instruct (4-bit quantized) on UF HiPerGator B200 GPUs to generate a Spanish description of each image. Spanish serves as a high-resource intermediate language. The caption prompt was designed to produce culturally specific vocabulary — naming objects like guampa, kambuchi, poyvi, and chipa by their local names rather than generic Spanish equivalents, since these terms match the vocabulary of the parallel training corpus and drive precise retrieval.
Stage 2 uses Gemini 2.5 Flash (temperature 0.0) with BM25 retrieval-augmented in-context learning. For each image, the prompt combines: (a) gold development-set caption pairs as demonstrations (leave-one-out, 20–49 examples); and (b) BM25-retrieved parallel training examples (0–80 examples). Parameters r (retrieval size) and d (dev examples) were tuned independently for each language.
Key Findings
The base AmericasNLP 2023 Guaraní corpus contains ~26,000 sentence pairs. Earlier DSR Lab work augmented this to 53,000 pairs through NLLB back-translation. Removing the augmentation reveals a dramatic effect: without synthetic data, all configurations cluster near 21 chrF++ regardless of r or d. With augmentation, the best configuration (r=40, d=49) reaches 51.34 — a +30 point gain. The synthetic data improves retrieval quality, not the translation model itself, by adding caption-style Spanish vocabulary to the BM25 index.
Retrieval must be tuned per language
The optimal retrieval configuration varied substantially across languages. For Guaraní (large augmented corpus), r=40 retrieved examples were highly beneficial. For Yucatec Maya, which had almost no parallel training data, r=0 was optimal — any retrieval from an out-of-domain corpus added noise. The principle: retrieval helps in proportion to corpus relevance, not corpus existence.
Unicode encoding was silently suppressing Bribri scores
Bribri reference files use NFD Unicode encoding (decomposed diacritics) while our pipeline generated NFC text (precomposed). Since chrF++ is character-level, visually identical characters in different encodings score as mismatches. After NFD-normalizing predictions, the Bribri dev score jumped from 11.50 to 17.02 — points lost entirely to a file encoding artifact. Combined with a stronger language-specific system prompt, the final Bribri dev score reached 19.99.
Links and Code
Team Repository: github.com/dhawan98/AmericasNLP2026-Gators-Submission
fficial Results: github.com/AmericasNLP/americasnlp2026/tree/main/results
Task Page: americasnlp.org