Jonathan Amith
Gettysburg College, PA·Computer Science
About 7,000 languages are spoken today, and the number is falling quickly. When a language falls out of use, its speakers lose cultural heritage, and science loses data about the range of human language.
- Fields
Computer Science
Social Science
Biology
- Topics
endangered languages
language documentation
documentary linguistics
- Grants
- 5 known grants, through Aug 2029
What they work on
- Build POWSM, a foundation speech model that transcribes any language as sequences of consonants and vowels, and prompt it with vector representations of sound correspondences so it can recognize low-resource varieties as transformed versions of well-resourced ones.
- Use weighted finite-state transducers that encode linguists' knowledge of sound changes to generate synthetic data for low-resource varieties, and train a language model on these data to decode POWSM output.
- Learn sound correspondences between language varieties automatically from transcribed audio, using a new form of unsupervised bilingual lexicon induction that draws on both text and audio, to support basic speech-to-speech translation.
- Build a single end-to-end neural network that converts recorded speech in Nahuatl, Mixtec, Amuzgo and Iñupiaq directly into a transcription, morpheme splits, underlying forms and glosses.
- Build linguistic knowledge into neural networks using differentiable weighted finite-state transducers and iterative self-training, and extend glossing schemes such as UniMorph to cover derivation, reduplication and noun incorporation.
- Test whether transfer learning can extend an end-to-end speech recognizer trained on more than 50 hours of one endangered language to related languages with only 10 hours of transcribed audio.
- Develop accurate speech recognition for a morphologically complex agglutinative language that cuts human transcription effort, and produce a time-coded transcribed corpus for five related languages.
- Document how four Indigenous language communities name, classify and use arthropods by pairing specimens collected by biologists and native speakers with native-language audio, macro photographs and CO1 gene sequencing.
- Turn transcribed and translated arthropod recordings into illustrated field guides, an image bank and museum exhibits, and compare the four communities to reveal patterns of historical contact.
- Build an open-source online database that aligns Indigenous names for plants, animals and material culture, with commentary from Indigenous scholars, botanists and biologists.
More about the research
Speakers often also lose detailed knowledge of local plants and animals. Speech technologies such as automatic speech recognition (ASR), software that turns speech into text, work well only for major languages with large training datasets. These five projects combine documentary linguistics, the recording, transcribing and translating of speech, with machine learning and biology. Much of the work involves Indigenous languages such as Nahuatl, Mixtec and Amuzgo. Two threads run through the projects. One is building AI tools that reduce the human effort needed to document low-resource languages. The other is recording Indigenous knowledge of plants, animals and material culture.
Why it matters
These tools could make speech recognition usable for endangered languages and for regional varieties of major languages. That could cut the hours of expert work needed to transcribe recordings and help more people use voice-powered services. The recorded corpora, field guides, exhibits and databases could preserve language and knowledge of local plants and animals for Indigenous communities, and make it available to linguists, biologists and botanists before it is lost.
Known grants
From NIH and NSF research grants. Labs often have other funding we can't see.
Collaborative Research: RI: Medium: Using Systematic Relationships and Phonetic Speech Foundation Models for Universal Speech-to-Text across Varieties of Languages
NSF · Sep 2025 – Aug 2029
Interdisciplinary approaches to comparative ethnoentomology and linguistics in endangered language communities
NSF · Jan 2025 – Dec 2027
Collaborative Research: RI: Medium: From Acoustic Signal to Morphosyntactic Analysis in One End-to-End Neural System
NSF · Aug 2022 – Jul 2027
Closest labs
David Harwath
University of Texas at Austin, TX·Computer Science·Early-career
Develop attention-based neural network segmentation layers that discover units in raw speech audio, applied hierarchically to recover structure at multiple levels such as phones, words and phrases.
Frederik Hartmann
University of North Texas, TX·Social Science
Build artificial neural networks that learn to reconstruct proto-forms, the ancestor words of a language family, from their descendant words, using large datasets spanning many language families.
Carol Espy-Wilson
University of Maryland, MD·Engineering
Extend a speech inversion tool, which already estimates tongue and lip movements from audio, to recover nasality and voicing by training neural networks on acoustic features.
The school
Gettysburg College: 1 labs on PI Pages.
Reach out
A start for your email. Fill in the brackets.