David Harwath
University of Texas at Austin, TX·Computer Science·Early-career
Speech recognition now powers digital assistants and automatic transcription, but building these systems normally requires enormous collections of recorded speech that humans have typed out as text. For most of the world's roughly 7,000 languages and their many dialects, no such transcribed datasets exist, so speech technology simply does not serve those speakers.
- Fields
Computer Science
Engineering
- Topics
speech recognition
spoken language processing
self-supervised learning
- Grants
- 1 known grant, through Jan 2028
What they work on
- Develop attention-based neural network segmentation layers that discover units in raw speech audio, applied hierarchically to recover structure at multiple levels such as phones, words and phrases.
- Add self-prediction layers and training objectives so higher, word-like layers predict the tokenization produced by lower, sub-word layers, allowing a model to learn a pronunciation lexicon without text.
- Build speech recognition systems without transcribed speech by treating automatically discovered speech units as "pseudo-text" in place of conventional transcripts.
- Apply the learned representations to textless speech-to-speech translation and to textless speech generation for dialog and image captioning, using self-supervised and multimodal learning.
More about the research
Children, however, learn to understand and speak long before they can read, which suggests speech can be learned without text at all.
Why it matters
Removing the need for transcribed speech could make speech recognition, translation and voice interfaces available for the thousands of languages and dialects that lack large annotated datasets, lowering cost and extending these tools to many more speakers than are served today.
Known grants
From NIH and NSF research grants. Labs often have other funding we can't see.
Closest labs
Jonathan Amith
Gettysburg College, PA·Computer Science
Build POWSM, a foundation speech model that transcribes any language as sequences of consonants and vowels, and prompt it with vector representations of sound correspondences so it can recognize low-resource varieties as transformed versions of well-resourced ones.
Fei Liu
Emory University, GA·Computer Science·Early-career
Uncover the hidden structure of informal, verbose transcripts of spontaneous speech, using deep neural networks and linguistic structure prediction to infer how topics and segments of a conversation relate to one another.
Michael Brown
Villanova University, PA·Psychology·Early-career
Identify which sound information listeners use to tell spoken words apart by recording brain responses in the first few hundred milliseconds after hearing speech.
The school
University of Texas at Austin: 49 labs on PI Pages, $1.1B a year on research, 871 research doctorates a year.
Reach out
A start for your email. Fill in the brackets.