Zhuoran Yang
Yale University, CT·Mathematics & Statistics
Large language models such as GPT-4 can take on new tasks without any retraining. A user simply shows a few examples in the prompt, or asks the model to reason step by step.
- Fields
Mathematics & Statistics
Computer Science
- Topics
large language models
in-context learning
chain-of-thought prompting
- Grants
- 1 known grant, through Jul 2027
What they work on
- Test whether large language models carry out in-context learning and chain-of-thought prompting as implicit Bayesian inference, and analyze how transformer attention, the mechanism that weighs relevant words, could approximate these estimators.
- Develop algorithms and theory that quantify the statistical errors of prompting-based estimators, separating error introduced during pre-training on text corpora from error introduced at the prompting stage.
- Apply the statistical framework to robotic control and autonomous driving, formulating principled methods that use pre-trained language models for complex sequential decision-making.
More about the research
Why this works so well is still poorly understood. This project aims to build a statistical theory of these prompting methods, so that people can use language models in a principled way rather than by trial and error.
Why it matters
Language models are increasingly used to guide decisions in robotics and transportation, yet prompting them remains largely guesswork. A statistical account of when and why in-context learning and chain-of-thought prompting succeed could make prompt engineering more reliable, and could inform better-grounded uses of pre-trained models in robotic control and autonomous driving.
Known grants
From NIH and NSF research grants. Labs often have other funding we can't see.
Closest labs
Craig Newgard
Oregon Health & Science University, OR·Computer Science
Develop a reinforcement learning framework in which two agents select features and a third selects an operation, with a shared value network, to generate new features in a traceable way.
Minge Xie
Rutgers University-New Brunswick, NJ·Mathematics & Statistics
Develop PCSI (Predictability-Computability-Stability Inference) for binary and multi-class classification, screening many algorithms by predictive performance and using bootstrap resampling to improve uncertainty estimates, robustness and accuracy.
Chi Jin
Princeton University, NJ·Computer Science·Early-career
Identify the general structures of function classes, such as those represented by neural networks, under which model-free reinforcement learning is tractable, and design provably efficient algorithms that also handle model misspecification.
The school
Yale University: 73 labs on PI Pages, $1.5B a year on research, 438 research doctorates a year.
Reach out
A start for your email. Fill in the brackets.