Carl Vondrick
Columbia University, NY·Computer Science·Early-career
Cameras are everywhere, from satellites and drones to phones and security cameras. Yet computers still struggle to reason about what these images show, especially without large amounts of human-labeled data.
- Fields
Computer Science
Engineering
Earth & Environmental Science
- Topics
computer vision
machine learning
artificial intelligence
- Grants
- 3 known grants, through Jul 2028
What they work on
- Build open-world recognition models for satellite, aerial and ground imagery from vision-language models without language annotations, using cross-modal contrastive learning and reasoning about change over time.
- Develop methods that turn natural-language questions into executable programs that solve visual tasks step by step, incorporating uncertainty, learning from feedback and adaptive computation.
- Create an AI framework that combines satellite, drone and internet images with self-improving large language models to generate interpretable hypotheses about environmental change, such as invasive species spread.
- Validate the planet-scale environmental framework on detecting submerged aquatic vegetation, which is difficult to see because of waves, murky water and weak signals through water.
- Learn to track hidden objects in video by using how sound travels through objects, combining acoustics with geometry and motion in unlabeled video.
- Represent videos as continuous functions in 3D space and use scene geometry to transfer learning between camera views, building spatial memory models for long videos of large spaces.
More about the research
These projects develop computer vision methods that learn from abundant unlabeled images and video. One project helps machines sense the space around them when visibility is poor. Two projects analyze satellite, aerial and ground images of the whole planet, so that scientists can ask questions in plain language and get answers they can interpret.
Why it matters
These methods could help scientists without machine learning expertise pull answers from vast satellite and drone image collections. Possible uses include tracking invasive species, ecosystem change, agriculture, climate and public health, problems that hit under-resourced communities hardest. Better tracking of hidden objects in video could also make autonomous vehicles, robots and security cameras more reliable in poor visibility.
Known grants
From NIH and NSF research grants. Labs often have other funding we can't see.
Collaborative Research: RI: Medium: Programmatic Foundation Models for Visual Analysis on a Planetary Scale
NSF · Aug 2024 – Jul 2028
Collaborative Research: ACED: Planet-scale AI for accelerating environmental science - Invasive species and beyond
NSF · Jun 2025 – Nov 2027
CAREER: Spatial Awareness for Machine Perception
NSF · Sep 2021 – Aug 2027
Closest labs
Bharath Hariharan
Cornell University, NY·Computer Science·Early-career
Build recognition systems that learn difficult visual concepts from as few as 1–2 labeled images and about 1,000 unlabeled ones, by drawing on a memory of past visual tasks across domains such as microscopy.
Noah Snavely
Cornell University, NY·Computer Science
Develop compositional implicit neural scene representations that pair a network's ability to capture fine detail with physical models of scene structure, such as light fields.
Xiaolong Wang
University of California-San Diego, CA·Computer Science·Early-career
Recognize objects in videos of everyday environments without a fixed category list, training with text supervision and self-supervision instead of detailed labels.
The school
Columbia University: 89 labs on PI Pages, $1.4B a year on research, 625 research doctorates a year.
Reach out
A start for your email. Fill in the brackets.