Junfeng Yang
Columbia University, NY·Computer Science
Developers often download pre-trained AI models from open online repositories, much as they download software libraries. Just as software can hide viruses or backdoors, these models can be tampered with.
- Fields
Computer Science
Engineering
- Topics
AI security
machine learning security
ML supply chain
- Grants
- 1 known grant, through Sep 2030
What they work on
- Detect model spoofing, where attackers upload malicious AI models under deceptive names, by developing anomaly detection based on naming conventions and model architectural signatures.
- Secure model deserialization, the loading of saved model files, by building automated least-privilege loading mechanisms and defining safe subsets of model-loading runtimes that block arbitrary code execution.
- Track how pre-trained models evolve and detect illicit modifications by building a model lineage graph that combines static and dynamic program analysis.
More about the research
A malicious model slipped into a scientific workflow or a business system could cause security breaches or silent errors. This collaborative project aims to give users ways to check whether a pre-trained model is trustworthy before using it. It combines software engineering principles with machine learning (ML) techniques to secure the whole 'supply chain': how models are chosen, loaded and managed.
Why it matters
Researchers, companies and government agencies increasingly build on pre-trained AI models shared online. Tools that detect spoofed models, block code execution during model loading and trace illicit modifications could help these users avoid installing tampered models. This would make AI-enabled systems more resilient to attack.
Known grants
From NIH and NSF research grants. Labs often have other funding we can't see.
Closest labs
Xing Gao
University of Delaware, DE·Computer Science·Early-career
Build a security testing system that combines dynamic execution and static analysis to examine every critical component of continuous software development pipelines and how each interacts with the people and organizations involved.
Alexandros Kapravelos
North Carolina State University, NC·Computer Science·Early-career
Define the security properties that continuous integration workflows should satisfy and design a Workflow Intermediate Representation, a common format that workflows from any CI platform can be translated into for uniform analysis.
Yueqi Chen
University of Colorado Boulder, CO·Computer Science
Refactor the Community Earth System Model (CESM) codebase by decoupling its computational and control logic and rewriting the control layer in Python, so that modern security practices can be integrated.
The school
Columbia University: 89 labs on PI Pages, $1.4B a year on research, 625 research doctorates a year.
Reach out
A start for your email. Fill in the brackets.