Publications
2024
Viewpoint Textual Inversion: Unleashing Novel View Synthesis with Pretrained 2D Diffusion Models
James Burgess, Kuan-Chieh Wang, Serena Yeung
ECCV 2024
[pdf] [project page] [code]
Depth-guided NeRF Training via Earth Mover's Distance
Anita Rau, Josiah Aklilu, Chris Holsinger, Serena Yeung-Levy
ECCV 2024
[pdf] [project page]
VideoAgent: Long-form Video Understanding with Large Language Model as Agent
Xiaohan Wang*, Yuhui Zhang*, Orr Zohar, Serena Yeung-Levy
ECCV 2024
[pdf] [project page] [code]
Diffusion-HPC: Synthetic Data Generation for Human Mesh Recovery in Challenging Domains
Zhenzhen Weng, Laura Bravo-Sanchez, Serena Yeung
3DV 2024
[pdf] [project page] [code]
Describing Differences in Image Sets with Natural Language
Lisa Dunlap*, Yuhui Zhang*, Xiaohan Wang, Ruiqi Zhong, Trevor Darrell, Jacob Steinhardt, Joseph E. Gonzalez, Serena Yeung-Levy
CVPR 2024 (Oral)
[pdf] [project page] [code]
Connect, Collapse, Corrupt: Learning Cross-Modal Tasks with Uni-Modal Data
Yuhui Zhang*, Elaine Sui*, Serena Yeung-Levy
ICLR 2024
[pdf] [project page] [code]
Hyperbolic Deep Learning in Computer Vision: A Survey
Pascal Mettes, Mina Ghadimi Atigh, Martin Keller-Ressler, Jeffrey Gu, Serena Yeung
IJCV 2024
[pdf]
Artificial Intelligence Identifies Factors Associated with Blood Loss and Surgical Experience in Cholecystectomy
Josiah G. Aklilu, Min Woo Sun, Shelly Goel, Sebastiano Bartoletti, Anita Rau, Griffin Olsen, Kay S. Hung, Sophie L. Mintz, Vicki Luong, Arnold Milstein, Mark J. Ott, Robert Tibshirani, Jeffrey K. Jopling, Eric C. Sorenson, Dan E. Azagury, Serena Yeung-Levy
NEJM AI
[pdf]
Analyzing Surgical Technique in Diverse Open Surgical Videos With Multitask Machine Learning
Emmett D. Goodman, Krishna K. Patel, Yilun Zhang, William Locke, Chris J. Kennedy, Rohan Mehrotra, Stephen Ren, Melody Guan, Orr Zohar, Maren Downing, Hao Wei Chen, Jevin Z. Clark, Margaret T. Berrigan, Gabriel A. Brat, Serena Yeung-Levy
JAMA Surgery 2024
[pdf]
Orientation-invariant autoencoders learn robust representations for shape profiling of cells and organelles
James Burgess, Jeffrey J. Nirschl, Maria-Clara Zanellati, Alejandro Lozano, Sarah Cohen, Serena Yeung-Levy
Nature Communications 2024
[pdf] [code]
A Human Mesh-Centered Approach to Action Recognition in the Operating Room
Benjamin Liu; Gilles Soenens; Joshua Villarreal; Jeffrey Jopling; Isabelle Van Herzeele; Anita Rau; Serena Yeung-Levy
Artifical Intelligence Surgery 2024
[pdf]
MicroBench: A Vision-Language Benchmark for Microscopy Understanding
Alejandro Lozano, Jeffrey Nirschl, James Burgess, Sanket Gupte, Yuhui Zhang, Alyssa Unell, & Serena Yeung-Levy
Preprint 2024
[pdf]
CryoViT: Efficient segmentation of cryogenic electron tomograms with vision foundation models
Sanket R. Gupte, Cathy Hou, Gong-Her Wu, Jesús G. Galaz-Montoya, Wah Chiu, Serena Yeung-Levy
Preprint 2024
[pdf]
Just Shift It: Test-Time Prototype Shifting for Zero-Shot Generalization with Vision-Language Models
Elaine Sui, Xiaohan Wang, Serena Yeung-Levy
Preprint 2024
[pdf]
Multi-Human Mesh Recovery with Transformers
Zeyu Wang, Zhenzhen Weng, Serena Yeung-Levy
Preprint 2024
[pdf]
Why are Visually-Grounded Language Models Bad at Image Classification?
Yuhui Zhang, Alyssa Unell, Xiaohan Wang, Dhruba Ghosh, Yuchang Su, Ludwig Schmidt, Serena Yeung-Levy
Preprint 2024
[pdf] [project page] [code]
Video-STaR: Self-training enables video instruction tuning with any supervision
Orr Zohar, Xiaohan Wang, Yonatan Bitton, Idan Szpektor, & Serena Yeung-Levy
Preprint 2024
[pdf]
Continuous perception benchmark
Zeyu Wang, Zhenzhen Weng, Serena Yeung-Levy
Preprint 2024
[pdf]
2023
LOVM: Language-Only Vision Model Selection
Orr Zohar, Shih-Cheng Huang, Kuan-Chieh Wang, Serena Yeung
NeurIPS 2023 Datasets and Benchmarks
[pdf] [project page] [code]
INSPECT: A Multimodal Dataset for Pulmonary Embolism Diagnosis and Prognosis
Shih-Cheng Huang, Zepeng Huo, Ethan Steinberg, Chia-Chun Chiang, Matthew P. Lungren, Curtis P. Langlotz, Serena Yeung, Nigam H. Shah, Jason A. Fries
NeurIPS 2023 Datasets and Benchmarks
[pdf] [project page] [code]
Generalizable Neural Fields as Partially Observed Neural Processes
Jeffrey Gu, Kuan-Chieh Wang, Serena Yeung
ICCV 2023
[pdf] [project page] [code]
Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models
Yuhui Zhang, Michihiro Yasunaga, Zhengping Zhou, Jeff Z HaoChen, James Zou, Percy Liang, Serena Yeung
ACL 2023 Findings
[pdf] [project page] [code]
Self-supervised Learning for Medical Image Classification: A Systematic Review and Implementation Guidelines
Shih-Cheng Huang, Anuj Pareek, Malte Jensen, Matthew P Lungren, Serena Yeung, Akshay S Chaudhari
Nature Partner Journals (NPJ) Digital Medicine
[pdf]
Robust Semi-supervised Detection of Hands in Diverse Open Surgery Environments
Pranav Vaid, Serena Yeung, Anita Rau
Machine Learning for Healthcare (MLHC) 2023
[pdf] [code]
Video Pretraining Advances 3D Deep Learning on Chest CT Tasks
Alexander Ke, Shih-Cheng Huang, Chloe P O'Connell, Michal Klimont, Serena Yeung, Pranav Rajpurkar
Medical Imaging with Deep Learning (MIDL) 2023
[pdf] [code]
NeMo: 3D Neural Motion Fields from Multiple Video Instances of the Same Action
Kuan-Chieh Wang, Zhenzhen Weng, Maria Xenochristou, Joao Pedro Araujo, Jeffrey Gu, C. Karen Liu, Serena Yeung
CVPR 2023 (Highlight)
[pdf] [project page] [code]
PROB: Probabilistic Objectness for Open World Object Detection
Orr Zohar, Kuan-Chieh Wang, Serena Yeung
CVPR 2023
[pdf] [project page] [code]
Diagnosing and Rectifying Vision Models using Language
Yuhui Zhang, Jeff Z. HaoChen, Shih-Cheng Huang, Kuan-Chieh Wang, James Zou, Serena Yeung
ICLR 2023
[pdf] [project page] [code]
CryoET Reveals Organelle Phenotypes in Huntington Disease Patient iPSC-derived and Mouse Primary Neurons
Gong-Her Wu, Charlene Smith-Geater, Jesus G. Galaz-Montoya, Yingli Gu, Sanket R. Gupte, Ranen Aviner, Patrick G. Mitchell, Joy Hsu, Ricardo Miramontes, Keona Q. Wang, Nicolette R. Geller, Cathy Hou, Cristina Danita, Lydia-Marie Joubert, Michael F. Schmid, Serena Yeung, Judith Frydman, William Mobley, Chengbiao Wu, Leslie M. Thompson, Wah Chiu
Nature Communications
[pdf] [code]
Comparing Spatial Patterns of Marine Vessels between Vessel-tracking Data and Satellite Imagery
Shinnosuke Nakayama, WenXin Dong, Richard G. Correro, Elizabeth R. Selig, Colette C.C. Wabnitz, Trevor J. Hastie, Jim Leape, Serena Yeung, Fiorenza Micheli
Frontiers in Marine Science
[pdf] [code]
Global organelle profiling reveals subcellular localization and remodeling at proteome scale
Marco Y. Hein, Duo Peng, Verina Todorova, Frank McCarthy, Kibeom Kim, Chad Liu, Laura Savy, Camille Januel, Rodrigo Baltazar-Nunez, Sophie Bax, Shivanshi Vaid, Madhuri Vangipuram, Ivan E. Ivanov, Janie R. Byrum, Soorya Pradeep, Carlos G. Gonzalez, Yttria Aniseia, Eileen Wang, Joseph S. Creery, Aidan H. McMorrow, James Burgess, Sara Sunshine, Serena Yeung-Levy, Brian C. DeFelice, Shalin B. Mehta, Daniel N. Itzhak, Joshua E. Elias, Manuel D. Leonetti
Preprint 2023
[pdf]
ZeroAvatar: Zero-shot 3D Avatar Generation from a Single Image
Zhenzhen Weng, Zeyu Wang, Serena Yeung
Preprint 2023
[pdf]
Open World Object Detection in the Era of Foundation Models
Orr Zohar, Alejandro Lozano, Shelly Goel, Serena Yeung, Kuan-Chieh Wang
Preprint 2023
[pdf] [project page] [code]
2022
Prostate Cancer Therapy Personalization via Multi-modal Deep Learning on Randomized Phase III Clinical Trials
Andre J Esteva, Jean Feng, Douwe van der Wal, Shih-Cheng Huang, Jeffry Simko, Sandy E DeVries, Emmalyn Chen, Edward Schaeffer, Todd Morgan, Yilun Sun, Amirata A Ghorbani, Nikhil Naik, Dhruv G Nathawani, Richard D Socher, Jeff L Michalski, Mack R Roach III, Thomas G Pisansky, Jedidiah Monson, Farah A Naz, James Wallace, Michelle M Ferguson, Jean-Paul T Bahary, James E Zou, Matthew Lungren, Serena Y Yeung, Ashley Ross, NRG Prostate Cancer AI Consortium, Howard M Sandler, Phouc T Tran, Daniel E Spratt, Stephanie Pugh, Felix Y Feng, Osama Mohamad
Nature Partner Journals (NPJ) Digital Medicine
[pdf]
Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning
Weixin Liang*, Yuhui Zhang*, Yongchan Kwon*, Serena Yeung, James Zou
NeurIPS 2022
[pdf] [project page] [code]
Adapting Pre-trained Vision Transformers from 2D to 3D through Weight Inflation Improves Medical Image Segmentation
Yuhui Zhang, Shih-Cheng Huang, Zhengping Zhou, Matthew P. Lungren, Serena Yeung
ML4H 2022
[pdf] [code]
Domain Adaptive 3D Pose Augmentation for In-the-wild Human Mesh Recovery
Zhenzhen Weng, Kuan-Chieh Wang, Angjoo Kanazawa, Serena Yeung
3DV 2022
[pdf] [project page] [code]
Adaptation of Surgical Activity Recognition Models Across Operating Rooms
Ali Mottaghi, Aidean Sharghi, Serena Yeung, Omid Mohareri
MICCAI 2022
[pdf]
ALGES: Active Learning with Gradient Embeddings for Semantic Segmentation of Laparoscopic Surgical Images
Josiah Aklilu, Serena Yeung
MLHC 2022
[pdf] [code]
Developing Medical Imaging AI for Emerging Infectious Diseases
Shih-Cheng Huang, Akshay S. Chaudhari, Curtis P. Langlotz, Nigam Shah, Serena Yeung, Matthew P. Lungren
Nature Communications
[pdf]
Using AI and Computer Vision to Analyze Technical Proficiency in Robotic Surgery
Janice H Yang, Emmett D Goodman, Aaron J Dawes, John V Gahagan, Micaela M Esquivel, Cara A Liebert, Cindy Kin, Serena Yeung, Brooke H Gurland
Surgical Endoscopy
[pdf]
2021
Holistic 3D Human and Scene Mesh Estimation from Single View Images
Zhenzhen Weng, Serena Yeung
CVPR 2021
[pdf] [project page] [code]
Unsupervised Discovery of the Long-Tail in Instance Segmentation Using Hierarchical Self-Supervision
Zhenzhen Weng, Mehmet Giray Ogut, Shai Limonchik, Serena Yeung
CVPR 2021
[pdf] [project page] [code]
DARCNN: Domain Adaptive Region-based Convolutional Neural Network for Unsupervised Instance Segmentation in Biomedical Images
Joy Hsu, Wah Chiu, Serena Yeung
CVPR 2021
[pdf] [project page] [code]
Staying in Shape: Learning Invariant Shape Representations using Contrastive Learning
Jeffrey Gu, Serena Yeung
UAI 2021
[pdf] [project page] [code]
GLoRIA: A Multimodal Global-Local Representation Learning Framework for Label-efficient Medical Image Recognition
Shih-Cheng Huang*, Liyue Shen*, Matthew P. Lungren, Serena Yeung
ICCV 2021
[pdf] [code]
Capturing implicit hierarchical structure in 3D biomedical images with self-supervised hyperbolic representations
Joy Hsu*, Jeffrey Gu*, Gong Her Wu, Wah Chiu, Serena Yeung
NeurIPS 2021
[pdf] [project page] [code]
Personalized Federated Learning with First Order Model Optimization
Michael Zhang, Karan Sapra, Sanja Fidler, Serena Yeung, Jose M. Alvarez
ICLR 2021
[pdf]
FlowVOS: Weakly-Supervised Visual Warping for Detail-Preserving and Temporally Consistent Single-Shot Video Object Segmentation
Julia Gong, F. Christopher Holsinger, Serena Yeung
BMVC 2021
[pdf] [project page] [code]
Using Deep Learning to Identify the Recurrent Laryngeal Nerve During Thyroidectomy
Julia Gong, F. Christopher Holsinger, Julia E. Noel, Sohei Mitani, Jeff Jopling, Nikita Bedi, Yoon Woo Koh, Lisa A. Orloff, Claudio R. Cernea, Serena Yeung
Scientific Reports
[pdf] [project page]
Deep Learning-Enabled Medical Computer Vision
Andre Esteva, Katherine Chou, Serena Yeung, Nikhil Naik, Ali Madani, Ali Mottaghi, Yun Liu, Eric Topol, Jeff Dean, Richard Socher
Nature Partner Journals (NPJ) Digital Medicine
[pdf]
A Real-time Spatiotemporal AI Model Analyzes Skill in Open Surgical Videos
Emmett D Goodman*, Krishna K Patel*, Yilun Zhang, William Locke, Chris J Kennedy, Rohan Mehrotra, Stephen Ren, Melody Guan, Maren Downing, Hao Wei Chen, Jevin Z Clark, Gabriel A Brat, Serena Yeung
Preprint 2021
[pdf]
2020
Medical Symptom Recognition from Patient Text: An Active Learning Approach for Long-Tailed Multilabel Distributions
Ali Mottaghi, Prathusha Sarma, Xavier Amatriain, Serena Yeung, Anitha Kannan
Machine Learning for Healthcare (MLHC) 2020
[pdf]
Using Computer Vision to Automate Hand Detection and Tracking of Surgeon Movements in Videos of Open Surgery
Michael Zhang, Xiaotian Cheng, Daniel Copeland, Arjun Desai, Melody Guan, Gabriel Brat, Serena Yeung
AMIA 2020
[pdf]
Automatic Detection of Hand Hygiene Using Computer Vision Technology
Amit Singh, Albert Haque, Alexandre Alahi, Serena Yeung, Michelle Guo, Jill R Glassman, William Beninati, Terry Platchek, Li Fei-Fei, Arnold Milstein
Journal of the American Medical Informatics Association (JAMIA)
[pdf]
Adversarial Representation Active Learning
Ali Mottaghi, Serena Yeung
CVPR 2020 Workshop on Visual Learning with Limited Labels
[pdf] [bibtex] [code]
Ethical and Legal Aspects of Ambient Intelligence in Hospitals
Sara Gerke, Serena Yeung, I. Glenn Cohen
Journal of the American Medical Association (JAMA)
[pdf]
2019
NIH Advisory Committee to the Director Artificial Intelligence Working Group Report
Rediet Abebe, Greg Corrado, Kate Crawford, Barbara Engelhardt, David Glazer, David Haussler, Dina Katabi, Daphne Koller, Anshul Kundaje, Eric Lander, Jennifer Listgarten, Michael McManus, Lawrence Tabak, Serena Yeung
NIH Advisory Committee to the Director Working Group on Artificial Intelligence
[pdf]
A Computer Vision System for Deep Learning-Based Detection of Patient Mobilization Activities in the ICU
Serena Yeung*, Francesca Rinaldo*, Jeffrey Jopling, Bingbin Liu, Rishab Mehra, N. Lance Downing, Michelle Guo, Gabriel M. Bianconi, Alexandre Alahi, Julia Lee, Brandi Campbell, Kayla Deru, William Beninati, Li Fei-Fei, Arnold Milstein
Nature Partner Journals (NPJ) Digital Medicine
[pdf]
2018
Temporal Modular Networks for Retrieving Complex Compositional Activities in Video
Bingbin Liu, Serena Yeung, Edward Chou, De-An Huang, Li Fei-Fei, Juan Carlos Niebles
ECCV 2018
[pdf] [project page] [bibtex]
Neural Graph Matching Networks for Fewshot 3D Action Recognition
Michelle Guo, Edward Chou, Shuran Song, De-An Huang, Serena Yeung, Li Fei-Fei
ECCV 2018
[pdf] [project page] [bibtex]
Dynamic Task Prioritization for Multitask Learning
Michelle Guo, Albert Haque, De-An Huang, Serena Yeung, Li Fei-Fei
ECCV 2018
[pdf] [bibtex]
Computer Vision-based Descriptive Analytics of Seniors’ Daily Activities for Long-term Health Monitoring
Jun-Ting Hsieh*, Zelun Luo*, Niranjan Balachandar, Serena Yeung, Guido Pusiol, Jay Luxenberg, Grace Li, Li-Jia Li, N. Lance Downing, Arnold Milstein, Li Fei-Fei
MLHC 2018
[pdf] [bibtex]
3D Point Cloud-Based Visual Prediction of ICU Mobility Care Activities
Bingbin Liu*, Michelle Guo*, Edward Chou, Rishab Mehra, Serena Yeung, N. Lance Downing, Francesca Salipur, Jeffrey Jopling, Brandi Campbell, Kayla Deru, William Beninati, Arnold Milstein, Li Fei-Fei
MLHC 2018
[pdf]
Bedside Computer Vision -- Moving Artificial Intelligence from Driver Assistance to Patient Safety
Serena Yeung, N. Lance Downing, Li Fei-Fei, Arnold Milstein
New England Journal of Medicine 2018
[pdf] [bibtex]
Scaling Human-Object Interaction Recognition through Zero-Shot Learning
Liyue Shen, Serena Yeung, Judy Hoffman, Greg Mori, Li Fei-Fei
WACV 2018
[pdf] [bibtex]
Tool Detection and Operative Skill Assessment in Surgical Videos Using Region-Based Convolutional Neural Networks
Amy Jin, Serena Yeung, Jeffrey Jopling, Jonathan Krause, Dan Azagury, Arnold Milstein, Li Fei-Fei
NIPS 2017 Machine Learning for Health Workshop (Best Paper Award)
WACV 2018
[pdf] [project page] [code] [data] [bibtex]