Publications
2026
UniT: Unified Multimodal Chain-of-Thought Test-time Scaling
Leon Liangyu Chen, Haoyu Ma, Zhipeng Fan, Ziqi Huang, Animesh Sinha, Xiaoliang Dai, Jialiang Wang, Zecheng He, Jianwei Yang, Chunyuan Li, Junzhe Sun, Chu Wang, Serena Yeung-Levy, Felix Juefei-Xu
CVPR 2026
[pdf]
PaperSearchQA: Learning to Search and Reason over Scientific Papers with RLVR
James Burgess, Jan Niklas Hansen, Duo Peng, Yuhui Zhang, Alejandro Lozano, Min Woo Sun, Emma Lundberg, Serena Yeung-Levy
EACL 2026
[pdf] [project page] [code] [benchmark]
Project Imaging-X: A Survey of 1000+ Open-Access Medical Imaging Datasets for Foundation Model Development
Zhongying Deng, Cheng Tang, Ziyan Huang, Jiashi Lin, Yunhe Chen, Junjie Ning, Chenglong Ma, Jiarui Liu, Wenxiang Li, Yu Zhu, Serena Yeung-Levy
arXiv
[pdf] [code]
CellFluxRL: Biologically-Constrained Virtual Cell Modeling via Reinforcement Learning
Dongxia Wu, Shiye Su, Yuhui Zhang, Elaine Sui, Emma Lundberg, Emily B. Fox, Serena Yeung-Levy
arXiv
[pdf]
Uncertainty Quantification for Distribution-to-Distribution Flow Matching in Scientific Imaging
Dongxia Wu, Yuhui Zhang, Serena Yeung-Levy, Emma Lundberg, Emily B. Fox
arXiv
[pdf]
Fine-tuning MLLMs Without Forgetting Is Easier Than You Think
He Li, Yuhui Zhang, Xiaohan Wang, Kaifeng Lyu, Serena Yeung-Levy
arXiv
[pdf]
Tool Verification for Test-Time Reinforcement Learning
Ruotong Liao, Nikolai Röhrich, Xiaohan Wang, Yuhui Zhang, Yasaman Samadzadeh, Volker Tresp, Serena Yeung-Levy
arXiv
[pdf]
iSight: Towards expert-AI co-assessment for improved immunohistochemistry staining interpretation
Jacob S. Leiby, Jialu Yao, Pan Lu, George Hu, Anna Davidian, Shunsuke Koga, Olivia Leung, Pravin Patel, Isabella Tondi Resta, Rebecca Rojansky, Derek Sung, Eric Yang, Paul J. Zhang, Emma Lundberg, Dokyoon Kim, Serena Yeung-Levy, James Zou, Thomas Montine, Jeffrey Nirschl, Zhi Huang
arXiv
[pdf]
Seeing Is Believing? A Benchmark for Multimodal Large Language Models on Visual Illusions and Anomalies
Wenjin Hou, Wei Liu, Han Hu, Xiaoxiao Sun, Serena Yeung-Levy, Hehe Fan
arXiv
[pdf]
Do VLMs Perceive or Recall? Probing Visual Perception vs. Memory with Classic Visual Illusions
Xiaoxiao Sun, Mingyang Li, Kun Yuan, Min Woo Sun, Mark Endo, Shengguang Wu, Changlin Li, Yuhui Zhang, Zeyu Wang, Serena Yeung-Levy
arXiv
[pdf]
RadDiff: Describing Differences in Radiology Image Sets with Natural Language
Xiaoxian Shen, Yuhui Zhang, Sahithi Ankireddy, Xiaohan Wang, Maya Varma, Henry Guo, Curtis Langlotz, Serena Yeung-Levy
arXiv
[pdf]
CellFluxV2: An Image Generative Foundation Model for Virtual Cell Modeling
Yuhui Zhang, Yuchang Su, Zoe Wefers, Shiye Su, He Li, Tianhong Li, Chenyu Wang, James Burgess, Alejandro Lozano, Emma Lundberg, Serena Yeung-Levy
bioRxiv
[pdf]
Computer Vision-Based Retrieval of Steps and Errors in Laparoscopic Cholecystectomy
Elaine Sui, Charlotte Egeland, Xiaohan Wang, Alfred Song, Rui Li, Joshua Villarreal, Anita Rau, Josiah Aklilu, Alan Brown, Jeffrey Jopling, Serena Yeung-Levy
medRxiv
[pdf]
ArtifactLens: Hundreds of Labels Are Enough for Artifact Detection with VLMs
James Burgess, Rameen Abdal, Dan Stoddart, Sergey Tulyakov, Serena Yeung-Levy, Kuan-Chieh Jackson Wang
arXiv
[pdf] [project page]
2025
CellFlux: Simulating Cellular Morphology Changes via Flow Matching.
Yuhui Zhang*, Yuchang Su*, Chenyu Wang, Tianhong Li, Zoe Wefers, Jeffrey Nirschl, James Burgess, Daisy Ding, Alejandro Lozano, Emma Lundberg, Serena Yeung-Levy
ICML 2025
[pdf] [project page] [code]
Data or Language Supervision: What Makes CLIP Better than DINO?
Yiming Liu*, Yuhui Zhang*, Dhruba Ghosh, Ludwig Schmidt, Serena Yeung-Levy
EMNLP 2025 (Findings)
[pdf] [code]
Using artificial intelligence to model expert panel diagnosis of cholecystitis severity
Griffin H. Olsen, Emmett D. Goodman, Josiah G. Aklilu, Sebastiano Bartoletti, Kay S. Hung, Janice H. Yang, Eric C. Sorenson, Jeffrey K. Jopling, Serena Yeung-Levy, Dan E. Azagury
Surgical Endoscopy
[pdf]
Zero-shot Action Localization via the Confidence of Large Vision-Language Models
Josiah Aklilu, Xiaohan Wang, Serena Yeung-Levy
arXiv
[pdf] [code]
The Impact of Image Resolution on Biomedical Multimodal Large Language Models
Liangyu Chen, James Burgess, Jeffrey J Nirschl, Orr Zohar, Serena Yeung-Levy
MLHC 2025
[pdf]
Systematic Evaluation of Large Vision-Language Models for Surgical Artificial Intelligence
Anita Rau, Mark Endo, Josiah Aklilu, Jeff Heo, Khaled Saab, Alberto Paderno, Jeffrey Jopling, Christopher Holsinger, Serena Yeung-Levy
arXiv
[pdf]
Artificial Intelligence-Powered 3D Analysis of Video-Based Caregiver–Child Interactions
Zhenzhen Weng, Laura Bravo-Sánchez, Zeyu Wang, Christopher Howard, Maria Xenochristou, Nicole Meister, Angjoo Kanazawa, Arnold Milstein, Elika Bergelson, Kathryn L. Humphreys, Lee M. Sanders , Serena Yeung-Levy
Science Advances
[pdf] [code] [data]
Apollo: An Exploration of Video Understanding in Large Multimodal Models
Orr Zohar, Xiaohan Wang, Yann Dubois, Philippe Hansen-Estruch, Licheng Yu, Xiaofang Wang, Felix Juefei-Xu, Ning Zhang, Serena Yeung-Levy, Xide Xia
CVPR 2025
[pdf] [website]
Automated Generation of Challenging Multiple-Choice Questions for Vision Language Model Evaluation.
Yuhui Zhang*, Yuchang Su*, Yiming Liu, Xiaohan Wang, James Burgess, Elaine Sui, Chenyu Wang, Josiah Aklilu, Alejandro Lozano, Anjiang Wei, Ludwig Schmidt, Serena Yeung-Levy.
CVPR 2025
[pdf] [project page] [code]
BIOMEDICA: An Open Biomedical Image-Caption Archive, Dataset, and Vision-Language Models Derived from Scientific Literature
Alejandro Lozano, Min Woo Sun, James Burgess, Liangyu Chen, Jeffrey J Nirschl, Jeffrey Gu, Ivan Lopez, Josiah Aklilu, Austin Wolfgang Katzer, Collin Chiu, Anita Rau, Xiaohan Wang, Yuhui Zhang, Alfred Seunghoon Song, Robert Tibshirani, Serena Yeung-Levy
CVPR 2025
[pdf] [project page] [code]
MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based Scientific Research
James Burgess*, Jeffrey J Nirschl*, Laura Bravo-Sánchez*, Alejandro Lozano, Sanket Rajan Gupte, Jesus G. Galaz-Montoya, Yuhui Zhang, Yuchang Su, Disha Bhowmik, Zachary Coman, Sarina M. Hasan, Alexandra Johannesson, William D. Leineweber, Malvika G Nair, Ridhi Yarlagadda, Connor Zuraski, Wah Chiu, Sarah Cohen, Jan N. Hansen, Manuel D Leonetti, Chad Liu, Emma Lundberg, Serena Yeung-Levy
CVPR 2025
[pdf] [project page] [code] [benchmark]
Foundation Models Secretly Understand Neural Network Weights: Enhancing Hypernetwork Architectures with Foundation Models
Jeffrey Gu, Serena Yeung-Levy
ICLR 2025
[pdf][project page]
Video Action Differencing
James Burgess, Xiaohan Wang, Yuhui Zhang, Anita Rau, Alejandro Lozano, Lisa Dunlap, Trevor Darrell, Serena Yeung-Levy
ICLR 2025
[pdf] [project page] [code]
Video-STaR: Self-training enables video instruction tuning with any supervision
Orr Zohar, Xiaohan Wang, Yonatan Bitton, Idan Szpektor, & Serena Yeung-Levy
ICLR 2025
[pdf]
Mechanistic Interpretability Meets Vision Language Models: Insights and Limitations
Yiming Liu*, Yuhui Zhang*, Serena Yeung-Levy
ICLR Blog 2025
[pdf]
Just Shift It: Test-Time Prototype Shifting for Zero-Shot Generalization with Vision-Language Models
Elaine Sui, Xiaohan Wang, Serena Yeung-Levy
WACV 2025 (Oral)
[pdf] [code]
Temporal Preference Optimization for Long-Form Video Understanding
Rui Li*, Xiaohan Wang*, Yuhui Zhang, Zeyu Wang, Serena Yeung-Levy
Preprint 2025
[project page]
Three Forms of Stochastic Injection for Improved Distribution-to-Distribution Generative Modeling
Shiye Su, Orr Zohar, Xiaohan Wang, Serena Yeung-Levy
arXiv
[pdf]
NegVQA: Can Vision Language Models Understand Negation?
Yuhui Zhang, Yuchang Su, Yiming Liu, Serena Yeung-Levy
ACL Findings 2025
[pdf] [project page]
Feather the Throttle: Revisiting Visual Token Pruning for Vision-Language Model Acceleration
Mark Endo, Xiaohan Wang, Serena Yeung-Levy
ICCV 2025
[pdf] [project page] [code]
NeuHMR: Neural Rendering-Guided Human Motion Reconstruction
Tiange Xiang, Kuan-Chieh Wang, Jaewoo Heo, Ehsan Adeli, Serena Yeung-Levy, Scott Delp, Li Fei-Fei
3DV 2025
[pdf]
DeforHMR: Vision Transformer with Deformable Cross-Attention for 3D Human Mesh Recovery
Jaewoo Heo, George Hu, Zeyu Wang, Serena Yeung-Levy
3DV 2025
[pdf]
Global organelle profiling reveals subcellular localization and remodeling at proteome scale
Marco Y. Hein, Duo Peng, Verina Todorova, Frank McCarthy, Kibeom Kim, Chad Liu, Laura Savy, Camille Januel, Rodrigo Baltazar-Nunez, Madhurya Sekhar, Shivanshi Vaid, Sophie Bax, Madhuri Vangipuram, James Burgess, Leila Njoya, Eileen Wang, Ivan E. Ivanov, Janie R. Byrum, Soorya Pradeep, Carlos G. Gonzalez, Yttria Aniseia, Joseph S. Creery, Aidan H. McMorrow, Sara Sunshine, Serena Yeung-Levy, Brian C. DeFelice, Shalin B. Mehta, Daniel N. Itzhak, Joshua E. Elias, Manuel D. Leonetti
Cell 2025
[pdf]
A clinically accessible small multimodal radiology model and evaluation metric for chest X-ray findings
Juan Manuel Zambrano Chaves, Shih-Cheng Huang, Yanbo Xu, Hanwen Xu, Naoto Usuyama, Sheng Zhang, Fei Wang, Yujia Xie, Mahmoud Khademi, Ziyi Yang, Hany Awadalla, Julia Gong, Houdong Hu, Jianwei Yang, Chunyuan Li, Jianfeng Gao, Yu Gu, Cliff Wong, Mu Wei, Tristan Naumann, Muhao Chen, Matthew P. Lungren, Akshay Chaudhari, Serena Yeung-Levy, Curtis P. Langlotz, Sheng Wang, Hoifung Poon
Nature Communications
[pdf]
Enhanced Trauma Video Review With Computer Vision: Trauma Resuscitation Phase Segmentation and Procedure Detection
Joshua A. Villarreal, Jaewoo Heo, Xiaohan Wang, Andrew Bain, Bahaa Succar, Dong-han Yao, Jeffrey K. Jopling, Serena Yeung-Levy, Ryan P. Dumas
Annals of Surgery Open 2025
[pdf]
A fine-tuned foundational model SurgiSAM2 for surgical video anatomy segmentation and detection
Devanish N. Kamtam, Joseph B. Shrager, Satya Deepya Malla, Xiaohan Wang, Nicole Lin, Juan J. Cardona, Serena Yeung-Levy, Clarence Hu
Scientific Reports 2025
[pdf]
Cryogenic electron tomography and elemental analysis of mitochondrial granules in human retinal ganglion cells
Gong-Her Wu, Cathy Hou, Andrew Thron, Hirenkumar Rajendra Patel, Liam Spillane, Sanket Rajan Gupte, Serena Yeung-Levy, Sahil Gulati, Christopher Booth, Yaping Joyce Liao, Wah Chiu
Structure 2025
[pdf]
A Systematic Review and Implementation Guidelines of Multimodal Foundation Models in Medical Imaging
Shih-Cheng Huang, Malte Jensen, Serena Yeung-Levy, Matthew P. Lungren, Hoifung Poon, Akshay S. Chaudhari
Research Square 2025
[pdf]
Transductive Visual Programming: Evolving Tool Libraries from Experience for Spatial Reasoning
Shengguang Wu, Xiaohan Wang, Yuhui Zhang, Hao Zhu, Serena Yeung-Levy
arXiv 2025
[pdf] [project page]
CryoHype: Reconstructing a thousand cryo-EM structures with transformer-based hypernetworks
Jeffrey Gu, Minkyu Jeon, Ambri Ma, Serena Yeung-Levy, Ellen D. Zhong
arXiv 2025
[pdf]
From Panel to Pixel: Zoom-In Vision-Language Pretraining from Biomedical Scientific Literature
Kun Yuan, Min Woo Sun, Zhen Chen, Alejandro Lozano, Xiangteng He, Shi Li, Nassir Navab, Xiaoxiao Sun, Nicolas Padoy, Serena Yeung-Levy
arXiv 2025
[pdf]
Automated Video-Based Analysis of Surgical Meta-competencies Using Computer Vision
Josiah Aklilu, Joshua A. Villarreal, Chloe K. Nobuhara, Charlotte Egeland, Xiaohan Wang, Elaine Sui, Alan Brown, Matthew Leipzig, Reid Dale, Anita Rau, Alfred Song, Shelly Goel, Eric Sorenson, Vanessa Palter, Roger Bohn, Teodor Grantcharov, Jeffrey K. Jopling, Serena Yeung-Levy
medRxiv 2025
[pdf]
Downscaling Intelligence: Exploring Perception and Reasoning Bottlenecks in Small Multimodal Models
Mark Endo, Serena Yeung-Levy
arXiv 2025
[pdf] [project page] [code]
SciVideoBench: Benchmarking Scientific Video Reasoning in Large Multimodal Models
Andong Deng, Taojiannan Yang, Shoubin Yu, Lincoln Spencer, Mohit Bansal, Chen Chen, Serena Yeung-Levy, Xiaohan Wang
arXiv 2025
[pdf] [project page] [code]
TTRV: Test-Time Reinforcement Learning for Vision Language Models
Akshit Singh, Shyam Marjit, Wei Lin, Paul Gavrikov, Serena Yeung-Levy, Hilde Kuehne, Rogerio Feris, Sivan Doveh, James Glass, M. Jehanzeb Mirza
arXiv 2025
[pdf] [project page]
No Tokens Wasted: Leveraging Long Context in Biomedical Vision-Language Models
Min Woo Sun, Alejandro Lozano, Javier Gamazo Tejero, Vishwesh Nath, Xiao Xiao Sun, James Burgess, Yuhui Zhang, Kun Yuan, Robert Tibshirani, Sean Huver, Serena Yeung-Levy
arXiv 2025
[pdf]
AccidentBench: Benchmarking Multimodal Understanding and Reasoning in Vehicle Accidents and Beyond
Shangding Gu, Xiaohan Wang, Donghao Ying, Haoyu Zhao, Runing Yang, Ming Jin, Boyi Li, Marco Pavone, Serena Yeung-Levy, Jun Wang, Dawn Song, Costas Spanos
arXiv 2025
[pdf] [project page] [code]
VisualOverload: Probing Visual Understanding of VLMs in Really Dense Scenes
Paul Gavrikov, Wei Lin, M. Jehanzeb Mirza, Soumya Jahagirdar, Muhammad Huzaifa, Sivan Doveh, Serena Yeung-Levy, James Glass, Hilde Kuehne
arXiv 2025
[pdf] [project page] [code] [dataset]
Closing the Modality Gap for Mixed Modality Search
Binxu Li, Yuhui Zhang, Xiaohan Wang, Weixin Liang, Ludwig Schmidt, Serena Yeung-Levy
arXiv 2025
[pdf] [project page]
Can Large Language Models Match the Conclusions of Systematic Reviews?
Christopher Polzak, Alejandro Lozano, Min Woo Sun, James Burgess, Yuhui Zhang, Kevin Wu, Serena Yeung-Levy
arXiv 2025
[pdf]
A Large-Scale Vision-Language Dataset Derived from Open Scientific Literature to Advance Biomedical Generalist AI
Alejandro Lozano, Min Woo Sun, James Burgess, Jeffrey J. Nirschl, Christopher Polzak, Yuhui Zhang, Liangyu Chen, Jeffrey Gu, Ivan Lopez, Josiah Aklilu, Anita Rau, Austin Wolfgang Katzer, Collin Chiu, Orr Zohar, Xiaohan Wang, Alfred Seunghoon Song, Chiang Chia-Chun, Robert Tibshirani, Serena Yeung-Levy
arXiv 2025
[pdf]
2024
How to build the virtual cell with artificial intelligence: Priorities and opportunities
Charlotte Bunne, Yusuf Roohani, Yanay Rosen, Ankit Gupta, Xikun Zhang, Marcel Roed, Theo Alexandrov, Mohammed AlQuraishi, Patricia Brennan, Daniel B. Burkhardt, Andrea Califano, Jonah Cool, Abby F. Dernburg, Kirsty Ewing, Emily B. Fox, Matthias Haury, Amy E. Herr, Eric Horvitz, Patrick D. Hsu, Viren Jain, Gregory R. Johnson, Thomas Kalil, David R. Kelley, Shana O. Kelley, Anna Kreshuk, Tim Mitchison, Stephani Otte, Jay Shendure, Nicholas J. Sofroniew, Fabian Theis, Christina V. Theodoris, Srigokul Upadhyayula, Marc Valer, Bo Wang, Eric Xing, Serena Yeung-Levy, Marinka Zitnik, Theofanis Karaletsos, Aviv Regev, Emma Lundberg, Jure Leskovec, Stephen R. Quake
Cell 2024
[pdf]
Why are Visually-Grounded Language Models Bad at Image Classification?
Yuhui Zhang, Alyssa Unell, Xiaohan Wang, Dhruba Ghosh, Yuchang Su, Ludwig Schmidt, Serena Yeung-Levy
NeurIPS 2024
[pdf] [project page] [code]
MicroBench: A Vision-Language Benchmark for Microscopy Understanding
Alejandro Lozano, Jeffrey Nirschl, James Burgess, Sanket Gupte, Yuhui Zhang, Alyssa Unell, Serena Yeung-Levy
NeurIPS 2024 Datasets and Benchmarks
[pdf]
Depth-guided NeRF Training via Earth Mover's Distance
Anita Rau, Josiah Aklilu, Chris Holsinger, Serena Yeung-Levy
ECCV 2024
[pdf] [project page]
VideoAgent: Long-form Video Understanding with Large Language Model as Agent
Xiaohan Wang*, Yuhui Zhang*, Orr Zohar, Serena Yeung-Levy
ECCV 2024
[pdf] [project page] [code]
Viewpoint Textual Inversion: Unleashing Novel View Synthesis with Pretrained 2D Diffusion Models
James Burgess, Kuan-Chieh Wang, Serena Yeung
ECCV 2024
[pdf] [project page] [code]
Describing Differences in Image Sets with Natural Language
Lisa Dunlap*, Yuhui Zhang*, Xiaohan Wang, Ruiqi Zhong, Trevor Darrell, Jacob Steinhardt, Joseph E. Gonzalez, Serena Yeung-Levy
CVPR 2024 (Oral)
[pdf] [project page] [code]
Connect, Collapse, Corrupt: Learning Cross-Modal Tasks with Uni-Modal Data
Yuhui Zhang*, Elaine Sui*, Serena Yeung-Levy
ICLR 2024
[pdf] [project page] [code]
Diffusion-HPC: Synthetic Data Generation for Human Mesh Recovery in Challenging Domains
Zhenzhen Weng, Laura Bravo-Sanchez, Serena Yeung
3DV 2024
[pdf] [project page] [code]
Hyperbolic Deep Learning in Computer Vision: A Survey
Pascal Mettes, Mina Ghadimi Atigh, Martin Keller-Ressler, Jeffrey Gu, Serena Yeung
IJCV 2024
[pdf]
Hospitalization Prediction from the Emergency Department Using Computer Vision AI with Short Patient Video Clips
Wui Ip*, Maria Xenochristou*, Elaine Sui*, Elyse Ruan, Ryan Ribeira, Debadutta Dash, Malathi Srinivasan, Maja Artandi, Jesutofunmi A. Omiye, Nicholas Scoulios, Hayden L. Hofmann, Ali Mottaghi, Zhenzhen Weng, Abhinav Kumar, Ananya Ganesh, Jason Fries, Serena Yeung-Levy, Lawrence V. Hofmann
NPJ Digital Medicine
[pdf]
Artificial Intelligence Identifies Factors Associated with Blood Loss and Surgical Experience in Cholecystectomy
Josiah G. Aklilu, Min Woo Sun, Shelly Goel, Sebastiano Bartoletti, Anita Rau, Griffin Olsen, Kay S. Hung, Sophie L. Mintz, Vicki Luong, Arnold Milstein, Mark J. Ott, Robert Tibshirani, Jeffrey K. Jopling, Eric C. Sorenson, Dan E. Azagury, Serena Yeung-Levy
NEJM AI
[pdf]
Analyzing Surgical Technique in Diverse Open Surgical Videos With Multitask Machine Learning
Emmett D. Goodman, Krishna K. Patel, Yilun Zhang, William Locke, Chris J. Kennedy, Rohan Mehrotra, Stephen Ren, Melody Guan, Orr Zohar, Maren Downing, Hao Wei Chen, Jevin Z. Clark, Margaret T. Berrigan, Gabriel A. Brat, Serena Yeung-Levy
JAMA Surgery
[pdf]
Orientation-invariant autoencoders learn robust representations for shape profiling of cells and organelles
James Burgess, Jeffrey J. Nirschl, Maria-Clara Zanellati, Alejandro Lozano, Sarah Cohen, Serena Yeung-Levy
Nature Communications
[pdf] [code]
A Human Mesh-Centered Approach to Action Recognition in the Operating Room
Benjamin Liu; Gilles Soenens; Joshua Villarreal; Jeffrey Jopling; Isabelle Van Herzeele; Anita Rau; Serena Yeung-Levy
Artifical Intelligence Surgery 2024
[pdf]
Ask, Pose, Unite: Scaling Data Acquisition for Close Interactions with Vision Language Models
Laura Bravo-Sánchez, Jaewoo Heo, Zhenzhen Weng, Kuan-Chieh Wang, Serena Yeung-Levy
Preprint 2024
[project page]
CryoViT: Efficient segmentation of cryogenic electron tomograms with vision foundation models
Sanket R. Gupte, Cathy Hou, Gong-Her Wu, Jesús G. Galaz-Montoya, Wah Chiu, Serena Yeung-Levy
Preprint 2024
[pdf]
Multi-Human Mesh Recovery with Transformers
Zeyu Wang, Zhenzhen Weng, Serena Yeung-Levy
Preprint 2024
[pdf]
Continuous perception benchmark
Zeyu Wang, Zhenzhen Weng, Serena Yeung-Levy
Preprint 2024
[pdf]
2023
LOVM: Language-Only Vision Model Selection
Orr Zohar, Shih-Cheng Huang, Kuan-Chieh Wang, Serena Yeung
NeurIPS 2023 Datasets and Benchmarks
[pdf] [project page] [code]
INSPECT: A Multimodal Dataset for Pulmonary Embolism Diagnosis and Prognosis
Shih-Cheng Huang, Zepeng Huo, Ethan Steinberg, Chia-Chun Chiang, Matthew P. Lungren, Curtis P. Langlotz, Serena Yeung, Nigam H. Shah, Jason A. Fries
NeurIPS 2023 Datasets and Benchmarks
[pdf] [project page] [code]
Generalizable Neural Fields as Partially Observed Neural Processes
Jeffrey Gu, Kuan-Chieh Wang, Serena Yeung
ICCV 2023
[pdf] [project page] [code]
Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models
Yuhui Zhang, Michihiro Yasunaga, Zhengping Zhou, Jeff Z HaoChen, James Zou, Percy Liang, Serena Yeung
ACL 2023 Findings
[pdf] [project page] [code]
Self-supervised Learning for Medical Image Classification: A Systematic Review and Implementation Guidelines
Shih-Cheng Huang, Anuj Pareek, Malte Jensen, Matthew P Lungren, Serena Yeung, Akshay S Chaudhari
Nature Partner Journals (NPJ) Digital Medicine
[pdf]
Robust Semi-supervised Detection of Hands in Diverse Open Surgery Environments
Pranav Vaid, Serena Yeung, Anita Rau
Machine Learning for Healthcare (MLHC) 2023
[pdf] [code]
Video Pretraining Advances 3D Deep Learning on Chest CT Tasks
Alexander Ke, Shih-Cheng Huang, Chloe P O'Connell, Michal Klimont, Serena Yeung, Pranav Rajpurkar
Medical Imaging with Deep Learning (MIDL) 2023
[pdf] [code]
NeMo: 3D Neural Motion Fields from Multiple Video Instances of the Same Action
Kuan-Chieh Wang, Zhenzhen Weng, Maria Xenochristou, Joao Pedro Araujo, Jeffrey Gu, C. Karen Liu, Serena Yeung
CVPR 2023 (Highlight)
[pdf] [project page] [code]
PROB: Probabilistic Objectness for Open World Object Detection
Orr Zohar, Kuan-Chieh Wang, Serena Yeung
CVPR 2023
[pdf] [project page] [code]
Diagnosing and Rectifying Vision Models using Language
Yuhui Zhang, Jeff Z. HaoChen, Shih-Cheng Huang, Kuan-Chieh Wang, James Zou, Serena Yeung
ICLR 2023
[pdf] [project page] [code]
CryoET Reveals Organelle Phenotypes in Huntington Disease Patient iPSC-derived and Mouse Primary Neurons
Gong-Her Wu, Charlene Smith-Geater, Jesus G. Galaz-Montoya, Yingli Gu, Sanket R. Gupte, Ranen Aviner, Patrick G. Mitchell, Joy Hsu, Ricardo Miramontes, Keona Q. Wang, Nicolette R. Geller, Cathy Hou, Cristina Danita, Lydia-Marie Joubert, Michael F. Schmid, Serena Yeung, Judith Frydman, William Mobley, Chengbiao Wu, Leslie M. Thompson, Wah Chiu
Nature Communications
[pdf] [code]
Comparing Spatial Patterns of Marine Vessels between Vessel-tracking Data and Satellite Imagery
Shinnosuke Nakayama, WenXin Dong, Richard G. Correro, Elizabeth R. Selig, Colette C.C. Wabnitz, Trevor J. Hastie, Jim Leape, Serena Yeung, Fiorenza Micheli
Frontiers in Marine Science
[pdf] [code]
ZeroAvatar: Zero-shot 3D Avatar Generation from a Single Image
Zhenzhen Weng, Zeyu Wang, Serena Yeung
Preprint 2023
[pdf]
Open World Object Detection in the Era of Foundation Models
Orr Zohar, Alejandro Lozano, Shelly Goel, Serena Yeung, Kuan-Chieh Wang
Preprint 2023
[pdf] [project page] [code]
2022
Prostate Cancer Therapy Personalization via Multi-modal Deep Learning on Randomized Phase III Clinical Trials
Andre J Esteva, Jean Feng, Douwe van der Wal, Shih-Cheng Huang, Jeffry Simko, Sandy E DeVries, Emmalyn Chen, Edward Schaeffer, Todd Morgan, Yilun Sun, Amirata A Ghorbani, Nikhil Naik, Dhruv G Nathawani, Richard D Socher, Jeff L Michalski, Mack R Roach III, Thomas G Pisansky, Jedidiah Monson, Farah A Naz, James Wallace, Michelle M Ferguson, Jean-Paul T Bahary, James E Zou, Matthew Lungren, Serena Y Yeung, Ashley Ross, NRG Prostate Cancer AI Consortium, Howard M Sandler, Phouc T Tran, Daniel E Spratt, Stephanie Pugh, Felix Y Feng, Osama Mohamad
Nature Partner Journals (NPJ) Digital Medicine
[pdf]
Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning
Weixin Liang*, Yuhui Zhang*, Yongchan Kwon*, Serena Yeung, James Zou
NeurIPS 2022
[pdf] [project page] [code]
Adapting Pre-trained Vision Transformers from 2D to 3D through Weight Inflation Improves Medical Image Segmentation
Yuhui Zhang, Shih-Cheng Huang, Zhengping Zhou, Matthew P. Lungren, Serena Yeung
ML4H 2022
[pdf] [code]
Domain Adaptive 3D Pose Augmentation for In-the-wild Human Mesh Recovery
Zhenzhen Weng, Kuan-Chieh Wang, Angjoo Kanazawa, Serena Yeung
3DV 2022
[pdf] [project page] [code]
Adaptation of Surgical Activity Recognition Models Across Operating Rooms
Ali Mottaghi, Aidean Sharghi, Serena Yeung, Omid Mohareri
MICCAI 2022
[pdf]
ALGES: Active Learning with Gradient Embeddings for Semantic Segmentation of Laparoscopic Surgical Images
Josiah Aklilu, Serena Yeung
MLHC 2022
[pdf] [code]
Developing Medical Imaging AI for Emerging Infectious Diseases
Shih-Cheng Huang, Akshay S. Chaudhari, Curtis P. Langlotz, Nigam Shah, Serena Yeung, Matthew P. Lungren
Nature Communications
[pdf]
Using AI and Computer Vision to Analyze Technical Proficiency in Robotic Surgery
Janice H Yang, Emmett D Goodman, Aaron J Dawes, John V Gahagan, Micaela M Esquivel, Cara A Liebert, Cindy Kin, Serena Yeung, Brooke H Gurland
Surgical Endoscopy
[pdf]
2021
Holistic 3D Human and Scene Mesh Estimation from Single View Images
Zhenzhen Weng, Serena Yeung
CVPR 2021
[pdf] [project page] [code]
Unsupervised Discovery of the Long-Tail in Instance Segmentation Using Hierarchical Self-Supervision
Zhenzhen Weng, Mehmet Giray Ogut, Shai Limonchik, Serena Yeung
CVPR 2021
[pdf] [project page] [code]
DARCNN: Domain Adaptive Region-based Convolutional Neural Network for Unsupervised Instance Segmentation in Biomedical Images
Joy Hsu, Wah Chiu, Serena Yeung
CVPR 2021
[pdf] [project page] [code]
Staying in Shape: Learning Invariant Shape Representations using Contrastive Learning
Jeffrey Gu, Serena Yeung
UAI 2021
[pdf] [project page] [code]
GLoRIA: A Multimodal Global-Local Representation Learning Framework for Label-efficient Medical Image Recognition
Shih-Cheng Huang*, Liyue Shen*, Matthew P. Lungren, Serena Yeung
ICCV 2021
[pdf] [code]
Capturing implicit hierarchical structure in 3D biomedical images with self-supervised hyperbolic representations
Joy Hsu*, Jeffrey Gu*, Gong Her Wu, Wah Chiu, Serena Yeung
NeurIPS 2021
[pdf] [project page] [code]
Personalized Federated Learning with First Order Model Optimization
Michael Zhang, Karan Sapra, Sanja Fidler, Serena Yeung, Jose M. Alvarez
ICLR 2021
[pdf]
FlowVOS: Weakly-Supervised Visual Warping for Detail-Preserving and Temporally Consistent Single-Shot Video Object Segmentation
Julia Gong, F. Christopher Holsinger, Serena Yeung
BMVC 2021
[pdf] [project page] [code]
Using Deep Learning to Identify the Recurrent Laryngeal Nerve During Thyroidectomy
Julia Gong, F. Christopher Holsinger, Julia E. Noel, Sohei Mitani, Jeff Jopling, Nikita Bedi, Yoon Woo Koh, Lisa A. Orloff, Claudio R. Cernea, Serena Yeung
Scientific Reports
[pdf] [project page]
Deep Learning-Enabled Medical Computer Vision
Andre Esteva, Katherine Chou, Serena Yeung, Nikhil Naik, Ali Madani, Ali Mottaghi, Yun Liu, Eric Topol, Jeff Dean, Richard Socher
Nature Partner Journals (NPJ) Digital Medicine
[pdf]
A Real-time Spatiotemporal AI Model Analyzes Skill in Open Surgical Videos
Emmett D Goodman*, Krishna K Patel*, Yilun Zhang, William Locke, Chris J Kennedy, Rohan Mehrotra, Stephen Ren, Melody Guan, Maren Downing, Hao Wei Chen, Jevin Z Clark, Gabriel A Brat, Serena Yeung
Preprint 2021
[pdf]
2020
Medical Symptom Recognition from Patient Text: An Active Learning Approach for Long-Tailed Multilabel Distributions
Ali Mottaghi, Prathusha Sarma, Xavier Amatriain, Serena Yeung, Anitha Kannan
Machine Learning for Healthcare (MLHC) 2020
[pdf]
Using Computer Vision to Automate Hand Detection and Tracking of Surgeon Movements in Videos of Open Surgery
Michael Zhang, Xiaotian Cheng, Daniel Copeland, Arjun Desai, Melody Guan, Gabriel Brat, Serena Yeung
AMIA 2020
[pdf]
Automatic Detection of Hand Hygiene Using Computer Vision Technology
Amit Singh, Albert Haque, Alexandre Alahi, Serena Yeung, Michelle Guo, Jill R Glassman, William Beninati, Terry Platchek, Li Fei-Fei, Arnold Milstein
Journal of the American Medical Informatics Association (JAMIA)
[pdf]
Adversarial Representation Active Learning
Ali Mottaghi, Serena Yeung
CVPR 2020 Workshop on Visual Learning with Limited Labels
[pdf] [bibtex] [code]
Ethical and Legal Aspects of Ambient Intelligence in Hospitals
Sara Gerke, Serena Yeung, I. Glenn Cohen
Journal of the American Medical Association (JAMA)
[pdf]
2019
NIH Advisory Committee to the Director Artificial Intelligence Working Group Report
Rediet Abebe, Greg Corrado, Kate Crawford, Barbara Engelhardt, David Glazer, David Haussler, Dina Katabi, Daphne Koller, Anshul Kundaje, Eric Lander, Jennifer Listgarten, Michael McManus, Lawrence Tabak, Serena Yeung
NIH Advisory Committee to the Director Working Group on Artificial Intelligence
[pdf]
A Computer Vision System for Deep Learning-Based Detection of Patient Mobilization Activities in the ICU
Serena Yeung*, Francesca Rinaldo*, Jeffrey Jopling, Bingbin Liu, Rishab Mehra, N. Lance Downing, Michelle Guo, Gabriel M. Bianconi, Alexandre Alahi, Julia Lee, Brandi Campbell, Kayla Deru, William Beninati, Li Fei-Fei, Arnold Milstein
Nature Partner Journals (NPJ) Digital Medicine
[pdf]
2018
Temporal Modular Networks for Retrieving Complex Compositional Activities in Video
Bingbin Liu, Serena Yeung, Edward Chou, De-An Huang, Li Fei-Fei, Juan Carlos Niebles
ECCV 2018
[pdf] [project page] [bibtex]
Neural Graph Matching Networks for Fewshot 3D Action Recognition
Michelle Guo, Edward Chou, Shuran Song, De-An Huang, Serena Yeung, Li Fei-Fei
ECCV 2018
[pdf] [project page] [bibtex]
Dynamic Task Prioritization for Multitask Learning
Michelle Guo, Albert Haque, De-An Huang, Serena Yeung, Li Fei-Fei
ECCV 2018
[pdf] [bibtex]
Computer Vision-based Descriptive Analytics of Seniors’ Daily Activities for Long-term Health Monitoring
Jun-Ting Hsieh*, Zelun Luo*, Niranjan Balachandar, Serena Yeung, Guido Pusiol, Jay Luxenberg, Grace Li, Li-Jia Li, N. Lance Downing, Arnold Milstein, Li Fei-Fei
MLHC 2018
[pdf] [bibtex]
3D Point Cloud-Based Visual Prediction of ICU Mobility Care Activities
Bingbin Liu*, Michelle Guo*, Edward Chou, Rishab Mehra, Serena Yeung, N. Lance Downing, Francesca Salipur, Jeffrey Jopling, Brandi Campbell, Kayla Deru, William Beninati, Arnold Milstein, Li Fei-Fei
MLHC 2018
[pdf]
Bedside Computer Vision -- Moving Artificial Intelligence from Driver Assistance to Patient Safety
Serena Yeung, N. Lance Downing, Li Fei-Fei, Arnold Milstein
New England Journal of Medicine 2018
[pdf] [bibtex]
Scaling Human-Object Interaction Recognition through Zero-Shot Learning
Liyue Shen, Serena Yeung, Judy Hoffman, Greg Mori, Li Fei-Fei
WACV 2018
[pdf] [bibtex]
Tool Detection and Operative Skill Assessment in Surgical Videos Using Region-Based Convolutional Neural Networks
Amy Jin, Serena Yeung, Jeffrey Jopling, Jonathan Krause, Dan Azagury, Arnold Milstein, Li Fei-Fei
NIPS 2017 Machine Learning for Health Workshop (Best Paper Award)
WACV 2018
[pdf] [project page] [code] [data] [bibtex]