Capital One endowed Associate Professor Research Scientist (20%) My research interests lie at the intersection of computer vision, computer graphics, and machine learning. Research Group | Teaching | Talks | Publications | Open Office HoursEmail: jbhuang@umd.edu (primary) | jbhuang@meta.com |
Prospective Ph.D. students starting in Fall 2025:
If you are not at UMD: We will be recruiting highly motivated Ph.D. students to join our lab. Please apply through the CS department and include my name as a potential advisor in your application.
If you are already a UMD student, please email me your CV, prior experiences, and how you would like to get involved.
Questions? Check out my answers to the Ph.D. Advisor Guide.
Prospective short-term students:
If you are undergraduate/gradudate students at UMD or at external institute looking to work on research with us, please fill the form. We will contact you if there is a match. (Please submit the form again if you used the old version before. Sorry!)
Prospective Undergraduate Students at UMD: Interested in engaging with computer vision research? Please fill out the form. My email inbox is flooded with various requests that I cannot handle. We will review the form and reach out via email if there is a match.
Prospective Faculty at UMD: The CS department is hiring multiple faculty! Find out more here. Please feel free to reach out if you have any questions.
Yao-Chih Lee | Yiran Xu | Yue Feng | Yi-Ting Chen |
Ting-Hsuan Liao | Hadi Alzayer | Songwei Ge | Kevin Zhang |
(with David Jacobs) | (with Christopher Metzler) | ||
Badour AlBahar (PhD 2022), now Assistant Professor at Kuwait University.
Chen Gao (PhD 2022), now Research Scientist at Meta
Yuliang Zou (PhD 2022), now Research Scientist at Waymo
Jinwoo Choi (PhD 2020), now an Assistant Professor at Kyung Hee Univeristy (Korea)
Sanjali Yadav, next a PhD student at University of Maryland College Park.
Esther Robb (MS 2021), next a PhD student at Stanford University.
Joseph Messou (MS 2020), next a PhD student at University of Maryland College Park.
Shih-Yang Su (MS 2020), next a PhD student at University of British Columbia.
Subhashree Radhakrishnan (MS 2018), next a Deep Learning Software Engineeer at NVIDIA.
Po-Han Huang (MS 2018), next a Deep Learning Software Engineeer at NVIDIA.
Sanket Lokegaonkar (MS 2018), next a Software Engineeer at Amazon AWS.
Adithya Nallabolu (MS 2017), next a Computer Vision R&D Engineer at Qualcomm.
Le Wang (BS 2014), next MS at Stanford, next ASIC/RTL Designer at Google.
Michael Qiu (BS 2014), next Senior Data Engineer at Capital One.
Anarghya Mitra (BS 2014), next Software Engineer at Google.
Zelun Luo (BS 2013), next a PhD student at Stanford University.
JunYoung Gwak (BS 2013), next a PhD student at Stanford University.
Danyang (Mike) Wang (BS 2012), next Analog Design Engineer at Analog Devices.
Linjia Chang (BS 2012), next MS at UIUC, now technical marketing engineer at Intel.
Sakshi Srivastava (BS 2012), next PhD student at UIUC.
Kevin Han (BS 2012), next PhD at UC Berkeley, now Senior Engineer at Pinnacle Photonics.
Yu-Ying Yeh, PhD student at University of California San Diego.
Yu-Lun Liu, PhD student at National Taiwan University, now at Assistant Professor at National Yang Ming Chiao Tung University.
Ishit Mehta, PhD student at University of California San Diego.
Chris Rockwell, PhD student at University of Michigan Ann Arbor.
Benjamin Attal, PhD student at Carnegie Mellon University.
Andreas Meuleman, PhD student at KAIST.
Xiaoming Zhao, PhD student at UIUC.
Badour AlBahar, PhD student at Virginia Tech.
Geng Lin, PhD student at University of Maryland College Park.
Chen Gao, PhD student at Virginia Tech, now Resarch Scientist at Meta.
Wenqi Xian, PhD student at Cornell Tech.
Xuan Luo, PhD student at University of Washington, now Resarch Scientist at Google.
Ting-I Hsieh (Intern 2020)
Yun-Chun Chen (Intern 2019), next PhD student at University of Toronto.
Chieh Hubert Lin (Intern 2019), next PhD student at UC Merced.
Meng-Li Shih (Intern 2018), next PhD student at University of Washington.
Jin-Dong Dong (Intern 2018), next PhD student at Carnegie Mellon University.
Wei-Yu Chen (Intern 2018), next PhD student at Carnegie Mellon University.
Chen Gao (Intern 2018), next PhD student at Virginia Tech.
Yen-Chen Lin (Intern 2017), next PhD student at MIT.
Hao-Wei Yeh (Intern 2017), next PhD student at University of Tokyo.
VideoGigaGAN: Towards Detail-rich Video Super-Resolution |
Magic Fixup: Streamlining Photo Editing by Watching Dynamic Videos |
IRIS: Inverse Rendering of Indoor Scenes from Low Dynamic Range Images |
UrbanIR: Large-Scale Urban Scene Inverse Rendering from a Single Video |
Rethinking Score Distillation as a Bridge Between Image Distributions |
Planar Reflection-Aware Neural Radiance Fields |
Fast View Synthesis of Casual Videos with Soup-of-Planes
Yao-Chih Lee,
Zhoutong Zhang,
Kevin Blackburn-Matzen,
Simon Niklaus,
Jianming Zhang,
Jia-Bin Huang, and
Feng Liu
European Conference on Computer Vision (ECCV) 2024
[Paper (PDF)]
[Project page]
Flash-Splat: 3D Reflection Removal with Flash Cues and Gaussian Splats |
Taming Latent Diffusion Model for Neural Radiance Field Inpainting |
Recent Trends in 3D Reconstruction of General Non-Rigid Scenes |
Modeling Ambient Scene Dynamics for Free-view Synthesis |
Seeing the World through Your Eyes |
TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusion
Yu-Ying Yeh,
Jia-Bin Huang,
Changil Kim,
Lei Xiao,
Thu Nguyen-Phuoc,
Numair Khan,
Cheng Zhang,
Manmohan Chandraker,
Carl S Marshall,
Zhao Dong, and
Zhengqin Li
Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024
[Paper (PDF)]
[Project page]
[Code]
[Demo]
Grounded Text-to-Image Synthesis with Attention Refocusing
Quynh Phung,
Songwei Ge,
Jia-Bin Huang
Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024
[Paper (PDF)]
[Project page]
[Code]
[Demo]
On the Content Bias in Frechet Video Distance |
FlowVid: Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesis |
LTM: Lightweight Textured Mesh Reconstruction of Unbounded Scenes Using Neural Fields |
In-N-Out: Faithful 3D GAN Inversion with Volumetric Decomposition for Face Editing |
Single-Image 3D Human Digitization with Shape-Guided Diffusion
Badour AlBahar,
Shunsuke Saito,
Hung-Yu Tseng,
Changil Kim,
Johannes Kopf, and
Jia-Bin Huang
ACM SIGGRAPH Asia 2023 (Conference track)
[Paper (PDF)]
[Project page]
[Code]
[Demo]
Visualizing Subtle Motions from Time-Varying Radiance Fields
Brandon Yushan Feng,
Hadi AlZayer,
Michael Rubinstein,
William T. Freeman, and
Jia-Bin Huang
Proceedings of International Conference on Computer Vision (ICCV), 2023
[Paper (PDF)]
[Project page]
[Code]
[Demo]
Expressive Text-to-Image Generation with Rich Text
Songwei Ge,
Taesung Park,
Jun-Yan Zhu, and
Jia-Bin Huang
Proceedings of International Conference on Computer Vision (ICCV), 2023
[Paper (PDF)]
[Project page]
[Code]
[Demo]
ClimateNeRF: Extreme Weather Synthesis in Neural Radiance Field
Yuan Li,
Zhi-Hao Lin,
David Forsyth,
Jia-Bin Huang, and
Shenlong Wang
Proceedings of International Conference on Computer Vision (ICCV), 2023
[Paper (PDF)]
[Project page]
[Code]
Dynamic Mesh-Aware Radiance Fields |
Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models |
Robust Omnimatte with 3D Background Modeling |
Neural-PBIR Reconstruction of Shape, Material, and Illumination |
Text-driven Visual Synthesis with Latent Diffusion Prior |
DisCO: Portrait Distortion Correction with Perspective-Aware 3D GANs |
Latent-Shift: Latent Diffusion with Temporal Shift for Efficient Text-to-Video Generation |
Shape-aware Text-driven Layered Video Editing |
Robust Dynamic Radiance Fields |
DC2: Dual-Camera Defocus Control by Learning to Refocus
Hadi Alzayer,
Abdullah Abuolaim,
Leung Chun Chan,
Yang Yang,
Ying Chen Lou,
Jia-Bin Huang, and
Abhishek Kar
Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
[Paper (PDF)]
[Project page]
[Video]
[Two Minutes Paper]
HyperReel: High-Fidelity 6-DoF Video with Ray-Conditioned Sampling
Benjamin Attal,
Jia-Bin Huang,
Christian Richardt,
Michael Zollhoefer,
Johannes Kopf,
Matthew O'Toole, and
Changil Kim
Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023 (Highlight)
[Paper (PDF)]
[Project page]
[Code]
[Video]
Consistent View Synthesis with Pose-Guided Diffusion Models
Hung-Yu Tseng,
Qinbo Li,
Changil Kim,
Suhib Alsisan,
Jia-Bin Huang, and
Johannes Kopf
Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
[Paper (PDF)]
[Project page]
[Video]
[Code]
Progressively Optimized Local Radiance Fields for Robust View Synthesis
Andreas Meuleman,
Yu-Lun Liu,
Chen Gao,
Jia-Bin Huang,
Changil Kim,
Min H. Kim, and
Johannes Kopf
Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
[Paper (PDF)]
[Project page]
[Code]
[Video]
A Safety-Performance Metric Enabling Computational Awareness in Autonomous Robots |
Learning Representational Invariances for Data-Efficient Action Recognition |
Temporally Consistent Semantic Video Editing |
Learning Instance-Specific Adaptation for Cross-Domain Segmentation |
Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformer |
Boosting View Synthesis with Residual Transfer |
Learning Neural Light Fields with Ray-Space Embedding Networks |
Neural Global Shutter: Learn to Restore Video from a Rolling Shutter Camera with Global Reset Feature |
Continuous and Diverse Image-to-Image Translation via Signed Attribute Vectors |
Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGAN
Badour AlBahar,
Jingwan (Cynthia) Lu,
Jimei Yang,
Zhixin Shu,
Eli Shechtman, and
Jia-Bin Huang
ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia), 2021
[Paper (PDF)]
[Project page]
[Code]
[Colab notebook]
Modulating StyleGAN for photorealistic human reposing and virtual try-on.
Learning to See Through Obstructions with Layered Decomposition
Yu-Lun Liu,
Wei-Sheng Lai,
Ming-Hsuan Yang,
Yung-Yu Chuang, and
Jia-Bin Huang
IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 2021
[Paper (PDF)]
[Project page]
[Code]
[Colab notebook]
News coverage: [New Scientists]
Using meta-learning to enable faster layered decomposition with test-time optimization.
AMICO: Amodal Instance Composition |
Dynamic View Synthesis from Dynamic Monocular Video |
Hybrid Neural Fusion for Full-frame Video Stabilization |
Automated Movement Assessment in Stroke Rehabilitation |
Robust Consistent Video Depth Estimation |
Space-time Neural Irradiance Fields for Free-Viewpoint Video |
PseudoSeg: Designing Pseudo Labels for Semantic Segmentation |
DropLoss for Long-Tail Instance Segmentation |
Flow-edge Guided Video Completion |
Semantic View Synthesis
|
DRG: Dual Relation Graph for Human-Object Interaction Detection |
NAS-DIP: Learning Deep Image Prior with Neural Architecture Search
|
FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning |
Shuffle and Attend: Video Domain Adaptation |
Learning Monocular Visual Odometry via Self-Supervised Long-Term Modeling |
Consistent Video Depth Estimation |
3D Photography using Context-aware Layered Depth Inpainting |
Learning to See Through Obstructions |
Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline |
Instance-aware Image Colorization |
Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation |
Unsupervised and Semi-Supervised Domain Adaptation for Action Recognition from Drones |
Reducing Footskate in Human Motion Reconstruction with Ground Contact Constraints |
Show, Match and Segment: Joint Weakly Supervised Learning of Semantic Matching and Object Co-segmentation |
DRIT++: Diverse Image-to-Image Translation via Disentangled Representations |
Tracking Persons-of-Interest via Unsupervised Representation Adaptation |
Portrait Neural Radiance Fields from a Single Image |
Few-shot Adaptation of Generative Adversarial Networks |
Why Can't I Dance in the Mall? Learning to Mitigate Scene Bias in Action Recognition |
Guided Image-to-Image Translation with Bi-Directional Feature Transformation |
CrDoCo: Pixel-level Domain Transfer with Cross-Domain Consistency |
SAIL-VOS: Semantic Amodal Instance Level Video Object Segmentation - A Synthetic Dataset and Baselines |
A Closer Look at Few-shot Classification |
Connecting the Digital and Physical World: Improving the Robustness of Adversarial Attacks |
Deep Paper Gestalt |
Source Form: An Automated Crowdsourced Object Generator |
Progressive Representation Adaptation for Weakly Supervised Object Localization |
Joint Image Filtering with Deep Convolutional Networks |
Multi-view Wire Art |
Diverse Image-to-Image Translation via Disentangled Representations |
DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistency |
Learning Blind Video Temporal Consistency |
VideoMatch: Matching based Video Object Segmentation |
Unsupervised Video Object Segmentation using Motion Saliency-Guided Spatio-Temporal Propagation |
iCAN: Instance-Centric Attention Network for Human-Object Interaction Detection |
DeepMVS: Learning Multi-View Stereopsis |
Deep Semantic Matching with Foreground Detection and Cycle-Consistency |
Semi-Automated Home-based Therapy for the Upper Extremity of Stroke Survivors |
Progressive Cyber-Human Intelligence for Social Good |
Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks |
Robust Visual Tracking via Hierarchical Convolutional Features |
Adaptive Correlation Filters with Long-Term and Short-Term Memory for Object Tracking |
Ensemble of Convolutional Neural Networks for Pose Estimation |
Semi-Supervised Learning for Optical Flow with Generative Adversarial Networks |
MaskRNN: Instance Level Video Object Segmentation |
Detecting Adversarial Attacks on Neural Network Policies with Visual Foresight |
Unsupervised Representation Learning by Sorting Sequences |
Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution |
HOMER: an interactive system for home based stroke rehabilitation |
Visual Analysis and Synthesis with Physically Grounded Constraints |
Temporally Coherent Completion of Dynamic Video |
Deep Joint Image Filter |
Tracking Persons-of-Interest via Adaptive Discriminative Features |
Unsupervised Visual Representation Learning by Graph-based Consistent Constraints |
Detecting Migrating Birds at Night |
A Comparative Study for Single Image Blind Deblurring |
Weakly Supervised Object Localization with Progressive Domain Adaptation |
Hierarchical Convolutional Features for Visual Tracking |
Single Image Super-Resolution from Transformed Self-Exemplars |
Image Completion using Planar Structure Guidance |
Towards Accurate and Robust Cross-Ratio based Gaze Trackers Through Learning From Simulation |
Transformation Guided Image Completion |
Saliency Detection via Divergence Analysis: A Unified Perspective ICPR 2012 Best Student Paper Award (Computer and Robotic Vision Track) |
Exploiting Self-Similarities for Single Frame Super-Resolution |
Single Image Deblurring with Adaptive Dictionary Learning |
Fast Sparse Representation with Prototypes |
Estimating Human Pose from Occluded Images |
Moving Cast Shadow Detection Using Physics-based Features |
A Physical Approach to Moving Cast Shadow Detection |
Image Recolorization For The Colorblind |
Learning Moving Cast Shadows for Foreground Detection |
Enhancing Color Representation for the Color Vision Impaired |
Information Preserving Color Transformation for Protanopia and Deuteranopia |
I set aside some time some weeks to meet with anyone (preferrably students from underrepresented groups). Free free to sign up below if you would like to chat with me on any topics.
06/18/2024 | Invited talk at CVPR 2024 workshop on Implicit Neural Representation for Vision |
06/18/2024 | Invited talk at CVPR workshop on The Future of Generative Visual Art |
06/18/2024 | Invited talk at CVPR workshop on SyntaGen: Harnessing Generative Models for Synthetic Visual Datasets |
06/17/2024 | Invited talk at CVPR workshop on Computer Vision for Fashion, Art, and Design |
12/06/2023 | Adobe GenTech seminar |
12/08/2022 | GRAIL Vision Seminar at University of Washington |
12/05/2022 | Faculty Job Workshop at University of Maryland College Park [Slides] |
11/21/2022 | Computer Science and Engineering AI Seminar at Ohio State University |
11/18/2022 | Invited guest lecture in CS197 at Harvard University [Slides] |
11/18/2022 | CS Department Seminar at Virginia Tech |
10/14/2021 | Invited talk at 2021 華仁全球講座 (Host: Jenq-Neng Hwang) |
08/26/2021 | Invited talk at Graphics And Mixed Environment Seminar (Host: Jun-Yan Zhu) |
07/20/2021 | Invited talk at School of Computer Science, Tel Aviv University (Host: Daniel Cohen-Or) |
07/13/2021 | Invited talk at VGG seminar, University of Oxford |
07/07/2021 | Invited talk at IMAGINE lab, Ecole des Ponts ParisTech |
05/10/2021 | Invited talk at National Yang Ming Chiao Tung University |
04/23/2021 | Invited talk at UT Austin (Host: Zhangyang Wang) |
04/20/2021 | School of Interactive Computing Seminar at Georgia Tech (Host: Frank Dellaert) |
04/08/2021 | CS Colloquium at Cornell University (Host: Noah Snavely) |
04/05/2021 | ECE Seminar at the University of Michigan |
03/29/2021 | CS Colloquium at University of Maryland (Host: Abhinav Shrivastava) |
03/22/2021 | CS Colloquium at University of North Carolina Chapel Hill (Host: Mohit Bansal) |
02/26/2021 | EECS Department Seminar at UC Merced (Host: Ming-Hsuan Yang) |
02/05/2021 | 3M Non-Tenured Faculty Award Symposium |
02/02/2021 | Visual Information Laboratory Seminar, University of Bristol (Host: Dima Damen) |
01/26/2021 | Computer Vision Seminar, University of Illinois, Urbana-Champaign |
01/20/2021 | 3D Representations Reading Group at MIT [Video] [Slides] |
04 / 2024 | Thanks Qualcomm for the Research Award! |
04 / 2024 | Thanks Google for the Research Scholar Award! |
09 / 2022 | Thanks Adobe, Meta, Google for the gift donation. |
06 / 2021 | Thanks Virginia’s Commonwealth Cyber Initiative for the grants on Detecting Disinformation and Misinformation (with Ruoxi Jia Adrienne Ivory) |
04 / 2021 | Thanks Center for Human-Computer Interaction for the Planning Grants for Large-scale Research Efforts(with Douglas Bowman, Joe Gabbard,Nazila Roofigari-Esfahan, and Hoda Eldardiry) |
04 / 2021 | Congrats to Esther and Meng-Li for getting into top PhD programs!Esther will join CS@Stanford University. Meng-Li will join CSE @ University of Washington. |
08 / 2020 | Thanks Facebook and Adobe for the gift donation. |
08 / 2020 | Thank NSF for the Smart and Connected Health (SCH) grant (1.1M for four years). |
03 / 2020 | Thank 3M for supporting our work with the 3M Non-Tenure Faculty Award. |
02 / 2020 | Thanks 4-VA for the supporting us with a collaborative research grant (with Vicente Ordonez at University of Virginia). |
09 / 2019 | Thanks NSF for supporting us with a medium Cyber-Physical Systems (CPS) grant (with Ryan K. Williams, Haibo Zeng, Changhee Jung). |
09 / 2019 | Thanks Rehabilitation Engineering Research Centers (RERC) for supporting us through the Rehabilitation Strategies, Techniques, and Interventions program (with Shirley Ryan Ability Lab, Thanassis Rikakis, Aisling Kelliher). |
04 / 2019 | Thanks SAMSUNG for continuing to support our research through the GRO award (with Alexander Schwing). |
02 / 2019 | Thanks Google for supporting our research with the Google Faculty Research Award. |
04 / 2018 | Thanks NSF for supporting our research with an CRII award. |
CVPR: Area Chair 2019, 2023, 2025
ICCV: Area Chair 2019, 2021, 2023
ECCV: Area Chair 2022, 2024
BMVC: Area Chair 2019, 2020, 2021, 2022
WACV: Area Chair 2020, 2022, 2023
IET Computer Vision: Associate Editor 2020-2022
IEEE Transactions on Pattern Analysis and Machine Intelligence: Associate Editor
SIGGRAPH: Technical Program Committee 2022, 2023
SIGGRAPH Asia: Technical Program Committee 2020, 2021, 2023
Eurographics: International Program Committee 2023
Computer Graphics Forum: Associate Editor 2021-2023
NeurIPS: Area Chair 2021, 2022, 2023, 2024
ICML: Area Chair 2022, 2024
ICLR: Area Chair 2023, 2024, 2025
AAAI: Area Chair 2023
TMLR: Action Editor 2022, 2023
Student Mentoring: CVPR 2021, 2022; ICCV 2021
Doctoral Consortium: ICCV 2021, CVPR 2023
LatinX in AI Mentoring Program: CVPR 2021, ICCV 2021
“A Conversation With …”: SIGGRAPH Asia 2021
CMSC 426 Computer Vision, Spring 2023, 2024
CMSC 733 Computer Processing of Pictorial Information, Fall 2022
CMSC 848K Multimodal Foundation Models, Fall 2024
CMSC 800 How to Conduct Great Research, Spring 2024
Advanced Machine Learning: Spring 2021, Spring 2020, Spring 2019
Advanced Computer Vision: Spring 2017
Computer Vision: Fall 2018, Fall 2017, Fall 2016
Deep Learning: Fall 2020, Fall 2019
Introduction to Programming: Spring 2018