Jia-Bin Huang
 |
Capital One endowed Associate Professor Research Scientist (20%) My research interests lie at the intersection of computer vision, computer graphics, and machine learning. Research Group | Teaching | Talks | Publications | Open Office HoursEmail: jbhuang@umd.edu (primary) | jbhuang@meta.com |
More about me: CV | CV of Failures | Awesome Tips | YouTube | Twitter | Threads
Prospective Ph.D. students starting in Fall 2024:
If you are not at UMD: We will be recruiting highly motivated Ph.D. students to join our lab. Please apply through the CS department and include my name as a potential advisor in your application.
If you are already a UMD student, please email me your CV, prior experiences, and how you would like to get involved.
Questions? Check out my answers to the Ph.D. Advisor Guide.
Prospective Visiting Students: For short-term visiting, please fill in the form here. We will contact you if there is a match.
Prospective Undergraduate Students at UMD: Interested in engaging with computer vision research? Please reach out via email with your CV, transcript, your level of commitment (e.g., 10-20 hours/week), and description of what types of projects you want to work on.
Prospective Faculty at UMD: The CS department is hiring multiple faculty! Find out more here. Please feel free to reach out if you have any questions.
Prospective Research Interns at Meta: We are actively recruiting Summer 2023 research interns in the Computational Photography Group at Meta. The offers will be given in a rolling basis so apply here soon!
Research Group
PhD students
 |
 |
 |
 |
| Yao-Chih Lee | Yiran Xu | Yue Feng | Yi-Ting Chen |
 |
 |
 |
 |
| Ting-Hsuan Liao | Hadi Alzayer | Songwei Ge | Kevin Zhang |
| (with David Jacobs) | (with Christopher Metzler) | ||
Undergraduate students
 |
 | ||
| Elizabeth Qiu | Sanjali Yadav | ||
Alumni
Former PhD Students
Badour AlBahar (PhD 2022), now Assistant Professor at Kuwait University.
Chen Gao (PhD 2022), now Research Scientist at Meta
Yuliang Zou (PhD 2022), now Research Scientist at Waymo
Jinwoo Choi (PhD 2020), now an Assistant Professor at Kyung Hee Univeristy (Korea)
Former MS/BS Students
Esther Robb (MS 2021), next a PhD student at Stanford University.
Joseph Messou (MS 2020), next a PhD student at University of Maryland College Park.
Shih-Yang Su (MS 2020), next a PhD student at University of British Columbia.
Subhashree Radhakrishnan (MS 2018), next a Deep Learning Software Engineeer at NVIDIA.
Po-Han Huang (MS 2018), next a Deep Learning Software Engineeer at NVIDIA.
Sanket Lokegaonkar (MS 2018), next a Software Engineeer at Amazon AWS.
Adithya Nallabolu (MS 2017), next a Computer Vision R&D Engineer at Qualcomm.
Le Wang (BS 2014), next MS at Stanford, next ASIC/RTL Designer at Google.
Michael Qiu (BS 2014), next Senior Data Engineer at Capital One.
Anarghya Mitra (BS 2014), next Software Engineer at Google.
Zelun Luo (BS 2013), next a PhD student at Stanford University.
JunYoung Gwak (BS 2013), next a PhD student at Stanford University.
Danyang (Mike) Wang (BS 2012), next Analog Design Engineer at Analog Devices.
Linjia Chang (BS 2012), next MS at UIUC, now technical marketing engineer at Intel.
Sakshi Srivastava (BS 2012), next PhD student at UIUC.
Kevin Han (BS 2012), next PhD at UC Berkeley, now Senior Engineer at Pinnacle Photonics.
Former Interns (as host or co-host)
Yu-Lun Liu, PhD student at National Taiwan University, now at Assistant Professor at National Yang Ming Chiao Tung University.
Ishit Mehta, PhD student at University of California San Diego.
Chris Rockwell, PhD student at University of Michigan Ann Arbor.
Benjamin Attal, PhD student at Carnegie Mellon University.
Andreas Meuleman, PhD student at KAIST.
Xiaoming Zhao, PhD student at UIUC.
Badour AlBahar, PhD student at Virginia Tech.
Geng Lin, PhD student at University of Maryland College Park.
Chen Gao, PhD student at Virginia Tech, now Resarch Scientist at Meta.
Wenqi Xian, PhD student at Cornell Tech.
Xuan Luo, PhD student at University of Washington, now Resarch Scientist at Google.
Former Visiting Students
Ting-I Hsieh (Intern 2020)
Yun-Chun Chen (Intern 2019), next PhD student at University of Toronto.
Chieh Hubert Lin (Intern 2019), next PhD student at UC Merced.
Meng-Li Shih (Intern 2018), next PhD student at University of Washington.
Jin-Dong Dong (Intern 2018), next PhD student at Carnegie Mellon University.
Wei-Yu Chen (Intern 2018), next PhD student at Carnegie Mellon University.
Chen Gao (Intern 2018), next PhD student at Virginia Tech.
Yen-Chen Lin (Intern 2017), next PhD student at MIT.
Hao-Wei Yeh (Intern 2017), next PhD student at University of Tokyo.
Publications
Visualizing Subtle Motions from Time-Varying Radiance Fields
Brandon Yushan Feng,
Hadi AlZayer,
Michael Rubinstein,
William T. Freeman, and
Jia-Bin Huang
Proceedings of International Conference on Computer Vision (ICCV), 2023
[Paper (PDF)]
[Project page]
[Code]
[Demo]
Expressive Text-to-Image Generation with Rich Text
Songwei Ge,
Taesung Park,
Jun-Yan Zhu, and
Jia-Bin Huang
Proceedings of International Conference on Computer Vision (ICCV), 2023
[Paper (PDF)]
[Project page]
[Code]
[Demo]
 |
ClimateNeRF: Extreme Weather Synthesis in Neural Radiance Field |
|
|
Dynamic Mesh-Aware Radiance Fields |
|
|
Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models |
|
|
Robust Omnimatte with 3D Background Modeling |
|
|
Neural-PBIR Reconstruction of Shape, Material, and Illumination |
|
|
Seeing the World through Your Eyes |
|
|
UrbanIR: Large-Scale Urban Scene Inverse Rendering from a Single Video |
|
|
Grounded Text-to-Image Synthesis with Attention Refocusing |
|
|
Text-driven Visual Synthesis with Latent Diffusion Prior |
|
|
DisCO: Portrait Distortion Correction with Perspective-Aware 3D GANs |
|
|
In-N-Out: Face Video Inversion and Editing with Volumetric Decomposition |
|
|
Latent-Shift: Latent Diffusion with Temporal Shift for Efficient Text-to-Video Generation |
|
|
Shape-aware Text-driven Layered Video Editing |
|
|
Robust Dynamic Radiance Fields |
DC2: Dual-Camera Defocus Control by Learning to Refocus
Hadi Alzayer,
Abdullah Abuolaim,
Leung Chun Chan,
Yang Yang,
Ying Chen Lou,
Jia-Bin Huang, and
Abhishek Kar
Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
[Paper (PDF)]
[Project page]
[Video]
[Two Minutes Paper]
HyperReel: High-Fidelity 6-DoF Video with Ray-Conditioned Sampling
Benjamin Attal,
Jia-Bin Huang,
Christian Richardt,
Michael Zollhoefer,
Johannes Kopf,
Matthew O'Toole, and
Changil Kim
Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023 (Highlight)
[Paper (PDF)]
[Project page]
[Code]
[Video]
Consistent View Synthesis with Pose-Guided Diffusion Models
Hung-Yu Tseng,
Qinbo Li,
Changil Kim,
Suhib Alsisan,
Jia-Bin Huang, and
Johannes Kopf
Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
[Paper (PDF)]
[Project page]
[Video]
[Code]
Progressively Optimized Local Radiance Fields for Robust View Synthesis
Andreas Meuleman,
Yu-Lun Liu,
Chen Gao,
Jia-Bin Huang,
Changil Kim,
Min H. Kim, and
Johannes Kopf
Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
[Paper (PDF)]
[Project page]
[Code]
[Video]
|
|
Learning Representational Invariances for Data-Efficient Action Recognition |
|
|
Temporally Consistent Semantic Video Editing |
|
|
Learning Instance-Specific Adaptation for Cross-Domain Segmentation |
|
|
Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformer |
|
|
Boosting View Synthesis with Residual Transfer |
|
|
Learning Neural Light Fields with Ray-Space Embedding Networks |
|
|
Neural Global Shutter: Learn to Restore Video from a Rolling Shutter Camera with Global Reset Feature |
|
Continuous and Diverse Image-to-Image Translation via Signed Attribute Vectors |
Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGAN
Badour AlBahar,
Jingwan (Cynthia) Lu,
Jimei Yang,
Zhixin Shu,
Eli Shechtman, and
Jia-Bin Huang
ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia), 2021
[Paper (PDF)]
[Project page]
[Code]
[Colab notebook]
Modulating StyleGAN for photorealistic human reposing and virtual try-on.
Learning to See Through Obstructions with Layered Decomposition
Yu-Lun Liu,
Wei-Sheng Lai,
Ming-Hsuan Yang,
Yung-Yu Chuang, and
Jia-Bin Huang
IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 2021
[Paper (PDF)]
[Project page]
[Code]
[Colab notebook]
News coverage: [New Scientists]
Using meta-learning to enable faster layered decomposition with test-time optimization.
|
|
AMICO: Amodal Instance Composition |
|
Dynamic View Synthesis from Dynamic Monocular Video |
|
Hybrid Neural Fusion for Full-frame Video Stabilization |
 |
Automated Movement Assessment in Stroke Rehabilitation |
|
Robust Consistent Video Depth Estimation |
|
Space-time Neural Irradiance Fields for Free-Viewpoint Video |
 |
PseudoSeg: Designing Pseudo Labels for Semantic Segmentation |
 |
DropLoss for Long-Tail Instance Segmentation |
|
Flow-edge Guided Video Completion |
|
Semantic View Synthesis
|
|
DRG: Dual Relation Graph for Human-Object Interaction Detection |
|
NAS-DIP: Learning Deep Image Prior with Neural Architecture Search
|
 |
FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning |
 |
Shuffle and Attend: Video Domain Adaptation |
|
Learning Monocular Visual Odometry via Self-Supervised Long-Term Modeling |
|
Consistent Video Depth Estimation |
|
3D Photography using Context-aware Layered Depth Inpainting |
|
Learning to See Through Obstructions |
|
Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline |
|
Instance-aware Image Colorization |
 |
Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation |
|
Unsupervised and Semi-Supervised Domain Adaptation for Action Recognition from Drones |
 |
Reducing Footskate in Human Motion Reconstruction with Ground Contact Constraints |
 |
Show, Match and Segment: Joint Weakly Supervised Learning of Semantic Matching and Object Co-segmentation |
 |
DRIT++: Diverse Image-to-Image Translation via Disentangled Representations |
 |
Tracking Persons-of-Interest via Unsupervised Representation Adaptation |
|
Portrait Neural Radiance Fields from a Single Image |
|
Few-shot Adaptation of Generative Adversarial Networks |
 |
Why Can't I Dance in the Mall? Learning to Mitigate Scene Bias in Action Recognition |
 |
Guided Image-to-Image Translation with Bi-Directional Feature Transformation |
 |
CrDoCo: Pixel-level Domain Transfer with Cross-Domain Consistency |
|
SAIL-VOS: Semantic Amodal Instance Level Video Object Segmentation - A Synthetic Dataset and Baselines |
 |
A Closer Look at Few-shot Classification |
 |
Connecting the Digital and Physical World: Improving the Robustness of Adversarial Attacks |
 |
Deep Paper Gestalt |
 |
Source Form: An Automated Crowdsourced Object Generator |
 |
Progressive Representation Adaptation for Weakly Supervised Object Localization |
 |
Joint Image Filtering with Deep Convolutional Networks |
|
Multi-view Wire Art |
 |
Diverse Image-to-Image Translation via Disentangled Representations |
 |
DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistency |
|
Learning Blind Video Temporal Consistency |
|
VideoMatch: Matching based Video Object Segmentation |
|
Unsupervised Video Object Segmentation using Motion Saliency-Guided Spatio-Temporal Propagation |
 |
iCAN: Instance-Centric Attention Network for Human-Object Interaction Detection |
 |
DeepMVS: Learning Multi-View Stereopsis |
 |
Deep Semantic Matching with Foreground Detection and Cycle-Consistency |
 |
Semi-Automated Home-based Therapy for the Upper Extremity of Stroke Survivors |
 |
Progressive Cyber-Human Intelligence for Social Good |
 |
Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks |
 |
Robust Visual Tracking via Hierarchical Convolutional Features |
 |
Adaptive Correlation Filters with Long-Term and Short-Term Memory for Object Tracking |
 |
Ensemble of Convolutional Neural Networks for Pose Estimation |
 |
Semi-Supervised Learning for Optical Flow with Generative Adversarial Networks |
 |
MaskRNN: Instance Level Video Object Segmentation |
 |
Detecting Adversarial Attacks on Neural Network Policies with Visual Foresight |
 |
Unsupervised Representation Learning by Sorting Sequences |
|
|
Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution |
|
|
HOMER: an interactive system for home based stroke rehabilitation |
 |
Visual Analysis and Synthesis with Physically Grounded Constraints |
 |
Temporally Coherent Completion of Dynamic Video |
 |
Deep Joint Image Filter |
|
|
Tracking Persons-of-Interest via Adaptive Discriminative Features |
 |
Unsupervised Visual Representation Learning by Graph-based Consistent Constraints |
 |
Detecting Migrating Birds at Night |
 |
A Comparative Study for Single Image Blind Deblurring |
|
|
Weakly Supervised Object Localization with Progressive Domain Adaptation |
|
|
Hierarchical Convolutional Features for Visual Tracking |
 |
Single Image Super-Resolution from Transformed Self-Exemplars |
 |
Image Completion using Planar Structure Guidance |
 |
Towards Accurate and Robust Cross-Ratio based Gaze Trackers Through Learning From Simulation |
 |
Transformation Guided Image Completion |
 |
Saliency Detection via Divergence Analysis: A Unified Perspective ICPR 2012 Best Student Paper Award (Computer and Robotic Vision Track) |
 |
Exploiting Self-Similarities for Single Frame Super-Resolution |
 |
Single Image Deblurring with Adaptive Dictionary Learning |
 |
Fast Sparse Representation with Prototypes |
 |
Estimating Human Pose from Occluded Images |
 |
Moving Cast Shadow Detection Using Physics-based Features |
 |
A Physical Approach to Moving Cast Shadow Detection |
 |
Image Recolorization For The Colorblind |
 |
Learning Moving Cast Shadows for Foreground Detection |
 |
Enhancing Color Representation for the Color Vision Impaired |
 |
Information Preserving Color Transformation for Protanopia and Deuteranopia |
Open Office Hour
I set aside some time some weeks to meet with anyone (preferrably students from underrepresented groups). Free free to sign up below if you would like to chat with me on any topics.
Recent talks
| 12/08/2022 | GRAIL Vision Seminar at University of Washington |
| 12/05/2022 | Faculty Job Workshop at University of Maryland College Park [Slides] |
| 11/21/2022 | Computer Science and Engineering AI Seminar at Ohio State University |
| 11/18/2022 | Invited guest lecture in CS197 at Harvard University [Slides] |
| 11/18/2022 | CS Department Seminar at Virginia Tech |
| 10/14/2021 | Invited talk at 2021 華仁全球講座 (Host: Jenq-Neng Hwang) |
| 08/26/2021 | Invited talk at Graphics And Mixed Environment Seminar (Host: Jun-Yan Zhu) |
| 07/20/2021 | Invited talk at School of Computer Science, Tel Aviv University (Host: Daniel Cohen-Or) |
| 07/13/2021 | Invited talk at VGG seminar, University of Oxford |
| 07/07/2021 | Invited talk at IMAGINE lab, Ecole des Ponts ParisTech |
| 05/10/2021 | Invited talk at National Yang Ming Chiao Tung University |
| 04/23/2021 | Invited talk at UT Austin (Host: Zhangyang Wang) |
| 04/20/2021 | School of Interactive Computing Seminar at Georgia Tech (Host: Frank Dellaert) |
| 04/08/2021 | CS Colloquium at Cornell University (Host: Noah Snavely) |
| 04/05/2021 | ECE Seminar at the University of Michigan |
| 03/29/2021 | CS Colloquium at University of Maryland (Host: Abhinav Shrivastava) |
| 03/22/2021 | CS Colloquium at University of North Carolina Chapel Hill (Host: Mohit Bansal) |
| 02/26/2021 | EECS Department Seminar at UC Merced (Host: Ming-Hsuan Yang) |
| 02/05/2021 | 3M Non-Tenured Faculty Award Symposium |
| 02/02/2021 | Visual Information Laboratory Seminar, University of Bristol (Host: Dima Damen) |
| 01/26/2021 | Computer Vision Seminar, University of Illinois, Urbana-Champaign |
| 01/20/2021 | 3D Representations Reading Group at MIT [Video] [Slides] |
Highlights and News
| 09 / 2022 | Thanks Adobe, Meta, Google for the gift donation. |
| 06 / 2021 | Thanks Virginia’s Commonwealth Cyber Initiative for the grants on Detecting Disinformation and Misinformation (with Ruoxi Jia Adrienne Ivory) |
| 04 / 2021 | Thanks Center for Human-Computer Interaction for the Planning Grants for Large-scale Research Efforts(with Douglas Bowman, Joe Gabbard,Nazila Roofigari-Esfahan, and Hoda Eldardiry) |
| 04 / 2021 | Congrats to Esther and Meng-Li for getting into top PhD programs!Esther will join CS@Stanford University. Meng-Li will join CSE @ University of Washington. |
| 08 / 2020 | Thanks Facebook and Adobe for the gift donation. |
| 08 / 2020 | Thank NSF for the Smart and Connected Health (SCH) grant (1.1M for four years). |
| 03 / 2020 | Thank 3M for supporting our work with the 3M Non-Tenure Faculty Award. |
| 02 / 2020 | Thanks 4-VA for the supporting us with a collaborative research grant (with Vicente Ordonez at University of Virginia). |
| 09 / 2019 | Thanks NSF for supporting us with a medium Cyber-Physical Systems (CPS) grant (with Ryan K. Williams, Haibo Zeng, Changhee Jung). |
| 09 / 2019 | Thanks Rehabilitation Engineering Research Centers (RERC) for supporting us through the Rehabilitation Strategies, Techniques, and Interventions program (with Shirley Ryan Ability Lab, Thanassis Rikakis, Aisling Kelliher). |
| 04 / 2019 | Thanks SAMSUNG for continuing to support our research through the GRO award (with Alexander Schwing). |
| 02 / 2019 | Thanks Google for supporting our research with the Google Faculty Research Award. |
| 04 / 2018 | Thanks NSF for supporting our research with an CRII award. |
Recent Community Services
Computer Vision
-
CVPR: Area Chair 2019, 2023
ICCV: Area Chair 2019, 2021, 2023
ECCV: Area Chair 2022;
BMVC: Area Chair 2019, 2020, 2021, 2022
WACV: Area Chair 2020, 2022, 2023
IET Computer Vision: Associate Editor 2020-2022
IEEE Transactions on Pattern Analysis and Machine Intelligence: Associate Editor
Computer Graphics
-
SIGGRAPH: Technical Program Committee 2022, 2023
SIGGRAPH Asia: Technical Program Committee 2020, 2021, 2023
Eurographics: International Program Committee 2023
Computer Graphics Forum: Associate Editor 2021-2023
Artificial Intelligence / Machine Learning
-
NeurIPS: Area Chair 2021, 2022, 2023
ICML: Area Chair 2022
ICLR: Area Chair 2023
AAAI: Area Chair 2023
TMLR: Action Editor 2022, 2023
Mentoring
-
Student Mentoring: CVPR 2021, 2022; ICCV 2021
Doctoral Consortium: ICCV 2021, CVPR 2023
LatinX in AI Mentoring Program: CVPR 2021, ICCV 2021
“A Conversation With …”: SIGGRAPH Asia 2021
Teaching
University of Maryland College Park
CMSC 426 Computer Vision, Spring 2023
CMSC 733 Computer Processing of Pictorial Information, Fall 2022
Virginia Tech
Advanced Machine Learning: Spring 2021, Spring 2020, Spring 2019
Advanced Computer Vision: Spring 2017
Computer Vision: Fall 2018, Fall 2017, Fall 2016
Deep Learning: Fall 2020, Fall 2019
Introduction to Programming: Spring 2018
