Zhizhang "Kevin" Hu
About Me
Hi, Welcome to my page! I am a research data scientist at Microsoft AI. I work on open-world multi-agent spatial intellegence, and multimodal large language models. I earned my Ph.D. in Electrical Engineering and Computer Science at University of California, Merced with a focus on multimodal learning and ubiquitous computing. Please feel free to contact me if you want to chat/ collaborate/ ask for the latest resume :-D.
Interests
- Multimodal Large Language Model
- Multi-agent Reasoning
Education
-
Ph.D. in Computer Science, 2024
University of California, Merced
-
M.Sc in Building Science, 2020
Carnegie Mellon University
-
B.Eng in Mechanical Engineering, 2018
Southwest Jiaotong University
Experience
Research Data Scientist II
Microsoft AI
Multi-agent spatial intellegence and open-world visual language reasoning.
Ph.D. Researcher
University of California, Merced
Work in following areas:
- Physics-informed multimodal sensing and learning algorithms.
- Causal inference for deep learning bias reduction.
- Smart healthcare with heterogeneous Internet of Things(IoT) systems.
- Representation learning and transfer learning algorithms.
Data Scientist Intern
Microsoft AI
Working with multimodal large language models' knowledge distillation and fine-tuning on referral visual grounding tasks. Advised by Dr. Jianguo Long and Pak Kiu Chung. Supervised and advised by Dr. Ming Tan.
Applied Scientist Intern
Amazon Inc.
Working on the visual cues-guided online text denoising for robust multimodal learning. Advised by Dr.Shasha Li, Dr. Ming Du. Supervised and advised by Dr. Arnab Dhua. Output paper: MM-LTP @CVPRW 2024.
Applied Scientist Intern
Amazon Inc.
Working on the vision-language multimodal learning for language-guided compostional image-image retrieval. Advised by Dr.Xinliang Zhu, Dr. Son Tran and Dr. Rene Vidal. Supervised and advised by Dr. Arnab Dhua. Output paper: ProVLA @ ICCVW 2023.
Recent Publications
Quickly discover relevant content by filtering publications.
(2024).
GraPhy: Graph-Based Physics-Guided Urban Air Quality Modeling for Monitoring-Constrained Regions.
BuildSys 2024.
(2024).
De-noised Vision-language Fusion Guided by Visual Cues for E-commerce Product Search.
2024 IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR) Workshops.
(2024).
IOTeeth: Intra-Oral Teeth Sensing System for Dental Occlusal Diseases Recognition.
IMWUT 2024.
(2023).
ProVLA: Compositional Image Search with Progressive Vision-language Alignment and Multimodal Fusion.
2023 IEEE/CVF International Conference on Computer Vision (ICCV) Workshops.
(2023).
CMA: Cross-Modal Association Between Wearable and Structural Vibration Signal Segments for Indoor Occupant Sensing.
IPSN 2023.
(2023).
Poster: Integrating On- and Off-body Sensing for Young Adults Failure to Launch (FTL) Behavior Profiling.
IPSN 2023.
(2022).
CIPhy: Causal Intervention with Physical Confounder from IoT Sensor Data for Robust Occupant Information Inference.
SenSys 2022 AIoT Workshop.
(2022).
MODES: multi-sensor occupancy data-driven estimation system for smart buildings.
ACM e-Energy.
(2022).
VMA: Domain Variance- and Modality-Aware Model Transfer for Fine-Grained Occupant Activity Recognition.
IPSN 2022.
(2022).
Demo: Real-Time Teeth Functional Occlusion Monitoring via In-Mouth Vibration Sensing.
IPSN 2022.
(2022).
Poster: Sedentary Posture Muscle Monitoring via Active Vibratory Sensing.
IPSN 2022 (Best Poster Award).
(2021).
AutoQual: task-oriented structural vibration sensing quality assessment leveraging co-located mobile sensing context.
CCF Transactions on Pervasive Computing and Interaction.
(2021).
Footstep-Induced Floor Vibration Dataset: Reusability and Transferability Analysis.
SenSys 2021 DATA Workshop.
(2021).
Poster: Vibration-based Indoor Occupant Gait Monitoring with Robot Vacuum Cleaners.
IoTDI 2021.
(2021).
Vibration-based indoor human sensing quality reinforcement via Thompson sampling.
Proceedings of the First International Workshop on Cyber-Physical-Human System Design and Implementation.
(2020).
A window-based sequence-to-one approach with dynamic voting for nurse care activity recognition using acceleration-based wearable sensor.
UbiComp/ISWC 2020 Workshop(Best Paper Award).
(2020).
Fine-grained activities recognition with coarse-grained labeled multi-modal data.
UbiComp/ISWC 2020 CML-IoT Workshop.
(2020).
Improving the Interoperability of gbXML Data Model through Redefining Data Mapping Rules of HVAC Systems.
ASHRAE Transactions.
(2019).
Device-free sleep stage recognition through bed frame vibration sensing.
BuildSys 2019 DFHS Workshop.
Contact Me
- 515010 NE 36th St, Redmond, WA 98052