Zhao Zirui
ZHAO ZIRUI
XR / VR / AI Interaction / Architecture
Education
Education background and core modules
Goldsmiths, University of London
MA Virtual and Augmented Reality
Core modules: Augmented Realities, Modelling and Animation, Virtual Reality, Approaches to Play, and Games Programming for Game Engines.
Harbin Institute of Technology
Bachelor of Architecture
Core modules: Future Spatial Structures, Green Building Systems Design and Research, Augmented Reality, VR Creative Image Design, and 3D Printing Technology and Applications.
Research Experience
Labs, internships, and research assistantships
Research Assistant, HKUST(GZ)
- Developed Unity 6 / Meta Quest 3 immersive and mixed-reality prototypes across VR artwork and AI language-learning scenarios, integrating sensor-driven interaction, voice AI, and scene-based learning flows.
- Contributed to Respire 2.0, an immersive interactive VR artwork using Vernier respiratory belt data to synchronize a 3D organic sculpture with users' breathing through real-time light and motion feedback.
- Built an MR English learning assistant with cafe conversation and presentation training scenes, integrating Tencent Cloud STT/TTS and Aliyun Qwen Flash for speech recognition, AI dialogue generation, voice playback, coach feedback, and audience Q&A.
Research Assistant, University of Auckland, Augmented Human Lab
- Developed an immersive VR system in Unreal Engine that transformed real-time 3D bird positions and species data into dynamic visual markers and spatialized birdsong, creating a visual and sonic expression of ecological data.
- Integrated a LiDAR point-cloud forest environment, data-driven event triggers, and an Ambisonic spatial audio system to build a data-driven ecological narrative experience planned for exhibition at the Len Lye Centre in New Zealand.
Research Intern, Narrative Space Lab, City University of Hong Kong
- Contributed to a multi-agent narrative game designed to strengthen sustainability awareness, studying how virtual communities shape individual attitudes toward sustainable behavior.
- Designed gameplay systems, AI prompting structures, dialogue logic, and visual assets to model emotional contagion and cognitive influence, supporting a CHI PLAY 2026 submission on multi-agent effects in sustainability attitudes.
Research Assistant, Oxford Cognitive Approaches to Psychosis Lab
- Developed a multimodal emotion-aware VR dialogue system in Unity with ONNX Sentis, combining speech and text analysis with VAD-conditioned prompting to generate empathetic AI responses.
- Built the dialogue core, prompt generation pipeline, and virtual avatar behaviors so AI responses could adapt tone, style, and content to the user’s emotional state through synchronized voice, facial expression, and body cues.
Research Assistant, SEE VR Lab, Goldsmiths, University of London
- Investigated the therapeutic potential of combining AI agents and VR for Social Anxiety Disorder through qualitative analysis and systematic review, comparing AI-supported intervention with traditional CBT outcomes.
- Handled Unity implementation and data analysis, using GAD-7 and STAI-S to evaluate how AI-enhanced VR counseling may reduce anxiety symptoms.
Research Intern, Virtual Reality Laboratory, Harbin Institute of Technology
- Built realistic natural scenes in Unity with HTC VIVE and collected student physiological data such as BVP and HR through E4 wristbands to evaluate how VR environments affect psychological restoration.
- Proposed design and construction methods for virtual restorative environments in academic contexts, examining how perceived restorative qualities in VR influence mental recovery.
Research Intern, Center for Digital Design and Innovation, Harbin Institute of Technology
- Calculated Green View Ratio for different indoor planting and green wall configurations and ran experiments on the restorative effects of campus green landscapes for university students.
- Used the Short-version Revised Restoration Scale to evaluate restorative benefit and provide data support for healthier, more restorative campus environmental design strategies.
Publications
Selected papers and proceedings
- Zhao, Z. (2026). Designing Constrained Generative NPCs with a Dynamic Evidence Graph for Investigative Narrative Worlds. In Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems (CHI EA '26). ACM. https://doi.org/10.1145/3772363.3798387
- Zhao, Z. (2026). Multimodal Emotion Perception for VR Dialogue: Towards Improved User Experience. In Proceedings of the IEEE VR 2026 Conference on Virtual Reality and 3D User Interfaces (IEEE VR 2026). IEEE.
- Zhao, Z., & Yin, C. (2025, July). Leveraging AI-driven VR environments to reshape urban planning and community engagement. Paper presented at the 19th International Conference on Computational Urban Planning and Urban Management (CUPUM 2025), London. https://doi.org/10.17605/OSF.IO/ABYQH
- Yin, C., Li, T., Xue, B., & Zhao, Z. (2025). Influence of virtual reality restorative environments on psychological well-being in university students: An evidence-based experimental study. Exploration of Digital Health Technologies, 3, 101155. https://doi.org/10.37349/edht.2025.101155
Work Experience
Industry experience and applied design
Unity Interaction Engineer, VITURE
- Develop Unity-based XR interaction prototypes and product-facing demos for VITURE spatial computing experiences, focusing on gestures, gaze, drag states, panel organization, hover feedback, and spatial UI interaction details.
- Support SpaceWalker OS3.0 interaction iteration and VITURE Unity SDK exploration by running sample projects, evaluating product experience, documenting interaction issues, and translating XR demo ideas into testable prototypes.
- Apply an AI-first development workflow with Claude Code and Codex for research, planning, implementation, review, and documentation, while maintaining human-led judgment on interaction quality, module boundaries, testing, and refactoring.
AI Product Design Intern, Feeling AI (Shanghai Yunwu Tech)
- Led the development of an AI-driven multi-character narrative demo by architecting a Western-themed world-building loop and NPC relationship networks. Defined semantic tagging and data structure standards to implement core dialogue, trust, and memory systems, validating the structured reasoning capabilities of LLMs in interactive storytelling.
- Contributed to product feature decomposition, AI character design, interaction-flow planning, and prototype documentation, translating the narrative experience into implementable modules and evaluating applications across games, education, and companion products.
3D / VR Designer, UCL Multimedia Anthropology Lab
- Worked on an immersive Unreal-based VR museum, digitally reconstructing more than 200 sets of Brazilian Indigenous cultural heritage while exploring visual storytelling strategies for local knowledge and digital heritage.
Parametric Design Intern, Xinjiang Architectural Design and Research Institute
- Participated in architectural concept and scheme design, using Rhino and Grasshopper for facade and massing studies, design optimization, technical coordination, and drawing revisions.
Architectural Design Intern, PTA Design Consulting, Shanghai
- Contributed to early-stage urban planning and mixed-use concept design through site analysis, zoning, spatial organization studies, planning regulation review, and circulation logic development.
Selected Projects
Selected works
Digital Heritage Project - Nanjing Da Bao En Temple
Led the design of an AR/VR heritage experience reconstructing the Nanjing Da Bao En Temple as a UNESCO-site digital heritage proposal, combining historical scene reconstruction, interactive storytelling, and AI-guided cultural dialogue. Coordinated 3D reconstruction, Unity/AR interaction planning, visual narrative assets, and competition proposal packaging for the global "Heritage Never Fades" submission.
FireEscapeVR - VR Escape Simulator Based on 3D Spatial Audio Localization (Unity)
Built a VR emergency-escape simulator in Unity using 3D spatial audio localization to guide users through evacuation tasks under limited-visibility conditions. Developed the immersive interaction interface, route guidance logic, and training-oriented scenario flow to explore how spatial hearing can support fire-safety education and emergency response practice.
Carbonis: Siren's Elegia - VR Interactive Film (Unreal)
Developed a symbolic underwater VR interactive film in Unreal Engine, blending mythology, ecological critique, spatial navigation, and environmental storytelling. Designed the submerged scene logic and interactive viewing flow for virtual art showcases; the project was also featured at the "Shanghai Three Days" exhibition and prepared for future museum and fashion-art presentation contexts.
Bloomiverse - AR Planting Adventure Game (Unity)
Created an AR eco-game featuring magical plants, object-triggered interactions, growth-state feedback, and playful environmental learning loops. Designed plant animation behavior, prop interaction logic, and technical art assets to connect mobile AR interaction with ecological awareness and user engagement.
Skills
Toolchain and technical strengths
XR / Spatial Interaction
Gesture Input, Gaze Targeting, Drag States, Hover Feedback, Spatial UI, Panel Organization, Meta Quest 3, OpenXR
AI Interaction
LLM Prompt Design, Multi-Agent Dialogue, Emotion-Adaptive Systems, Conversational Memory, Agentic Coding
Cloud AI
Tencent Cloud STT/TTS, Aliyun Qwen Flash, Voice Interaction Pipelines, AI Tutor Dialogue Flow
Research & Evaluation
Physiological Data Collection (HR/BVP), GAD-7, STAI-S, SPSS, Qualitative Review, User-Centered Prototyping
Programming Languages
C#, C++, Python
Game Engines
Unity3D (C#), Unreal Engine (Blueprint & C++), NPC Logic & System Architecture
Design Tools
Maya, ZBrush, Rhino, Blender, Cinema 4D