Home

Research
     1. Multimodal XR Micro-Interactions While Walking

Multimodal Interaction
        1. Mondrian’s Symphony:The Harmony of Color, Sound, and Touch
       2. Deep Sea Explorer: A multimodal video game Ubicomp/ISWC 2026*
UX DESIGN
  1. Careshare Kitchenette
  2. M-POS
  3. HuaWei Energy-Efficient Metrics Modal *
  4. Spotify-Re:tasted Daylist
  5. E-Together(Grand Prize)
  6. Fluffy’s Revenge: 2D pixel roguelike game


installation
  1. Photoplantmorph
  2. Language Box


About

Info



I am a HCI researcher and interaction designer with a strong focus on interaction design, UX design, HCI, vR/AR, multimodal XR interaction, media tech and ICT4S.

My expertise lies in research-driven prototyping, user study design, empirical evaluation, and the development of precise and refined interactive systems. I am particularly interested in bridging technical implementation and human-centered research, transforming experimental findings into actionable design insights, product improvements, and context-sensitive interaction solutions.


Read more →

2. Deep Sea Explorer:
“A multimodal video game"



Have you ever imagine use “Multimodal controls” in a swimming games as if  in real deep sea?

This project presents the development and evaluation of a video game experience that compares multimodal input “Gesture + head movements” with traditional keyboard controls in a deep-sea environment. 

While further research is required for the development of a full game, our results show keyboard control is preferred for ease of use, while multimodal input is favored for its novelty and enhanced interactivity, making the game more fun and immersive.

team project 2024.12-2025.01


Role: Designer & researcher


Ubicomp/ISWC 2026 · keyu ren*, yanliang gong* · Ubicomp/ISWC 2026 Proceedings



Report



Final works