Botao ‘Amber’ Hu
  • Research
  • Design
  • Journey
  • Lenses
  • About
Interact w/ me

Wizards in the Muggle World

Created
Sep 22, 2025 5:44 PM
Tags
Mixed Reality Design
Hidden
Date
October 17, 2025 → October 22, 2025
Location

Bergen, Norway

URL
https://cscw.acm.org/2025/
Venue

CSCW 2025

image

“Wizards in the Muggle World” — Towards Immersive Mixed Reality Street Play

Fig. 1: “MOFA: The Duel” gameplay on the streets of SoHo, Manhattan, New York City
Fig. 1: “MOFA: The Duel” gameplay on the streets of SoHo, Manhattan, New York City

What happens when you can duel like a wizard on real city streets? In our CSCW paper, we explore Immersive Mixed Reality Street Play: co‑located, movement‑based play in public spaces using see‑through head‑mounted displays. Imagine Harry Potter-style spellcasting — but instead of Hogwarts, you and your wizard friend are dueling on a sidewalk, in a park, or under the vaulted ceiling of a train station.

Fig. 2: “MOFA: The Dragon” in the public atrium of World Trade Center Transportation Hub, New York City.
Fig. 2: “MOFA: The Dragon” in the public atrium of World Trade Center Transportation Hub, New York City.

Why immersive mixed reality street play?

As mixed reality headsets gradually evolve from lab curiosities to everyday devices, the tech industry has teased concept videos for years that imagine cities turning into shared playgrounds (e.g., “Discover Pokémon in the Real World with Pokémon GO!”). However, today’s AR games still mostly rely on conventional smartphone touchscreen interactions. A significant gap exists between a glossy demo and real people moving, negotiating space, and being watched by strangers. Real-world implementations of immersive mixed reality street play remain largely unexplored. Our work brings this imagined future into the present and examines what actually happens — socially, ethically, and practically — when embodied MR play enters public life.

We call this kind of gameplay immersive mixed‑reality street play (IMRSP) and study it with a research‑through‑design game probe named MOFA—the Multiplayer Omnipresent Fighting Arena. (Fun fact: “MOFA” is pronounced like mófǎ (“magic”) in Chinese.) MOFA distinguishes itself with an “omnipresent” feature: gameplay can occur spontaneously anywhere — even without Wi‑Fi, 5G, or internet — via nearby peer‑to‑peer connectivity instead.

MOFA adapts a magic theme inspired by the long lineage of magic fantasy fiction. It uses intuitive magic gestures (a wand wave) to cast spells. The “Wizard vs. Muggle” framing reflects the inherent power imbalance of mixed reality: players have a clear wizardly role in mixed reality, while passersby cannot instantly read the augmented layer.

On a SoHo street, spectators hold up phones — one showing a “Get the HoloKit app” prompt — while a nearby headset‑wearing player duels.
Fig. 3: The video demonstrates “MOFA: The Duel” gameplay on the streets of SoHo in Manhattan, New York City.
Over‑the‑shoulder view in a bright public atrium: a person puppeteers the AR dragon on a phone as three headset‑wearing players attack with glowing spells.
Fig. 4: The video demonstrates “MOFA: The Dragon” in the public atrium. The Dragon is puppeteered by a handheld device, fighting against three players wearing mixed reality headsets.

A street‑ready game probe

In MOFA, three short game modes let people jump in quickly:

  • The Training (solo): practice spellcasting against an AI opponent (see Fig. 6).
  • The Duel (1v1): face‑to‑face wizard fights with dodging and counters. (see Fig. 1 and Fig. 3)
  • The Dragon (co‑op): three+ players hunt a dragon while a handheld “puppeteer” drives the creature — letting spectators join with phones. (see Fig. 2 and Fig. 4)

Under the hood, MOFA runs on an open-source smartphone‑based optical see‑through mixed reality headset HoloKit paired with Apple Watch motion sensing for casting; the devices co‑locate over peer‑to‑peer networking (We open-sourced a Unity package: MultipeerConnectivity for Netcode GameObjects) so games can pop up “anywhere.” The wand is both an input prop and a social affordance: it makes big, public gestures feel explainable to others.

If you peek at Fig. 5, you’ll see the first‑person UI and the wand‑to‑watch spell system.

Press enter or click to view image in full size

Multi‑panel schematic of MOFA’s Duel mode: wand‑and‑watch gesture detection, spell/shield types, first‑person UI, co‑location/networking flow, round structure, and phone‑to‑HoloKit MR conversion. Optional extended description: Panels illustrate (1) wand + Apple Watch casting; (2) spell categories and circular shield UI; (3) first‑person HUD; (4) QR/peer‑to‑peer connection and spectator join; (5) round‑based rules and effects; (6) converting monocular phone AR to stereoscopic HoloKit MR.
Fig. 5: Detailed game and user experience design, using “The Duel” as an example. It illustrates: (1) thefirst-person perspective of the game with all UI elements users can view through the MR interface; (2)interaction design details using Apple Watch to detect spell-casting gestures; (3) magic spell and shieldsystem design; (4) co-location and networking user flow: how two players connect and how spectators join towatch in third-person view; (5) detailed round-based dueling game mechanism design; and (6) conversionprocess from monocular screen-based AR view into immersive stereoscopic MR view based on HoloKit.

How we studied it

We took MOFA “into the wild” across 20 sites and focus here on five representative scenarios — campus courtyards, expo floors, urban parks, iconic public buildings, and a scenic mountainside. In total, 65 people played; we recorded 265 minutes of gameplay and 88 minutes of quick post‑play interviews right on the spot. For example, Fig. 6 shows street‑level deployments in an urban park in Brooklyn.

Press enter or click to view image in full size

In a riverside park beneath a bridge, a headset‑wearing player practices by casting orange spell effects at an AI opponent on open grass.
Fig. 6: “The Training” in a public park in DUMBO, Brooklyn.

We complemented fieldwork with a desktop analysis of online forum reactions to our real gameplay videos — useful for seeing how people respond when they haven’t tried it themselves.

What we saw on the street

1) From shy to showtime. Multiplayer MR in public quickly became social theater. Players cheered, warned teammates (“watch out!”), celebrated wins with fist‑bumps, and even developed a ritual: clicking wands together before/after play (see Fig. 7). Spectators gathered — a classic honeypot effect — snapped photos, and asked to join (helped by the spectator view and the “dragon puppeteer” role).

2) AR beats VR (outside). Many preferred MR’s “see-through” feeling over fully immersive VR: you can see friends, engage with the environment, keep your bearings, and still be in a game.

3) It’s a workout. Especially in The Duel and The Dragon, people dodged, squatted, and sprinted — then laughed about how winded they were. MR play can double as bite‑sized exercise.

4) Place matters. The setting acts like a “stage.” Open parks and plazas encouraged bigger movement and felt more acceptable; narrow sidewalks or crowded concourses felt tense. Landscape scale (that mountain scene!) amplified drama and joy.

Press enter or click to view image in full size

Four‑photo strip showing a playful custom: players tap wooden wands together before or after matches across city, indoor, mountain, and park settings.
Fig. 7: Players developed a playful ritual: clicking wands with teammates or opponents before and after play, reminiscent of wand interactions in the Harry Potter films.

Where the friction shows

“A dream you dream alone is only a dream. A dream you dream together is reality.” — Yoko Ono

1) Awkwardness isn’t uniform. We found a gradient: playing together in public felt least awkward; solo play felt stranger; just watching without any MR view felt most awkward. Social proof matters — two people doing a visible, legible thing makes it easier for everyone.

2) On‑site joy vs. online doubt. People who played described MR street play as “fun,” “amazing,” and “magical.” Online commenters, reacting only to video, voiced stronger concerns about social appropriateness, safety, and device form factor — echoing the broader history of public reactions to wearables. Both views are real; the gap tells us public familiarity and bystander comprehension are pivotal.

3) Safety and ethics aren’t optional. Even with see-through vision, fast, expressive movement in public requires care. People worried about collisions, and (in phone‑in‑headset setups) theft risk. Power asymmetries of mixed reality also affect bystanders’ privacy and comfort. Designing visible etiquette and safety cues is part of the work.

Four design moves for responsible MR street play

A bystander records on a tablet that displays a third‑person AR view of glowing spells as two players duel outdoors near a bridge.
Fig. 8. Spectators can use iPads to view the augmented reality experience from a third-person perspective and capture recordings of what they see.

1) Design for context, not just content. Favor open parks, plazas, and pedestrianized zones; avoid busy sidewalks. Let the app “nudge” players toward suitable spaces when possible.

2) Use social affordances. Props, roles, music, and clear costumes/signifiers make public gestures legible to non‑players. Our wand was crucial (see Fig. 9): it explained gestures to onlookers and anchored the narrative for players. Provide spectator views and lightweight “join‑in” roles (e.g., phone‑based puppeteer) (see Fig. 8).

3) Safety‑by‑design. Keep the real‑world view largely unobstructed; avoid mechanics that invite physical contact; favor strong visual feedback (e.g., shields shattering) over tiny HUD meters. Consider dynamic play‑zone indicators that adapt to crowd density.

4) Normalize togetherness. Lean on multiplayer to reduce embarrassment and increase delight. Guide pairs/groups into co‑play; let bystanders become participants in seconds.

Close‑up of a hand wearing a smartwatch and holding a wand mid‑flick; the watch registers a motion spike from a downward wave that triggers a spell.
Fig. 9: A downward hand wave produces a spike the smartwatch detects, triggering spells in MR. The wand serves as a tangible prop — conveying ritual meaning and providing embodied haptic feedback — while making the gesture legible to bystanders.

What this means

Our findings suggest IMRSP can turn ordinary streets into momentary stages where strangers meet, cheer, and sometimes join the fun. But mainstream adoption will be slower than concept videos suggest — not for lack of tech alone, but because public norms, safety practices, and bystander rights are still being negotiated. The good news: careful design and transparent etiquette go a long way.

We’ve open‑sourced MOFA to help others build and study these futures in situ. If you’re a game designer, researcher, or city steward, we invite you to prototype with us: try small, visible deployments, include spectators, and treat norms as a first‑class design material. The magic isn’t only in the graphics — it’s in how people play together in future streets.

Press enter or click to view image in full size

Black promotional banner with a bright orange spell circle and the paper title announcing a CSCW 2025 presentation in Bergen, Norway. Optional extended description: Graphic features the full paper title and author list alongside the glowing spell motif.
Fig. 10. We will present “Towards Immersive Mixed Reality Street Play: Understanding Co-located Bodily Play with See-through Head-mounted Displays in Public Spaces” at CSCW 2025 in Bergen, Norway.

This article accompanies our CSCW 2025 paper:

Botao Amber Hu, Rem RunGu Lin, Yilan Elan Tao, Samuli Laato, and Yue Li (2025). “Towards Immersive Mixed Reality Street Play: Understanding Co-located Bodily Play with See-through Head-mounted Displays in Public Spaces”. https://doi.org/10.1145/3757679 or preprint https://arxiv.org/pdf/2505.12516

MOFA was an award-winning game that we demonstrated at CHI 2023, where it won Best Interactivity Demo, and at SIGGRAPH 2024 Immersive Pavilion, where it received Best in Show.

About the author

Botao Amber Hu is a social computing researcher and experiential futures designer. He is a PhD candidate in Human Centred Computing group at University of Oxford’s Department of Computer Science. As a designer, he creates experiential futures using social mixed reality as his primary medium. He also serves as a visiting lecturer for Mixed Reality Design at China Academy of Art while directing Reality Design Lab. He is the inventor of open-source mixed reality headset HoloKit.