Dancing with Digital Physics: How Mixed Reality is Nudging Group Movement

Imagine putting on a Mixed Reality (MR) headset and stepping into a room with friends. Between you hangs a shared, virtual rope. You cannot physically touch it, but as you swing your arms, a deep "whoosh" fills the room, rising in pitch as you move faster. Soon, you and your friends are moving in perfect sync, guided entirely by an invisible, digital force. This is the experience of GravField, a co-located MR performance system. Presented at CHI 2026, GravField explores a compelling question at the intersection of spatial computing and human interaction: How can virtual objects shape our collective movements without scripting our actions or overriding our agency? The Power of "Live Nudging" In behavioral economics, a "nudge" is a subtle environmental change that reliably influences behavior without forbidding options. GravField translates this concept into the physical, digital realm. Instead of relying on prescriptive on-screen arrows or puppet-like electrical muscle stimulation , GravField shapes movement using "Mixed Reality Objects" (MROs). During a session, a human facilitator known as an "Object Jockey" (OJ) orchestrates the experience. Much like a club DJ reads a crowd, the OJ live-configures the "digital physics" of these virtual objects in real time. They can introduce a virtual Rope that invites cooperative swinging , a Spring that encourages an oppositional push-and-pull dynamic , or a Magnetic Field that prompts fluid, orbital navigation around the room. Feeling the Intangible Through Synesthesia Because virtual objects lack physical weight, GravField relies on tightly coupled, synesthetic audiovisual feedback loops. The "digital physics" are conveyed intimately through sound and sight. For instance, if the OJ increases the virtual mass of the rope, participants must use more dramatic motions—like a full 360-degree spin—to generate a satisfyingly loud "twang". Participants in the study described this phenomenon as "hearing the force," instinctively leaning into the interaction. The Live Nudging Spiral By observing 25 workshop participants, the researchers identified a fascinating behavioral loop they call the Live Nudging Spiral. It unfolds in six distinct stages: 1. Learning: Participants tentatively probe the unfamiliar virtual object to understand its implicit rules and physics. 2. Internalizing: Through perceptual coupling, they master the interaction, discovering the bodily movements that produce the richest audiovisual feedback. 3. Coordination: The group organically falls into emergent social entrainment, synchronizing their swings or creating sequential relay motions without being told to do so. 4. Exploration: Confident in their shared rhythm, participants deliberately experiment with exaggerated gestures or sudden vocalizations to test the system's boundaries. 5. Boredom: Eventually, physical fatigue or cognitive saturation sets in, and the group's engagement plateaus. 6. Re-configuration: Sensing this lull, the Object Jockey intervenes, subtly tweaking parameters (like gravity or elasticity) to defamiliarize the object, sparking renewed curiosity and restarting the collaborative cycle. # Designing for Agency What makes GravField so compelling is its steadfast commitment to preserving human agency. The system maintains a state of "metastable play," where patterns of coordinated movement constantly form, dissolve, and re-form. Participants are never commanded on what to do; they are simply provided an affordance landscape that they can interpret, play with, and even creatively subvert. As spatial computing and programmable realities become increasingly integrated into our daily lives, GravField offers a brilliant blueprint for the future. It proves that technology does not have to rigidly control our bodies to bring us together. Instead, by carefully designing digital physics and respecting our autonomy, mixed reality can become an empathetic, collaborative partner.