As AI assistants evolve from easy textual content interfaces to multimodal companions, we’re seeing a shift towards extra proactive, located help. Latest improvements like Venture Astra and Gemini 3.1 Flash Reside already permit customers to debate their bodily environment in actual time, typically using visible bounding field overlays to determine objects in a digicam feed. Whereas these overlays are extremely efficient for 2D screens, the transition to immersive platforms like Android XR presents a novel problem: how will we transfer past flat UI to create a very embodied, spatially conscious dialogue?
To bridge this hole, we introduce AgentHands, printed at CHI 2026, a analysis prototype that brings the ability of co-speech gestures to the 3D world. In human communication, our arms do extra than simply level; they describe shapes, mimic actions, and emphasize factors, all synchronized with our voice. By leveraging the spatial understanding capabilities of Prolonged Actuality (XR), AgentHands replicates this pure synergy. Following up our prior analysis in Human I/O and Wise Agent, AgentHands additional equips AI brokers with expressive, synchronized hand gestures that rework summary verbal directions into intuitive, bodily demonstrations, making conversations about your environment extra pure and interesting.
