An experimental XR prototype submitted to Google's Gemini Competition that transforms user questions into polished pitch decks through AI-powered workflows. The system provided transparent AI reasoning, allowing users to intervene and refine outputs for better results. I developed the AI agent architecture and user experience design, while Stijn Spanhove created the immersive XR environment.
The prototype combined multiple cutting-edge technologies to create a novel interface for AI-assisted presentation creation. Using the Gemini API as the core reasoning engine, the system processed natural language questions and generated comprehensive pitch deck content. Whisper handled speech-to-text conversion for voice interactions, while React Three Fiber and WebXR provided the immersive 3D interface running on Meta Quest 3. Imagen 2 generated visual assets for the presentations, and Google Text-to-Speech enabled natural voice feedback. The key innovation was the transparent reasoning interface that showed users exactly how the AI was processing their requests, allowing for granular interventions at each step of the content generation process. This approach addressed a common challenge in AI systems: the black box problem where users can't understand or influence the AI's decision-making process. The submission demonstrated the potential for XR interfaces to make AI workflows more intuitive and controllable.
2024