Voice interaction is an exciting frontier in VR, and when Meta Reality Labs asked us to make an open-source demo to show off their new SDK, we were more than happy to pitch them a lot of left-field ideas. The result was Whisperer, a short playable VR story where you, playing as a ghost, help the main character rekindle her passion for gardening.

We devised a whispering gesture made with your hands to begin interaction, and had it double as a selection tool, allowing you to speak directly to particular objects. We set up natural language processing to take your commands to help our Hero Bea navigate through three separate scenarios.

The project launched as part of the Meta Connect conference and soon after was available in the App Lab as well as open-source on GitHub, to inspire others to use their voice in VR. We may have even left a couple easter eggs hiding out in the code.