Description

Silver Unicorn was developed at Reality Virtually hackathon at MIT media lab in October 2016. This project was awarded the 1st prize in Getting things done and doing business (engineering) category. Silver Unicorn is a personal companion robot with mixed reality, tangible and voice interface. The main objective of the project was to showcase human-digital-physical-natural interface. I was responsible for developing the interactivity modules and networking components on Hololens and building and deploying the application.

Demo

https://www.youtube.com/watch?v=n057VRxPspU

Technical Details

• Application was built in Unity and deployed on to Hololens using Visual Studio.
• Application uses the Universal Windows Platform (UWP) networking packages.
• Application uses the inbuilt gaze, gesture and voice control on Hololens.
• Hololens interactivity scripts programmed in C#.
• A desktop application containing a TCP server to receive voice commands from Hololens and a Bluetooth classic client using Sphero SDK to send motor commands to the robot.
• The app makes use of Hololens coordinate system & spatial mapping to place 3D spatial UI.
• The robot consists of a marker recognized by Vuforia running on Hololens.
• The marker reveals a virtual “Silver Unicorn” which has predefined interaction with the spatial menu.
• The speech menu enables the voice interaction on Hololens which sends out data to the TCP server on keyword recognition.

Technology & Tools

Microsoft Hololens, Mixed Reality, Unity, C#, TCP/IP, Vuforia, Sphero robot