Using Quest 3 for Scene Understanding

Developed a Unity App and deployed it on a Quest 3 to access the camera pose and pre-scanned scene mesh. After some hacky ways to obtain the camera feed of the headset, I ran panoptic segmentation on it and ray-traced from the headset pose to the mesh to label the objects in the scene according to the panoptic masks.