Prototype for AR Avatars on the Magic Leap 2
I prototyped an idea of what IRL avatars might look like with AR, using the Magic Leap 2. I used SOTA pose-recognition models and a low level game engine (StereoKit) on Android, which was pretty difficult on the ML2. The final prototype used Mediapipe C++, Pytorch, and Stereokit, and ran in realtime on device. I also had to develop custom algorithms for keypoint depth estimation and IK. I also ported h.264 video streaming code to the Magic Leap device, and patched cling (the C++ interpreter) to work on the Magic Leap.
View Code / View Presentation Slides
