Artists or game developers who would like to get a view of what is possible using Meta Quest Pro's eye tracking and face tracking capabilities can see them implemented in this app. A fantastical character representing an alien will mirror your facial movements including smiles, eye winks, eye brows, etc.
Face tracking is encoded using blendshapes that are based on the Facial Action Coding System (FACS). You can see these blendshape activations in a histogram displayed to the left of the mirrored avatar. By observing how these blendshapes activate, you can get a good feel for how the blendshapes combine to represent natural facial expressions..
Once the movements are detected, the designer can choose to modify the strength of the activation so that it meets the creative intent for the character. For instance, if you have a character with a very large mouth, you might not want to fully activate the mouth blendshapes as it might make the character look menacing. Using a control slider on the right side of the application, the user of this app can change the strength of the activations to see their impact on the retargeted alien character.
This application is an implementation of the Presence Platform’s Movement SDK which is provided by Meta to allow all developers access to the Eye Tracking, Face Tracking, and Body Tracking capabilities. The code and asset models that are the basis for this application are available for download at: https://github.com/oculus-samples/