Apple files patent for mixed reality headset that can track your facial expressions

Apple has filed a new patent for a mixed reality headset that could feature an array of sensors to track the eyes, gestures and even facial expressions of its users.

Titled “Display System Having Sensors,” the patent filing — first reported by Variety — shows that the company intends to combine these inputs for a system “that provides 3D virtual views of a user’s environment augmented with virtual content.”

The application was filed with the US Patent & Trademark Office (USPTO) on March 21, 2019. It was published last week on July 18.

The proposed system could have hand sensors, “head pose sensors” that can capture information about the user’s position, orientation, and motion in the environment, “eyebrow sensors” to track expressions of the user’s eyebrows, and “lower jaw sensors” to track movement of the mouth and jaw.

The 💜 of EU tech

The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!

Augmenting its capabilities further, the head-mounted display (HMD) can also supposedly capture data about the user’s environment — such as depth, lighting, etc. — that are fed as inputs to a controller of the mixed reality system, which then renders virtual content based on the information received.

An interesting addition is the “video see through” camera that captures high-quality video of the user’s environment to display the user with a virtual view of their real environment on the HMD.

CNET had revealed last April that Apple was working on a mixed reality headset that’s capable of running AR and VR applications. According to the report, the project — codenamed T288 — is expected to be released next year.

There’s still a lot of variables, and there’s no assurance that Apple is indeed building a headset matching these specifications. But the new patent application throws reasonable light on the the tech giant’s plans for its AR/VR product lineup.