Apple is starting to put the pieces together for its next big breakthrough

Nearly 15 years after the iPhone launched the smartphone revolution, Apple is putting the finishing touches on what it hopes will be its next game-changing device: a headset that combines the digital and real worlds.

According to three people familiar with the project, the corporation has hired Hollywood directors such as Jon Favreau to create video material for a headset that will be released next year. According to these sources, Favreau, who is an executive producer of Apple TV+'s "Prehistoric Planet," is trying to bring the show's dinosaurs to life on the headset, which resembles a pair of ski goggles and aspires to provide virtual and augmented reality experiences.

Separately, according to two people familiar with the project and documents reviewed by The New York Times, Apple plans to unveil software tools that will allow apps to add new camera and voice functionality, laying the groundwork for a hands-free interface that customers will eventually be able to navigate on the headset, at its annual developer conference on Monday.

Trudy Muller, an Apple representative, declined to comment on future projects. Favreau's representative was unavailable for comment.

Apple will enter a new competition to define the future of mixed reality with the projected headgear. Microsoft, Google, and Facebook's parent company, Meta, are all working on software and hardware that will allow 3D digital images and the physical world to coexist in the future.

Last year, Meta CEO Mark Zuckerberg changed the name of his firm from Facebook to Metaverse, a concept in which the online, virtual, and actual worlds collide to form a new reality. He and others believe it has the potential to become the next generation of computing, succeeding the smartphone era dominated by Apple's iOS and Google's Android software, much as Windows and Macintosh did decades ago.

Carolina Milanesi, a technology analyst with Creative Strategies, a technology research firm, said, "This is the next frontier." "This is about a new computing experience for Apple, as well as a chance to engage consumers with a device and new experiences that build on what they've done with content."

The mixed-reality effort is likely to be eclipsed during Apple's Monday speech, which will focus mostly on the company's existing software systems. Analysts believe Apple will also unveil a new MacBook Air with smaller display borders and improved processors than existing models.

Apple's virtual-reality content and software development is critical to developing experiences that will give its future headset meaning. The Apple Watch, the company's most recent big new product, was released with nearly 3,000 apps, but it struggled to gain traction since few of those apps were usable, according to tech critics. Meta's Quest virtual-reality headset, which sold over 10 million units last year, has had similar issues because many people see it as a gaming device.

Apple has pursued products that appeal to a wide range of potential customers and have a variety of functions, from the original Macintosh to the iPad. Last year, it sold an estimated 240 million iPhones, accounting for over half of its total revenues of $366 billion. Analysts believe that in order for the headset to be worthwhile, it will need to have features that extend beyond the limited world of video games.

For years, Apple CEO Tim Cook has talked about the potential of augmented reality. He informed investors in 2016 that the corporation was actively investing in it and that it was a "huge commercial prospect." Many Apple employees were reading "Ready Player One," a futuristic novel about virtual reality, at the time, and discussing the potential of developing their own mixed-reality world.

Mike Rockwell, a Dolby Technologies engineer, was hired by Apple to spearhead the project. Two persons familiar with the project stated that his early attempts to construct an augmented-reality product were hampered by a lack of computational capacity. Apple has been forced to postpone its release until next year due to ongoing battery issues, according to those insiders.

Apple's augmented-reality effort has sparked debate within the company. At least two members of the company's industrial design team said they departed partly because they were concerned about designing a device that would alter how people connect with one another. As public worry about children's screen usage grows, such concerns have grown within the company.

With Rockwell at the helm, the device would be one of the first from Apple to be directed by its engineering staff rather than Steve Jobs, the company's co-founder, who died in 2011, and Jony Ive, the company's previous design chief, who left in 2019. Ive and his designers were in charge of the Apple Watch project, which determined how it appeared, worked, and was promoted.

Favreau's programming demonstrates how Apple is attempting to set itself apart from Meta's product. It also demonstrates how the company is leveraging the Hollywood contacts it has built since launching Apple TV+ in 2019.

"An incredible headgear may provide a better experience than an 80-inch TV," said Matt Miesnieks, CEO of LivingCities.xyz, a metaverse business.

Apple's new software tools rely on a multiyear effort to encourage developers to create augmented-reality apps. ARKit, which allows developers to use the iPhone's camera and motion sensors to place digital objects in the real world and let people to interact with them, was launched in 2017.

According to a poll of more than 500 developers conducted by Creative Strategies, over 70% of Apple's developers claimed they did not use the tool.

According to a source familiar with the project, a toolkit that Apple is slated to unveil at the conference would provide software developers new powers to initiate shortcuts inside their apps using Siri and QR codes, interactions that will be used in future headsets.