Apple's SHARP AI model creates 3D scene from single photo in under a second for Vision Pro

Apple Unveils SHARP to Generate 3D Scenes from Photos

Apple has published a research paper introducing SHARP, an AI model that creates a 3D scene from a single image in under a second. Titled "Sharp Monocular View Synthesis in Less Than a Second," the paper details how Apple's new AI tool generates 3D scenes from photos using advanced neural networks 1

1

. The Cupertino tech giant trained the model on approximately 8 million synthetic images created in-house and 2.65 million licensed photographs, enabling SHARP to discern depth and scale with striking accuracy 1

1

.

How SHARP Converts 2D Photographs into 3D Spatial Scenes

Apple's new AI model operates by predicting what a photorealistic 3D scene would look like by analyzing the image's nearby viewpoints 2

2

. The system uses 3D Gaussian Splatting, a technique that represents scenes as millions of tiny colored blobs called "splats" with defined position, size, orientation, color and transparency 2

2

. Unlike traditional methods requiring hundreds of images from various angles, SHARP can turn a photo into a 3D scene using just one image. The 3D Gaussian representation produced by SHARP maintains consistency in scale and distance, avoiding the stretching and warping common in 2D-to-3D conversions 1

1

.

Real-Time 3D Rendering on Standard Hardware

The Sharp Monocular View Synthesis process completes the 3D mapping in less than a second on a standard GPU via a single feedforward pass through a neural network 2

2

. Once the initial mapping is complete, the image can be rendered in real time, producing high-resolution photorealistic 3D scene outputs 1

1

. According to the research paper, the representation is metric with absolute scale, supporting metric camera movements while maintaining real-world consistency in distances and scale 3

3

. This speed and efficiency distinguish SHARP from competing 3D generation tools currently being developed by Google, Meta, and smaller developers 3

3

.

Vision Pro Integration and Spatial Photo Features

While SHARP remains a proof-of-concept, the tool appears to be a natural evolution of Apple's existing spatial photo features introduced in iOS 26 1

1

.

The model can quickly turn 2D images into 3D gaussian splats that can be viewed on Vision Pro, Apple's mixed reality headset 3

3

. If integrated into the Photos app on iPhone, users could explore their images in immersive 3D environments. The code has been published on GitHub, allowing developers and researchers to test the tool and experiment with the technology 2

2

. However, there's no indication yet of when or if SHARP will be integrated into iOS 26 or macOS Tahoe 1

1

.

Limitations and Future Implications

SHARP does have constraints that users should consider. The model focuses on rendering nearby scenes, meaning fidelity degrades when users stray too far from the original viewpoint 1

1

. The AI model only accurately renders nearby viewpoints rather than unseen parts of the scene, limiting exploration beyond the initial perspective 3

3

. Despite these limitations, the speed at which SHARP operates makes it a promising starting point for consumer applications. With Apple's control over both hardware like iPhone and Vision Pro, plus software ecosystems, some observers believe Apple's AI work may have been underestimated in the competitive landscape of 3D generation tools 3

3

. As AI-driven 3D tools continue evolving across the industry, watch for potential announcements about SHARP integration into Apple's operating systems and devices in future updates.