Computational pictures and in-camera focus stacking can assist photographers maintain extra of the picture in focus, however what if a digital camera might give attention to every part suddenly utilizing solely optics? Researchers at Carnegie Mellon University used 4 totally different lenses and 3D show tech to create a Franken-lens that may maintain each pixel in a photograph in focus.
Traditional optics can solely give attention to one distance vary at a time, adjusting a single focus airplane to carry objects in focus. Techniques like focus stacking can assist, however are based mostly on modifying and computational pictures – not optics.
But the research group reverse-engineered 3D display technology to essentially enable each pixel to have a different focal plane, creating a lens that can keep everything in focus without computational photography. The result is that the objects and background are all in focus, a method that the researchers dubbed Spatially-Varying Autofocus.
The key to the weird lens is Split-Lohmann, an optical association made for 3D shows for interactive gaming and flicks, also developed by Carnegie Mellon researchers in 2023. But as an alternative of utilizing the Split-Lohmann for a show, the group used the thought so as to add a Spatial Light Modulator to the lens.
See how the prototype works within the video beneath:
ABOVE: Watch the prototype digital camera lens in motion
The Spatial Light Modulator tilts contained in the lens to redirect the sunshine, which modifications the focal airplane. But this modulator can be managed regionally.
The researchers used a depth map created from the digital camera’s autofocus to regulate the modulator to match the objects within the body. This allows the focus to be managed for every pixel, somewhat than utilizing one focal airplane for your entire picture.
Light comes by the lens, bounces off the customized Spatial Light Modulator and heads for the digital camera’s sensor. The result’s a picture the place each object within the picture is in focus, with none modifying tips.
The researchers’ prototype lens is T-shaped, with the sunshine getting into from the left (so the photographer is definitely taking footage of one thing to their left, not proper in entrance of them).
Light is available in from the top-left of the T by the variable lens, by one relay lens, by a beam splitter and cubic section plate, by one other relay lens, then to the Spatial Light Modulator, bouncing again the place that beam splitter then redirects the sunshine into one other relay lens and, lastly, to the digital camera, which is a Canon EOS R10.
The researchers used a wide range of lenses on the primary entry level (the video beneath appears to point out a Nikon, which feels a bit blasphemous on a Canon, but I digress) and three Samyang 85mm f/1.4 AS as “relay lenses.”
While the technique is all optics and no computational photography, the process still requires at least two images. The first one is to create the depth map to adjust the Spatial Light Modulator to the objects in the scene, and the second one is to actually take the photo of everything in focus.
The researchers were able to use both contrast detection and phase detection autofocus to create those depth maps.
The team’s prototype with the R10 was able to take photos with everything all in focus at a speed of about 3fps. But the group also created another prototype using a machine vision sensor to improve the speed to 21fps.
While genres like portraits seldom want everything in focus, the unusual lens could potentially be used in genres like macro photography, where keeping enough of the scene in focus is a challenge.
But perhaps even more interesting is that the researchers’ prototype was also able to take those photos using a wide open aperture. Narrower apertures can help keep more (though not all) of the image in focus, but cut back the amount of light entering the lens. The unusual T-shaped lens could keep that wide aperture while still getting everything in focus.
Another potential use of the technology is to customize the Spatial Light Modulator, which would give the photographer the ability to pick and choose which objects to leave in focus.
As an example, the researchers demonstrated this concept by eliminating a thin wire fence from the focal map. This enables the background to be sharp along with the subject, but blurs the thin wire enough that it nearly disappears in the photo.
This disappearing trick only works with very thin objects like wire fences, but it’s an interesting concept that feels a bit like editing a photo before it is taken.
The team – which included Yingsi Qin, Aswin C Sankaranarayanan and Matthew O’Toole – presented their work on the 2025 International Conference on Computer Vision.
You might also like…
Browse the best Canon lenses or the best macro lenses.