Years within the Making, Glass Imaging Is Delivering on its Promise to Rework Smartphone Pictures

Glass Imaging’s spectacular GlassAI Neural picture sign processing (ISP) expertise is closely featured within the brand-new Honor 600 smartphone, promising to enhance the zoom pictures expertise on the flamboyant new telephone.

What Does Glass Imaging Do?

PetaPixel has lined Glass Imaging’s AI expertise quite a few instances in the previous few years, together with a visit to Glass Imaging’s California headquarters in 2024. The firm’s main focus is utilizing AI expertise and extra clever picture sign processing to dramatically enhance picture high quality from small picture sensors and cell digicam programs.

By leveraging its refined algorithmic expertise, a selected imaging pipeline, and on-device processing energy, GlassAI will help smartphones overcome the bodily limitations of small sensors and lenses.

In the case of the Honor 600, GlassAI applies its neural community processing to the telephone’s zoom imaging, “helping to recover fine detail, reduce noise, and preserve natural color and texture across the zoom range.”

GlassAI Inside the Honor 600 Smartphone


”We’re honored to associate with Honor and to have our Neural ISP expertise featured within the 600 sequence,” says Ziv Attar, CEO of Glass Imaging. “Honor continues to set a high bar for mobile imaging, and we’re proud to contribute to that experience and showcase our product in the market.”

As Glass Imaging tells PetaPixel, its neural ISP expertise is especially useful when coping with small pixels.

“In general, one advantage we offer is in dealing with small pixels. Sub-micron pixels encode high-frequency data in complex ways that traditional ISPs fail to decode — that information isn’t lost, just hard to recover. By modeling the specific point spread function (PSF), sensor, and noise profile of each module, GlassAI corrects optical degradations at their source instead of relying on generic approximations,” Shivansh Rao from Glass Imaging’s machine studying crew tells PetaPixel.

“Specifically, smartphone cameras run into a hard trade-off as pixels shrink: to keep diffraction in check at tiny pixel sizes, designers tend to widen the aperture, but a wider aperture pulls in steeper rays at the edge of the lens that introduce more geometric aberration. A conventional ISP does not model the optics, so it can’t undo that blur; it generally only smooths or sharpens around it. Because we model the actual optics and sensor physics per module, we correct those degradations directly instead of masking them,” Rao continues.

As Glass Imaging has defined earlier than, the corporate performs in depth end-to-end coaching for demosaicing, denoising, deblurring, and multi-frame fusion concurrently, so that every one these vital picture processing steps are carried out in parallel relatively than one after one other.

“A traditional ISP runs [these] as a chain of discrete steps, and each step discards information the later steps can never recover,” Rao says. “By training the whole pipeline end-to-end on the RAW, we avoid that compounding loss.”

Understandably, given the sometimes jarring, unrealistic, and unpleasant-looking outcomes from many conventional computational pictures fashions and modes, photographers are a little bit bit skeptical of something AI-related of their telephones. Glass Imaging hopes to keep away from the pitfalls of extra conventional computational pictures by counting on an actual RAW picture knowledge sign at each step of the method. The firm’s mannequin just isn’t creating new element from skinny air, however relatively recovering “genuine detail” from actual knowledge captured by the precise picture sensor.

“GlassAI Neural ISP is trained to recover genuine detail the optics actually captured but a standard pipeline would lose, so the output stays realistic to what was in front of the lens. That avoids the over-sharpened, artificial, or ‘AI-cleaned’ look you get when computational steps are guessing at detail,” Rao says.

In the case of the Honor 600, Glass Imaging’s expertise takes the place of a devoted telephoto lens. While many class-leading smartphones nowadays function three rear cameras, most important, ultra-wide, and telephoto, extra mid-range fashions designed for purchasers on a tighter finances usually eschew a devoted telephoto digicam. This can also be the case for a lot of very costly folding or ultra-thin telephones, the place house is at a premium and a devoted telephoto digicam comes with too many different trade-offs.

So as an alternative of utilizing an extended lens with a devoted sensor, the Honor 600’s zoom perform crops from its 200-megapixel most important digicam.

“On the Honor 600, zoom isn’t handled by a dedicated telephoto lens — the phone reaches into a scene by cropping its high-resolution main camera. That puts the load on the sensor and main lens: the 200 MP main camera uses a 16-in-1 ‘Hex Bayer’ pattern, so its native pixels are tiny (around 0.56 μm) and normally binned into 2.24 μm super-pixels,” Rao says.

Overcoming the Physical Limitations of Small Pixels

“When you zoom, you’re effectively shooting at those native sub-micron pixels, which is exactly where image quality hits a real physics problem.”

While zooming in on a 200-megapixel sensor might present the info for an honest simulated telephoto shot, cropping like this additionally reveals sensor-level points, akin to diffraction and aberrations that may in any other case hurt picture high quality. A blur spot may even cowl a number of pixels, for the reason that pixels themselves are so small.

“The result is that simply adding more pixels stops helping, at least with naive sharpening in a traditional ISP, which can’t recover the lost detail,” Rao tells PetaPixel.

“GlassAI addresses this in two ways. First, we incorporate lens aberration correction into the pipeline and train on the actual point spread function (PSF), sensor, and noise of that specific module, which matters more and more as pixels shrink. In our own controlled study, neural restoration improved resolution (MTF50) by over 50% as pixels shrank from 0.75 to 0.35 μm (covering the range of the Honor 600’s sensor), while a traditional ISP largely stalled: since it has no model of the optics, the residual blur stays in the image and the extra detail from denser sampling is lost,” Rao explains.

“That gap is very hard to close with a stepwise pipeline — by the time it sharpens, the real detail has already been discarded. Second, as mentioned, we train an end-to-end network on the RAW data, avoiding the information losses of a traditional multi-stage (demosaic, denoise, sharpen) ISP pipeline,” Rao continues.


Put one other approach, by exactly modeling your complete imaging pipeline, together with the particular sensor and lens used within the Honor 600’s 200-megapixel most important digicam, Glass Imaging can get well particulars that will usually be misplaced.

“We’re approaching what we call the ‘telephoto physics wall’: the point where silicon scaling outpaces what conventional optics and traditional ISP can cleanly deliver, and further resolution gain is limited,” the Glass Imaging Deep Optics crew defined earlier this 12 months in an extremely detailed technology article.

As the corporate reveals by means of in depth testing, its neural ISP considerably outperforms conventional picture sign processing, delivering sharper, cleaner photographs. The efficiency hole widens as pixel measurement decreases, opening new potentialities for smartphone producers.

Glass Imaging Can Improve Image Quality From Any Sensor

Speaking of smartphone makers, whereas Glass Imaging can not talk about any specific future, unannounced merchandise, Rao says the crew is working exhausting to develop new applied sciences for a lot of gadgets throughout the smartphone business and past. Glass Imaging’s expertise provides promising advantages for any digicam, particularly these with smaller sensors and lenses, akin to wearables, drones, automotive, medical, and extra.

“You should expect to see Glass Imaging in more places over time,” Rao concludes.

---

Image credit: Honor 600 product photographs by Honor. Real-world pattern photographs by Cyrus Tabar.