In the last decade, smartphone photography has significantly progressed thanks to innovations in camera hardware, computational imaging, and AI. However, capturing photos in low light has always remained challenging, until Samsung Electronics debuted its revolutionary Nightography feature. First introduced in the Galaxy S22 series, Galaxy users can now enjoy even more vivid, highly optimized photos, even in the dark, with enhanced Nightogaphy for the Galaxy S23 series.
Helping users say goodnight to washed-out, blurry night images was a group effort driven by Samsung’s experts. Mr. John Seok-Jun Lee, Head of Computational Camera Part at Samsung Research America (SRA), and Narasimha Gopalakrishna Pai, Head of AI Computational Imaging Part at Samsung R&D Institute India-Bangalore (SRI-B), had to say about their joint development of Nightography, the best way to take crisp photos and videos from dusk until dawn.
Nightography helps users to take vivid photos in low-light environments. The S23 camera captures a series of images with different brightness levels, then applies multi-frame processing (MFP) technology. This technology analyzes the images, removes the frames with noise or blur, and then merges the remaining frames into a single clear shot with HDR. Then an AI-powered ISP algorithm on NPU enhances the photos’ color, tone, and details.
Nightography image quality on the Galaxy S23 has improved tremendously with enhanced brightness while still controlling the noise, clarity, and saturation around light sources, like neon signboards. By leveraging the exclusive snapdragon 8 Gen 2 for the Galaxy S23 series.
The R&D Revolution: How did MPI and SRI-B Collaborate to Develop Nightography?
SRA’s mobile processor innovation lab created several algorithms to overcome the challenges of taking pictures in low light environments. By analyzing and merging multiple captured inputs, Signal to Noise Ratio (SNR) and dynamic range are improved and blur caused by camera or object motions is optimized. To remove high noise levels and enhance photo details further, SRA MPI also collaborated with MX Visual Solution Team to develop and optimize the AI image signal processing (ISP)
SRI-B was responsible for the algorithm pipeline development, optimization and AI model acceleration using all the available compute units on the platform via multicore CPU, NPU and GPU. SRI-B also developed an AI denoising module that reduces motion blur in the final output. SRI-B also created ‘Night portrait’ for the Galaxy S23 selfie camera, which captures night photos with beautiful color blooms.
Getting Hands-on with Nightography
In the future, the number of pixels in a smartphone camera will continue to increase. In line with this trend, SRA MPI plans to solve the problem of low light photography in high mega-pixel cameras through next-generation AI MFP technology. In addition, Samsung is looking to increase the quality of video nightography to match that of photo nightography.
The overall goal is to improve the image quality and user experience by continuing to improve Nightography and expert RAW across all image resolutions. Starting with the Galaxy S23 series, Samsung is planning to explore how to apply the power of Nightography to other premium devices.