Doubling the Signal Quality of Smartphone Camera Pulse Oximetry Using the Display Screen as a Controllable Selective Light Source
Christian Holz and Eyal Ofek. EMBC 2018.
Microsoft Research, Redmond, WA.
Figure 1: The proposed camera oximetry technique in operation
Our Android app captures oximetry reflections from the finger using the display as an illuminant in solely the red and green spectrum (thus appearing yellow). This produces significantly more accurate oximetry values than when using the rear camera and the flash for illumination.
Abstract
Recent smartphones have the potential to bring camera oximetry to everyone using their powerful sensors and the capability to process measurements in real-time, potentially augmenting people's lives through always-available oximetry monitoring everywhere. The challenge of camera oximetry on smartphones is the low contrast between reflections from oxyhemoglobin and deoxyhemoglobin. In this paper, we show that this is the result of using the camera flash for illumination, which illuminates evenly across bands and thus leads to the diminished contrast in reflections. Instead, we propose capturing pulse using the front-facing camera and illuminating with the phone’s display, a selective illuminant in the red, green, and blue band. We evaluate the spectral characteristics of the phone display using a spectroradiometer in a controlled experiment, convolve them with the sensitivity curves of the phone’s camera, and show that the screen’s narrowband display illumination increases the contrast between the reflections in the desired bands by a factor of two compared to flash illumination. Our evaluation showed further support for our approach and findings.
Publication
Figures
Figure 3. Spectral distribution of intensities of the LCD screen under different illumination colors (simulated wavelenghts) and the phone camera flash.
Graphs are normalized to the maximum intensity recorded in the direct condition. Curves illustrating flash illumination are normalized separately from those depicting display illumination. It is evident that while the flash light illuminates the entire visible spectrum at a comparable intensity apart from a drop in the teal range, the LCD produced peaks of intensities in only the red, green, and blue bands. The different illumination colors tested during the experiment resulted in a more or less pronounced intensity of each of these distributions, but did not change illumination peaks along the spectrum.
Figure 4. Normalized spectral display intensities and camera sensitivities.
The figure reveals that the green illumination peak largely intersects with an area in which deoxyhemoglobin reflects and lets more light pass than oxyhemoglobin, similar to the near-IR spectrum. Areas in which this is the case are highlighted yellow, whereas the opposite areas are highlighted in beige. As shown in Fig. 3, the flash light illuminates both types of areas, thus reducing the relative intensity of light reflected by deoxyhemoglobin over that reflected by oxyhemoglobin, which diminishes the contrast between both. We also see that in the red spectrum, the illuminant creates intensities in a band in which oxyhemoglobin reflects more light than deoxyhemoglobin, ideal for sensing the former.
| illumination and capture bands | oxy/deoxy difference | ||
|---|---|---|---|
| WOLED & Rcamera | / | WOLED & Gcamera | = 0.63% |
| Rdisplay & Rcamera | / | Gdisplay & Gcamera | = 0.40% |
| Flash & Rcamera | / | Flash & Gcamera | = 0.17% |
Table 1. The ratio between light reflections from deoxyhemoglobin and oxyhemoglobin in different simultaneous illumination and capture bands.
This table assumes that level of illumination is sufficient to be resolved by the camera. Our tests have shown that the camera observes no reflections in the blue band, neither for the display illuminant nor the flash illuminant, due to the tissue's absorption of blue light. The ratios listed above show that signal quality using flash as the illuminant is lower than display due to the flash’s wide illumination across bands. The more narrow-band display illumination generates more than twice the difference in reflections between oxyhemoglobin and deoxyhemoglobin and thus a better signal.
Our measurements also reveal another insight for camera oximetry using flash illuminants: The strong light intensity of a flash saturates the red channel quickly, which requires the camera to reduce its sensitivity to resolve a signal in the green band. Since the digitizer has to perform tone mapping across the entire sensor, not for individual color bands, as the fundamental camera sensor is a gray level sensor with a corresponding color-band filter atop each pixel. Therefore, the red signal gets diminished by the amplification of the green signal, which requires a trade-off in sensitivity. Unlike the flash, the display produces brighter illumination in the green than in the red band, causing the resulting recording to contain a bigger signal-to-noise ratio in both, the red and the green signal
Figure 5. The normalized light intensities of an OLED screen and an LCD screen are similar; OLEDs are relatively stronger in the red and green band.
The LCD screen on the Nexus 5 and the OLED screen on the Galaxy S6 produced comparable spectra. The intensity of the illumination of the OLED screen is higher, showing red and green illuminations that are twice as strong compared to the Nexus 5 LCD screen. This is due to a brighter display in the S6, which boosts our signal quality even further due to the stronger illumination in the appropriate bands. As shown in Table 1, using the OLED for illumination produces a 3.7x higher contrast in the camera signal (0.63% vs. 0.17%) than the signal recorded from reflections using the flash.