Digital Camera Myths, Mis-statements, and Misunderstandings

On Wednesday, 17 Jan. 2024 Wayne Prentice will be presenting as part of the Society for Imaging Science and Technology (IS&T) Rochester NY chapter seminar series. This meeting will be hybrid; in-person, and by Zoom video conference.

Place: Irondequoit Public Library, (Room 216) 1290 Titus Ave, Rochester, NY 14617 (in person) Time: 6:00 pm
On-line: https://lnkd.in/gg8vxVj2 (Meeting 859 2672 3473)

The digital cameras are deceptively complex. Understanding camera operation/design requires some knowledge of the parts, including, radiometry, optics, sensors, sensor design, image processing, color science, perception, and image/video encoding. It is easy to miss something.

This talk was inspired by interactions with co-workers and clients. I have observed that subtle, yet important, points when missed can lead to suboptimal product and design decisions. The goal of this talk is to fill in some of those gaps.

Wayne has worked in the imaging industry for over 35 years, for systems ranging from X-ray, CAT scanners, MRI, extra-terrestrial imaging, and digital cameras. Much of his digital camera experience came from 17 years working at Kodak R&D, for digital cameras. At Kodak Wayne became the lead image scientist and manager for Digital Camera R&D group. He has worked as an independent contractor over the past 5 years, providing solutions to a wide range of imaging challenges, mostly developing custom camera applications, computer vision, and HDR imaging.

Super-Resolution: Computational and Deep Learning-Based Approaches

On Wednesday, April 26, 2023, Majid Rabbani and Prasanna Reddy Pulakurthi presented as part of the Society for Imaging Science and Technology (IS&T) Rochester NY chapter seminar series.

Abstract: Super-resolution refers to obtaining an image higher than the camera sensor’s resolution. Super-resolution can be applied to a single low-resolution image or a single frame within a low-resolution captured sequence. The approaches can be either computational-based (using physical modeling of the capture process, subpixel motion estimation and image registration, and regularized spatially variant deblurring) or example-based using machine learning and deep convolutional networks. This presentation provides a brief overview of the field’s history and evolution while addressing its challenges and future directions.

Video file of the presentation (YouTube), https://youtu.be/czlEG-QkKRI

Presentation File (PDF), /burnsdigitalimaging/ISandT/ISandT%20Super-Resolution%20Rabbani%20and%20%20Pulakurthi%20.pdf

L’Oréal Skin Color Chart, Colour Coordinates, Dermatology (updated)

chart1

Skin Color Chart and CIELAB coordinates

Some time ago L’Oréal* developed a colour chart showing various shades of skin colour, based on a sampling of the spectral reflectance characteristics of women’s (healthy) skin around the world.

For a current project, we are addressing the requirements for, and control of, image capture for dermatologists. By understanding important optical (colour) characteristics for this imaging application, we can tailor system design and evaluation methods for improved performance. The chart and similar databases can provide information on system tolerances, and sources of variability.

Report: A short report on the region of (colour) signal-space occupied by the chart is available here, Coordinates of L’Oréal Skin Color Chart (updated) (PDF)

Color Chart: L’Oréal webpage, A new geography of skin color
_____
*L’Oréal Paris, the cosmetics company

Space Imaging Heritage in Rochester NY

On 15 Feb. Bob Fiete presented a talk as part of the Society for Imaging Science and Technology (IS&T) Rochester NY chapter seminar series.

When we see the images taken from satellites on the evening news, from the Russian invasion of Ukraine to the distant galaxies captured by the James Webb Space Telescope, most people don’t realize the significant role played by a group of talented people in Rochester, NY to make these images a reality. This talk will look at the history of imaging technologies and the pioneering accomplishments in Rochester, NY that give us these amazing images from space.

Dr. Bob Fiete is Chief Technologist and Senior Fellow at L3Harris with over 40 years of experience in imaging science. He was Director of R&D for ITT Space Systems Division and has chaired conferences and seminars on imaging and optics. He has briefed the White House and US House of Representatives, and worked with law enforcement on cases image exploitation.

Here is the video,

https://youtu.be/5hfotnao6to

Herbarium Project Cake

As Society for Imaging Science and Technology (IS&T)‘s Archiving conference opens tomorrow, a reminder of the scope of imaging projects for collections.

From Don Williams, part of the Smithsonian Institution‘s celebration of their National Herbarium Collection Digitization Project (4 million items scanned). Cake (right), and the item (left) I found based on the cake’s ‘metadata’. Cake photo courtesy of Ken Rahaim.

Live from Jezero Crater, Mars: Part 2 The Sound of the Machine

Computing the sounds of Perseverance Rover

Previously, I presented frequency-domain analysis is applied to recordings, and showed results based on Perseverance Rover audio files released by NASA. Since NASA posted the original (with-Rover) and filtered (without-Rover) versions, we can use these to compute (estimate) the sounds of the Rover itself.

Based on the previous examination, I found

  • short silence at the start
  • 2 msec. delay due to the filtering (original and filter records were not synchronised)

The Estimate: The above diagram shows how the computed Rover sounds were computed. A bit more detail is shown here.

Estimating the sounds of the machine

The results are shown below, along with the original and filtered versions.

Original (top), filtered (middle), and New computed Rover (bottom) audio signals

We again note that the components of the apparent Martian atmospheric sounds (at 6-8 and 10-11 sec.) are evident in all records. We can interpret this by saying that the computed signal does not completely isolate the Rover sounds, but this is to be expected.

As in Part 1, we can perform the frequency analysis on the computed signal. This is based on a moving window, a length 2000 FFT with a one-third overlap. The results for an example segment are shown.

Power-spectral density for part of the computed audio signal.

The video (2.5 min.) includes the spectral analysis, here. (You might want to try it with headphones or earbuds)

Video,https://youtu.be/eKJFEHfv3pg

Here is a version without me talking, Link

As before, consistent high spikes in the spectra indicate the presence of periodic components (tones) that may be due to, e.g., vibrations, motor rotation, etc. These are seen at 5-7 kHz and near 13 kHz.

– Peter Burns

Live from Jezero Crater, Mars …

[ For my technical friends, this may be seen as straightforward. For others, I hope you find this a useful example of the type of analysis that is behind the design and maintenance of many modern products and services]

First sounds from Mars: Recently NASA posted the first audio recordings from the Perseverance rover. In order for us to hear the ambient Martian sound, two versions of the audio file were posted;

  • the original, containing ‘rover self-noise’
  • a version with the rover self-noise filtered out

Commenting on these sounds, Matt Heck suggested that, in general, attaching a microphone to engines is a useful part of monitoring and diagnosis of problems, wear, etc. He also suggested identifying conditions by frequency analysis to monitor resonant vibrations (frequencies). While this is often done in design, it is rare in normal product usage.

Our friend Fourier: A common analysis tool for signals of many kinds is Fourier analysis. Simply put, expressing signal/image content in terms of frequency components is another way to describe them. Whenever you hear someone say, ‘I don’t have the bandwidth to take on that task’, they are (usually inadvertently) giving a nod to the French mathematician, Joseph Fourier* … but I digress.

The Sound of the Machine: Since NASA posted the original (with-Rover) and filtered (without-Rover*) versions, let’s take a look at these signals. The audio signals are plotted here. The clip runs 18 sec., sampled at 48 kHz.

We note that the components of the apparent Martian atmospheric sounds (at 6-8 and 10-11 sec.) are evident in both records. Filtering signals such as these, under field conditions, is naturally challenging and it is not surprising that the atmospheric and machine sounds would not be completely isolated.

Fourier (Frequency-domain) Analysis: Consistent with Matt Heck’s suggestion, I computed the Signal (frequency) spectrum for the estimated Rover sound. [Since the statistics are not stationary, a spectrum based on a moving window is commonly used. In this case, a length 2000 FFT was used with a one-third overlap]

Shown below is the power spectral density (magnitude-squared) for a single half-second record, for both original and filtered versions. I have found the highest three components, as candidates for identification. For simplicity, the y-axes are auto-scaled, so the actual amplitude values for the two plots are different.

Example spectrum computed for the part of the First Sounds audio file (not the filtered version)
Corresponding spectrum from the filtered audio file.

Variation: The results of the above analysis are subject to normal signal variation and background noise. So, to get a feel for the consistency (or underlying signature) of, e.g., the characteristic sounds that Matt Heck was referring to, I offer this short video.

The video includes the spectral analysis, here. (You might want to try it with headphones or earbuds)

Video (2min). https://youtu.be/d326SKx7wUg

Here is a version without me talking, Link

Consistent high spikes in the spectra indicate the presence of periodic components (tones) that can be tied to particular mechanisms (e.g., vibrations, motor rotation, etc.). For the original recording, we observe these at 5-7 kHz and near 13 kHz.

Other observations: 0.22 sec. of silence at the beginning, and a small delay (phase-shift) introduced by the filtering.

– Peter Burns

——

* Full name: Jean-Baptiste-Joseph Fourier