Next level digital sensing and beyond

Unlimited digital sensing unleashed for imaging, audio, and driverless cars: A new technique could unleash the potential for digital sensors like cameras to sense all light intensities, and microphones to sense all audio ranges.

Have you ever photographed a beautiful sunset or recorded a live gig on your phone, only to yield over-saturated images and fuzzy, stop-start playback?

This is because everyday digital sensors like cameras, microphones, and even more scientific instruments like seismometers, radar and ultrasound systems, are subject to saturation points: they cannot detect entities such as light, sound, quakes, temperatures, and other stimuli beyond certain physical limits.

Now, Imperial College London and Technical University of Munich (TUM) researchers have developed a technique that joins new hardware and algorithms to unleash the full potential of digital sensors like these.

Applications range from consumer photography and scientific and medical imaging to space exploration. For example, the technique could increase the range on instruments like cameras and environment sensing for self-driving vehicles, the precision and sensitivity on seismometers for detecting quakes on Earth and Mars, and better range on dosimeters, which measure harmful ionising radiation, for example after nuclear accidents.

Further applications include improved ultrasound imaging and high dynamic range (HDR) X-ray imaging for medical examinations as well as improved precision and range of sensors which detect hurricanes and other natural dangers. It could also help improve non-destructive scanning of objects like airport baggage and testing cracks in gold bars.

Lead author of the research Dr Ayush Bhandari from Imperial’s Department of Electrical and Electronic Engineering said: “We need sensors that capture the full range of what our environment has to offer, and there is boundless potential for sensors to measure signals beyond current human and technological limits.”

Electronic sensing devices contain analogue-to-digital converters (ADCs), which convert information like light and sound from cameras and microphones into digital signals. However ADCs are bound by voltage limits, and saturation occurs when an incoming signal exceeds those limits. Saturation is commonly experienced as ‘bleached’ looking images, or audio that pops and skips, particularly when the stimulus suddenly ‘spikes’.

Dr Bhandari added: “Our new technique lets us capture a fuller range of stimuli in countless examples of digital technology, with applications ranging from everyday photography and medical scanners to extra-terrestrial exploration, bioengineering, and monitoring natural disasters. The hardware-software co-design approach opens up new scientific frontiers for further research.”

‘Folding’ the signal

To carry out the study, the researchers experimented with ADCs that use ‘modulo’ sampling to test whether using a different type of voltage, called moduli, could help sensors process a greater range of information. A modulo refers to the remainder produced when the voltage of a signal is divided by the ADC’s maximum voltage.

They built a prototype with an inbuilt algorithm that triggers the ADC to switch to modulo voltage once the stimulus limit is reached and ‘folds’ these signals into smaller ones. Using this, the researchers were able to convert modulo measurements into smaller conventional digital signals that can be read by existing sensors.

This method allowed the ADCs to process a much wider range of information than was previously possible. It could even provide ‘unlimited sampling’ that accurately captures signals whose amplitudes far exceed ADCs’ voltage limits.

Sky’s the limit

Study co-author and Dr Bhandari’s undergraduate mentee at Imperial, Thomas Poskitt said: “Today, we are surrounded by digital sensors which form a crucial part of the digital revolution. All digital sensors have maximum and minimum limits to what they can detect, but we’ve found a way to breach the upper limit with no theoretical maximum: the sky is the limit!

“By taking a modulo of the signal, we keep the voltage within the limit and reconstruct the full signal, even without knowing how many times the voltage has exceeded the limit. This can unlock a high dynamic range for any sensor which could, for example, allow cameras to see what humans cannot.”

One important application of the technique will be improving cameras on driverless cars. Cameras on cars driving through a tunnel become saturated by the sudden surge of light as cars emerge, causing a loss of seen information and risking the car’s safety.

Indeed, the technology could eventually help develop sensors that can process signals beyond what humans can sense, like ultraviolet, infrared light and other hyperspectral bands, using modulo imaging sensors.

The researchers say their findings overcome perceived limitations in both digital sensing and the ways in which different disciplines can work together to fix common issues. Dr Bhandari said: “By combing new algorithms and new hardware we have fixed a common problem – one that could mean our digital sensors perceive what humans can, and beyond.”

Study co-author Professor Felix Krahmer from TUM said: “The key feature of our approach is that if a signal takes the voltage past the threshold, the hardware switches the signal from voltage to modulo, essentially resetting itself to let in a wider range of signals. What’s new about the current paper is that it presents the first unified approach with both a hardware protoype adapted to computational features of the reconstruction method and a recovery scheme successfully addressing the challenges of the circuit implementation.”

This work was based on the hypotheses set out in Dr Bhandari’s doctoral thesis and 2017 paper which was granted a US Patent in 2020 and was funded by UK Research and Innovation (UKRI) FLF program and European Partners Fund.