RMIT researchers have unveiled a device that possesses the ability to ‘see’ and create memories, akin to the way humans do. Manufacturers’ Monthly sits down with team leader Professor Sumeet Walia to learn how the device can reshape the landscape of autonomous technologies.
A team of engineers at RMIT University have created a small neuromorphic device that mimics the intricate processes of the human brain and eye. With its potential to revolutionise a multitude of industries from self-driving cars to forensics to autonomous operations in hazardous environments, this invention has captured the imagination of experts and enthusiasts alike.
The inception of this research can be traced back over the course of seven to eight years, according to lead researcher Professor Sumeet Walia.
With a background as an electronics engineer, Professor Walia is passionate about engineering new material systems and nano-electronic or photonic devices to translate research into real world impact. After his PhD in electronic materials from RMIT University, he delved into research on atomically thin materials.
“These materials are thousands of times thinner than human hair,” Walia explained.
“When you thin down a material to that level, new optical and electrical properties emerge due to quantum confinement effects,” he said. I was interested in learning how this translates into different electrical properties and interactions with different colours of light. We began developing miniaturised photoreceptors that can detect different wavelengths of light,” Walia added.
Each wavelength of light can provide a broad range of information. Ultraviolet (UV) light can be used in forensics or to detect the decay of food. Visible light captures information like human eyes. Infrared (IR) light detects things that the visible spectrum of light is unable to do.
“Our eyes capture visible wavelengths of light,” Walia explained. “There are photoreceptors and colour receptors in human eyes that capture that information and translate it into electrical signals. “After some pre-processing in the optical nerve, the information is transferred to the brain to create memories,” Walia said.
For instance, when driving a car, the human brain will rapidly use this information to make decisions. If a pedestrian is in your way, you need to stop or swerve around them. These complex operations occur within a few milliseconds, and that is what Professor Walia and his team are attempting to replicate with their research. A device that can match the unparalleled efficiency of human vision and brain in processing information.
“Over the course of our research, we realised that we had the first piece of the puzzle – detecting light and capturing light,” Walia explained. “We wanted to take it a step further into data processing and computation and understand how the human brain interacts with that information. This led us to engineer materials to not just capture light, but also store and process information,” he said.
Why would this be useful?
The ways current systems operate, including smart systems or machine learning artificial systems, are both expensive and time-consuming.
“If you mounted a sensor on a satellite, and you are imaging an area of the ocean,” Walia began explaining. “To obtain any information, you collect images frame-by-frame and then send them down to earth via optical downlinks. “Many of these images are redundant and you will still require a person and a computer to process these images. When something of interest happens in those waters, it can potentially take two or three days,” he said.
There is a massive carbon footprint issue with the data processing industry, which is responsible for four per cent of global carbon emissions, according to Walia. This is because the data is being processed using high-power computers, for redundant information much of the time. The type of sensors developed by Professor Walia and his team can process information to enable ultra-fast decision making – thus saving time and energy – and revolutionising industries that rely on quick, informed responses.
The ingenious core: Doped indium oxide
At the heart of the device lies a sensing element known as doped indium oxide. Just as the retina captures light, this chip’s precisely engineered doped indium oxide replicates this process, capturing visual information with a level of precision that mirrors nature itself.
But the ingenuity doesn’t stop there. This device takes inspiration from the human optical nerve, transmitting pre-packaged information to subsequent processing stages. And it doesn’t end with transmission – the chip seamlessly stores and classifies this data, mirroring the human brain’s intricate memory system.
“Indium oxide is the atomically thin material I mentioned earlier,” Walia said. “We can tailor the material based on the wavelength of light you want to capture. “In this case, our team deployed the indium oxide in a way that it not only captures the UV part of light,” he said.
Professor Walia and his team deliberately introduced defects, which allows the device to trap charge carriers—electrons that carry electrical current. This process, known as doping, adds certain chemical elements to a semiconductor to change its electric conductivity. This unique approach enables the device to retain memory of captured events, contributing to longer periods of information retention without requiring a constant electrical signal refresh.
Walia clarified, “The way material physics works – if you shine light that the material absorbs, you will see a jump in current. “If you remove that light, it will come back to its original value. This way you’re not really storing that information, because it’s back to its original value. What defect engineering allows us to do is, you shine light, there’s that jump in current, but when you remove that light, or that stimulus, the current does not jump straight back to its original value. Those charges get trapped in those defect states, thus retaining memory of that event,” he said.
According to Professor Walia, achieving light capture with material that was atomically thin was an important breakthrough for his team.
“When we talk about miniaturised materials, it’s important to not only capture the light but also store and process it in the same thing. This way, you are eliminating quite a few auxiliary components that prevent miniaturisation at a systems level perspective
“In 2018 or 2019, we published our first paper where we showed this ability to use defects with a different material. With indium oxide, we have just scaled that up to a four-by-four array. We have 16 pixels that are capturing light simultaneously and processing information. Our next steps will include scaling that further to a level of, for example, a digital camera.”
Applications across industries
The potential applications of this remarkable device are staggering in their diversity and impact.
For self-driving cars, the device could serve as an onboard decision-maker, analysing visual information and making split-second judgments akin to human drivers. Beyond transportation, the device’s applications could be used for autonomous operations in different terrains in sectors such as mining, defence, and space exploration.
“Space junk, for example, is becoming a huge problem due to collisions between debris and active satellites. Our technology can help maneuver satellites if they are in the direct path of collision. This can be done autonomously, as opposed to collecting information, relaying it down to earth, and then someone making that decision,” Walia said.
The research can also play a vital role in food shelf-life assessments. Walia envisions handheld devices capable of imaging and analysing visible and ultraviolet wavelengths, offering real-time feedback on the quality of food or the presence of forensic evidence.
“It’s sometimes hard to understand the stages of decay in food products. Handheld smart detectors could be used in packaging or over food processing lines to check whether degradation has already begun and use this to make decisions about shelf life. Currently, food is discarded after that expiry date, which means millions of tons of reasonably good food is wasted,” he said.
Similarly in forensics, UV imaging would provide information that is not visible to the human eye. UV is already used in crime scenes, but a device with real-time feedback is quicker and can significantly remove human error.
The neuromorphic device can be deployed in various sectors of manufacturing in Australia and beyond, such as precision manufacturing and can be used for automation and decision-making on the go. Imagine self-monitoring production lines that can instantly detect flaws and defects, autonomous robots that navigate intricate environments, and supply chain systems that optimise routes and resource allocation on the fly. The possibilities are as vast as they are transformative.
Recently, Professor Walia and his team received funding from the Australian Research Council under the National Intelligence and Security Discovery Research Grants 2023, for a defence project that will take their technology further. The expected outcome of the project is an autonomous vision device that highlights changes in the scene using visible and infrared wavelengths.
When asked about challenges, Walia said every industry would have requirements to work around.
“From a fundamental perspective, can you actually engineer the right materials to capture different wavelengths of light? The food and forensic industry would say UV is more important than visible light. But conversely, the medical industry might want to identify disease causing pathogens from the microscopic images using visible light. Security applications, and even cars ¬– when driving at night – would require IR. We can use different materials to capture UV, visible, and infrared, and then tune the chip function based on that.
“Another challenge for deployment would be scalability and integration for different wavelengths. Could we, for example, have three different chips, which you can switch between depending on your use case? Integration in this case would be to have smart neural network algorithms that harvest data to recognise patterns or numbers or images.”
A sustainable vision for the future
At the core of Walia’s vision is sustainability—both for the planet and its inhabitants. He emphasises the interconnectedness of planetary and population health, asserting that the technology’s adoption could contribute to reducing emissions and optimising resource utilisation across various industries.
“The key thing for me is to take our research out of the lab and take it out to the real world. We are very interested in speaking with industry partners to see how our technology could be of significant benefit to them. not looking at one specific industry to deploy our technology. My vision is to explore pathways with different sectors that feed into that sustainability chain.”
As the RMIT University team continues to advance their research and push the boundaries of neuromorphic innovation, the world awaits the transformative impact of this pioneering device.