Summary: Researchers developed a robotic sensor using artificial intelligence to read braille at a remarkable 315 words per minute with 87% accuracy, surpassing the average human reading speed. This sensor employs machine learning algorithms to interpret braille with high sensitivity, mirroring human-like reading behavior.
Though not designed as assistive technology, this breakthrough has implications for developing sensitive robotic hands and prosthetics. The sensor’s success in reading braille demonstrates the potential of robotics in replicating complex human tactile skills.
Key Facts:
- The robotic braille reader uses AI and a camera-equipped ‘fingertip’ sensor to read at double the speed of most human readers.
- The robot’s high sensitivity makes it an ideal model for developing advanced robot hands or prosthetics.
- This advancement challenges the engineering task of replicating human fingertip sensitivity in robotics, offering broader applications beyond reading braille.
Source: University of Cambridge
Researchers have developed a robotic sensor that incorporates artificial intelligence techniques to read braille at speeds roughly double that of most human readers.
The research team, from the University of Cambridge, used machine learning algorithms to teach a robotic sensor to quickly slide over lines of braille text. The robot was able to read the braille at 315 words per minute at close to 90% accuracy.
Although the robot braille reader was not developed as an assistive technology, the researchers say the high sensitivity required to read braille makes it an ideal test in the development of robot hands or prosthetics with comparable sensitivity to human fingertips.
The results are reported in the journal IEEE Robotics and Automation Letters.
Human fingertips are remarkably sensitive and help us gather information about the world around us. Our fingertips can detect tiny changes in the texture of a material or help us know how much force to use when grasping an object: for example, picking up an egg without breaking it or a bowling ball without dropping it.
Reproducing that level of sensitivity in a robotic hand, in an energy-efficient way, is a big engineering challenge. In Professor Fumiya Iida’s lab in Cambridge’s Department of Engineering, researchers are developing solutions to this and other skills that humans find easy, but robots find difficult.
“The softness of human fingertips is one of the reasons we’re able to grip things with the right amount of pressure,” said Parth Potdar from Cambridge’s Department of Engineering and an undergraduate at Pembroke College, the paper’s first author. “For robotics, softness is a useful characteristic, but you also need lots of sensor information, and it’s tricky to have both at once, especially when dealing with flexible or deformable surfaces.”
Braille is an ideal test for a robot ‘fingertip’ as reading it requires high sensitivity, since the dots in each representative letter pattern are so close together. The researchers used an off-the-shelf sensor to develop a robotic braille reader that more accurately replicates human reading behaviour.
“There are existing robotic braille readers, but they only read one letter at a time, which is not how humans read,” said co-author David Hardman, also from the Department of Engineering.
“Existing robotic braille readers work in a static way: they touch one letter pattern, read it, pull up from the surface, move over, lower onto the next letter pattern, and so on. We want something that’s more realistic and far more efficient.”
The robotic sensor the researchers used has a camera in its ‘fingertip’, and reads by using a combination of the information from the camera and the sensors. “This is a hard problem for roboticists as there’s a lot of image processing that needs to be done to remove motion blur, which is time and energy-consuming,” said Potdar.
The team developed machine learning algorithms so the robotic reader would be able to ‘deblur’ the images before the sensor attempted to recognise the letters. They trained the algorithm on a set of sharp images of braille with fake blur applied. After the algorithm had learned to deblur the letters, they used a computer vision model to detect and classify each character.
Once the algorithms were incorporated, the researchers tested their reader by sliding it quickly along rows of braille characters. The robotic braille reader could read at 315 words per minute at 87% accuracy, which is twice as fast and about as accurate as a human Braille reader.
“Considering that we used fake blur the train the algorithm, it was surprising how accurate it was at reading braille,” said Hardman. “We found a nice trade-off between speed and accuracy, which is also the case with human readers.”
“Braille reading speed is a great way to measure the dynamic performance of tactile sensing systems, so our findings could be applicable beyond braille, for applications like detecting surface textures or slippage in robotic manipulation,” said Potdar.
In future, the researchers are hoping to scale the technology to the size of a humanoid hand or skin.
Funding: The research was supported in part by the Samsung Global Research Outreach Program.
About this AI and robotics research news
Author: Sarah Collins
Source: University of Cambridge
Contact: Sarah Collins – University of Cambridge
Image: The image is credited to Neuroscience News
Original Research: Closed access.
“High-Speed Tactile Braille Reading via Biomimetic Sliding Interactions” by Parth Potdar et al. IEEE Robotics and Automation Letters
Abstract
High-Speed Tactile Braille Reading via Biomimetic Sliding Interactions
Most braille-reading robotic sensors employ a discrete letter-by-letter reading strategy, despite the higher potential speeds of a biomimetic sliding approach.
We propose a complete pipeline for continuous braille reading: frames are dynamically collected with a vision-based tactile sensor; an autoencoder removes motion-blurring artefacts; a lightweight YOLO v8 model classifies the braille characters; and a data-driven consolidation stage minimizes errors in the predicted string.
We demonstrate a state-of-the-art speed of 315 words per minute at 87.5% accuracy, more than twice the speed of human braille reading.
Whilst demonstrated on braille, this biomimetic sliding approach can be further employed for richer dynamic spatial and temporal detection of surface textures, and we consider the challenges which must be addressed in its development.
