A year 11 student has developed computer software to help people with upper body paralysis use hands-free computer operation.
We take advantage of how easy it is to use a mouse or trackpad. But for those who don’t have use of their hands, computer navigation is trickier.
“Currently, people are using things such as switch-based technology, brain computer interfaces and even eye gauging technology. But the problem is there’s all this required personalised hardware that costs hundreds and thousands of dollars,” said Sam Kantor, a student at Sydney’s Moriah College.
“If we just create intelligent software that uses the kind of hardware that people have accessible to them, the prices can be reduced to as much as zero.”
For his science fair, Kantor developed Eye Connect, a program that offers a software solution to the issue of hands-free computer operation. As of writing, the project has been in development for 12 months, including six months developing the prototype and six months of testing.
His software works by helping people with upper body disabilities control their computer using eye movements through their computer’s webcam. It can be used across several different operating platforms, such as Linux, Macs and Windows.
The idea for his software came about when he was in year 10 and taking an accelerated semester of computer science at the University of New South Wales.
During the course, he was given free rein to create anything he wanted as part of the final mark.
At first Kantor explored computer vision, but he realised it could be more than just a simple vision project and a real product that could help people with disabilities.
Kantor has developed two types of technology, depending on the person’s level of paralysis. Elements include blink-based control, with Kantor developing his own blink detection algorithm.
This is done by capturing facial images and converting the eye to a grayscale picture. Each pixel on this image is then analysed, and his software can display real-time eye analysis on the user’s screen.
The software then finds clusters of pixels within the eye region that contrasts with the skin colour around the eye. The main eye shape is then isolated and Kantor’s algorithm uses coordinate geometry to determine the eye width and height to determine whether the eye is open or closed.
“[The technology] didn’t exist yet, so I had to create reliable methods of low detection using computer vision,” Kantor said.
“The eventual outcome after lots of testing and modifying the code was around 99 per cent accuracy, which is unprecedented currently in the field of computer vision.”
Kantor has been working on the project by himself and has developed it without any external funding. He recently met with occupational therapists at the Prince of Wales Hospital, and has been able to receive feedback about the software.
“They seemed very excited about this software and how it can truly help people without requiring the kind of expensive and obtrusive hardware that they currently use,” he said.
“It really acts as a more convenient and often even more effective method without requiring the same amount of [money that] current human interaction systems do.”
The software has now been released on SourceForge, an open source directory that allows people to download it free of charge.
Kantor has carried out human testing, but he would like to conduct trials with real patients to improve the software and make it compatible with other devices, such as mobile phones and infrared communication for electronics such as televisions.
“I think that the way to conduct those trials with people suffering upper body paralysis will be through forging a relationship with the Motor Neuron Disease Association and various local hospitals,” he said.
“So that’s an effort that I’m trying to work on, and hopefully I’ll be able to conduct those trials in the upcoming months.”
Kantor recently travelled to Phoenix to take part in the Intel International Science and Engineering Fair (ISEF), representing Australia as part of the New South Wales Young Scientists delegation, where he competed against roughly 1500 students from around the world.