Summer’s here, which means Australians are flocking to the beach — and this drone technology is keeping swimmers safe by detecting sharks from high in the air.
University of Technology Sydney computer scientist Professor Michael Blumenstein has spent years teaching artificial intelligence algorithms to “see” the world from a drone.
He’s one of the creators of SharkSpotter, a technology that can detect sharks and other marine life swimming off Australian beaches.
The platform uses the real-time feed from a high-resolution camera attached to a drone. The software processes the video stream and analyses it, looking for sharks in the ocean.
“It was always around empowering lifeguards to do their job better, to provide safety on the beach and to save lives,” Blumenstein said.
SharkSpotter is trained to recognise several objects, including 16 different types of marine life and humans, using deep learning. Blumenstein said the technology is given “supervised training”, where researchers provide labelled samples to learn from.
“You’re showing patterns to a learning algorithm and it absorbs those patterns,” Blumenstein said.
“In our case, you can show it a whole bunch of sharks. [Then] you could easily set it loose [on] a totally different day on a different beach and still get pretty high accuracy of detecting a shark in the ocean.”
The software has been tested on more than 50 beaches around the country.
“Humans that used to detect sharks off helicopters, which was the old way of doing it with binoculars — the accuracy was around 35 per cent,” he said.
“We’re getting up to around 90 per cent [accuracy].”
Retraining the algorithm
Blumenstein’s team has wrestled with a lot of technical challenges concerning variables like glare and turbid water. He said the technology has maintained a high accuracy on comparable beaches.
“We’ve been lucky in Australia, that the ocean water’s very clean; the visibility is usually very good,” he said.
In other parts of the world, the algorithm has needed to be retrained.
“If you’ve got the process and the pipeline of activity properly sorted, you can actually retrain fairly quickly,” Blumenstein said.
“But it’s an extra effort.”
One of the trickiest locations was the notoriously shark-infested Reunion Island, in the Indian Ocean.
“The water there was much more turbid. The colour was different. But we got it to work through adjustments in how we conducted the training and the way we deployed the drone.”
The researchers also trained the AI to detect crocodiles in rivers.
Blumenstein said many people have suggested integrating LiDAR or another remote sensing method that could see deeper into the ocean. But the team is yet to hit a technical challenge it couldn’t overcome using vision alone.
And the team has always planned to integrate other technologies if needed.
“But we just haven’t needed it,” he said.
Blumenstein recently returned from Vietnam, where he launched an extension of the SharkSpotter technology designed for disaster management. The AI is trained to detect human survivors after floods or other natural disasters in urban areas.
Early trials suggest the technology is as accurate as SharkSpotter, Blumenstein said.
“We’re not detecting anything but the shape of a human,” he said.
“This is just ‘could you find humans in wreckage post-flood’. And we found that the technology was able to translate.”
In 2018, a Little Ripper drone used by the SharkSpotter team performed what is believed to be the world’s first drone rescue.
The aircraft dropped an inflatable rescue pod beside two teenagers struggling in heavy surf.