Think like a farmer, act like an engineer. That’s the motto of innovators looking to solve big problems in agriculture.
Salah Sukkarieh is Professor of Robotics and Intelligent Systems at the University of Sydney, and Director of Research and Innovation at the Australian Centre for Field Robotics. He has worked on autonomous systems for ports, mines, aerospace, and, most recently, agriculture.
He recalls that when he started working on drone technology there were not many aerospace companies in Australia working on drones, and those that were were not interested in drones for agriculture or the environment as the business case didn’t stack up financially. Australia’s size and the remoteness of many rural areas have also been deterrents.
Things have changed now. There is strong interest from the agriculture industry in the use of robotics and automation to support farmers, and he is surprised by the number of students who are interested in working on these projects.
“There are engineers who have a farming background who come out of the woodwork and say, “I would really like to be part of this process,” he said.
“There are also engineers who are interested in humanitarian engineering. They feel this is something that would be of social benefit so even though they’ve never worked on a farm before, they have a desire to work in this space.”
Autonomous industries
The Australian Centre for Field Robotics (ACFR) was established at the University of Sydney in 1997, with Sukkarieh being within it in one form or other since its inception.
As an undergraduate he worked on the control system for an underwater drone. The Centre then landed one of its most significant projects – the development of AutoStrads with Patrick Stevedores. These were large robots capable of straddling a shipping container and moving it around the terminal autonomously.
“I did my PhD under that project. That’s where the fascination around autonomy and its industrial application really started,” Sukkarieh said.
“We were building inertial units and looking deeply into GPS systems from scratch, helping the sensor companies identify how they can improve their systems while at the same time testing them in an operational context.
“It was at the very early stages and a lot of my research was on designing and coding statistical estimators and data fusion techniques that would fuse the information from these sensors in order to determine the location, speed and attitude of the straddle carrier; and being able to provide that accurately and robustly under different conditions.”
From there he started working on navigation systems for unmanned air vehicles or drones and this got him thinking about agricultural applications. About 15 years ago, a weed contractor asked him if it would be possible to use drones to detect aquatic weeds around rivers and creeks.
He thought about it for a while and tinkered with machine learning algorithms to try and detect the weeds. He demonstrated the capability, but it didn’t progress beyond that.
“It was something that was probably a bit ahead of its time,” he said.
“We showed that we could do it but it was just too expensive at the time. And the level of sophistication that you needed from an operator was too high. You still needed a good team of engineers who had PhDs to run the system.”
He wondered if the technology would be more useful in larger open areas such as grazing livestock, so he knocked on the door of Meat & Livestock Australia (MLA) in 2005 and asked their thoughts on the idea.
“With any of the industry research projects there’s got to be a project champion within the industry who can see an application for that,” he said.
“The MLA research officer could see what drones could be used for. So we started this project with them using drones for collection of data and tackling weeds over large areas.”
The process of testing drones on farms naturally got Sukkarieh talking to a range of farmers who were intrigued by the technology and started suggesting other problems they would love a solution for. It became obvious that ground robots would do a better job than drones because if they can detect a weed, then they can simultaneously do something about it, and maybe other things that the farmers wanted.
“I was also getting many contacts from farmers outside of the grazing livestock industry, mainly in row crops like grains and vegetables,” he said.
The ground robots made even more sense in that situation so Sukkarieh approached Hort Innovation to see if they were interested in funding the research. An initial small project started in 2012 which then led to a series of larger scale projects, before a formal Centre partnership was announced in October 2016.
Down on the farm
One of the first ground robots the team came up with was the Ladybird, a science and research platform that conducts various on-farm crop intelligence and manipulation tasks.
It has numerous sensing systems including hyperspectral, thermal, infrared, panoramic vision, stereovision, LiDAR and GPS, which collectively provide a wide range of information on crop properties that can then be further assessed to determine things like yield and health.
“It was fundamentally a research platform, so we wanted to have the capability of being able to install different sensors without restriction,” Sukkarieh said.
“It’s a much bigger platform to work on because we had to deal with farms that had different row widths. It was also the first platform that we really tested the solar electric systems on board. We wanted to make sure it had extra batteries and solar panels, as well as charging systems that would allow for efficient 24/7 operation on a farm.”
He said the industry got excited about the technology and wanted an operational robot so Ladybird transformed into RIPPA (Robot for Intelligent Perception and Precision Application) and it became the operational platform: lighter, more rugged and easier to operate.
Working hand-in-hand with RIPPA is VIIPA (pronounced Viper), the Variable Injection Intelligent Precision Applicator. It is used for autonomous spot spraying of crops and weeds at high speed using a directed micro-dose herbicide, pesticide or fertiliser.
“We wanted to look at a few things,” Sukkarieh said.
“Firstly, how do you build low-cost, light-weight robots? If you look at the history of agriculture, you have to produce more and more crops over larger and larger land sizes, so the vehicles get bigger. But there’s greater fuel costs, greater maintenance costs, and there’s significant compaction of soil. We wanted to focus more on small robots: less compaction, more agility in operations, less input costs.”
Another goal was to move from fossil fuels into solar electric. One challenge on a farm is being away from mains supply. And the automobile industry was driving some of those costs down.
The third idea was to build the data analytics and automated decision systems that would sit on a robot to grant it intelligence to sense and then make decisions.
VIIPA was designed as an intelligent subsystem that can operate independently of RIPPA but in collaboration with it.
“VIIPA can tell RIPPA, slow down a little bit, or speed up, and RIPPA can be doing other tasks at the same time,” he said.
“The reason why we went down two different paths is because VIIPA doesn’t have to sit on the robot. It can sit on the back of the tractor or in the back of the vehicle, maybe even handheld.”
Another robot they are working on is SwagBot, for the grazing livestock industry.
“We needed something that’s more robust; can go over logs, and ditches and move a lot faster than RIPPA or Ladybird could,” he said.
“It started with philanthropic funding to the University. MLA then got interested in the process and have helped fund us over the past year to advance it to the next level, which is really looking at complete autonomy. We’re investigating how we can make this robot work 24/7 on large land sizes, around animals, within complex terrain.”
The Digital Farmhand is a low-cost robot designed for small holder farmers here and overseas. It has a more modular design without the customised elements developed for certain specific tasks.
“The question was, what would we need to do to dramatically lower the cost of robots?” Sukkarieh said.
“You want to put as much sensing activity in as with the other platforms. You want the capability of remote control, or being able to tow it behind a tractor, or having it work autonomously, and being able to build that with off-the-shelf components.”
Next generation
Sukkarieh said they also have a project around STEM in agriculture robotics to encourage the next generation of farmers and engineers in that space.
“We have a few projects now around Ag robotics STEM for rural kids and for the Indigenous community, and all of these focus on robotics and coding in an agriculture setting,” he said.
“We have a project where we look at needs of digital agriculture in developing countries and what does introducing digital technology do to help them grow food better, but at the same time we are trying to understand what the social implications are of introducing this type of technology.
“To me, the fascination now is more around that relationship between the socio-economic and technical aspects and what that means for small holder farmers in Australia and in developing countries. That is quite fascinating because it’s a very complex area and requires a transdisciplinary approach.”