From 2016-2019, a team from the Australian Centre for Field Robotics at the University of Sydney undertook projects focusing on advancing the application of robotics, intelligent sensing systems, precision agriculture automation and more in Australia’s vegetable industry.
In November 2019, Vegetables Australia spoke to Project Lead Professor Salah Sukkarieh about the research, and the potential impact it has for the future of on-farm robotic automation.
Over the past three years, many vegetable industry members would have seen the LadyBird™ and RIPPA™ (Robot for Intelligent Perception and Precision Application) robots in both trial-farm and commercial-farm settings.
A strategic levy investment under the Hort Innovation Vegetable Fund, Using autonomous systems to guide vegetable decision making on-farm (VG15003) was undertaken to research and develop novel sensors, algorithms and robotic technologies that would help reduce production cost and increase on-farm productivity in the vegetable industry, in particular brassica, lettuce and baby leaf.
The project was awarded to the Australian Centre for Field Robotics at the University of Sydney and, led by Professor Salah Sukkarieh, started in January 2016. It was divided into four themes: Sensing, Automated Decision Support (ADS), Crop Interaction and Farm Automation Standards.
The project used the LadyBird robotic platform, designed and developed by the Australian Centre for Field Robotics under a previous investment entitled An Intelligent Farm Robot for the Vegetable Industry (VG12104). The robot was fitted with a number of different sensors including Thermal, Hyperspectral, Vision and Laser, and experiments were conducted on the University of Sydney farm.
Evaluating and testing autonomous systems developed in VG15003 in Australian vegetable production systems (VG15059) started in June 2016 and focused on Evaluation, Market Analyses, Economic Viability and Industry Engagement of the four themes in VG15003. The project used the RIPPA™ robotic platform, also designed and developed under project VG12104. It was customised for testing and evaluation of these technologies on commercial operational farms.
“The two projects worked together: VG15003 focused on the research and demonstration of novel sensing and algorithms, and VG15059 operationalised that research and it demonstrated to growers on-farm,” Professor Sukkarieh explained.
Both investments drew to a close in February 2019, having supported and progressed the development of new technologies for the vegetable industry.
Key findings
The experiments undertaken during project VG15003 collected large amounts of data and analysed for evaluation of plant health and estimation of water index.
“Initially experiments were done for lettuce and the scope was expanded to cauliflower and broccoli. Machine learning algorithms were developed that could observe plant structure, distinguish weeds from crops, detect water stress or damage, and map soil properties such as conductivity and moisture,” Professor Sukkarieh said.
“The data collection exercise demonstrated the capacity to rapidly collect high spatial and spectral resolution data on a routine basis (weekly), as well as the software pipeline that could extract this data and make meaningful representations of the plants and environment in general.
“A number of important characteristics were determined from these results, including the ability to get close to 100 per cent confidence in the machine learning systems extracting plant information (shape and size), as well as greater than 90 per cent accuracy in detecting weeds green-on-green when in uncluttered environments (using spatial and shape differences), and greater than 75 per cent in cases where weeds were similar to early established plants.”
During VG15059, Professor Sukkarieh and his team ventured on-farm to demonstrate RIPPA’s capability for detecting weeds in real-time among crops and real-time weeding using mechanical prongs as well as directed spraying technologies that minimised chemicals.
“The software pipeline developed was also used to detect foreign objects and we retrofitted an industrial vacuum cleaner that could suck up those foreign objects. We demonstrated the capability of actively sampling water conductivity near plant roots. A user interface was developed that not only allowed growers to control the robot, but also to see results of plant statistics, history, weed pressure and water availability,” Professor Sukkarieh said.
“Both projects went through continuous cycles of experimentation and design improvements; especially for operational envelope enhancements.”
Most of prototype tests were conducted at the University of Sydney’s Lansdowne farm, and operational trials were conducted in Gatton, Cowra, Griffith, Werribee, Gippsland and Launceston. More than 660 growers attended, and feedback was collected through end-to-end demonstrations.
Addressing issues
This was the first time extensive full production, autonomous data collection of individual plants in the field had been conducted – which posed challenges to Professor Sukkarieh and his team.
“There were are a number of technical challenges that we overcame, including registration of changing plant data over multiple weeks and the effect of lighting and weather on autonomous data gathering,” Professor Sukkarieh said.
“Plant biology means that the algorithms that were developed had to deal with changing plant architecture over time which hadn’t been dealt with before in the research community.”
This was also the first time that the robotics and automation work extended beyond just automating a platform for operation on-farm and moved into crop intelligence. Grower input also contributed to the program, as Professor Sukkarieh explained.
“We received very positive feedback in our trials as well as a number of suggestions on operational improvements that were incorporated into the program. As the project moved through its various stages of development, the growers became very interested in the capability that the system could also provide in the future for pest/disease detection.”
According to Professor Sukkarieh, VG15003 and VG15059 both demonstrated that a continuous cycle of research and development through to operational demonstration with the growers was key.
“This is to ensure that the research aligned with the grower interests, and that continuous feedback was provided to the research. The next steps in robotics would include better crop intelligence algorithms for detecting pests/diseases and novel tools for precision action on plants and ultimately harvesting – and these should follow the same approach,” he said.
“There are technologies that are ready for commercialisation, including automated platforms for weeding and crop intelligence (plant size and distribution). These should now transition into the hands of the growers for end-user feedback.”
The bottom line
Professor Sukkarieh said that the three projects – VG12104, VG15003, VG15059 – have placed the Australian vegetable industry at the leading edge of on-farm robotic automation.
“We are now in position to commercialise many of the technologies developed in these programs. The industry now should look further forward to understand what future vegetable farm operations would look like with the belief that on-farm robotic solutions are coming,” he said.
“This will help provide a vision on how best to use these technologies (beyond weeding and crop intelligence), how intelligent on-farm systems should work with each other, and how these intelligent assets will support supply chain optimisation.”
For more information:
Professor Salah Sukkarieh
Email: [email protected]
www.sydney.edu.au