From apple-picking robots to automated solar tracking systems, RMIT University engineers are using digital twins to understand real-world consequences of complex change.
Digital twins are often conflated with immersive 3D visualisations, but graphical rendering is only one layer of presentation.
According to Professor Ben Cheng, Academic Lead for Research Development at RMIT University’s School of Engineering, what defines a digital twin is more on its computational fidelity and live data integration, rather than how it looks.
“A true digital twin depends on automated, bidirectional communication, not a static simulation or a one-way digital shadow,” Cheng said. “Sensors capture real-time operational states from the physical asset to update the model, and the model’s outputs are then fed back to guide real-world operations.”
Within this system, information from the physical world is fed into the model, allowing the testing of various possibilities. The system then delivers outcomes that can be used to inform, train or control the real-world system.
This closed-loop approach lets RMIT researchers move beyond passive monitoring to build digital twins that adapt and act across manufacturing, energy and infrastructure systems.
The RMIT team has recently brought together various technologies, including spatial capture, drone photogrammetry, the internet of things, physics-based simulation, and robotics integration, to reproduce reality on a screen and create a digital environment that can be interrogated from multiple angles and for various outcomes.
Rather than proving their systems in the enclosed environment of a mechanical plant or factory, RMIT worked with Food Agility and Agriculture Victoria on the Sustainable Orchard of the Future project, applying digital twin principles to an outdoor environment far less controlled than a building.
Creating a digital twin of nature
“We knew the underlying principles that we used for indoor could be employed for outdoor; we just needed to add more parameters,” said Associate Professor Toh-Yen Pang, Deputy Head of Biomedical Engineering at RMIT.
Outdoor complications, he said, include sunlight, glare and reflection, constant weather changes, unevenness of ground, growth of trees, leaf movement and more.
The creation of the orchard in a digital space involved the capture of around 3000 high-resolution images, taken through the use of drone photogrammetry from numerous altitudes and angles. Drone flight paths had to be designed to maximise overlapping reference points to ensure accuracy.
“The process of photogrammetry is to find common points between the images and then use them to help construct a 3D model,” said RMIT Project Manager, George Wu.
The result is not just a digital representation of an orchard, though. Once all relevant information has been added, it becomes an environment with physics embedded into it.
“So, the terrain that the orchard sits on, the robot that moves through it in the digital environment can feel every dip and rise,” Wu said. “And the apples have gravity, so the harvesting robot can learn how to grab and drop them. Various forces can be adjusted and configured.”
Once the model is designed, Wu said, a wide range of scenario analyses are possible.
Enabling a high-performance orchard
One of the first questions asked of the researchers was how the orchard’s orientation affected sunlight reaching the trees and their fruit.
This led to a further query around how agrivoltaics – the combination of agricultural production and solar energy generation – might help or hinder the levels of sunlight reaching the trees.
Solar panels cast shadows, so if they are mounted in a way that enables their automatic movement during sunlight conditions, they can serve double duty as sunlight controllers. The RMIT digital twin enabled testing of various configurations, angles and outcomes.
If the panels keep direct sun off a section of the crop, they can also prevent the ripening of the fruit for a few extra days.
“That means this farm can hire a smaller group of people to do the harvesting over a slightly longer period, meaning they don’t need to fight for more people within a very short harvest window,” Cheng said.
That is just one of the potential benefits offered by a digital twin. Using NVIDIA Omniverse and GPU-accelerated techniques such as ray tracing, the team can analyse how light interacts with the leaves, and also train a robot to grasp, pick and handle the fruit and ensure a clear pathway.
“Doing most of it in simulation to validate it first before deploying it in the real world can significantly de-risk the farm’s work,” Wu said.
The digital system can also become a place that stores generations of gathered knowledge from farmers, including around how quickly specific types of apples ripen in the sun, for example.
“Farmers carry decades of experience and knowledge that’s never been written down,” Pang said. “When those farmers eventually retire, all that knowledge retires with them too.”
The digital twin ecosystem turns that knowledge into an evidence-based library, giving the industry a central record it can trust.”
Digital twin: Know what you want
RMIT’s capability extends beyond orchards, it’s built on a deeper understanding of complex systems across manufacturing, infrastructure, energy, logistics and more.
The technology can be used to inform any complex project, but it must begin with the challenge rather than the system.
“When we are approached, usually we spend at least a few days on site, next to their system, observing what happens as they tell us their pain points,” Cheng said.
“What we always worry about is that we could build a model that actually doesn’t capture what we want to solve, so first we need a clear definition of the problem.”
For engineering and technology organisations facing complex-system challenges, Cheng said, there is an opportunity to collaborate with RMIT to explore how digital twins might enable ideas, including automation and robotics, to be validated and refined before the risk and cost move into the real world.
Organisations facing their own complex-system challenges can contact Professor Chi Tsun (Ben) Cheng at [email protected], or connect with him on LinkedIn, to discuss how a digital twin could transform the way they operate.
