What It’s Like to Build AI Solutions at Carbon Robotics
In the agricultural world, weeds aren’t a minor issue — they’re a threat to farmers’ livelihoods.
Weeds compete with crops for critical resources like water, sunlight and nutrients, limiting crop growth and reducing harvest yields. And, in some cases, weeds look very similar to crops, making it difficult for farmers to identify which plants should be eradicated.
The teams at Carbon Robotics want to help farmers deal with this challenge more effectively. That’s why they developed their new Large Plant Model, which gives farmers precise control over the company’s LaserWeeder, a tool that uses AI, computer vision and robotics to identify crops versus weeds.
What Does Carbon Robotics Do?
Carbon Robotics harnesses the power of computer vision, AI and robotics to build autonomous agricultural equipment that’s designed to eliminate chemicals, reduce costs and increase yields.
Deep Learning Engineer Shosei Anegawa, who helped develop the model, said that the new solution is trained on over 150 million plant images, providing a new classification paradigm that has greatly improved farmers’ ability to identify weeds that look nearly identical to crops.
For Anegawa, working with AI at Carbon Robotics is unique in the sense that off-the-shelf models aren’t suitable for the solutions he and his peers build. That means they get to build custom models that can detect tiny objects, distinguish classes that look similar, and run in real time to weed at maximum efficiency, offering an immersion in deep learning that he considers one of his favorite aspects of his job.
And the best part? Anegawa and his teammates get to see their ideas become products that positively impact farmers.
“Walking behind a machine and watching a model or feature you designed work to kill weeds is a feeling I can’t see myself getting tired of,” Anegawa said.
Below, Anegawa shares more about the company’s new Large Plant Model and what it’s like to work on AI at Carbon Robotics.
How Carbon Robotics Uses AI in Agricultural Technology
Explain how AI is used at Carbon Robotics. How is it connected to your product or product development?
We developed our Large Plant Model, trained on over 150 million plant images, which powers the LaserWeeder’s classification and targeting. Its primary purpose is to give farmers precise control over the LaserWeeder’s classification and shooting behavior, and to adapt to various field conditions and crops with a single model. We also employ other deep learning-based models for tracking and targeting, as well as for perception in our autonomous tractor kit.
What Is the Large Plant Model?
Carbon Robotics’ Large Plant Model is an AI model trained on more than 150 million plant images that powers the LaserWeeder’s classification and targeting. It helps farmers identify weeds that look very similar to crops and lets them define plant behavior on the fly without retraining the model.
Tell us about a project or milestone involving AI/ML and its impact on Carbon Robotics.
Our most impactful milestone so far has been the release of our Large Plant Model, which powers a feature called plant profiles. Before the LPM, our approach to weed detection and classification followed a standard object detection paradigm where we trained a model to localize and classify crop or weed — one of four weed categories defined by us — depending on the crop the farmer was weeding in. For example, farmers would often want to not shoot certain plants like barley, which are planted to protect the crop. Or, there might be specific weed species that both fall under our “grass” category but need vastly different laser shoot times to kill. In addition, the concept of what a crop is changes between farmers and fields, meaning we need to have a family of models for each different crop.
To address this, the Large Plant Model uses a new approach where farmers can easily group and define behavior on species on the fly without any retraining. This new classification paradigm was a massive improvement in dealing with weeds that evolved to look almost identical to crops. This also opened up the option for us to roll into most new crops and be weeding within minutes.
How Carbon Robotics Approaches Data Management
Are there any unique approaches to AI or data management that you or your team have?
I came from a research background, and one of the biggest differences of working at Carbon has been that we have our own in-house labeling tools. This allows us to quickly add to or pivot our labeling strategies, which isn’t something that’s possible in a research environment where the datasets are often fixed. This opens up a whole new dimension of problem-solving. We can inject specific data into models, or even change how a model behaves relatively quickly to solve a specific problem.
What It’s Like for Engineers to Use AI at Carbon Robotics
What do team members working with AI/ML at Carbon Robotics get to do that they might not be able to elsewhere? Why should an applicant interested in AI join your team?
One of the more interesting aspects of the problems that we’re solving is that off-the-shelf models that would often be used for similar tasks, such as the “You Only Look Once” algorithm for object detection or the “Distillation With No Labels” framework for self-supervised learning, don’t perform well or lack some properties that are necessary for us. This requires us to build custom models that can detect tiny objects, distinguish classes that look extremely similar, and still run in real time to weed at maximum efficiency. This is also true for other parts of the machine that we use AI for, such as targeting and tracking. Using and designing custom models and pipelines to adapt to agriculture-specific challenges is one of my favorite parts of working on deep learning at Carbon.
How Carbon Robotics Engineers See the Real-World Impact of Their Work
What technical aspects of your work are you proud of, either individually or as a team?
What I’m most proud of is how tight the loop is between an idea and a machine having a real impact on a farmer’s field. Everyone is always working on experiments to solve specific problems that customers run into, whether in their spare time or while dealing with more pressing issues, and they’re highly impactful when they hit customer machines. We also go out to the field ourselves to test experiments and new features we’ve designed, which gives us much more insight into model behavior and lets us steer our work in better directions. Walking behind a machine and watching a model or feature you designed work to kill weeds is a feeling I can’t see myself getting tired of.
Frequently Asked Questions
What Does Carbon Robotics Do?
Carbon Robotics builds autonomous agricultural equipment by combining computer vision, AI, and robotics. Their primary goal is to help farmers eliminate the need for chemical weed killers, reduce operational costs, and increase crop yields.
What is the Large Plant Model?
The Large Plant Model is an advanced AI model trained on over 150 million plant images. It powers Carbon Robotics' LaserWeeder by providing a new classification system that allows the machine to identify weeds that look nearly identical to crops. A key feature of the LPM is that it allows farmers to group and define plant behavior on the fly without needing to retrain the model.
How does Carbon Robotics use AI in agriculture?
Carbon Robotics integrates AI into their agricultural technology in several key ways. The team use its Large Plant Model to drive the LaserWeeder's ability to differentiate between crops and weeds, allowing the machine to decide exactly which plants to shoot with lasers and which to leave alone.
Beyond weed control, the company utilizes deep learning models for perception in their autonomous tractor kits to help the machinery navigate fields safely and effectively.
