Edge AI is the implementation of artificial intelligence in an edge computing environment, which allows computations to be done close to where data is actually created, rather than at a centralized cloud computing facility or an offsite data center.
This localized processing allows devices to make decisions in milliseconds without needing an internet connection or the cloud. Essentially, as the device produces data, the algorithms on board can put the data to use immediately.
Edge AI Definition
Edge AI is the implementation of artificial intelligence in an edge computing environment. That means AI computations are done at the edge of a given network, usually on the device where the data is created — like a camera or car — instead of in a centralized cloud computing facility or offsite data center.
For example, let’s say you have a smart coffee pot that can produce custom drinks for each user, learning from their preferences as time goes by thanks to AI. But unlike most smart devices, this coffee pot isn’t connected to the internet and all of the algorithms it uses to process data are generated within the coffee pot itself — this is thanks to edge AI.
This kind of technology can be used far beyond the reach of your kitchen countertop. Edge AI’s ability to more securely produce real-time analytics at higher speeds, lower costs and with less power has made it an attractive alternative to cloud computing AI — and companies across industries like manufacturing, healthcare and energy are taking advantage.
What Is Edge AI?
Put simply, edge AI is a combination of edge computing and artificial intelligence. Therefore, to understand edge AI, one first must understand edge computing and artificial intelligence.
What Is Edge Computing?
Edge computing is a distributed computing framework that brings computations and data storage closer to actual devices rather than off-site data centers. Edge computing can increase speed and response times thanks to the proximity of data to devices.
These days, smart devices are everywhere. Everything from the watch on your wrist to the car in your garage is capable of performing autonomous computing and exchanging data with other smart devices — a concept commonly known as the internet of things, or IoT.
All that data flying back and forth puts a heavy strain on data centers. But edge computing is meant to ease that burden by moving some of the processing closer to its point of origin. So, rather than traveling to the cloud, the job is done on “on the edge,” so to speak.
The “edge” simply refers to the device being used. This can be a phone, a camera, a car, a medical device or a television. So edge computing is when the computer is inside or near that device. And, like any other computer, edge computers are designed to process standard data.
What Is Artificial Intelligence?
Artificial intelligence is a branch of computer science that encompasses a wide range of technology like machine learning and deep learning. The goal of artificial intelligence is to build smart machines that work and perform tasks that humans normally do without human oversight.
Putting it all together, edge AI is essentially “doing that work and making decisions locally,” according to Aaron Allsbrook, the founder and CTO of edge AI startup ClearBlade. “It’s all the rules. It’s the processing, it’s gathering the data, it’s understanding the data — not in big servers or in the cloud, but in your house, in a work or a job site, or in a parking garage,” he told Built In. “It’s about moving the different mathematical algorithms, and running those predictions at the edge.”
Advantages of Edge AI
Edge AI is Cost Effective
Rob May, the founder and CTO of Dianthus, a company that builds machine learning tools for e-commerce startups, first became aware of edge AI as a partner at venture capital firm PJC, when Deeplite, a company that brings AI computation abilities to edge devices like drones and cameras, was pitched to him as an investment opportunity. He was skeptical, and even passed at first.
“When I thought about the edge AI market initially, I thought it was going to be a small market,” he told Built In. But as he thought about it more and more, he came to realize that it’s a very “cost-driven” sector. So, when Deeplite pitched to PJC again, he agreed to invest in it.
“Let’s say you have a small security camera, and you want that camera to do some kind of analysis. You might not be able to keep that camera cloud-connected all the time, compared to just having a chip in there and a $2 microcontroller,” he continued. “[Edge AI], in a lot of cases, can be a much better model.”
Edge AI Processes Data Faster
Cost effectiveness is just one of edge AI’s many draws. Integrating artificial intelligence at the edge makes real-time data analytics possible, preserves bandwidth, and reduces latency often experienced in cloud connected machine learning models. Things can be sped up by several milliseconds, which can make all the difference when it is being used in something like an autonomous car. Plus, because data stays in the device and doesn’t get moved back and forth between private data centers, it is often more secure. And, with security, comes privacy, which has become of increasing importance across the board, especially when it comes to IoT devices.
Still, edge AI is still “pretty new,” said Allsbrook. “I would say it’s emerging. I think we’re just beginning to see what’s possible.”
Edge AI Disadvantages
Edge AI Requires Ongoing Training
Edge AI systems can be challenging because, like other AI models, they have to be trained on a regular and ongoing basis — just using data from edge devices. This often means creating a data set by transferring data from a huge number of edge devices to the cloud, which can be rather complex depending on the bandwidth available and connectivity to the edge devices.
Extra Security Measures Need to Be Taken With Edge AI
Security is also an area of concern, just in a different way. While edge computing can make systems more safe by keeping processing local, the infrastructure and devices themselves need their own security measures. This can include access control, traffic monitoring, data backup, antivirus and anti-malware software and even encryption.
Edge AI Has Been Slow to Catch On (But That’s Changing)
The edge AI market has not grown as quickly as some may have hoped. Since PJC’s initial investment in Deeplite and, by extension, the larger edge AI space, May said the market has been “a little slower than anticipated” in terms of general adoption. He attributes this to the longer design cycles of the devices that edge AI actually requires — things like drones, phones, cars. He says another big part of this is the general lack of awareness that this space even exists among certain tech companies.
“You have to be aware of something, and then you have to build it into a cycle. And that might take 18 months to get something out the door, from a design perspective,” May said. “I think all that kind of adds up. But I’m hopeful that it’s going to pick up more and more.”
And it seems to already be happening. According to Allied Market Research, the edge AI market was worth just $9 million in 2020, but it is projected to be worth nearly $60 million by 2030. Meanwhile, in 2020, Gartner predicted that 75 percent of enterprise data will be generated and processed at the edge by 2025, bypassing the cloud entirely.
And companies in this space are continuing to garner attention from investors. AI chip startup Axelera, for example, raised $27 million in funding, and brought on Jonathan Ballon, the former VP and general manager of Intel’s edge AI and internet of things group, as its chairman. In the near future, Del Maffeo says Axelera has plans to focus on the security, retail and industrial automation industries because all three of them invest heavily in computer vision tech and cameras, which its chips are specifically designed for. For example, the chips can be used for cashierless checkouts at Amazon Go grocery stores, or in security cameras in smart cities, or in collaborative robots that require vision to prevent accidents with their human coworkers.
“Edge computing can help people improve the quality of their life,” Del Maffeo said. “We are at a tipping point where new technologies can really make a huge impact on the quality of life of people around us. … If you look forward 10 to 15 years, the world is going to change a lot all around us, and edge AI can positively impact it.”
How Does Edge AI Technology Work?
With edge AI, machine learning algorithms can run directly at the edge of a given network, close to where the data and information needed to run the system are generated, such as an IoT device or machine equipped with an edge computing device. Edge AI devices use embedded algorithms to monitor the device’s behavior, as well as collect and process the device data. This allows the device to make decisions, automatically correct problems and make future performance predictions.
Edge AI can run on a wide range of hardware, from existing central processing units, or CPUs, to microcontrollers and advanced neural processing devices. Some of the most-used edge computers are made by large tech companies like Intel, NXP and Qualcomm.
Cloud AI vs. Edge AI
While there are similarities between cloud AI and edge AI, they have distinct differences.
Cloud AI is when data is processed and stored on the cloud. This offers software engineers more flexibility in the design and structure of cloud AI systems, but it requires an internet connection to function. Cloud AI’s dependence on internet connectivity can potentially lead to efficiency and security issues.
Since edge AI processes and stores data locally and without internet connection, the technology can produce real-time data and make independent decisions.
May says that the high levels of computing power achieved by edge AI devices could eventually make edge AI even bigger than cloud AI. But this will largely depend on the kinds of chips that come to market, and how cheap and efficient they are.
One startup working toward this is Axelera, which is in the midst of designing a chip that offers high computing performance and usability at what co-founder and CEO Fabrizio Del Maffeso says is a “fraction of the price and power consumption,” of the tech we typically see today. So, instead of moving data from the memory to the CPU to the memory again (which is typical of most computers), the chip merges the memory with the CPU in what is called “in-memory computing,” he explained. “We are modifying the memory and making the calculation inside the memory. This allows the chip to be very efficient because you don’t move so much data.” By essentially embedding computing elements inside the memory, he continued, the computer’s neural networks can be accelerated.
Axelera certainly isn’t the only company innovating in this space. The edge AI hardware market is projected to grow from 920 million units in 2021 to more than 2 billion units by 2026. And, according to one estimate, the edge AI chip market alone is expected to grow by some $73 billion by 2025.
The Importance of Edge AI
Overall, the edge AI space is growing right alongside the increasing ubiquity of artificial intelligence in general, which is continuing to optimize even the most mundane parts of daily life — from predicting the best day for trash collection in a city, to helping small businesses run more efficiently. And despite its technological complexities, the ultimate goal of edge AI is a fairly simple one: to get closer to devices themselves, thus reducing the amount of data that needs to be moved around.
This, in turn, preserves bandwidth and reduces latency, or the surface area data must travel to be processed. Plus, because this data doesn’t have to travel far, edge AI makes real-time analytics possible, which can have a big impact on future innovations in this already fast-moving space.
“Edge AI is much more real time. It’s looking at the data as it comes off of the live feeds, that makes predictions in the live feeds,” Allsbrook said. “If we can do AI at the edge, we can make really cool predictions pretty dynamically without having to move data to centralized clouds or IT.”
Edge AI Examples
Edge AI is perhaps most popular for its industrial IoT applications, but that isn’t its only use case. Other examples of edge AI can be found in DevOps, robotics, consumer technology and more.
Health Monitoring Devices
Edge AI allows hospitals and other healthcare providers to reap the benefits of artificial intelligence without having to transmit sensitive patient information unnecessarily. All the data collected from health monitoring devices like cardiac trackers and blood pressure sensors can be processed and analyzed locally, enabling real-time analytics that help medical professionals provide better care to patients.
Edge AI Implementation Software
Allsbrook’s company ClearBlade specializes in implementing edge AI in industrial IoT devices, particularly as it relates to predictive maintenance, or forecasting when certain equipment is going to fail. Companies can make inferences directly on the ClearBlade platform off of the live IoT data feeds thanks to its engine called Onyx, which allows people to build models in lots of different languages and with various types of tools.
Allsbrook says he’s also noticed that people are using Onyx to repurpose their old models using edge AI, bringing them onto new technology without having to build a fresh model or hire a new team of data scientists.
Self-Driving Cars
When a 4,000-pound autonomous vehicle is driving down a busy road, every millisecond counts. The rapid data processing enabled by edge AI allows the system to respond quickly to the world around it — ideally making it a safer, more reliable contraption.
Security Cameras
Edge AI’s use of computer vision, object detection and facial recognition makes some security cameras, like ones from Vmukti, particularly effective. They allow for two-way audio, digital zoom and remote monitoring from any location.
Smart Homes
From video doorbells to voice controlled light bulbs and refrigerators that monitor things like food consumption and expiration dates, smart homes contain a web of IoT devices that are meant to work together to make the residents’ lives easier. Instead of these devices having to send all the data from the house to a centralized remote server for processing, edge AI allows all of this to happen onsite, making it faster and more secure.
The Future of Edge AI
If and when we do have all our smart devices equipped with AI on the edge, May said it’s likely we’ll be able to notice it in our everyday lives. He predicts more things will have voices embedded in them. And this tech will likely work faster and be cheaper.
“When you think about the ability to build intelligence into every device, no matter how small, whether it’s internet-connected or not, I think that’s a pretty powerful opportunity,” May said. “It’s hard if all of that has to be cloud-connected all the time,” he continued, so “edge AI is one of the key technologies that’s going to help with this sort of ambient computing.”
In fact, he thinks eventually edge AI could be even bigger than cloud AI, so long as it’s able to maintain high levels of computing power. That’s not to say this technology will completely oust cloud computing. Companies will likely still need to keep all their software-as-a-service, or SaaS, applications, database applications and infrastructure in the cloud.
Instead, Del Maffeo predicts the two will complement each other as artificial intelligence continues to get more sophisticated.
“Your phone is getting more and more powerful. Your TV is becoming a computer,” he said. “I mean, computers are everywhere. It’s a classic trend.”