Traditional machine learning models like ChatGPT and Claude AI are trained by data mined from millions of users. But as calls for data privacy and AI safety continue to mount, AI is shifting away from data scraping in favor of decentralized techniques. Newer AI models are being trained collaboratively — using raw data that never leaves its source device, be it a smartphone, laptop or IoT device.
This decentralized approach is called federated learning, which allows edge devices to collaboratively train models without sharing their data. The method enables organizations to protect data while leveraging the power of machine learning.
Federated Learning Definition
Federated learning is a distributed machine learning technique that allows devices to train an AI model collaboratively while avoiding sharing data with a central server. Each device trains a local model on its data and shares only model updates, ensuring privacy and data security while advancing the machine learning process.
What Is Federated Learning?
Federated learning is a decentralized way to train machine learning models by involving edge devices that collaboratively train models without sharing their data.
Google introduced federated learning as a term in 2017. The development came in the aftermath of major data security breaches like the Cambridge Analytica-Facebook scandal, after which the public took a heightened interest in protecting their privacy.
“One of the challenges in modern machine learning is the need for large amounts of data to build modern-scale learning models,” said Suhas Diggavi, a computer engineering professor at the University of California, Los Angeles. “There [are] a lot of legitimate concerns about users' data being used for this purpose.”
Today, common examples of federated learning in practice include predictive tools like Gboard’s next-word prediction, emoji suggestion and autocorrect. Google uses federated learning to improve Gboard’s performance without sending personal data — like text conversations — to its central servers. This approach protects user privacy while optimizing model accuracy.
“You probably don’t want every word you type on your keyboard stored in the cloud — in federated learning, [your phone] performs small computations with local data, [which it sends as a training update],” said Justin Kang, a computer science PhD candidate at the University of California, Berkeley. “This local update is nearly impossible for a human to interpret.”
How Does Federated Learning Work?
Federated learning decouples machine learning from private data storage, relying on decentralized devices like smartphones or IoT sensors. After local data updates hone the global model, the improvements are distributed back to the devices for further local training.
“This process is repeated multiple times until the model reaches a desired level of accuracy or performance,” said Salman Avestimehr, Tensor Opera AI CEO and University of Southern California professor.
Federated Learning Steps
The federated learning process involves the following steps:
- Initialization: A global model is initialized on a central server.
- Local training: Each device trains the global model using its local data.
- Update sharing: Devices share model updates with the central server.
- Model aggregation: The server aggregates the updates from various devices to improve the global model.
- Global model update: The improved global model is returned to the devices, and the process repeats.
Federated Learning Frameworks
Multiple frameworks have been developed to support federated learning across various devices and networks. Frameworks provide the necessary APIs to develop real-world applications. Examples include:
- TensorFlow Federated: Developed by Google, TFF is an open-source framework for building federated learning applications.
- PySyft: PySyft is a federated learning framework built on PyTorch that supports privacy-preserving technologies like secure multi-party computation and differential privacy.
- Flower: Flower is an open-source federated learning framework designed to integrate with machine learning libraries like TensorFlow and PyTorch.
Applications for Federated Learning
Federated learning use cases fall into two categories: cross-device learning and cross-silo learning:
Cross-Device Applications
“In cross-device learning, the goal is to train AI models using data collected from various edge devices,” Avestimehr told Built In. Examples include:
- Smartphones: One of the earliest applications of federated learning is in smartphones, particularly in personalized services like keyboard predictions, voice assistants and personalized photo stories.
- Transportation: To improve autonomous driving systems, data from multiple vehicles are used to train models for object detection, route planning, traffic prediction and EV battery range prediction without sharing sensitive information.
- Manufacturing: Federated learning can optimize manufacturing processes by leveraging data from IoT devices to train predictive maintenance models.
Cross-Silo Applications
“In cross-silo learning, federated learning is used to train models on data held by different organizations or institutions, often referred to as silos,” Avestimehr said. In this scenario, the data remains local while institutions collaborate to strengthen the robust model. Examples include:
- Healthcare: Sensitive medical data, like patient records and diagnostic imaging, cannot be shared across institutions due to privacy regulations like HIPAA. Federated learning holds great promise in the healthcare sector — enabling hospitals and research institutions to collaborate on building more accurate predictive models for disease detection, personalized treatment plans and drug discovery.
- Financial institutions: Banks and financial services companies utilize federated learning to detect fraud across multiple locations without exposing customer data.
Challenges of Federated Learning
Common pitfalls of federated learning include:
Communication Efficiency
Since model updates must be exchanged between devices and the central server — or between devices in decentralized architectures — bandwidth and latency can significantly affect machine learning performance. Techniques like model compression, quantization and asynchronous updates can reduce the communication load, but these solutions can introduce trade-offs in model accuracy.
Data Protection
Although federated learning enhances data privacy by keeping data local, research has shown that AI models are not immune to cyberattacks. “It’s possible [to attack] the AI model and learn information about users,” Kang told Built In.
Techniques like model inversion attacks can be used to infer sensitive information from model updates — and bad actors may even be able to intentionally corrupt federated learning using malicious devices. To mitigate these risks, federated learning often incorporates techniques like differential privacy and homomorphic encryption, which further protect user data during training.
Data Heterogeneity
Federated learning systems must handle the inherent heterogeneity of data and devices, including differences in processing power, memory and network conditions. This can lead to imbalanced updates and varying contributions to the global model. Even the unique way users interact with their devices can skew model accuracy.
Lack of Guaranteed Privacy
“A major misconception about federated learning is that it guarantees privacy for the users’ data,” Kang said. Companies like Google strive to guarantee privacy using algorithms like Federated Stochastic Gradient Descent (SGD) — which amplifies privacy by sampling data from random devices — and Federated Learning with Dynamic Regularization (FedDyn) — which allows systems to maintain accuracy in diverse environments by better handling data variability.
User Participation
“Another significant challenge is to incentivize users to participate and therefore help each other,” Diggavi told Built In. To encourage users to share their data, companies must drive user motivation through free services or ad-free content.
Benefits of Federated Learning
Despite its challenges, federated learning offers significant benefits:
Data Security
By decentralizing data storage and processing, federated learning reduces the risks associated with central data breaches. If individual devices are compromised, the global dataset remains protected since individual data points are not stored in one location.
Privacy
The bottom line: Users don’t want personal data like voice recordings or interactions with smart home devices to be sent to enterprise companies’ central servers.
“If [you] combine federated learning with provable privacy methods, [you] could give a privacy guarantee while enabling collaborative learning models,” Diggavi said. This privacy-preserving approach is particularly valuable in fields like healthcare and finance, where data confidentiality is paramount.
Frequently Asked Questions
What is federated learning?
Federated learning is a distributed technique where devices collaboratively train a model by sharing only updates, not data, ensuring privacy and security while enabling decentralized machine learning.
What are examples of federated learning?
Examples of federated learning include Google's Gboard improving typing predictions without accessing user data, healthcare institutions collaborating on predictive models while keeping patient data private and autonomous vehicles sharing driving insights to enhance AI systems without exchanging raw sensor data.
What are the benefits of federated learning?
Federated learning reduces communication costs, enhances data privacy, enables collaboration across organizations and improves machine learning models without centralizing sensitive data.
