Human in the loop in machine learning means pairing humans and machines to speed processes, efficiently sort through masses of data, prevent bias and fine-tune training models.
HITL in ML, as it’s sometimes abbreviated, is used in four stages as a machine model is being built and perfected. First in training the model, that is, showing it how to sort through the data that’s creating the model; second, testing the model to ensure it uses the data correctly; deployment, or actually commissioning the model to do its job; and monitoring, or making sure the model continues to do its job correctly, and stepping in to make fixes if it is not.
New to machine learning? Read this for an understanding of the role humans play — and will continue to play, sources say — in this subcategory of artificial intelligence. An ML expert? Read on to discover, perhaps, more ways your company can put HITL in ML to work to create better products.
What Is Human in the Loop?
What Is Human in the Loop?
“Human in the loop incorporates human knowledge in training and tuning machine learning models,” said Chen Zhang, CTO at Rain, a voice technology company based in New York. HITL improves model accuracy, handles scenarios when the AI model isn’t confident enough to do so and overrides erroneous AI decisions.
It also helps overrule biased AI decisions, Zhang said. “For example, training data could contain mostly white people’s faces for a face recognition algorithm, or contain mostly Australian accents for a speech recognition algorithm,” he said. “Humans in the loop help root out those biases in favor of generalizing a model,” he said.
How Does Human in the Loop Work?
Because the Singularity hasn’t quite arrived, humans play a role in every part of the process, from labeling the data to tuning the model to fix anomalies such as overfitting, the term for inaccurate predictions, to “edge cases,” which are scenarios the machine has not previously encountered.
Julia Valentine, managing partner of AlphaMille, a New York-based technology consultancy, details four main ways humans step into the machine-learning process; in other words, add the human in the loop factor.
Training
In many cases, humans train the model by demonstrating how tasks should be accomplished, Valentine said. Humans also evaluate and validate the results when they accept or correct the model. “At the basic level, that’s reinforcement learning,” she said. Just as a supervisor might take over a trainee’s task if time is of the essence, this approach works faster than traditional supervised learning algorithms, Valentine said.
Testing
Humans test the model results and determine if the model performs as expected, Valentine said. This is not without risks, though. One scenario: If the human training the model has unconscious bias, the model will be trained to have a similar bias, she said. Human brains and knowledge are required to test the models and make sure they are ethically sound.
Deployment
It is difficult to foresee every potential use case, especially for edge cases, Valentine said. Too, issues can arise when a model encounters incomprehensible or imbalanced data. “Humans use their imagination to think through extreme scenarios and decide whether the model is ready for deployment, and also when to instruct it to raise a red flag,” Valentine said.
Monitoring
When models successfully pass through training, testing and deployment stages, constant or intermittent human monitoring might be required to prevent costly mistakes or evaluate model drift and harmful biases.
8 HITL/ML Terms to Know
- Edge case: A scenario that a machine has not encountered
- Entity: Specific data asked for to fulfill a request
- Intent: What customers are looking for when they type into a search bar
- Overfitting: Inaccuracies that occur when analyses too closely follow a set of data
- Underfitting: Insufficient data to capture a trend
- Training model: A data set that teaches a machine learning algorithm to make a prediction or execute a task
- Supervised training model: A model that relies on labeled data
- Unsupervised training model: A model that uses raw or unlabeled data
Check out Built In’s new tech dictionary for more industry terms and definitions.
Human in the Loop Examples
Here’s how the experts Built In spoke to for this story use HITL in ML at their own organizations.
Sift Through Mountains of Data
AlphaMille helps investment firms, including VC and private equity investors, incubators, family offices and investment banks, automate private-market deal discovery; that is, discern what’s out there and what’s available to buy. At any given time, thousands of companies can be available, and more come on the market every day, Valentine said. “It is humanly impossible to sift through that much information even with dozens of analysts,” she said. “Machine learning plus human in the loop is a great use case for deal discovery.”
Refine Machine Learning Models
Fintech startup DFD Partners matches asset and wealth managers with their next clients, an approach that is “data inspired, tech infused,” according to its website. “We’re now transitioning out of having something that ‘works’ to something more refined,” said Devon Drew, founder and CEO.
The company’s mission is to help asset managers who lack the infrastructure of a giant company scale quickly. DFD’s ranking algorithm, which ranks advisors on how good of a match they are with the company’s users (asset managers), is central to the company’s mission. “It automates the time-intensive process of vetting potential clients for the asset managers,” he said.
“Human in the loop helps create a more amazing client experience.”
By holding focused feedback sessions, DFD gets input from wealth and asset managers, accredited investors and other industry experts to refine the model and thus help users get more accurate results.
This approach builds on DFD’s original sorting algorithm. “We knew how important it would be to have something that used the latest tech available to get more accurate results, which is when we switched over to ML,” Drew said. “Human in the loop helps create a more amazing client experience.”
Generate More and Better Labeled Data
Rain, the voice technology company, uses machine learning models to recognize intents (figuring out what customers mean from what they say to a voice assistant) and entities (data points related to that intent, such as a menu item or car part).
Rain is using ML to develop an automotive technician voice assistant and uses models to determine the correct intent — is the customer looking for info related to a specific car repair? — and to disambiguate between synonyms that map to the same entity, for instance the various ways to ask for ‘wheel nut torque’ on a vehicle, Zhang said. “Humans help to provide more and better labeled data continuously in order to improve these models,” he said.
Execute Text Clustering Tasks
Rain is also using unsupervised learning models for text clustering tasks, he added. To do so, it’s analyzing what users actually say to voice assistance and grouping them into linguistically similar utterances. That saves time because the entire log doesn’t need to be reviewed one query at a time. It turns lengthy utterances into feature requests, discovers linguistic diversity (such as accents) and also spots user frustration.
“Humans help to evaluate the results from these models, and then transform the results into data that is consumable for other AI models,” Zhang said.
Human in the Loop: What’s Next
HITL and ML might sound futuristic and cool (what tech person doesn’t love a good acronym?), but it’s actually as old as industry itself. “The interaction between human and machine began in the industrial revolution,” noted Julia Valentine of AlphaMille.
“The interplay between AI/ML and humans will continue to evolve.”
Human in the loop would cease to be only if humans ceased to be, “and we’re far too optimistic to contemplate that,” she said. “The interplay between AI/ML and humans will continue to evolve.”