With recent reports on deep learning language models like Google BERT and OpenAI’s GPT-3, it’s easy to assume that true AI is here and ready to revolutionize businesses everywhere. Although AI is already remaking businesses, it’ll take a few more years for the smarts exhibited in things like GPT-3 to make their way into deep, narrow domains. That’s because the advances we’ve seen over the last couple of years draw upon general, internet-scale learning. They can’t help you develop a new cancer treatment or design the wing of a new aircraft because they haven’t been trained on deep, domain-specific data.
Will there be a GPT or BERT to provide human-level insight for your domain? Eventually, but not for a while. Models like BERT and GPT-3 draw upon the enormous data offerings of the public internet, giving them a superhuman breadth of knowledge. Unfortunately for us, these mega-models are better at “faking” their understanding than actually knowing what they’re talking about. This is fine when you ask them to do something low-stakes like write a poem or talk about a historical event. It’s less acceptable when you need to know about the side effects or possible drug interactions of a new breast cancer drug.
This problem arises because AIs rely on oceans of data to function. Most domain areas aren’t vast oceans, however. Instead, they’re large lakes, at best. This relative lack of data may seem surprising. You read all the time that businesses are drowning in data. The more specific your problem gets, however, the less data your AI has to draw upon. The solution to this problem may seem a bit counterintuitive. Rather than (expensively) attempting to build a big, domain-aware model, it’s currently better to use models trained on smaller, more specific sets of content to solve narrow problems within an industry. Chatbots or product sales predictions are two good, current examples. This is because the pre-training and data annotation required to build these solutions is minimal, and human input can take you over the finish line. By contrast, only the world’s biggest companies have the capabilities to handle domain-scale models.
So, when looking to AI to solve a business problem, most companies should be thinking in terms of problems that have obvious ROI cases that can be fielded for a reasonable amount of money. The technological advances that will make domain-specific AI possible for regular businesses will come in time. After all, four years ago, there was no BERT or GPT3 at all. But they’re here now, and the technology and tools that enabled their creation will eventually filter down to less monolithic use cases. But for this to occur, two things need to happen:
- Data mark-up tools will need to get smarter.
- The cost of building big models will need to come down significantly.
As with any emerging technology, the cost element will take care of itself over time, so a bit of patience is all that’s required on this front. The data mark-up question is much more challenging. It’s not currently feasible for companies to build the data sets required to create smart domain-aware models. Too much data and too much human input is needed.
Technological advances in machine learning aren’t stopping or even slowing, however. It’s likely that, within a few years, systems will be able to use small amounts of sample content to pre-train data discovery models to go out and find the necessary domain-specific data. When these models can do their job at scale, domain-level AI is within reach. So, be patient and use tried-and-true machine learning techniques to solve smaller, domain-specific problems now, and keep your eyes on the rapidly advancing field. Deep smarts in narrow domains will be here soon enough; it’s hard to say when this will be, but it’s probably sooner than most people believe.
