Dee learning is getting a lot of hype at the moment. People want to use neural networks everywhere, but are they always the right choice? We'll take a look at some of the disadvantages of using them.
Table of Contents:
- Understanding the Deep Learning Hype (Data, Computational Power, Algorithms, Marketing)
- Neural Networks vs. Traditional Algorithms (Black Box, Duration of Development, Amount of Data, Computationally Expensive)
Understanding The Hype Around Deep Learning
There are four primary reasons why deep learning enjoys so much buzz at the moment: data, computational power, the algorithm itself and marketing.
Massive amounts of available data gathered over the last decade has contributed greatly to the popularity of deep learning. This has allowed neural networks to really show their potential since they get better the more data you fed into them.
By comparison, traditional machine learning algorithms will certainly reach a level where more data doesn’t improve their performance. The chart below illustrates this perfectly:
2. Computational Power
Another very important reason for the rise of deep learning is the computational power now available, which allows us to process more data. According to Ray Kurzweil, a leading figure in artificial intelligence, computational power is multiplied by a constant factor for each unit of time (e.g., doubling every year) rather than just being added to incrementally. This means that computational power is increasing exponentially.
The third factor that has increased the popularity of deep learning is the advances that have been made in the algorithms. These recent breakthroughs in the development of algorithms are mostly due to making them run much faster than before, which makes it possible to use more and more data.
Finally, marketing has played an important role. Neural networks have been around for decades (proposed in 1944 for the first time) and have experienced peaks and valleys in popularity. The phrase "deep learning" gave it all a fancy new name, which made a new awareness (and hype) possible. As a result, many people wrongly believe deep learning is a newly created field.
Additionally, major breakthroughs in the field of machine learning, including the controversial "humanoid" robot Sophia from Hanson robotics have led to increased media coverage and awareness.
Neural Networks vs. traditional Algorithms
Should you use neural networks or traditional machine learning algorithms? It's a tough question to answer because it depends heavily on the problem you are trying to solve. Consider the "no free lunch theorem," which roughly states there is no "perfect" machine learning algorithm that will perform well at any problem. For every problem, a certain method is suited and achieves good results, while another method fails heavily.
Personally, I see this as one of the most interesting aspects of machine learning. It's the reason why anyone working in the field needs to be proficient with several algorithms and why getting our hands dirty through practice is the only way to become a good machine learning engineer or data scientist. That said, helpful guidelines on how to better understand when you should use which type of algorithm never hurts.
The main advantage of neural networks lies in their ability to outperform nearly every other machine learning algorithm, but this comes with some disadvantages that we will discuss and lay our focus on during this post.
Again, decide whether to use deep learning or not depends mostly on the problem at hand. In cancer detection, for example, a high performance is crucial because the better the performance the more people can be treated. But there are also machine learning problems where a traditional algorithm delivers a more than satisfying result.
Disadvantages of Neural Networks
1. Black Box
Arguably, the best-known disadvantage of neural networks is their “black box” nature. Simply put, you don’t know how or why your NN came up with a certain output. For example, when you put an image of a cat into a neural network and it predicts it to be a car, it is very hard to understand what caused it to arrive at this prediction. When you have features that are human interpretable, it is much easier to understand the cause of the mistake. By comparison, algorithms like decision trees are very interpretable. This is important because in some domains, interpretability is critical.
This is why a lot of banks don’t use neural networks to predict whether a person is creditworthy — they need to explain to their customers why they didn't get the loan, otherwise the person may feel unfairly treated. The same holds true for sites like Quora. If a machine learning algorithm decided to delete a user's account, the user would be owed an explanation as to why. I doubt they'll be satisfied with “that’s what the computer said."
Other scenarios would be important business decisions. Can you imagine the CEO of a big company making a decision about millions of dollars without understanding why it should be done? Just because the "computer" says he needs to do so?
2. Duration of Development
Although there are libraries like Keras that make the development of neural networks fairly simple, sometimes you need more control over the details of the algorithm, like when you're trying to solve a difficult problem with machine learning that no one has ever done before.
In that case, you might use Tensorflow, which provides more opportunities, but it is also more complicated and the development takes much longer (depending on what you want to build). Then a practical question arises for any company: Is it really worth it for expensive engineers to spend weeks developing something that may be solved much faster with a simpler algorithm?
3. Amount of Data
Neural networks usually require much more data than traditional machine learning algorithms, as in at least thousands if not millions of labeled samples. This isn’t an easy problem to deal with and many machine learning problems can be solved well with less data if you use other algorithms.
Although there are some cases where neural networks do well with little data, most of the time they don’t. In this case, a simple algorithm like naive Bayes, which deals much better with little data, would be the appropriate choice.
4. Computationally Expensive
Usually, neural networks are also more computationally expensive than traditional algorithms. State of the art deep learning algorithms, which realize successful training of really deep neural networks, can take several weeks to train completely from scratch. By contrast, most traditional machine learning algorithms take much less time to train, ranging from a few minutes to a few hours or days.
The amount of computational power needed for a neural network depends heavily on the size of your data, but also on the depth and complexity of your network. For example, a neural network with one layer and 50 neurons will be much faster than a random forest with 1,000 trees. By comparison, a neural network with 50 layers will be much slower than a random forest with only 10 trees.
At the end of the day neural networks are great for some problems and not so great for others. In my opinion, deep learning is a little over-hyped at the moment and the expectations exceed what can be really done with it, but that doesn’t mean it isn't useful. We're living in a machine learning renaissance and the technology is becoming more and more democratized, which allows more people to use it to build useful products. There are a lot of problems out there that can be solved with machine learning, and I'm sue we'll see progress in the next few years.
One of the major problems is that only a few people understand what can really be done with it and know how to build successful data science teams that bring real value to a company. On one hand, we have PhD-level engineers that are geniuses in the theory behind machine learning, but lack an understanding of the business side; on the other, we have CEO’s and people in management positions that have no idea what can be really done with deep learning, but think it will solve all the world's problems in short time. We need more people who bridge this gap, which will result in more products that are useful for our society.
Niklas Donges is an entrepreneur, technical writer and AI expert. He worked on an AI team of SAP for 1.5 years, after which he founded Markov Solutions. The Berlin-based company specializes in artificial intelligence, machine learning and deep learning, offering customized AI-powered software solutions and consulting programs to various companies.