I have been waging a war across multiple fronts and fatigue has finally set in — they’re winning.
For months — years — I’ve battled, fighting the enemy’s persistent attacks with my catlike reflexes. Click x. Boom! I refuse to fall into their traps, which are perfectly crafted to reduce my defenses to rubble and my brain to jelly. And yet they continue their relentless attack on every front, wielding their weapons wherever I turn. But I’m a stubborn soldier, and in this war, I’ll fight to the bitter end.
Yes, this is a war. A war against recommendations and the engines that power them.
What Is a Recommendation Engine?
Recommendation engines generally optimize toward some goal like greater engagement with the content. These systems are critical to how we interact with products and media online because they narrow down huge lists of content (e.g. videos, songs, games, products) which would otherwise be incomprehensible to users. However, these engines also play an important role in our digital lifestyle, representing one of the largest intrusions of artificial intelligence into our daily lives. And here’s where things can turn sour.
YouTube forces repeats of Big Bang Theory after I went on a small binge. Geeky physicists are funny, but their jokes get old. Spotify still recommends calming songs after I played some meditation music...six months ago. Amazon forces the same products down my throat despite buying them only weeks earlier. Seriously, if I buy a toilet seat from you, don’t keep trying to sell me toilet seats.
It’s frustrating to see the same content over and over again at the expense of content I actually want. To make matters worse, these manipulative systems play to my weaknesses, and for that, I hate recommendation algorithms.
Seriously, if I buy a toilet seat from you, don’t keep trying to sell me toilet seats.
In a perfect world, recommendation algorithms would introduce me to new products and content I would love. They would help me create new ideas. They would inspire me. The algorithms would be like my best friend, telling me their new favorite things. Instead, they’re my worst enemy.
Why Can’t We Be Friends?
Often recommendation algorithms lean towards showing the most popular items. They focus on what will pull in the most clicks. When algorithms are optimized for more time on-site or other similarly shallow metrics, then this is bound to happen. Many people would love to mix up their lives — to improve themselves! But within these frameworks it becomes difficult to do so, as these algorithms continually reinforce our bad behavior (sometimes for profit, sometimes for user engagement and interaction). Google’s Fit app adjusts my weekly goals down when I don’t hit them, and I defer my New Year’s resolutions to next year (sometimes without even noticing!).
It’s not all doom and gloom, though. Some services do a good job at recommendations. For one, I find those served by Yelp are spot on. I get a good mixture of restaurants related to the ones I’ve previously visited, as well as better options which help me improve my health. That’s especially important for me. I want to stay well-fed when fighting a war.
Of course, there’s something to be said about popular recommendations. When I’m ordering groceries online from Amazon, recommending the most purchased item (toilet paper) is fine. I wasn’t looking for a revelation on that front. Yes, there are other problems with this system, such as the monopolization of commodity products, but that doesn’t affect me, the end user.
Recommendations aren’t the responsibility of the program, but the programmer.
So what’s the difference between a good recommendation and a bad one? The companies that do the best work combine many layers of recommendations. They take your history and find what other users with a similar history liked. They find products that are similar based on how users viewed them. They find items that have similar content. They find products that are used by people in the same area. The best even add their own magic based on their industry expertise to help users along the way. These hybrid systems are powerful and a welcome relief amongst homogeneous competitors.
Netflix’s recommendation engine is a good example of a hybrid system. The end goal is to have a score for each piece of content for each user, but how do we understand which content a user wants? We start with the content a user views and then identify similar content through the application of natural language processing (NLP) on the show descriptions and genre. Netlflix then combines this with collaborative filtering techniques such as singular value decomposition (SVD) or probabilistic matrix factorization (PMF) in which the preferences of other users becomes a proxy for a given user’s preferences, thus giving us another set of scores. Netflix’s own knowledge of what’s currently popular augments these algorithms with a simple score based on total views of a show in the last seven days. We can apply this combination of scores evenly or weight them through experimentation.
Recommendation algorithms are now pervasive in all of our technologies. Whether or not we admit it, these systems influence our desires and consumer tendencies. Even so, recommendations aren’t the responsibility of the program, but the programmer. Creators must think more about the user experience and what the user really wants. Those who implement these systems must think more criticlly about how their products impact users’ lives.
There are certain steps engineers and designers can take towards creating better systems. For the most part, technologists can focus more on user-centric design — one that gives the best experience for the user. But how?
Brokering a Peace Deal
Firstly, recommendations need explanation. For example “You’ll love this new song because you listened to Taylor Swift’s album on repeat last week” (ahem). With this information, users can better understand behavioral traps they may fall into, and make decisions about what they want recommended to them (which may or may not be more T. Swift...).
Booking does a good job at this, providing users reasons for recommending hotels, such as “great for two travelers.” It’s simple, but effective.
I’ll keep hoping that, on the horizon, an army of engineers will appear in all their nerdy glory to give these algorithms the power to recommend something I’ll actually love.
Next, build recommendations based on more complex criteria. For example, books aren’t just a few tags; they’re written in a certain style, voice, tone, structure...the list goes on. They evoke a particular mood for the reader. As such, recommendations should be based on these various components and then the user can choose what they find most important (“I’m interested in x,” or “I’m not interested in y”).
This may sound complicated, but we already have the infrastructure. With the right user interface, such as a simple sliding scale representing the various components, users can even fine-tune their own recommendations. Giving the power to the user lets them do what they do best: find stuff they love. This data can then be fed back into algorithms to improve the system further.
Until companies improve their algorithms and start giving more thought to long-term user needs (over immediate gratification), I must endure. I’ll keep pushing my counterattacks through the click of an x or a not-so-subtle thumbs down. I’ll keep hoping that, on the horizon, an army of engineers will appear in all their nerdy glory to give these algorithms the power to recommend something I’ll actually love.
