How Netflix, Lyft, and Yahoo use Contextual Bandits for Personalization

Inspiration from some of the world's biggest tech orgs

Ryan Lucht

Before joining Eppo, Ryan spent 6 years in the experimentation space consulting for companies like Clorox, Braintree, Yami, and DoorDash.

Contextual bandit algorithms are a powerful machine learning tool, designed for making many automatic, optimal decisions. They balance the trade-off between exploration (trying out new options to gather more information) and exploitation (choosing the best-known option based on current information) in real-time as new information becomes available.

Unlike traditional bandit algorithms, contextual bandits take into account additional context or features and choose the right decision for each individual user - they personalize experiences. Common applications of contextual bandit algorithms include content recommendations, dynamic pricing, and other adaptive experimentation use cases.

If it’s hard to picture where contextual bandits are applied in practice, not to worry. Here are three famous — and very important — use cases from tech giants like Netflix, Lyft, and Yahoo.

How Netflix uses contextual bandits to personalize title artwork

Netflix uses a powerful recommendation system to surface the content on their platform that you’re most likely to engage with, but their personalization of your user experience doesn’t stop there. You may have noticed that any given TV show or movie on Netflix can have multiple different featured images as you’re browsing.

Netflix employs contextual bandit algorithms to personalize artwork for each user, enhancing their browsing experience by presenting the most engaging images. The platform dynamically selects different artwork variations for the same content, aiming to capture individual preferences and viewing habits. The goal is to increase the likelihood of a user engaging with a title.

__wf_reserved_inherit — From the Netflix Tech Blog: "Artwork for Stranger Things that each receive over 5% of impressions from our personalization algorithm. Different images cover a breadth of themes in the show to go beyond what any single image portrays."

Why does Netflix reach for a contextual bandit algorithm in this case instead of another recommendation system? For starters, it’s a much smaller action space: instead of choosing between tens of thousands of titles to recommend, there are only a modest number of artwork options to display for each title. A recommendation system would probably be overkill for the size of the task at hand.

In practice, the Netflix team saw a significant lift to their core metrics by implementing a contextual bandit for personalization. They wrote extensively about their motivation and experience implementing it on the Netflix Tech Blog - a highly recommended read for anyone interested in contextual bandits.

Using contextual bandits for dynamic pricing at Lyft

While Lyft uses standard multi-armed bandit algorithms in some marketing use cases on their website, they get far heavier mileage from contextual bandits, which power their ability to rapidly respond to changes in “environmental and market conditions”.

If you’ve ever seen the Wait and Save option when requesting a Lyft ride (opting for a longer pickup time in exchange for a discount), you’ve seen a contextual bandit algorithm at work. Lyft uses contextual bandits to dynamically set the pickup window using the context from data on local driver availability and ride demand, and optimize for the best combination of reliability on the pickup time and cost savings.

‎They also use contextual bandits to allocate marketing incentive budgets (discounts to riders, or additional pay for drivers) over longer timescales. Even inside their core pricing algorithms, bandits are used to dynamically tune certain parameters since they can adapt to changing conditions faster than other machine learning tools. (You can do this with Eppo’s Contextual Bandits too - use the outputs of existing AI/ML models as context for the bandit to optimize on. A great use case!)

Yahoo personalizes news suggestions with contextual bandits

Yahoo has also published several papers detailing how they effectively utilize contextual bandits to enhance the personalization of news article recommendations on the Yahoo! Front Page. The "Today Module," a highly prominent section of the Yahoo! Front Page, features a rotating selection of high-quality news articles curated by human editors. The primary goal of the Today Module is to present users with the most engaging articles based on their interests to maximize user satisfaction and click-through rates (CTR).

In this setup, contextual bandits are used to dynamically rank and highlight the most relevant articles for individual users. By leveraging contextual information like demographic information and past behavior, the bandit continuously learns and updates its strategy as new news articles are published (and as users repeatedly visit the site).

The real-time advantage contextual bandits have in balancing exploration and exploitation is crucial in a rapidly changing environment like news. Compared to other approaches like a recommendation system, bandits can quickly adapt to shifts in content popularity and user preferences. Yahoo reports that their usage of contextual bandits has shown substantial improvements in metrics.

Build your own contextual bandit

Contextual bandit algorithms are a powerful tool for personalization and adaptive decision-making. Whether it's personalizing artwork, dynamically pricing rides, or tailoring news suggestions, Netflix, Lyft, and Yahoo all found that these algorithms help drive engagement and customer satisfaction. With their ability to adapt quickly to changing conditions and learn from new data, expect to see contextual bandits play an ever-increasing role in personalization efforts across companies.

If you need an easy way to stand up bandit algorithms without weeks of work from an ML team, you’re in luck. Eppo’s Contextual Bandits will do the hard work for you - just supply actions and their contexts to the Eppo SDK, tell us about the business metric you want to optimize, and let us do the rest. To learn more, get a demo of Contextual Bandits now.