Eppo News
Measure Your Marketing Spend with Eppo Geolift
Use gold standard incrementality testing to evaluate the true contribution of marketing to your company's bottom line
Learn more
Joining The "Experimentation Avengers": Eppo's Customer Data Scientist Team
A day in the life of an Eppo Customer Data Scientist
“What on earth is a Customer Data Scientist?” I pondered when I stumbled upon the LinkedIn posting that would eventually lead me to my next career move.
Despite my unfamiliarity with the title, I was interested in joining Eppo. I had been following the company for a while—consistently impressed with the keen insights and statistical rigor of its technical content and appreciative of CEO Che Sharma's clear knowledge and explanation of challenges in the field in his appearance on The Data Scientist Show.
So, I applied. After all, I was already a data scientist; adding the word “customer” couldn’t be that different, right?
As I went through the interview process, I learned more about the role. At a high level, customer data scientists work closely with existing and potential customers to empower them to use Eppo effectively. We’re the “white glove service” team that makes Eppo stand out as a trusted strategic partner, not just another generic SaaS provider. When I realized the caliber of experts already on the CDS team, I started to feel like I was joining the experimentation version of the Avengers.
A big part of the role is teaching. This ranges from giving demos on how to use Eppo to giving statistical recommendations on how to analyze an experiment. This part of the job appealed to me because I greatly enjoyed being a teaching assistant in graduate school. I love the challenge of explaining technical concepts in a way that empowers others to succeed–whether that’s helping a student with a homework problem or helping a company with a product decision.
The role also has a hands-on component. CDSs often write SQL queries to help customers understand the data pipeline. They also work on Python projects that automate elements of the onboarding process or demonstrate key statistical concepts.
The multifaceted nature of the CDS role appealed to me, so I decided to accept an offer. I have now been a CDS for over two months, and it has been an incredible experience so far. In this post, I’ll discuss the key aspects of the day-to-day responsibilities of this role.
Explaining some (clearly hilarious) concepts in Bayesian statistics to the editor of this blog
Enabling Customer Success
One of my favorite parts of being a CDS at Eppo is helping customers run trustworthy experiments at scale. As the gold standard for uncovering scientific truth, controlled randomized experiments are the single best way for organizations to quantify the effectiveness of new ideas in terms of key business metrics. An experimentation program that reliably discerns successful ideas from unsuccessful ones undoubtedly yields a significant ROI and democratizes idea generation. This is Eppo’s core value proposition.
Despite the undeniable business value of trustworthy experiments, the road to success is full of traps and pitfalls that often turn an effort to follow the scientific method into statistical theater. Examples include data quality problems, sample ratio mismatch, and p-hacking. Although Eppo’s software is robust against these issues, there are many cases in which attentive support from an experimentation expert makes a huge difference. This is where CDSs come in–they work on the front lines with companies using Eppo and provide hands-on support, ranging from assisting data investigations to providing statistical guidance.
As CDSs at Eppo, we pride ourselves on a detail-oriented white-glove approach to helping customers. If a customer notes that an experiment's results are unexpected, we partner with the customer’s data teams and provide hand-crafted SQL queries to accelerate investigations. If a customer has questions about the statistical methodology, we make ourselves available to meet with them to answer their questions thoroughly. Our detailed and transparent support ultimately fosters a sense of trust in the product and the numbers it reports.
Interacting with multiple data teams
Most data science roles involve working in a specific industry with a particular tech stack. At Eppo, I work with multiple data teams across different industries every day.
Not only has this given me the opportunity to work with many talented data scientists and engineers, but it has also exposed me to a much wider range of data science problems than in any of my previous roles. While my background is in the gaming industry, where I’ve tackled tasks like implementing CUPED from scratch for a smaller in-house platform, my teammate Bertil has run experiments on massive experimentation platforms at Meta and Booking.com. Lukas (who leads the CDS team) built an experimentation tool from scratch at Angi, while Heather (our Solutions Engineer) worked at Optimizely and Twilio Segment for years.
The metrics relevant to a gaming company are quite different from those applicable to, say, a delivery service. Some customers use frequentist statistical methodologies, while others use Bayesian. In some industries, it is ideal to randomize experiments at the user level, while in others, it is better to randomize at a coarser grain. CDSs at Eppo gain hands-on experience with many different flavors of experimentation.
Interacting with multiple data teams is also a great way to stay current on trends in the data world. Customers and prospects often ask about how Eppo integrates with various tools, and truthfully, this was the first I had heard of a few of them because I never encountered them in previous roles. Now that I work as a CDS at Eppo (and can also lean on our incredible collective expertise), I have a better sense of the different tech stacks that companies use and am familiar with a wider range of tools.
A Culture of Continuous Learning
Joining Eppo has been a game-changer for my growth as a data scientist and statistician. As a CDS at Eppo, I am always learning about new data science concepts and technical papers.
Part of this is because of the nature of Eppo’s product. In most companies, data science is a means to an end; data scientists often build models or perform analyses to improve a product. At Eppo, data science largely is the product. A major value proposition of our software is that it enables our customers to leverage the best statistical methods for analyzing experiments. As a result, we make it a priority to stay on top of new developments in statistics and causal inference research. With the advent of larger datasets and more computing power, this has been quite an exciting area of research in the last decade.
One of the best examples of Eppo’s culture of continuous learning is the weekly Eppo Statistics Reading Group discussion. This is an optional meeting in which one of us chooses a technical article to discuss with the team. These discussions are quite casual and welcoming; we have an unofficial rule that nobody can spend more than an hour preparing, which generally results in a laid-back conversation in which we are all learning something new together. I am blown away by how enlightening these discussions have been. If you’d like to get a sense of the topics we have covered, check out Sven Schmit’s reading group summaries on LinkedIn.
I have also been grateful to learn from the experiences of my colleagues at Eppo. Many of the data scientists and engineers worked on internal experimentation platforms across a diverse range of companies before joining Eppo. As a result, the team is quite passionate about the product, as we have first-hand knowledge of the pain points it alleviates. It has been fascinating to hear how my colleagues approached ubiquitous experimentation problems in previous roles as well as domain-specific challenges.
Another aspect of the CDS role that has accelerated my learning is the quality of questions we receive from our customers. Many of the teams we work with include incredibly sharp data scientists who ask about advanced topics such as the tradeoffs associated with using sequential tests, the optimal way to spend alpha across a set of metrics, and how to choose an appropriate Bayesian prior.
Ability to make an impact
As a relatively young company, Eppo is growing rapidly. CDSs have plenty of opportunities to make an impact, given their combination of skills in customer interactions, statistics, and analytics engineering.
It’s satisfying to meet with customers, understand their needs, and propose technical solutions that help shape the product roadmap. In my short time at Eppo, I have already had the opportunity to partner with the engineering team to design the implementation of a new feature. One of my favorite parts of working as a CDS at Eppo is the freedom. When not meeting with customers, I’ve been able to work on projects that align with my interests and add value to the product.
The Eppo team on stage at the iO Theater in Chicago
Flexibility and Work-Life Balance
One of the best parts of working at Eppo is that it is fully remote. I love working remotely because it is so flexible. The lack of a commute adds so much time to my day and saves me from the stress of rush hour traffic. It’s nice to be able to fit in a workout during the day, move over my laundry between meetings, and spend time with my pets. Working remotely also comes with geographic flexibility. As a CDS at Eppo, I can work from anywhere. That means I can live where I want within the US without having to relocate to a city with a high cost of living.
Although working remotely is amazing, you don’t have as much opportunity to socialize in person with co-workers. Fortunately, Eppo makes up for this by having regular team meetups about once a quarter, which are a ton of fun (I just came back from my first meetup in Chicago). A favorite saying among Eppo employees is, take your work seriously, but don’t take yourself too seriously. Our Slack conversations capture this perfectly, as checking my messages typically amounts to sifting through a mixture of statistics insights, silly banter, and hilarious memes. All of this leads to a close-knit team culture despite the physical distance.
Eppo’s culture also encourages a healthy work-life balance. I’ll admit that when I first saw that Eppo has “unlimited PTO,” I had concerns about what that would mean in practice. From what I’ve seen in my two months at Eppo, taking PTO is completely normalized; in fact, our onboarding docs suggest that we take five weeks of PTO per year. I have already taken multiple days off for an out-of-state bachelor party. We also recognize 13 holidays per year in addition to a holiday break the last week of December.
Conclusions
Overall, my journey at Eppo has been a blast so far. A typical day involves answering questions from our savvy customers, nerding out about statistics with my coworkers, and laughing at memes in our Slack channels.
I believe that any experienced data scientist who is passionate about statistics, experimentation, and teaching will find the CDS role a perfect fit. It offers the rare opportunity to learn from a wide array of companies simultaneously and work at the forefront of developments in applied causal inference.
Table of contents
Ready to go from knowledge to action?
Talk to our team of experts and see why companies like Twitch, DraftKings, and Perplexity use Eppo to power experimentation for every team.
Get a demo
