The most crucial mind shift in a data role? Focus on impact

Data teams often focus on output when the critical thing to optimise is impact. Here are some tips on how to have impact no matter what data role you're in

At Monzo, where I work, it's calibration season: this is where pay and promotions are decided. Before anyone in the data team writes their review, we share three simple but often overlooked steps for every person:

1. Focus on impact

2. Instead of saying “delivered X”

3. Use “delivered X, and it had Y impact”

The difference between “delivered X” and “delivered X and it had Y impact” might seem subtle, but it’s not. 

The thing is, too often, data teams focus on output, but this is a mistake. 

The key thing to optimise is impact.

For instance, you might have discovered a small (but overlooked) change that could improve the signup conversion rate by 1%. This could mean that 100,000 new customers sign up each month. This could be much more valuable than building an entirely new machine learning system. 

Why it can be hard to have impact in a data team

If you’re in a data team, I bet you often find yourself stretched. Maybe you’re putting out fires, having to pull ad-hoc queries for no good reason or using data as an afterthought for a senior executive who has already made up their mind. 

There are four traits in organisations that often distract data people in unproductive ways: 

1. Data people are too accessible to other people. Unlike engineers who are often protected by a product manager and an established way of working, work comes from all sides to data people. Prioritisation can be difficult, especially when dealing with senior stakeholders

2. Everyone has an opinion about data. Data is at a crossroads where it’s easy enough to understand that everyone has an opinion. This means that we often find ourselves caught up in endless discussions that go nowhere and have to spend too much time redoing analysis

3. Too many people are overly dependent on data people to do their job. It’s not uncommon for a sales executive to ask for data to “deep dive into what’s going on in the business”. Sometimes that’s okay, but understanding your business area shouldn’t be outsourced to the data team

4. Data people too often go for the new and shiny thing. This one is on us. Data people tend to focus on technically complex solutions and forget about business impact, and often don’t navigate through small experiments

Seth Rosen @sethrosen

Them: Can you just quickly pull this data for me? Me: Sure, let me just: SELECT * FROM some_ideal_clean_and_pristine.table_that_you_think_exists

To avoid these traps, you have to think hard about where you have the most impact and spend your time there. 

How (and how not) to have impact

If you’re having impact in your data role, you probably spend most of your time:

• Making tools better, faster and available for everyone to use. For example, by making your dashboarding tool faster

• Making data available. Build high-quality dashboards and data models that make it easier for people to self serve the data they need

• Generating insights based on hypotheses and experiments from data and follow-up on if they’re acted upon

• Directly driving a top KPI such as revenue through machine learning models applied thoughtfully

• Doing data pulls for the right reason. Sometimes there’s no way around it, and spending time helping other people get the data they need is worth it

If you’re not having an impact, perhaps you spend a lot of your time:

• Always being available to do ad-hoc requests for whoever speaks the loudest. Instead, learn to say no and learn to prioritise early on (pro-tip: get your stakeholders to prioritise for you)

• Forgetting that ‘simple and done’ is often better than ‘overly complex and halfway there’. If you build a machine learning model, ask yourself first how a simple rule-based approach would perform

• Only doing data work and spending 95% of your time heads down in SQL. Instead, lift your head up and also pay attention to what’s happening outside of the data team

I’d recommend once in a while trying to log where you spend your time over a few weeks. If you find yourself spending the majority of time on the latter, stop and think.

How to focus on impact in different data roles

To use a cheesy analogy, if you’re in data, you’re an Avenger. No matter which one you are, you have your own superpowers.

If you’re further to the left, you work on tooling and systems to make data better for everyone. If you’re further to the right, the more likely it is that your work directly impacts a top-line KPI.

All roles have an important part to play. Here’s how an impact-driven data person thinks:

Data Engineer

You live in a world somewhere between the data and engineering teams and are probably more comfortable than most with writing production code. You’re an unsung hero, and if you do your job well, many people may not even notice you’re there (and that’s a good thing). You work on building and maintaining the data platform, which includes how data comes in, gets processed, and becomes available for use.

Impact-driven work:

• Streamline how data goes from production systems into the data warehouse, so other data people don’t even have to think about it

• Building faster and safer ways for people to find, access, and work with data. Customer rights and data security should be respected by default

• Building a framework for how to measure data quality across the company

Non-impact work:

• Blindly optimising and forgetting the use cases and not keeping the end-user in mind

• Inventing a data orchestration framework from scratch without looking at best practices. 

• Not listening to user issues and feedback and forging ahead

Analytics Engineer

You create value by making data available to other data consumers in an easy, reliable and scalable manner. You’re one of the stewards of the data warehouse and probably understand the data relationships and caveats better than most. People come to you when the spaghetti needs to be untangled. You’re like Mr Wolf from Pulp Fiction. Someone you can throw into any situation well knowing that they’ll get it sorted

Impact-driven work:

• Building data models that are high quality using best practice engineering standards such as testing and version controlling

• Making sure data is performant and ready on time

• Regularly knowledge sharing with analysts, data scientists and machine learning scientists to give them comfort in how to use the data to investigate or explore ideas

• Create onboarding material that helps new data joiners more easily get up to speed with data models in your area

Non-impact work:

• Over optimising the performance of data models if the business use case is unclear

• Working in a silo and not regularly catching up with your end-users

• Not documenting your work thoroughly

Data Analyst

You’re the child Sherlock Holmes and Marie Kondo never had; You’re curious and investigate trends by diving deeper into data than most, and you bring assurance to the business through well thought out dashboards or reports. You clearly understand the business, the most important priorities and metrics, and how you fit in. People often come to your desk (virtual or not) to help them understand what’s happening in their area, and dozens of people use dashboards you’ve created every day.

Impact-driven work:

• Investigating what caused a sudden spike in churn and coming up with a clear hypothesis for what could be done about it

• Build and maintain well-thought-out dashboards that most people in the business use that you religiously look at each Monday morning. You watch these dashboards like a hawk if anything is broken

• Having beers with a person from the sales team, sharing some insights you’ve seen in the data and getting their take from having been on the phone with customers during the week 

• Help a sales executive get some data through a “quick data pull” for an important client presentation later that day. From talking to your sales friend over beers, you know this client will be one of your company’s largest

Non-impact work:

• Getting sucked up in ad-hoc requests spending most of your time satisfying a stakeholder who’s working on a not-so-important passion project of theirs

• Doing a logistic regression to predict which of 100 survey responses in a spreadsheet are negative sentiment (you should probably just do it manually)

• Sizing the opportunity in the Georgian market for the fifth time because the go-to-market lead can’t make up their mind if it’s a good idea

Data Scientist

You’re the proud owner of the job title that’s been crowned as the sexiest job title in the 21st century (no pressure)! You bring statistical knowledge and rigour to data to help make sense of it in places where it’s not always obvious. You help product teams evaluate their recent launches through A/B experiments. You find the best order to call people in for a lead list. You investigate what’s caused NPS to drop by 5 points in the last quarter.

Impact-driven work:

• Building a metric that can be used for running A/B tests and telling the product team that an experiment in the signup flow actually improved signups by 5% instead of 1%

• Having coffee with the user researcher to discuss findings she made watching user tests and how you can translate this into an experiment

• Building high-quality dashboards for your team (as an analyst would)

Non-impact work:

• Obsessing over the scientist’s part of your role and thinking that dashboards or simple analysis are beneath you 

• You spend weeks running (insert name of advanced statistical test) to understand the correlation between sales and NPS in a way nobody understands 

• Taking data quality for granted and thinking that it’s not also your job to contribute

Machine Learning Scientist 

You’re a razor knife. You’re more technical than most, and you spot opportunities to train machines by feeding them data at every turn. You’re frequently offered modelling jobs. You know when machine learning is the right tool, and more often than not, you argue for starting with a simpler rule-based approach. You’re close to the business and the product teams and really understand your domain. If you’re doing well, you probably have a few models (with your name to them) that are generating meaningful value directly to a top-line KPI. 

Impact-driven work:

• Throwing yourself into a business area and in a two-week sprint building ten different models to solve business problems to get an indication of which leans themselves best for machine learning

• Learning basic Go so you can deploy code directly in the backend without being dependent on backend engineers

• Joining product team’s daily standup to stay in the loop of what their key problems are and make friends with engineers to understand how data is emitted

• Spend a lot of time with the rest of the data team; those clean data models from the analytics engineer are wonderful. The insights from the rule-based approach that the data scientists made also prove very handy for indicating where the next machine learning opportunity is

Non-impact work:

• Creating a detailed Jupyter Notebook that only you can reproduce (if the rest of your data team uses SQL, you probably should do the same)

• Improving the accuracy of your model by 0.01% by spending weeks tuning the last parameters in your XGBoost model (if you’re working on Google’s Ad Rank algorithm I’ll make an exception)

• Learning about Reinforcement Learning and telling everyone else about it in your lunch breaks

I’ll leave with a few parting tips that apply to all data roles.

Try not to be too busy. You’re a creative worker, and a lot of your best ideas come from you having the time to explore them.

If you’re in an environment where you’re held back by tools, consider whether you’re in the right place. Aim to be in an environment where any data person can have an idea on the way to work and explore it end-to-end by midday (hint hint).

Understand that no matter which Avenger you are, you have your own unique superpowers. As a data analyst, you don’t have to aspire to become a data scientist - the value you bring through insights and high-quality dashboards is massive. As a data scientist, you don’t have to aspire to be a machine learning engineer - the knowledge you bring in statistics is often just as valuable. 

Make friends with other people outside of data. They’re often your best source of inspiration. Whether in sales, customer support, research or somewhere else, go hang out where these people are. Go to their meetings, be in their Slack channel or have a beer or a coffee with them. 

You’re lucky. It’s an excellent time to be in this field. The modern data stack is on the rise (although still messy at the core), but we’ve only seen the tip of the iceberg. Most companies have realised the importance of data already, and as more companies adopt a modern approach to data, including fully-fledged analytics engineering teams and invest heavily in data quality, this means that all other roles will be more fun too.

What a time to be in data.