The Math of Burnout: Why Your Data Team's Capacity Never Recovers

Intro

The math of burnout

0:04 Thank you for having me today and I’m really excited to be here. This presentation is an interesting experiment that we’re going to have today. I didn’t bring any technical solutions but I did VIP coding. So if it’s going to work, we’re going to have success. If not, I have enough data engineers in the room to jump in and support us. I hope. Right. We have data engineers here. Good. So the math of burnout in my entire career as a data engineer then an analyst and a product one thing that I always saw when I worked with team was that we arrive into this moment that we asking ourselves where we went to what have we done what have we finished and looking at the Jira was amazing. You could see tickets. They got open. They got closed. We picked up some stuff from the sprint. Things moved around. And then one day I met a team that I was their product manager that kept losing their time and every week we had a plan and they arrived to it and they couldn’t finish it. And at Q1 we have decided that we have a big data strategy for this year and we need to actually change the way that we’re doing things regarding tracking. huge project. We were thinking about it. It’s going to take us between a queue and two to finish. And we gave this estimation for the data strategy. Everybody wrote it down. Everybody were happy and we started to execute it basically on the first week of Q1 together with OKRs that we were committing to because that’s what you need to do when you’re working in the tech industry. The problem started in the second or third sprint because we realized that we’re not reaching to actually what we have committed to finish by Q1. So we can move to the execution on Q2. Meaning a lot of our stakeholders is going to get stuck. And what I did was what every good product manager does usually. I called the agile coach and I screamed, I need help. Something is sinking and the project not going to be finished on time. And they arrived and they started

Planned week vs real week

2:03 asking questions. And what we actually discovered was that while our plan of the week was very clean as you can see on the top, the reality was a little bit different. The reality was that we somebody or one of the engineers started working on what we needed or what we committed and planned. Then there was a support ticket that he needed to go to. Then there was another meeting. there was another incident and we kept basically losing the week that we planned of what we’re going to achieve as tickets into a lot of micro h reactive actions that were actually disturbing us and everything was urgent always there was nothing that we could have done differently and the agile support the agile coach together with us decided that you know what we have a great idea let’s make a support ticket system everybody wants support from the data team will submit a support ticket. Anybody can guess how it ended up a complete disaster because people started to freak out. What? Now you’re asking us to fill up tickets for getting our support? Our dashboard is not working. And this is when things started to crack because we realized that we cannot actually make everybody happy while doing support on the one side and the other side. we need to do proactive work that we have committed to and we started to think about solutions and we didn’t have any and only years later what I realized was that actually what happened to us was what I’m call the nonlinear approach of data we’re starting and building up a new pipeline a new dashboard a new a metric that everybody’s starting to use and then it means that we’re getting more and more maintenance that we need to do so we cannot to be proactive while we need to maintain things that still existing. And it’s starting with four pipelines growing to 40, growing to 1,000 even in some cases. I’ve seen that also happening, god forbid. And

Every launch adds upkeep

4:04 each new product that we’re actually adding onto our ecosystem is dragging us down because it’s blocking us from actually reaching what we want to actually achieve, which is the proactive work that we have committed for Q1 or Q2 or for the entire year. And this is when I realized that something needs to change. We cannot actually continue in an approach that we keep accumulating products without actually asking what we keep. Because the cost of building a dashboard is relatively low, let’s say, because it’s not the same as building a feature on a website where it’s exposed to users. It’s internal. We just build a dashboard, we release it, we forget about it. We do the same thing with a pipeline. We committing, we ship. But actually all of them adding more and more cognitive load to whatever we’re working on. And then I started to think about what we actually being measured based on. We are being measured based on tickets we’re closing on our Jira. So everybody happy and see that tickets got closed and we are being measured by the dashboards that we have shipped this year and everybody also happy with that because if you shipped dashboards and people can use it, it’s great. But actually the question was what is the value of these dashboards? What is the value of the tickets that we close? Is

Value is not a ticket closed

5:17 it helping in decision changing? Is it helping in cost optimization? Is it helping us in growth? And this is a question that we never ask ourselves whenever we creating the ticket. And this is a lot of the problems that we are facing when we’re talking about data products. We just execute it because somebody asked us to do it because they see a value. But there was never a validation of what it’s going to save or what it’s going to create as a growth. And then we can make a decision that is not based on gut feeling that it’s going to change something because marketing now needs more data. And the entire process that have I I’ve built in the last few years is actually how we changing our activities during the retro from talking about feelings or ticket close to actually how we changed and created the impact in the organization. Because one of the things that still hunting me until today is this lottery that every morning I used to wake up and it’s like okay what’s going to happen today? Is it going to be an incident? It’s going to be a great support ticket that I need to solve or is it just that I will be able to actually work on what I committed that I’m going to deliver today. And a lot of

Live audience exercise

6:25 these parts have connected into a system that I am using today and I will invite you to join. So there is a QR code here. This is the part that maybe going to fail, maybe not. You can pull out your phones, feel free, scan it, and it’s going to ask you a very simple question and reflect on it based on your past week. So, how many hours did you have planned work? How many hours you were basically doing meetings, incidents, support, and uh ad hoc. And we see the result. Everybody scanned. We’re waiting now for the live results. Anybody else needs the QR code still? Try really to reflect about the hours that you had for the past week and how it went. Okay, we’re warming up meaning some people submitted their answers.

8:16 Okay, 30 seconds. Now happen the magic at the back. We have a lot of processing and fixing. Okay. So interesting from the people who fill up the survey 42% are data engineers which is already a good start. So it means that the data going to work if something breaks. We have 21 answers in total. And here are the results which are very interesting. In this room right now what we’re saying is 45% of our day out of 40 hours is uh basically reactive work meaning meetings meaning ad hoc. I hope all of you saved the results. I forgot to say it so you can compare it to the rest of the room and it’s about 18 hours of the 40 hours a week. This is a very typical thing that I see when I’m working with teams. It’s between 45 to 65% is reactive work which means that the rest of the time is just what left for you for doing proactive work. is a very big difference than what we all think about that we have 80% reactive sorry 80% proactive and the 20% is just the reactive with the small maintenance.

9:49 Now this is an average. If you’re going

How much work was unplanned

9:51 to do it basically in a a a smaller teams, you’re going to see different numbers which going to be representing better how you’re actually acting with the day-to-day. And if we looking at the breakdown of it. So there is nobody that is in the 0 to 20 which means building what I am trying to create in here is basically the structure of how all of you are spread. So each dot is somebody who submitted the survey 0 to 20% is very good which means it’s not a new team where you’re working at you actually have already products and you’re already maintaining them. A lot of you are in the sustainable area. So 13 13 people answer that and seven in the fragile what it means the fragile area it means that basically right now what you’re having is a situation where you’re having more and more support ticket than what you’re doing as a proactive which means you’re running more and more on reactive job and it means that the products that you are owning are more than what you actually can maintain because it means that you spending more and more time on maintaining things and fixing them and reacting to things that happening in the So maybe people who got sustainable want to raise their hand and the fragile and fragile.

11:22 Think about basically what have brought you to this situation. Which products actually dragging you down? So what we basically looking here now is

Where unplanned time goes

11:34 how you spending your time in an average in the room and you can see that the biggest part is meetings then it’s support and the rest is the rest of the disturbances that you had during uh the conversation and this is again a very big pattern that I keep seeing we are being in more and more meetings because we need to get more and more context specifically now it’s becoming even more and more critical with AI that we investing more time in understanding what the stakeholders needs and how we actually fixing things and how we moving forward. In the support part, it’s maintaining basically the product existing and trying to figure out how we’re running forward with them and making sure that the users can maintain So what can you take out of this? One person in this room right now is a beginning. It’s a data point that you can refer to and look at when you’re going to take it tomorrow for your team. Example, you can already start seeing the pattern within the team and see how different people in the team are working. How many of them are doing more the reactive and how many doing more the proactive. The question that you need to ask yourself when you see it, it’s not anymore. So when you’re doing it in the retro, it’s not longer, yeah, we feeling that we are busy or this is what we see now. Yeah, we had a lot of tickets. What you need to ask yourself is

One week is a data point

12:54 what actually causing us and holding us back from reaching the more proactive shares. What can we actually remove or what we can redirect to actually enable us as a team to move forward and then in a month from now what you will be able to say is what have we changed actually based on these numbers and how the number have moved. Are we more in the proactive part or are we more in the reactive part? And this is basically becoming like an index that you can actually discuss from retro to retro and also do a question of why it happened. Is no longer a feeling is no longer okay we were very busy this week and this why we didn’t manage to bring the sprint into delivery. It’s actually coming with the right areas. One of the things that I actually realized in many cases of the when I’m doing this survey with team is that there are a lot of tickets that being logged for big support tickets that we need to do but there is a lot of small and micro operations that we’re doing over the day. So somebody just came to our table ask us question. This already took us from the cognitive load of what we were building or where we were to actually answer somebody’s question. There is a lot of cases where basically we going and we investing in an incident right now which not all of us need to be there. We actually could have left one engineer and we all got involved because we thought that this is an acouic incident that we all need to be there. And the question that needs to come in the retro now is why we were all there. What actually changed or would have changed if we would have done it.

14:28 And then of course the question is how we moving forward creating the right structure that next month the situation going to be better. So we’re going to be less involved in certain things. So we don’t need all of us to be in meetings. We don’t need to be all of us inside an incident or all of us need to get involved in a support. We can structure it in a better way. And this way we avoiding also the feeling of people that okay now you need to document everything because you actually going to be documenting it in the retro whenever you have these questions and you’re going to be able to actually ask the why it happened that we arrived to this amount of proactive versus reactive and how it So before I am finishing I will leave you with three questions you can ask each other now and it’s basically why did it land that high or that low. So feel free to ask each other have a discussion about it. What happened that week? So try to reflect is it something that was when you filled it up now the questionnaire was it a one time is it something that is constantly happening and how you actually moving forward and is it an unusual thing or is it a usual thing that happens? So really try to reflect with each other maybe how the results are reflecting on each of you and there is with the late camera so we got a little bit more answers later on unplanned is 47 and the planned is 46%.

What Data Really Costs

15:56 There are some changes into the numbers and if you’re interested to learn a little bit more about it self-promotion I’m sorry I do marketing I have a market here [laughter] I see the difference you can find my book h this is the QR code you can order it on Amazon and I would love to share there a little bit more about uh this system and how it’s Thank you, Leo. Any question from the audience here? Nobody. The vibe coding did it did. I think it went well.

Q&A

16:45 But this this port I did the 50 pages in one go. It was you writing it. I I wish. So, anybody that also put uh Leah on the I have one. Oh, we are one. Thank you. Thank you for your talk. So, I have a question. Maybe it’s a if I would have listened to this talk 5 years ago or 3 years ago and now what do you think has changed with all of the new AI and all of the things that are now more the agents are going to do? So all of the things that you said in terms of problems has been something that yes all the data teams had these problems very much but do you think this is changing now with the new way that AI or is it going to get worse or how do you see it that’s a tough question so it’s I don’t think that it changed much because for us as data at the end of the day data still is a nonlinear it’s not going to change the only thing that changed are the tools that we are using. So if in the past it used to be I don’t know Postgrace and some AWS servers and now it’s snowflake and dbt the technology changed but the problem stays because it’s a cultural thing that we need to solve actually rather than a tools adding more tools into this conversation not going to solve it. So this is this is this perspective with I don’t think that it’s going to get any better because actually still the demand for data is getting higher and now the trust levels and we all feel it here in Berlin are very very low in the data that people consume. So they’re asking more questions. Why am I investing this amount of money in data and technology?

18:45 And we see a lot a lot changes in the tax stack based on this and also with the people. [clears throat] But the problem is still the same as I see it and it’s not going to change anytime soon if we not going to change our So wouldn’t the agents be the one who will solve those supports issues so people can work more on the proactive ones? depends on what [laughter] agent can solve a lot of stuff. Are they solving it correctly? I don’t think not yet. Maybe in the future yes but then it means also the team is going to be much smaller than what we see today. This is a different conversation that we need to And I think actually then our still focus as human is going to be on solving the specific problems that agent cannot access and don’t have the right depth to actually solve. Thank you. Any other question? So, what time is it? Should we ask some questions? Hard questions. Do you want to ask hard questions? I don’t know. I was uh more worried about you know I was more thinking about the burnout of uh you were talking about the team I was more about you know personal you know the story [laughter] and so on as individual but um there’s also burnout for teams I guess right it it always was there I was in a burnout of a team and we lost the team right at some point people just started to resign because they couldn’t take it anymore and then there is already somebody that is pointed as the toxic that caused all of it. Even if he wasn’t toxic, he just said the truth.

20:37 Anybody wants to share anything without microphone? Uh [laughter] I’m sorry. Thank you. Uh honestly, I just wanted to share that I’ve been in a situation like this. I was a part of the team and half of the team quit because we couldn’t take it anymore. And uh our man manager truly tried to do something about it. We tried to make uh uh process simpler for us and more complicated to stakeholders but nobody liked it and uh at the end it didn’t uh it wasn’t simpler for us too. So I just wanted to ask what could be done in this situation except for asking ourselves like what did we do wrong? Is there any point where we could get out of this burnout like how should we stop these people who keep asking questions or the problem is in the data team you need to read the book. No [laughter] okay okay so I can I can find it there right because it’s really it’s really important. I still feel the pain because it was my favorite team and unfortunately didn’t work out at the So I I had the same thing and this why I actually arrived to writing this set of systems that will try to actually reduce the risk of it. One of the things is really to communicate clearly and not by feelings where the team is and what’s holding them back. And I think this is something that we are missing quite often when we communicate to stakeholders. We’re talking a lot about the tech stack. We’re talking a lot about a commitment at Q4 for what we’re going to do during the next year and we’re not really reflecting what is disturbing us from moving to wherever we need to arrive to. And this is a lot of not adding more layers of security. This is what product managers do. This is what I did and I failed because I had this support ticket system and everybody hated me. I was the toxic at the end.

22:46 the team anyway resigned. But the idea was behind it to try to actually protect the team and it’s not going to work because you cannot just say, “Okay, now everybody needs to do tickets.” If it’s really 15 or half an hour or 1 hour from your time, log it down and write it down yourself later on. Create a ticket so you’ll be able to remember that you did it and reflect on it and then come and say based on this 60% of my time went on reactive work and this is the products that were dragging us down. So start mapping actually the real products and the real issues that you facing. So if there is a certain product that keep creating an incident on a weekly basis on a bi-weekly basis figure out how you stopping doing it. Instead of keep building and fighting on the proactive work focus on fixing what is right now breaking and try to fix it that it’s going to work in a better way. And I think this is where the team is changing from I’m an enabler and I’m just sitting here and you tell me what you need me to do to actually an an impact operator. We actually creating impact in operation by seeing that this tool is dragging incidents because incidents cost money and not only that it’s costing us money to fix things or the loss of data that we need to recmp compute or whatever that happens there. It’s also costing simple people stress and sleepless 10 girl by the book. [laughter] But I I guess the answer should be you have to buy the book because if you buy it’s better if you just read it, right?

24:17 Of course. And you can read it in one It’s [laughter] it’s more effective. You know this any other questions? Alberto, you want to ask a question? No. No, not really. [laughter] But you see, so I was just curious about like our vote or like the the this group vote and like what would be your evolation of like this uh distribution of the vote like regarding the meeting percent and active work and the tickets. So what would be like uh from your point of view a good goal of like this category division? So like if it takes now 40% of the meetings is it too much or too less and what will be like ideal to go. So there is no idea because each team working differently right I don’t know which teams are here if it’s a platform team in compared to data science team in compared to analytics team each of them have different purposes seeing high meetings this is a sickness that we all have in all over. So this is usually what you see very much occupying the the top three categories. Let’s say I see in many teams for example when it’s coming to analytics that incidents are usually the top uh when it’s coming to a support platform somehow is always in the top there and ad hoc is also a lot of analytics team in the top three that you see meetings always there because we cannot avoid meetings and actually I think that since we have these AI agents everybody wants to have more meeting now how do we actually automating stuff how we making things working much better than what they were before and that’s the reality. It’s a shift that happening right now and it’s going to continue into the future. But now you have agents so you can solve it faster.

26:09 Right. [laughter] All right. I see you. Thank you. Thanks for the presentation. Yeah.