# SiriusXM's Data-Driven Platform Prioritization Framework

**Podcast:** Engineering Enablement by DX
**Published:** 2026-06-15

## Transcript

Welcome back to the Engineering Enablement Podcast.
I'm your host, Justin Riok.
This episode was recorded live at DX Annual, where we were joined by Eleanor Millman, Senior Staff Product Manager, and Mina Tawadras, Associate Director of Product Management at SiriusXM, where they've been building a data-driven prioritization framework for platform engineering in a large-scale internal developer platform organization.
Eleanor and Mina walked through how SiriusXM approached one of the hardest problems in platform engineering, deciding what to build, in what order, and how to align teams around shared priorities.
They share the prioritization model they developed across developer speed, reliability, security, cost, and platform efficiency, along with how they evolved that framework using developer data, assumption tracking, and AI-assisted workflows.
Thanks for listening.
Hello, I'm Eleanor.
This is Mina.
I guess I have an intro again here.
I led product for SiriusXM's platform engineering org for two years, and Mina is its current product lead.
We're excited to be here today to talk to you or talk you through a prioritization framework we developed, as well as how data can increase confidence in what you are building and how AI can make all of this easier.
You'll be getting a how-to guide on how to prioritize developer productivity projects, especially when you have few to know product managers, which of course can be common in internal product spaces.
So I will set the stage for our story.
A few years ago, SiriusXM created a 50-person platform engineering org to build a brand new AWS-based platform upon which a whole new tech stack and app rebrand would run.
We were and still are serving 700, 1,000 internal developers across the full SDLC.
We have broad language and runtime support, so Scala, Java, Python, TypeScript, a few others.
We have deep capabilities.
While we do support the typical back-end service owners, we also have some more unusual use cases.
Our Databricks-based data platform also runs on our...
Cloud Poplar.
Also, we require infrastructure as code for higher environments.
And many of the developers we still support at that time were inexperienced with owning infrastructure.
Some also were not that familiar with AWS.
And while many knew Terraform, a number had never used AWS CDK and CloudFormation, which is what we landed on for our IAC requirements.
Lastly, we had few to no mandates other than this infrastructure as code requirement, which meant that we had to support a broader range of tools.
Typical product problem.
We were struggling with this question, what should we build and in what order?
And one of the challenges of being a platform engineering team, as I suspect many of you know, is that oftentimes you have to make impactful product decisions without having the product staffing that would help you make the best decisions.
The team did a great job before I joined as the first and at that time only product manager, but they were finding that as the number of potential projects grew, their methods of prioritization were not scaling.
And I want to call out that platform engineering has prioritization challenges because it's a non-traditional product area.
For most externally facing products, you build a feature, you release it to users, and you can often measure the revenue a feature generates.
But platform is twice removed from revenue and end users, so impact is much more complicated than just revenue.
We knew we cared about development speed, runtime reliability, security and compliance, and cost reduction, but we're unsure of how to combine them.
And yet, When I looked around to see what others had done in the prior, fund for prioritization in the space, I didn't find anything at the time.
So we improvised.
We took an iterative approach to developing our prioritization framework.
We started with this formula, which is very loosely based on rice, a common prioritization formula.
So we defined impact as how much value a project brings to users, basically value per user times number of users.
We made urgency a function that blows up as a project's due date nears.
We try hard to avoid putting an urgency date, but sometimes you have true hard deadlines.
In our case, for example, that company-wide launch date I mentioned.
An effort is defined as developer months or developer weeks, whatever you'd like needed to deliver a project.
So this basic formula prioritized high impact or urgent items that took little effort to build.
Basically, low hanging fruit.
So we ran projects through that formula and it worked okay.
But we realized that impactful larger projects were being deprioritized too much.
So we used this math trick to dull the effect of larger efforts.
That change made the project prioritization ranking feel more correct, but we still were missing at least one element.
Sometimes a project is required because internal stakeholders demand it.
For example, security says you have to set up an audit trail or you won't be in compliance.
There's no arguing with that.
So we added this internal business need, which is really just a fudge factor, to take this into account.
Just like urgency, we start to use this variable as little as possible, but it was required to correctly prioritize a few projects.
And so this is the formula that we use today.
But let's dive more into impact because that's really where all the complexity is.
So as I mentioned earlier, we knew we cared about these four impact factors.
So development speed, how quickly developers can deliver features, bug fixes, et cetera, runtime reliability.
For us at SiriusXM, listeners are our end users because we're a media music company.
So we defined runtime reliability as giving these listeners a great experience through low latency, low error rates, and low downtime.
Security and compliance, just what you'd expect.
For example, implementing security scans in the CICD pipelines would have impact in this category.
And then cost reduction.
For us, we chose to define it as reducing infrastructure costs, which is mostly our AWS and Datadog bills.
As with the formula I just discussed, we tested this idea out by scoring the projects in our queue, and we realized we were missing a few impact factors.
First of all, platform engineering our efficiency.
So of course, when we become more efficient, that means that we can deliver value faster to our users.
So refactors, making our own code reviews more efficient fell into this category.
Another example, our software delivery team wanted to do some work to be able to make it easier for themselves to write automated tests for our shared GitHub action workflows.
They struggled to prioritize this project until we added this impact factor.
Next was user trust.
So as I mentioned earlier, we had few mandates, so we had to foster adoption of platform engineering offerings rather than require it of our users.
Having our users trust us makes that easier.
So any project that originates with a user feature request would get a small bump in this category.
Also, anything that prevented something we own from going down would also benefit.
So for example, we...
self-host our GitHub instance.
So if that went down, we would really lose user trust.
So projects that help keep our GitHub instance up and functioning well receive a few points in this area.
And lastly, we in platform engineering really wanted and want to be data-driven, but of course it takes significant engineering effort to be data-driven.
We use this category to prioritize projects around enabling telemetry in our offerings and building out our basic metrics platform.
So there we go.
We defined seven impact factors.
The next question was how to weight them.
So pre-launch of that app rebrand and new tech stack I mentioned, we knew that stakeholders cared most about development speed and then, of course, delivering a high-quality experience to those listeners, our end users.
So we started out with these weights, 30% for development speed, 20% for runtime reliability, and then the rest equally 10% each.
But post-launch, the company started focusing on cost reduction, especially as traffic was flowing through the new AWS-based platform.
The great thing about the framework is that it was easy to change the weights to reflect these changed priorities.
Of course, we increased the focus on cost.
You can see it went from 10% to 20%.
Also, we increased the weight of our platform engineering impact factor since we had been building up a bunch of tech debt as we were so focused on getting our users to that launch date and we wanted to work that down again.
Lastly, while we were making some progress in being data-driven, we looked around and were not anywhere where we wanted to be, so we also bumped up being data-driven a bit.
Now, if you raise some weight, you must reduce others.
And there was definitely a moment when it felt very uncomfortable because who wants to reduce any focus at all?
But you got to.
And so what really helped us to do this, to make those decisions, we looked at developer data.
So we're, of course, a DX customer.
And so the DX quarterly survey was very useful as we made these decisions.
For example, we saw that developers reported the systems they maintained were high quality and had few incidents.
So we felt comfortable reducing the focus on runtime reliability.
Not that we don't care about it, but for future projects, we didn't feel like we had to invest in it the same way.
as we had before.
Likewise, data showed us that we were doing actually quite well on security and compliance, so we are also able to reduce the focus on that.
And now, every quarter since then, we take a look at these impact factor weights after we define our platform engineering OKRs, and we tweak them as needed to get alignment.
OK, finally, let's talk about scoring specific projects.
So we use relative scoring for the impact factors.
As you can see in the legend at the bottom, We give a zero for no impact, one for low impact, two for medium, three high, and five if a project has massive impact in a particular impact factor category.
We estimate these impact factor values using user data where possible.
We particularly like using the DX studies feature.
which is basically targeted surveys.
We find that a really fast and easy way to go get this data, but you can just go talk to users.
You can send out your own surveys, whatever you need to do to get a little more of a sense of how a project may affect a certain user group.
Just to talk through one example, let's say Project E.
So you can see that it has a one for speed.
So that means that it has some but low impact on development speed.
It has a three for reliability.
So it has a high impact on that end user latency, error rate, et cetera.
Zero for our own efficiency.
So must not affect it there.
A little bit of impact, low impact on security.
Five, so massive impact on cost.
And two, so definitely somewhat helps with user trust and must not have been helping us be data-driven.
The very last thing that we added in this first big iteration was some lightweight documentation on the prioritization scoring of each project so that we could explain to the Platform Engineering org and any stakeholders why we scored a project a certain way.
So we just wrote them down very quickly in blurbs, put them in a column in our project management software.
And let's talk quickly through an example.
So this was pre-launch.
We had very little cost attribution, and that did not feel good.
So we really realized we had to have a project around AWS cost attribution.
It did have an urgency date.
We knew we absolutely had to be able to explain the surge in cost post-launch.
So we put the date in there.
We gave it a one, so low impact for development speed, because if we didn't make this, there was some cost attribution, but not very much.
Without a more granular cost attribution, we knew that engineering teams would have to spend quite a lot of time explaining cost spikes to users, or sorry, pardon me, to finance.
We felt like we could save developers a bit of time by doing better beforehand.
We thought there was no impact to runtime reliability and security and compliance, so they got zeros.
Cost reduction, we gave it a five because, of course, the first step in reducing costs is to be able to visualize those costs and understand where you're spending money.
We gave it a one for platform engineering efficiency.
We thought it would help us be a little bit more efficient because same thing, we're going to have to explain our costs just as much as the engineering teams will.
User trust.
We said, low, small impact.
We felt like a modern cloud platform, of course, should have cost attribution.
How can we look our users in the eye and ask them to trust us if we don't offer this?
So we acknowledge that we would build some user trust by building this.
And lastly, being data-driven, of course, being able to visualize costs and where money is going allows us to make data-driven decisions around cost.
So there you have it.
That was the first iteration.
And Mina is going to talk you through some additional changes we made.
So I came to Cloud a little later than Eleanor.
So I was able to look at this model and see where we can make some improvements and where we can make some iterations.
And as the teams grew, we kind of grew in size.
We had multiple teams.
So challenges arose that we thought we could kind of iterate on as well.
So what was probably the most difficult about this model was when we had these like cross-team priorities and there were conflicting assumptions between teams.
You have something that requires capacity across two teams as backlogs, and then they start scoring the work, and you get such dramatic different scores from team A versus team B.
And you really need that buy-in from all those teams to feel totally bought in on why we need to have to dedicate backlog capacity to this functionality.
So it got me thinking of this problem as a larger problem, and I kind of learned about an interesting study that I think demonstrates this pretty well.
There's a study in the 90s called the curse of knowledge.
The idea there is if you know something really well or you feel like you know something very well, you actually overestimate how much somebody else knows about that same topic.
So the study itself, what they would do is they would kind of split up two groups.
They'd call them tappers and listeners.
Tappers were told to just tap out a familiar tune, like happy birthday or something like that.
And they were asked to estimate.
what's the percent chance the listener group is going to get that melody right?
So tappers thought, all right, everyone knows the song.
It's in my head.
I'm tapping it out.
About half the listeners will probably guess the song.
It was actually close to 2.5%.
So we really overestimate our ability or somebody else's knowledge when we feel like we have knowledge in that space.
So this was kind of what was happening in platform engineering in these scoring kind of conversations.
You had a team that was like, I know the users.
I've talked to them for like two or three months now.
I know this space.
They would score it a certain way.
That other team has different exposure to that same user base.
They built their own assumptions, and they had a very hard time understanding how someone else couldn't see their worldview the same.
So let's go ahead and paint a concrete picture of this.
Here's an example project.
Let's say we were scoring something around improving our deployments, right?
We might have two teams working on this score, and we might say this is such a large initiative that we're going to need a little bit of work from you, DevX.
We're going to do a little bit work from our delivery team.
Let's go ahead and score it.
DevX might go ahead and say our dev speed impact, back to that rubric, is actually a five.
Super high impact.
Why?
Because the current deployment time is actually 20 minutes.
We've looked at DX survey data.
It's pretty close to 20 minutes.
And this is going to hit 700 of our development team today, let's say.
Then you have delivery on the other side saying, okay, dev speed, probably a medium impact because actually, if you look at our GitHub actions data here, It's closer to five minutes using actions data.
And actually, if you look really deeply into this problem, it probably only affects Java developers.
So both are actually right if you think about it, but they're actually coming at it from totally different assumptions.
And the result here is kind of like more of like a heated debate about whether or not we should do this work because you're arguing and debating about priorities versus the assumptions.
So going deeper into this, we're able to kind of like break down or think of assumptions and how they fail us in kind of three.
Three ways, let's say.
So one way is kind of like when you have two assumptions in conflict, right?
The example there is a pretty good one.
Somebody thinks deployment time is 20 minutes.
Somebody thinks it's five minutes.
There is two existing assumptions that are existing right now in conflict.
There's also assumptions can be invisible.
This could be a situation where, again, like think of all the floating assumptions going on in your businesses today.
They're not written down.
Or maybe what happens often is somebody builds these assumptions.
over time and they might leave the team and that assumption goes out the door with them and finally assumptions can be stale um there's you know situations where you might uh believe you know something because you did solid research on it you did some studies on it six months ago we know how quickly technology changes or user base changes and that assumption all of a sudden becomes stale and it's not as useful as it once was so we kind of noticed looking at these three different like assumption failures And we kind of like came up with a hypothesis, right?
What if we treated assumptions the same way we treated code?
And if we go back to those three assumptions, our three assumption failure options, you could actually see how code could kind of resolve all three of them.
And this is something we've been experimented with.
So conflict resolution, as an example, all of a sudden, if you think of like, you had a difference in code between two developers, there is actually a mechanism in code to go ahead and resolve that.
You submit a PR.
you could go ahead and have that PR reviewed.
And then you get to a point where the proper assumption is actually stored in code and you have a moment to tracking of the conversation that led us to that place.
If you think about assumptions being invisible, this is, again, the reason why things are great in code is it makes this more discoverable.
We store it now in a central repository.
Everyone has access to the repository.
The assumptions are much more visible.
They're version controlled.
We can know the data source.
We know who was the last person to validate it.
And data freshness, again, if you are bringing it in as code or bringing an assumption in as code, every assumption has a timestamp, an author, a history.
So you can actually see, well, that assumption was last committed like 12 months ago.
We should probably revisit that.
So what does this actually look like?
Here's an example of an assumptions document.
This is one iteration we had early on where you would literally, again, put your assumption in code.
Again, probably from the engineering world, like ADRs, like this is very similar, but kind of take a product lens to it.
And why not actually store and list your assumptions in code?
And we'll talk about the power that this kind of brings in an AI-enabled organization.
So some things that we kind of found super useful, the tricky thing here is you need this data to be flexible enough to store many different types of assumptions.
So it's hard to go ahead and format things very specifically in table format.
So keeping with loose format was helpful for us.
But kind of the key areas here is what is the assumption?
In human readable language, I don't think you have to over-engineer to try to think of like how a computer might read this and I'll explain why But also what is the data source?
Like a very critical one and also when was it last updated?
And you can also see here we've had experiments with a few other things like what's the confidence level of this assumption?
That we're still like going back and forth on so This was kind of interesting, like being able to move things into a code base and dealing things with PRs, making things more visible.
It was pretty cool.
So we were able to codify these assumptions and it shifted the debates, which was the goal, from debating about priorities to debating about assumptions.
And I think that was really helpful for us because priority debates, I'm sure many of people have been in these like what's more important today, they're emotional, they're positional.
It's like my project matters more, right?
People dig in and it becomes about team identity and political capital.
Assumption debates are very different.
They're often a lot more analytical.
The questions you might ask are, what do we believe about our users?
What data supports it?
When was this data last validated?
You're not arguing anymore about which project matters.
You're arguing about what you know today about your shared reality.
So when we made this shift, the temperature kind of dropped in these arguments.
People stopped defending kind of their roadmap or like, this is the most important thing.
And you start actually being able to talk and engage about the assumptions that said, and you kind of like built this base together of a shared reality.
So this created another problem.
Every solution creates more problems, which is great.
You can imagine, you're probably already thinking to yourself, okay, I'm going to store every assumption in code.
That's a big file.
And also, am I going to expect people to read this file every time they come up with a new assumption to see if one already exists?
What's actually going on here?
So it helped with the conversations, but these are the problems.
Users weren't always able to look up for existing related assumptions.
It's very tricky.
Unless you are trying to literally look up deployment lead time and you want to know what the existing assumption is on the organization, you do a keyword search for deployment lead time.
But a lot of the assumptions that we use in product, they're hard to correlates or hard to like know that something is even related to the initiative you're working on.
So it's harder to find the existing assumptions that already existed when you're talking about reading a flat document.
So, you know, it's been almost the whole talk and we have not used the two-letter word yet.
So how does AI fit in?
And we started using AI to help with this process.
So what we've started to do is we've been able to use AI to surface those assumptions when the connections aren't obvious.
So imagine you're scoring an IDE plug-in rollout and AI surfaces, in this case, like our AI will go ahead and read your assumptions repository and it might surface that, hey, did you know that your, you know, contractor headcount, which is 20% of your employee base, doesn't have access to that IDE.
Those are very difficult assumptions to actually tease out if you're trying to read through a whole document.
But with AI, it's become a lot easier to just go ahead and leverage the LLM to go ahead and read through that.
read through your initiative and try to come up with some connections to that so now what we're hoping for kind of the future of this is like people are writing epics product briefs or scoring initiatives we use ai to query the current assumptions repository and we surface that relevant data survey results and related past initiatives they've got us to think like what is our position here because like we we as humans we still like to make judgments right so Where we see the best fit for this model, which I'll kind of show in a bit, is we still use AI for recall.
So think of it as like the best way to recall information that is hard to access with our limited brain power.
But we use humans still for judgment, especially on like, hey, is this actually something we believe in?
Do we actually want to prioritize it?
We are still the judgment layer, but we make a lot better calls when we help use, when AI is used for getting that data out of the system.
So here's kind of like what this tool ended up looking like.
So it started off in like kind of a matrix form of like a scoring matrix that we saw earlier.
I'm going to show like some pseudo screenshots of what this actually looks like today.
So this is like our feedback loop, if you've heard about it.
So a product manager, an engineer, an architect will go ahead, enter a product idea into your favorite, you know, back and forth generative.
coding chatbot of choice that we cannot suggest what we use because we're a public trader company.
Then that same tool would go ahead and ask guidance questions, fill in the gaps about the initiative because it knows what it's trying to store.
It has a goal in mind.
I have to be able to collect this data to be able to make this decision more easier for my human.
It finds assumptions that already exist in the assumption library, but then we have a human loop over here where the human actually has to validate that assumption.
You have to read through it and you have to say, do I actually agree with this?
Is this too stale?
Is there something here that's wrong that I want to submit back to the repository for future?
If that is true, we update the assumption.
It brings back a score and then same thing.
Is this a score you actually believe in?
Do you actually believe in the priority of this work or not?
And then you kind of cycle and the assumptions library grows and grows and gets more powerful.
We've expanded to like a shorter version, like a rapid loop where it's literally just.
send the product idea, give me a score.
So we were kind of like right now have these two models floating out there and we're just checking usage on each and seeing what people prefer today.
So this is fake, but this is what our like chat tool, which is one way we've integrated this in, works.
So kind of same example here, but just now laid out.
Somebody might say, hey, I have a...
Skills marketplace that I want to go ahead and deploy to this IDP or internal developer platform.
I want it to be a self-service catalog where teams can discover plugins and integrations for their favorite tool.
AI assistant comes back and says, all right, first, I need to understand more about this project.
Because again, it has the context of how to score the work.
It asks guidance questions.
This has been pretty valuable to get people to really think, why are we building this?
And you can go ahead and just...
Users would go ahead and just give it the extra information on that initiative if it wasn't able to pull it from a PRD or another brief-like document.
The AI system would come back and actually list out the assumptions for you that it was able to grab.
And kind of the interesting thing here is we list out the line of code that it's coming from.
So, okay, I found five assumptions that apply.
We have, this is all fake data, so don't worry about, but like, you know, active engineers, and it's coming from this platform and it's line 102.
We also go ahead and try to figure out, does this match any OKRs this quarter?
So OKRs are stored in code as well.
And it will tell you what percent alignment.
This contributes to a KR strongly.
This one contributes to an objective strongly.
And you still have to, again, validate or invalidate any of this.
And any conversations you have with that tool, you actually could push it back.
So you could say, all right, actually, there's one assumption you have over there.
I actually differ.
It's actually 50, I think, talking about.
plugin requests per month, sorry per month, and where'd you get it from?
I got it from like a Jira board somewhere.
So then you store that back in code and then there's a reference point of where this assumption got generated.
And the key here is this is going to go back to the repository and it's going to require a human PR approval because we're still trying to enforce this idea of like humans being a loop at the most critical time.
And then finally, all right, we've updated our assumptions.
That's great.
Now the next product manager, the next engineer would get a lot of value from this newly generated, newly fresh, up-to-date assumptions.
But now what's the score?
That was kind of the idea of this topic.
Like what's the actual priority here?
So we'll go ahead and spit out the score for you.
And again, same kind of flow.
You as a human have to validate, do I agree with the score?
Is there some other context that's missing here?
So we always keep humans in the loop, both at the validation of the assumptions.
as well as the validation of the score.
And when that gets done, we send it off to a central repository.
So now there's more context for the next scoring initiative because it's kind of interesting.
All right, well, last time we scored an issue related to this, it got like a three for dev speed.
So it has that context as well to hopefully like increase or improve its ability to score work a little bit more effectively.
So that's what we have working today.
And I'm going to go ahead and give the clicker back to Eleanor to close us off.
Yeah, so...
We hope you try it out if it sounds like something interesting.
Should you want to start tomorrow, you should do three things.
First of all, you should define your rubric.
You heard me talk about the seven impact factors that we've used.
You can use them as is or change them based on what works for your organization.
And then figure out what weights make sense.
So you can look at company OKRs and developer data to align on weights with key stakeholders.
You should revisit these frequently, of course, as company conditions change.
Next, number two, score your projects writing down assumptions as Mina described.
On your first run, please always try to do this with people synchronously so that you can have good conversations that surface assumptions.
Then document those assumptions writing down sources, dates, owners.
Of course, do treat those assumptions as code, putting them into source control and creating an easy mechanism for users to contribute to and change them.
Lastly, number three, integrate your assumptions into your existing AI workflows so that AI can help recall that information when scoring and you can easily loop in humans to ensure that the assumptions are validated.
So thank you so much.
Please reach out to Mina and me on LinkedIn if you want to connect on this and other platform engineering and developer productivity topics.
And of course, we are happy to take questions.
Perfect.
Okay, just a friendly reminder.
Just a friendly reminder, you can submit questions for the app.
So you'll just go to your main page and then select session Q&A.
We're in session 4B for those of you who are trying to figure out where to leave those questions.
As you both could imagine, there's many of questions coming through.
One of the first questions is what happens if you're looking to evaluate something that would totally blow up costs?
Do you still give it a rating of a five?
Do you ever think about changing your model?
How do you guys approach that?
You know, we we definitely had this conversation like should they be able to have negative?
Should we because cost reduction?
Of course, we give points for a project when it reduces costs.
So should we do opposite prioritization if it's going to increase costs?
We had many conversations like this in the end.
We decided not to we figured that there were other mechanisms to decide whether that project made sense.
Hopefully if it was going to increase costs, folks have thought about the benefits in other ways.
But there's no doubt that's the sort of thing.
that you want to talk about.
And one thing I really liked about this framework is it made us have those conversations.
By making something, trying to make something more quantified, you have to debate with your coworkers what makes sense for your org.
Yeah, and I think now what we have work, again, keeping a more iterative model, if your users start scoring some work and you start seeing it over and over and over again, this is not, we're not going to ever hit moonshot items or we want to park some capacity for.
Some things that might blow up costs, but have so much developer speed, you can iterate on the weights or even the rubric itself.
So I think it's very good when you start to keep open to iterating those initial weights as well.
How often are you all iterating on this?
Do you add things to this project as new ideas come up, or are you only going through this during your planning cycles?
So on the AI tool, we are iterating as we speak.
Every time a user scores something, we go ahead and get immediate feedback from that user.
We even have a parallel agent right now that's actually running in store and work as we go.
And we're just using that primarily for feedback on the product, not necessarily for the scores themselves.
So we're iterating on this all the time.
Okay, so what happens when the framework says that...
something that a big stakeholder wants shouldn't be the priority.
Have you had any examples where that's happened and how have you handled that conversation?
Yeah, I would say that, yes, I think many of us in this room probably have had that happen in general.
There's a number of ways that we've handled it depending on the project.
First of all, Sometimes it always gets us to have a conversation, which I think is a really positive thing when we're talking about products.
We would look and certainly ask the stakeholder, why do you think this is so important to the point of everything Mina said about assumptions?
Maybe the stakeholder has certain assumptions that we don't hold.
Maybe a bit of user research can go and help us align ourselves together.
So that's one possibility.
Another possibility is that maybe the framework, early on it was maybe the framework itself is wrong.
Maybe that stakeholder really values something that needs to show up in the framework.
Hopefully you would not forget security and compliance, but should you, maybe InfoSec will show up and be like, oh my gosh, you've got to be secure.
That might prompt you to realize that you're not.
correctly thinking about the value you bring to your company.
Lastly, I would say that was that internal business value thing.
Sometimes, hopefully rarely, you just got to let a stakeholder demand it.
Finance might require something, security might require something, and you have to do it.
But hopefully as an org, you can mostly focus on the value you bring to the developers you support or the users you support and minimize just random asks from stakeholders.
I could add an example that I could think of.
a really large initiative.
It was like a six-month project going to change the world type of initiative.
So there's a lot of investment behind it, a lot of belief behind it.
And we launched, we were able to get something out there, and we gave user value, but not to the impact that was hypothesized.
So we used the formula for the follow-up, right?
Like, okay, we did V1, we have all these amazing features that everyone is so excited or was so excited for for V2.
And we have all this momentum.
We just did this giant launch.
Let's go, go, go to build V2.
And by using the formula, we were actually able to have the conversations with stakeholders that were like, this is the right time to continue building because we have the momentum.
But when you plug into the formula, we look at the developer speed and then we look at other initiatives.
They're outranking V2 of that product, which is a painful thing.
It's a painful thing to say, let's slow down momentum on this thing that I firmly believe in.
But when you have it down like in metrics and like in numbers and you're able to like explain it, it's able to deal with the really heavy stakeholder conversations easier than without those numbers.
Because again, it's going back to like, it's very difficult to debate priorities.
You can't, it's hard to tell a stakeholder like a, you know, this is not the most important thing for us.
But if you start just throwing out assumptions and throwing out data and saying, this is important, but.
It's going to give us X value.
And comparatively to this, that'll give us Y value.
Now we're just talking about value.
And we do like have the discussions about the assumptions instead.
So I found the formula was really helpful in that example.
A really creative way to be able to help pull emotion and ego out of some of these really big and hard decisions that everyone is having to make right now.
So really, really cool.
What about...
outside of the developer space?
How are you, are you leveraging this in any way to help in other roles outside of developers?
Or is that something that you're also thinking through?
I did a quick response and I'll tee it off to Eleanor, but like we have, we're actually doing similar to a lot of companies here.
We are looking at like the non-developer, the citizen builder, whatever you want to call them, the persona that does not code every day.
I'm one of them.
So we have some really exciting work in that area coming up.
And we ran it through that exact rubric that you saw.
Eleanor ran it through a few weeks ago.
It showed me the score.
And I was like, that's really low.
Because the rubric is prioritizing developer speed.
And we still believe developers are those that were traditionally hired developers.
So I think the rubric, as you see today, really opens up this question.
around how do you support these users and what you have to change about your prioritization skills to get there.
So I think it's a very interesting time and I think, Nella, you've done some work looking into this area.
Yeah, it's just another example of it.
Don't see this as a really firm framework.
We have to prioritize this project this way because that's what the framework says.
only either what we see is prior to like if we see a project prioritized a certain way it feels right great and we can explain why and if it feels wrong in our gut we go and figure it out so exactly I scored this project that I'm working on now to help non-developers deploy apps they make and it was really low and it just felt wrong and of course it's because we are not including those users because they're not typical platform engineering users so I think the framework will probably at least for us only be more useful as we first push forward the conversation of are these users part of our user base?
I hope the answer will be yes.
And then when they are part of our user base, well, wait a second, what do they care about and what value can we bring to them?
Because runtime reliability, for example, like these users probably won't bring that, but they're making proof of concepts that may help our overall company do better.
And then maybe we'll be making internal tools to help.
non-developers be more productive?
I don't even know.
So the point is we're probably going to have another round of conversations of what impact factors should we consider?
What are their weights?
What's the benefit to the company?
So I'm excited to go through that.
An exciting time for sure.
Thank you both so much for being here.
Super interesting stuff.
I know so many in this room have probably hundreds of more questions for you both, so I'm sure you'll be very busy the rest of the day.
Thanks again for being here.
Everyone else in the room, we will take about a five-minute break.