# AI-Native Observability: Agents, OpenTelemetry, and UX Shifts

**Podcast:** The AI Native Dev - from Copilot today to AI Native Software Development tomorrow
**Published:** 2026-02-10

## Transcript

The interaction patterns in software will totally change with agents.
A lot of the agents are chat interfaces.
They are kind of giving you an answer.
My view on it is that it will be more like an interactive mode between the user and the agent.
As an example today.
I'm using Google Presentation and there is this Nano Banana feature.
The result is pretty disappointing because it creates an image out of your presentation.
I think the way you want it as a user is that I say, hey, help me make my presentation slide nicer.
And then I can still go in and change the size or the text.
Before OTEL, every vendor, including myself as Instana, did their own format.
And OpenTelemetry actually was the first approach to standardize this format.
If you look at a lot of vendors, they say we support OpenTelemetry.
What that means is that you can send them OpenTelemetry data.
They take that data and convert it into their internal format.
What we have built is the data is always OpenTelemetry because all the models by default understand the format and therefore can really work with it similar to what they can do with code.
How do you think about the career path in sort of software engineering today?
And what's a good thing for someone to do today if they're sort of early in their career?
Yeah, so...
Before we jump into this episode, I wanted to let you know that this podcast is for developers...
building with AI at the core.
So whether that's exploring the latest tools, the workflows, or the best practices, this podcast's for you.
A really quick ask, 90% of people who are listening to this haven't yet subscribed.
So if this content has helped you build smarter, hit that subscribe button and maybe a like.
All right, back to the episode.
Hello, everyone.
Welcome back to the AI Native Dev.
Today, we're going to go plunging...
deep into the world of DevOps and observability and, you know, what does that look like in the world of AI?
And to dig into that, we have Mirko Novakovic, who was the founder of Instana, sort of has built an observability platform, sold it to IBM.
That's where we met Mirko way back then.
And today is the founder and CEO of Dash Zero, building that out.
So Mirko, thanks for coming on to the show.
Yeah, thanks for having me.
So just to dig in, start by giving us a bit of context of like...
Dash Zero, the company you've built and are running today, tell us a few words about it and some of its core.
Yeah, so we are an AI native observability platform.
We actually started, when we started, we were not promoting AI native at the beginning.
We were open telemetry native.
And that turned out to be a good foundation for AI, by the way, which we can discuss.
So the idea was there is a new standard in observability code open telemetry, which standardizes the format of the telemetry data, logs, metric traces, end user events, and also standardizes the tagging system on that telemetry data, which is called the semantic convention.
So host name now is host underscore name or host dot name.
And that...
That is the way we have started.
So we built a full platform for logs, traces, metric, and user monitoring, taking only open telemetry data into it and keeping it as open telemetry and making the most out of the semantic convention by creating context.
So if you look at a trace, you see all the logs, you see the metrics, the underlying infrastructure, everything in context based on the semantic convention.
And we try to make a very easy onboarding flow.
PLG type of sales motion and having it easy to use.
Yeah, so part of it is kind of just a next-gen observability platform that sort of gives you all the tools and then it kind of combines all this sort of the broadened scope of what observability captures today with the traces and logs and all that jazz.
But I think that core of...
OpenTelemetry or OTel is really interesting.
I guess maybe you can sort of say a few words about OTel itself and its distinction between when is it a format and when is it the specific fields, right?
Because I know when OpenTelemetry came around, there was a lot of...
maybe like a first set of companies that sort of built on top of it and were maybe overly optimistic at not just the format, but like the type of information that would just magically, I'll just connect to the system.
If they already have OTEL, I will have the info I need.
And they learned not all the information is always there, but some did standardize.
So maybe, can you say a bit about like, what have you seen in terms of OpenTelemetry?
Where has it standardized just formats and what data is sort of typical to find?
in any hotel-based capture.
Yeah.
So let's start by why I think it's also meaningful, right?
Before hotel, it is every vendor, including myself as Instana, did their own format, right?
So we had an agent, and so you specified your own format, and you sent your own format.
This is the old-school agent.
This is the tracing agent, not an AI agent.
Exactly.
That's the old-school agent that you install on your host, basically, that captures the data, right?
The CPU utilization, the logs, and the traces.
Exactly.
It's not an AI agent.
The benefit was that...
you could make everything you needed for your platform right into the format and you could do some other magic so for example if the agent was running on the host you could add the host name because you were running on it and just send it over right and open telemetry actually was the first approach to standardize this um this format right so that you have no proprietary data and and i think the main drivers were the cloud providers Because if you're running something like AWS Lambda or any managed service and you were to provide telemetry data of that service, how do you do that?
You either provide it in 20 formats like Datadog, New Relic, DareZero, or you have a standardized format that everybody can understand.
And that's how it started.
So I would also say if you look at a lot of vendors, they say we support open telemetry.
What that means is that you can send them open telemetry data.
But I would say still 90% of the vendors, they take that data and convert it into their internal format.
Because that's how the platform is built.
So you can send it, but then in the system, essentially the tagging system, et cetera, is gone.
Because it's now data.format or format of any other vendor.
And I think what we have built is something where the data is always open telemetry.
It stays open telemetry.
The naming is still OpenTelemetry in the tool.
And yeah, coming back to your question, I think there's only one tag that's mandatory and that's service name.
So the only tag, and so essentially everything that emits telemetry data is a service, right?
in that sense.
And so if you have a service, I don't know, payment service, and you send a log, you add the service.name equals payment service.
And if you have a metric of that, you do the same.
And now you can correlate everything on a service level by saying, give me all the metrics, logs, traces of the service name.
Yeah, that's the only thing that's mandatory.
And that's also a bit of a problem.
I would say for vendors, because we also see that a lot of the data we get from customers at the beginning does not have all the information that would be needed.
For example, if you now ask, give me all the logs of that pod or that host, if that log does not have the pod name and the host name as a tag, we can't do it, right?
Because again, going back to proprietary agents, there we could add that information to the log or...
Yeah.
ourselves but now we are relying on the customer sending us the right context and if we don't get that context we can't really recreate it right so um there are companies like ollie garden are only focusing on quality of telemetry data, right?
They actually look at data and then they will tell you, oh, by the way, here we see it's running Kubernetes, but we are missing the Kubernetes cluster name and the pod name.
And so we can give you hints of doing better, right?
By the way, I think there is also a big chance for AI.
We could actually look at data and then proactively...
Add configuration data or add even...
Populate kind of those fields.
But it's a bit of an inverted version, which is sort of the core of it is to say whoever it is that is operating the system, they have metadata that they store on top of it.
Instead of having that metadata be extracted or inferred or captured in the observability system or in an external system, otherwise monitoring those services, you...
You kind of collaborate or sort of push the customer, help them more relevant.
Maybe AI does that more and populate that information at the source, you know, to sort of decorate the services with the right.
Exactly.
We, for example, also built an open source project, a Kubernetes operator.
And by using that operator, we do all that configuration stuff for you.
So if you use that operator, we will essentially make sure that all the telemetry data has the right Kubernetes information, host information set.
And we also auto-inject the agents, the agents, for example, to Java or Node.js into your runtimes on the fly, right?
So that you get all this.
Agent is such a confusing word right now.
Like, again, these are agents.
These are not the AI agents.
These are the observability agents.
Exactly, exactly.
Yeah, yeah, yeah.
Really, like, I don't know how we got ourselves in this trouble.
But today, in observability, you definitely have ambiguity around the word agent.
Or insecurity, right?
Yeah, that's true as well.
So, I mean, at this point, listeners might be a little bit like, isn't this an AI startup?
An AI podcast?
What's going on here?
So, you mentioned that Otel proved actually kind of quite useful when it came to agents.
Tell us a bit more about that.
Why was it useful?
It was useful.
I think it started by when these LLMs came out and we started experimenting with it, that we saw that we get really good results with those platforms like Cloud, for example.
We use Cloud internally, giving us really good results by saying, hey, analyze this trace, right?
Right.
put in their trace which is literally a text format of tags right yeah which is specified by open telemetry and now the thing is because it is actually open source it's openly documented it's an open standard all these models are trained on the data and they literally understand that host.name is a host name and they can start uh basically understanding the context getting what is uh http status code 404.
You then know that it's actually a problem, right?
And now it can really analyze these things.
So OpenTelemetry turned out to be really useful because all the models by default understand the format, understand OpenTelemetry, and therefore can really work with it similar to what they can do with code, right?
It's like it's a text structure, text format, very well specified.
It has a syntax.
It has semantics.
And so it can actually really do interesting things and analyze telemetry data.
I mean, we were really impressed by the output.
And this is still format-oriented, right?
Because there isn't, I don't know at least of any kind of big bodies of telemetry data, even open telemetry data, that are just available for the LLMs to train on.
Generally, I think in the world of DevOps.
like traces, you know, everything you collect, even in dash zero, it's in your system.
It's not published anywhere, unlike maybe like GitHub for a code.
It's not published.
There are a few repositories where you can find a large amount of logs and spans.
As examples, right?
Yeah.
People just posted as a, like, were they custom created or is it like donated logs, you know, from people?
Donated.
It's kind of donated.
Yes.
It's kind of donated.
And by the way, a funny story is when we started, there is an open telemetry sample application, also open source.
And the sample application has very well documented problems in it, right?
Right.
Different types of errors.
And when we started with LLMs, we asked the LLMs about problems in that sample application because it was running.
And it always gave us perfect answers.
And we were super excited at the beginning, but it turned out the LLM was also trained with the problems documented on the actualist.
So it could do it because it could cheat the test because it knew the problems up front.
So results were amazing at the beginning, but then we figured out, okay, it's actually not that amazing with other things.
And then we kind of connected the dots that it was already trained on the documentation of the problems.
Drilling into that a little bit, so like the LLMs were naturally better, you know, that was like a nice kind of benefit of choosing open technologies that the LLMs came pre-made or pre-ready to process open telemetry data.
They're not very good at analyzing traces.
They're not very good at like understanding time series data.
They don't, you know, the volume of data a little bit is there.
I guess.
On those fronts, like hotel, no hotel, that doesn't terribly matter, right?
Or I guess what has been your experience in terms of the native support of just drop this trace into whatever, Claude or Chachapiti and get some result?
I think there are two separate problems, right?
One is you have one trace and there is a problem in it, right?
In the erroneous fan or something.
If you do drop that trace into, for example, Claude, I think it will definitely come up with an analysis to tell you, hey, I see there's a problem with this trace.
And depending on the metadata it gets, for example, it's a database problem.
It gets a database status code in there from an Oracle database.
It will look up that error code and will give you context on that error.
So they are really good at saying, hey, this is actually...
I don't know, exhaustion of a connection pool in the Oracle database, and you should do this and that based on the documentation.
This is essentially what you would do as a human, right?
You would see that code and then search for it, and it does the job for you.
Where it is not really good if you have thousands or millions of tracens to figure out anomalies inside of it.
A, because they can't really do that large amount of data, right?
The volume.
So what you have to do there is you have to provide the agent.
Now we are talking about the AI agent.
You have to give the AI agent the right tools to do the analysis, right?
So we have a functionality, for example, called triage.
What triage does is it compares, you give it a million traces and ask, is there any anomaly in it for erroneous traces?
And it would look at all the tags and would tell you, oh.
The ones with the error always have this customer ID as a tag, and then it will return that result.
And now we provide that tool to the agent through an MCP server, right?
And the agent can now, the AI agent can now use that tool, that triage tool, and it will use it, right?
Autonomously, it will say, okay, there's a problem.
Let's figure out if there are any anomalies.
So let's use that triage feature.
Yes, you have to build your API, essentially, in a way that it works for the agent and that the agent can use it.
Yeah, to be consumed.
So I think maybe let's delineate.
So we're going to talk a little bit about sort of the way in which AI meets observability needs and sort of that type of analysis.
And I think there are sort of three pillars here to talk about.
One is what we started talking about here, which is more...
the kind of AI power, like how do you use AI and agents to be smarter, right, to sort of, you know, offload more of that work, provide good functionality.
The second is about agent as a consumer.
And, you know, I think you started now talking about the agent will consume.
So I want to disambiguate a little bit of which agent is that.
Is it your agent or is it like a client's agent?
And then maybe we go a little bit more philosophical to talk about sort of product and talk about scope and the likes.
So maybe let's start.
We started digging a lot more like the other lens can do this, it can do that.
So why don't we talk like you have a bunch of agent-powered observability.
I don't know how you call them.
I'll let you say that in a second.
But under the mantle of agent zero is the capabilities.
So tell us about that.
What are useful things to do today with AI when it comes to this world?
I mean, when we started, Agent Zero was just one AI agent.
But then over time, we figured out that there are actually a lot of use cases where AI agents make sense.
And now Agent Zero is just a platform for agents.
And we have different ones.
And I think the most prominent in the whole space, there's also a category, AI SRE agents, is essentially troubleshooting, right?
I mean, you get a call 3 a.m.
in the morning.
There's an outage.
You have a problem.
And now you want AI to support you figuring out what the problem is.
And it turns out they are actually pretty good at it.
So that's the SRE agent.
We call that agent the seeker.
So it's an agent.
We give every agent a name.
And so that's the seeker.
And the seeker is essentially the troubleshooting agent that helps you with any kind of problem figuring out in the data what the root cause of the problem is.
So it will tell you.
And this is, I guess, the root of the root cause analysis agent.
Is it mostly about sort of discovering the data?
Is it more about analysis?
I guess what would you say are the core competencies of that agent?
I think it must do both, right?
First, it must understand the underlying system and the dependencies because the agent literally has to figure out what to look at, right?
Yeah.
And then it's about...
digging into the data and figuring out if it's a log or a span or a metric that is the root cause of the problem, if there's any correlation.
So it's a step of things.
Normally, you would see the agent figuring, okay, which services are affected of the problem and what is the underlying infrastructure?
And then it would ask our server, give me the spans and logs, the erroneous logs of that service and give me the CPU time of that.
underlying Kubernetes pod.
And so it gathers all the data, analyzes it, and tries to narrow it down.
So that's essentially how these RCA root cause analysis agents work.
And again, they work pretty well, right?
Because most of the errors are text-based, and it's easy for an agent to look at the data and figure out what to do next.
Yeah, I think the domains that are sort of getting the most success are sort of the very text-heavy domains, and this one definitely qualifies there.
How do you think about, like, you built your agent.
I'm sort of breaking my own sort of ordering there of the pillars of it, but, you know, talk about building your own agent.
You're describing a sequence of actions.
I can imagine, you know, Claude Code or, like, any sort of other agent that I have myself going off and using tools to do these things and get the right steering.
I guess, how do you think about...
building your own agent versus integrating into whoever it is that the customer has an agent?
Why have your own?
And do you think that's a long-term or short-term reality?
It's something I'm not 100% sure of how it will work.
So we do both, right?
We integrate into agents like Cloud Code or Cursar or whatever through our MCP server.
So there the use case is, or the idea is, the developers inside of the IDE essentially is cursor or something.
And now they can ask a question, figure out the errors of that service in production and suggest we fix this in code, right?
So it will connect to our MCP server, do the analysis that we just discussed, narrow it down, can then use the functionality of cursor to match it with the right code.
and see if it could suggest a fix in the code, right?
I think that's a natural kind of use case for a developer.
Why would the developer log into a second tool?
Why wouldn't you stay inside of the tool, right?
So that's one sort of integration.
But then there's the other use case that you get a Slack message or a PagerDuty message, and it says, hey, we have a higher error rate on your payment service, and you want to click on it.
and get a full analysis in context of the data, right?
With the dashboards, with the spans, with the logs, you want to analyze it.
And there the agent is inside of our tool.
That's the agent zero then.
And it will guide you through the UI, right?
And I'm just of a meeting with my product team because I do think that this is something we are working on right now, that the interaction patterns in software will totally change with agents.
I think at the moment we are still at the beginning where a lot of the agents are chat interfaces and they are kind of giving you an answer.
My view on it is that it will be more like an interactive mode between the user and the agent.
As an example, today I'm using Google presentation and there is this nano banana feature which says, hey, do you want me to update your slides?
And I think the result is pretty disappointing because the result is that it creates an image out of your presentation.
And now the slide is an image.
It is nice because it looks like, but now it's not interactable anymore for me as a user.
You can continue editing it yourself.
Exactly.
And I think the way you want it as a user is that I say, hey, help me make my presentation slide nicer.
And then I can still go in and change the size or the text, right?
And that's the same for us.
At the moment, the agent will tell you this and that is the root cause.
But what you really want is I think that it will show you, it will actually do the filtering and everything inside of the tool, will pinpoint it.
And now as a user, you can say, yeah, but I removed this error and now do the analysis again, right?
Because I know that this problem is not really the problem.
it's more an interactive mode, right?
I think that's something we are creating right now.
So at the moment, we are still at this nano-banana type of integration, right?
It's nice, but I don't think it's the final way.
we want to see more as an interactive mode, right?
Which encoding you already have, right?
It creates code.
You can now change the code.
Yeah.
And even though there's kind of the conversation right now, for instance, on the sort of the IDE versus the sort of the terminal base, you know, what is the preferred path on it?
But it sounds like you're dividing it into, you know, maybe the sort of the lowest level is the tools.
It's just the ability to...
kind of access different parts of the system.
That's probably just your APIs and you have the MCPs.
On top of that, sounds like some sort of analytical tools of figure out what the next thing might be or like, you know, agentic tool.
So there might be like, as always with software, there might be like composable layers of tools.
And those you can call them now from here, it's like it's an interface question.
And maybe there is a case of, you know, a developer at Cloud Code at their terminal.
Maybe there's a case of like a headless, you know, when an incident got called, something automatically ran.
And I think for both of those, it probably doesn't really matter if it's, like for the first one, clearly it's wherever the user was.
And so if they're in cloud code, then you should integrate as a tool.
The middle layer probably can run either way, right?
You can invoke a headless cloud code to sort of...
have whatever the relevant kind of mcp tools to do the analysis uh but uh but really it's your sort of agentic process of it so it's just as well run in your system but but the other part is if you're really in a hey i'm troubleshooting this has the review exercise of like how do i collaborate with the ai uh over here and that's the part that's uh i guess we're sort of most shaping uh uh at the moment it reminds me a bit of um i had a conversation here with uh with meryl the ceo and founder of uh of graphite and we talked about reviews and like to some extent i think code review as it is done today my opinion a lot of that is is would be a bit nonsensical in the sense that you you want the same reviews the same questions to happen before but review is an action of like reviewing the results of the ai's kind of coding is probably more fundamental than ever and will kind of continue to grow in importance.
And so figuring out the right UX, what is it that you are reviewing, how is it that you're collaborating, is kind of an interesting question to sort out.
Does that sound aligned?
Absolutely.
I also think for us it's an existential question, right?
Because if it turns out that the user will use observability in other tools, I think...
then we are just a database, right?
And that will mean that it's a race to the button in terms of pricing.
And so the value then is not generated anymore in our tool, it's generated in Cursor or any other SRE agent tool or whatever.
So for us, it's really existential to see, okay, how can we provide more value inside of our tool than you get in a Cursor or in an AI SRE agent?
Because if we can't provide more value, I think it is...
is this an existential threat, right?
Because at the end, people will pay for the value, not for the data.
And if the value is somewhere else generated, then we will at the end be a database, right?
And then it's not good.
So that's really how I think about it.
So for me, it's really about, okay, if I can create this interaction with the user and can provide real value, that's awesome, right?
And also, I have to say, you probably know, but if you look at large organizations, we have hundreds of users registering for observability, but only a fraction of them are really using it.
Because you need to kind of be an expert to do troubleshooting.
So I think we have the chance with Agentic AI to enable almost 100% of the users to get value out of it because it can help you following the right actions inside of the tool and it can guide you.
So I think that's really...
powerful and then you get more value, right?
Because if 100% of the users can get a troubleshoot.
They can self-service and they can build those.
The most clear demonstrations of AI productivity in sort of the entire software development lifecycle, including observability, are cases where you used to have a dependence on another team and you didn't anymore, right?
Or in general, where you skip a whole step.
Some of it is, you know, a product or a support or someone else logs a bug and a background agent, you know.
figures out and you only meet the solution at the pull request maybe at some point even that gets auto resolved so that's an example sometimes it's a something that needed to go from the back end team to the front end team yeah and then get deployed but now one of those teams can take it all the way so you know it and so i think the the opportunity that you're describing here makes a lot of sense which is the more self-sufficient the more sort of single owner can take it all the way through uh the more the more productivity you get.
And now the question becomes just like, what's the scope of the product and what value do you get from which tool?
And there's a second, we just talked about the root cause analysis agent, but there's a set of other agent, which I call it's mostly removing toil, right?
It's removing tasks that you really don't like, right?
And as a simple example is dashboards, building dashboards for a service.
And then you have an update in your service and you have that service on 50 dashboards.
And now you have to update all these 50 dashboards manually to have that new metric on it.
And I think that's where AI is also super powerful.
It can create and suggest dashboards for a problem or service.
It can automatically update those dashboards.
Same for alerts.
You have alert rules, something changes, and you need to update all these alert rules.
And I think that's where we also created agents that help you create update dashboards, create update alerts, and do those things, right?
I think that makes a ton of sense.
And another big point is adding more context.
And that means we are adding other data sources to observability to understand the context better of a problem.
As an example, we connect to linear and JIRA, or we connect to your notion.
And by doing that, we can actually see, oh, well, maybe there was, I have a database issue with the schema.
Maybe there was a task that updated the schema and I can correlate it to it, right?
Or maybe there's a documentation in Notion with a schema update, which I can use and pinpoint user to it.
Or I can use the code, the connection to GitHub to pinpoint to a change in the code, et cetera, right?
I think adding more context.
is also very important now because the agents can use that context to create better answers.
Yeah, and this comes back to the root cause analysis.
That agent might come along and figure it out, but you have to relate, like give it access and relate it to it.
And I guess organizationally, one of the challenges that I hear talking to the enterprise side of it, talking to the companies using these different tools is sort of this...
infinite mesh of connectivity between all these different tools like the number of tools i need to connect to my linear i need to connect and then there's going to be another subset of tools that might want to connect to dash zero as a source of information i want to read something about the logs because i'm writing code and i want to get that information doing it and all this sort of cross-pollination of it and so it's interesting i don't know what your view is you know at tesla we uh we more and more sort of think about context as a as a core competency that's a bit agent agnostic.
And so there are statements about your system, statements about your knowledge, your practices, how a system should operate, but also like what should your code do and what are your coding best practices and how to use a certain library.
And all of those things right now are being derived or sort of intelligently AI-powered, assessed and extracted out of systems by so many different agents as we all figure out what's going on.
And we we kind of store them inside i guess how do you think about the challenge of over there clearly is is like where do we go right are you going to curate context in 20 different spots or do you want to curate them in some central uh central environment and so i guess my my question back maybe to the existential thing as well is what what do you think is is the core of observability knowledge you know if if the coding side revolves around the code or maybe the product functionality like an observability You're a database, but what's the most important long-term insight or sort of knowledge that you think lives in operations and not earlier in the system?
I don't know if that makes sense.
No, it makes total sense.
I think I'm just not sure if I have 100% of an answer to it right now, but I think the question makes total sense.
And I do think there are multiple things, right?
One is, as we discussed, the tools that you provide.
And that can be a lot of knowledge and context already, right?
How do you figure out what are anomalies and traces and how do you figure out these tags or things?
How to ask the questions.
I think also there will be a lot of knowledge about how do you evaluate models and so the evaluation that you have and how do you essentially make sure that the...
root cause analysis does what you think it should do, right?
And also giving some guidance.
There are more and more of these to-do lists that you give so that the workflow is not every time different, but you have the knowledge if there is a certain type of problem, how to do the analysis, right?
And which step you should follow on the wider scope.
I think that's something.
And then I think last but not least, it is really the user experience.
the way you integrate the agents into a collaborative mode, right?
Basically coming from a single player mode also to a multiplayer mode where at the end of the day, if you troubleshoot, you normally have these war rooms, right?
And multiple people are working on that problem.
And agents are now part of the war room, right?
You maybe have one or two, three agents working with you in parallel, investigating things, giving you information.
that information is then processed by a user.
You give more context to the agent.
And that way, I think that will be a big part of it, right?
Who figures out the best user experience, the best way of doing it?
And to your last question, I like the idea to have that knowledge somewhere persisted, right?
About a system, about certain things, how it works.
I don't know if we would be the system.
I also like the idea that it should be a centralized.
I mean, you had these enterprise architecture management tools before, right?
Where you had the whole system documented or should document your whole landscape and give hints.
I think I can see why you would have a lot of that context into a centralized tool where all the agents can access and get some.
understanding of how things work and how they are configured, rules, et cetera.
I think that makes sense, right?
The enterprise architecture management tool for agents, right?
That has it in a way that agents can work with it and understand it and understand certain things.
I think it makes sense.
Yeah, I think it's a good analogy.
It's like so many things when you're sufficiently old like us, you sort of see a bunch of things, a whole bunch of practices that were...
were good practices.
They were just too hard to maintain at pace and at scale.
And now with agents, maybe we do those.
And so those sort of central systems of record, well attuned for consumption, we'll touch on consumption by whom in a sec, is an opportunity to develop.
Yeah, when I started my career, I worked for IBM and we had a different set of documents we had to maintain.
in every project.
And one of them, I really loved it was a document called architectural decisions.
And essentially, whenever you did a decision, I use this in that framework or whatever, you had to document it and explain why, right?
Because whenever you had a new team member join the team, they ask you, why did you use this framework?
Doesn't make sense, right?
You can pinpoint them to the document because there was actually a reason for it.
And once you read it, you understand.
I think that's also...
would be good for agents, right?
If they come in and say, oh, why did you do this and that?
But maybe there's a reason for it the agent cannot understand.
And so you should have some sort of kind of categorized knowledge for those agents, right?
Yeah, and curated and in a place that you can also choose to change it over time on it.
And then on top of that, what we see is, you know, you write it down, but not all agents listen the same.
And so you need to...
I guess as I think about AI Native, I think AI Native is primarily about delegating work to AI and or tasks on it.
And I think when you think about delegation, sort of really the two things when you think about the human analogy that you need to do is, okay, did you equip, you know, the...
the entity that you're delegating to with enough information that it's sort of plausible that they will get it done right like what is your intent what is the knowledge around the system so what information is available to them and then two is how would you verify their work and so to me that's basically context and spec and evals you know those are like the two uh so you have to have some means the evals are clearly not going to be comprehensive just like with humans you know you're not going to know how to verify every single task that your employee did because then like sort of pointless to have them.
But you do need to sort of spot check, I guess, and sort of assess it.
And so with the agents, like here's the information.
Did you listen?
Did you understand?
You can force them to listen, but understanding it depends on the model.
So let's maybe talk a little bit about the human side.
And to start that, if I talk about AI native, this was like a little bit of my definition.
When we spoke about what is an AI native product, a lot of a lot of your focus was on this notion of building agent first.
So how do you think about the future of your product in terms of its primary constituency?
You know, is it agents?
Is it humans?
How do you delineate the two?
Yeah, the way we do now design of our product is really that we think of everything first in the perspective of an AI agent, right?
And also what an AI agent could probably do with that information and how we can...
make life easier for the user then, right?
So that's probably the most important thing for us at the moment is really always say, okay, can an agent do that work?
How would that look like?
And how would that information feed back to the user that it's understandable?
And also I would call it traceable, right?
So I think for us, essentially in observability, one of the most important thing we got from our users is that they want to understand why the...
AI agent has come to a certain conclusion and you really want to follow it, right?
And if we say, yeah, the problem is here, the database, then often the question is, oh, why did you come up with that conclusion, right?
And then you want to follow the steps, essentially, how the agent was getting there.
So it's always about, okay, what can the agent do and how do we make that understandable?
to the user and where are the interaction points of the user with that agent right so but it's not anymore that we say we built a tool where the user is the prime primary yeah basically uh interaction point with with us we think that the agent is the primary action point now yeah it should do most of the work and the user should just interact with the agent at points where the user has the knowledge uh to to make the agent better, right?
So the interface, I guess another way to say this is you think the primary UX for the user will be to interact with the agent.
And so there's like a layer of functionality that is aimed quite heavily at the agent to achieve this.
And then for the user, you have to solve for a different problem.
And I guess some things go away.
I don't know.
Is there an example that jumps to mind of like...
how this, I guess, kind of led you to build something a little bit differently?
Let's think of a very simple thing, a dashboard, right, for a service.
So you have a service, a payment service and a dashboard.
It used to be the normal thing is you put on the red metrics where you show how many calls do you have to that service?
What was the response time?
How many errors did you have?
And I think today...
the primary information we give you on a service is a textual description of the status of the service.
It would tell you, hey, your service is operating fine.
It is in the range of the performance of the last 30 days.
But we found two errors which came up recently that we haven't seen before, which are suspicious.
And then you say, okay, let's investigate them.
And then...
from there.
So you're not looking at charts or anything anymore.
You're looking at that.
And then the next thing is that you interact with the agent.
You say, okay, investigate those problems for me.
And so now it jumps to the, let's say, to the trace view and analyzes, already set all the context, right?
It has filtered down to exactly those problems.
It will show you the right information, the context.
And now the next thing is it describes, look at here.
It's only for that customer in that scenario.
And here we go.
Right.
So the steps are now very simple.
You get all the context.
Product design.
It's not anymore about charts, numbers.
Yeah.
And those things, because the agent will do the work for you to look at those charts, essentially.
Yeah.
And give you a summary of the things that are important for you to look at.
Yeah, and I really like that.
And I think it comes back to the first principles of like, why did we have the chart in the first place?
Well, the task wasn't give me a chart.
The task really was I want to understand what the status of my service is.
And charts are just the way we get used to looking at them.
And by the way, charts are good for users, not good for agents, right?
That's an interesting thing also, right?
We created charts because we as humans, we are really good at looking at a chart and seeing a spike, right?
Right.
Where the agent will actually look at the underlying data and do a deep analysis of the data.
But we cannot look at 5,000 data points and then see this.
We don't see the spike in 5,000 data points, right?
That's not how our brain works.
We look at charts.
But that's a good point, right?
An agent doesn't need charts anymore.
The charts are just for the user.
Yeah.
And essentially, we've created the chart.
for the user to pinpoint an anomaly.
So now when the agent already tells you, I found an anomaly, do you want me to investigate it?
The chart gets useless, right?
Because you don't need it anymore.
But this is exactly how I think we have to rethink, as you said, first principles.
Why have you started with it?
And we normally build a user experience to the weaknesses of the human brain, right?
And now we can optimize it because the agent can...
do the heavy work, right?
The heavy lifting and we just follow it, right?
I love that.
But also I love how it actually sort of touches both sides of the weaknesses and strengths, right?
You're saying one part is there's an opportunity to give a user, a human user, a better answer than a chart.
You know, here's the actual conclusion of the chart.
But also that if you...
if you work from outside and you're trying to have the agents just rely on the information that is available to the human, it will actually do a worse job because the agents are not as good at understanding the charts as humans.
So I love that sort of analogy and example.
And I guess the next question will be on, well, who is this for and how does it maybe change the sort of the profile of the individual?
Because you were already alluding to Today, there's a certain level of expertise for people that really truly operate the DevOps dashboards.
So some of that is also getting down to contextualizing, I guess, the answer to people with different components.
Just like as a human, you'd explain, you'd use different words to explain the same scenario to people with different level of proficiency.
I mean, normally, troubleshooting came down to a few people in an organization who have a very...
At one, they need a very broad understanding of the overall system because a lot of developers and people have only a deep understanding of their service, but not if you have a microservice environment with a thousand services.
There are only a few people who really understand how everything works together.
And normally in troubleshooting scenarios, you need that understanding to narrow down where to look at, right?
What could be the problem?
I always say or joking, if there is a big problem, systems normally look like a Christmas tree because everything is blinking, right?
Because everything is somehow connected.
So everything is red and then it doesn't help anymore because if everything is red, you still don't know where it's coming from, right?
And then you need this expert knowledge of people say, yeah, but it's probably here because I know there's this database that's connected to everything else.
And I think that's why...
you only have these power users being really effective with observability because you need that understanding, right?
And I think that's where the agent comes in because agents are good at understanding the wider scope and narrowing it down.
And now we can enable everyone to basically having that knowledge about the overall system because the agent can give you that context, right?
And now everyone can troubleshoot.
I'm not saying that we are already there yet.
But I think the ultimate goal would be to enable every developer, every SRE, everyone who needs to have that knowledge troubleshooting and understanding the system quickly, right?
Without having that expert knowledge or new people coming in, right?
Which is also a problem in enterprises.
Now these two people who have the knowledge leave the company.
And as we all know, it's normally not documented in Notion Confluence or somewhere.
It's in the head of the people.
And now that knowledge is gone, right?
And so this to me comes back to that sort of context.
We use the word context.
The human analogy is really the separation between intelligence and knowledge, right?
And so you want to enable intelligence.
The agents kind of give you some intelligence.
You can look at a lot of logs.
You can find the spikes.
You can go.
You can search.
You can find related information.
But then there's also knowledge of what is the system, what is.
what have been the sort of the past incidents that we've had so that you know that it's always that sort of finicky database, you know, that's sort of in the corner, you know, how do you build those?
And I think one of the challenges that when people conflate intelligence and knowledge, and, you know, like I sometimes say, it's like assume intelligence, not mind reading, you know, like there's no...
They can be super intelligent, but if you do not equip them with knowledge, it's anywhere between the setup to failure to just highly inefficient.
Like every time, they're going to have to learn that.
I'm curious about the people side.
So maybe like there's a bit of a closing as we kind of run out of time here.
So this is all good and well, and it's the future.
Maybe bringing it down to today, what is the journey that you're seeing customers or walking customers through?
in terms of adopting this approach?
Because a bunch of this, there's a fair bit of change involved in what you've described.
And people don't always like change.
And also there's limitations of the technology today.
So it can't quite do everything amazingly yet.
I would say, first of all, it's really changing the way users work.
There was a public statement by the CIO, the Telegraph in London.
He made it actually on LinkedIn.
He said that Dash Zero and Agent Zero are changing the way They do incident.
The playbook for incident resolution are changing because now the first place is to ask the AI and not anymore going through the list of steps you do to troubleshoot.
And I think so.
So the agents are already changing the way we do incident resolution, et cetera, because they are super helpful, right?
Probably also how.
coding agents are changing the way we code, we probably first ask an agent to do a suggestion and then we iterate with it or how we do a post on LinkedIn, first ask Gemini or whatever, right?
So I think that's already happening for sure.
What I would say today is that it's really a learning curve on both sides, right?
I mean, we have a chat interface and we, for example, we also look heavily on what the users are asking our system, right?
Because it's the first time that we essentially, a user can do something with the system that we haven't designed for them, right?
Normally, you can only use functionality that you have literally designed in the user experience.
But now they can ask questions like, oh, give me a usage report of the dashboards.
We never thought about this.
And actually, it turns out that the agent can answer that question.
So we now know, oh.
Actually, users are interested in usage of dashboards.
Right.
So we get feedback through that interface also to understand what kind of functionality we should bake into the tool, right?
Because the tool today is probably how ChatGPT, when they started, they saw people asking coding questions, and then you could see, oh, maybe we build something like the coding agents, right?
It's a use case.
And so we see that too.
But at the moment, I think it's a...
It's learning on both sides, right?
It's what works, what doesn't work, how does it change my procedures, my playbooks, right?
We see customers already changing their playbooks on incident resolution, but I think over time, it will become an integral part of observability.
I think even today, we can't imagine anymore having observability without AI.
Yeah.
Yeah, no, I love that.
And I guess it's an extreme version of what we're doing in general when you're building a product, you should listen to your users.
But because you're providing users with an interface that is much closer to what they would say to a support person or to your rep, right, or to an engineer on the team, than the free form.
version of it.
You don't have to decipher what they meant from the clicks on the page or the dashboard they created.
You're getting a more verbatim version of like, this is the question I had.
Can you answer it?
Especially if you never had this option before, they could never click on it.
You would never get that answer without talking to the customer.
Yeah, absolutely.
So I think, excited to see this evolve.
I mean, I find the world of DevOps in general is like a bit cautious around AI and observability feels like the pioneering part of it because it has a lot of data analysis and a lot of like, a lot of toil to that, you know, to kind of take away.
So it's exciting to sort of see Agent Zero and in general kind of AI native approaches and sort of the changes of the practices.
very sort of keen to see that.
And I'm curious about sort of the closing of the loop.
And we'll see that as, I guess, as the systems become more reliable.
So definitely, we'll be kind of close in tracking.
let you go here though, I want to ask one kind of a personal question I'd like to ask many of the guests here.
Typically I would ask, well, if you had a son that, you know, was going to university, would you recommend that they go take computer science?
And when we're talking, it's like, it turns out you have a son that has just started sort of studying.
So I guess, how do you think about kind of the career path in sort of software engineering today?
I guess kind of it sounds like you still think it's a good idea to go into a computer science degree today.
How do you think, where do you think this is headed?
And what's a good thing for someone to do today, right, if they're sort of early in their career?
I mean, I definitely recommended it to my son.
And the reason for it is that I think that the main thing you get trained doing a computer science or engineering study is it trains your brain in understanding problems, math, etc., right?
I mean...
I was a coding geek since I was a child, right?
So I learned coding myself, not at university.
I studied computer science.
But what you learn there is you learn to solve hard mathematical problems, physics, etc.
And I think that will not go away, right?
I think having a brain that helps you analyze problems and structure things, I think that's still really relevant.
And I just think he should do that.
to get that basic training and then figure out what he wants to do.
And he's actually, he loves hardware.
And I think that's a very interesting spot in the future, right?
Robots, drones, all these things are also a very interesting combination with AI and software.
So yeah, I think it's still relevant.
Yeah.
I think it's super interesting and I get different roles on it.
And I absolutely agree around the...
deal with hard problems, you know, like learn to overcome adversity to begin with.
It's just a hard degree to get through, you know, just like life lessons of that.
Hopefully you have some good teachers, you learn how to tackle problems.
I guess maybe what I would add is, like, I think the importance, as it stands, if you go to a computer science degree, you come out, if you didn't do any programming on the side, you're probably not a very good programmer, you know, by that time, you know, like you're not a, but so you learn a lot, you need to learn alongside the degree.
And I think that is probably more important than ever because I think where I put very little faith is in the university's ability to adapt anywhere near fast enough to teach the students the knowledge part of what they need from an AI perspective and kind of the AI era.
So I guess the sort of the emphasis there is, you know, do that, but then, you know, alongside the degree.
learn who's dash zero or like get some experience with some sort of agentic development tools to absolutely yeah I agree Mirko thanks a lot for coming on to the show and sharing these insights and looking forward to see the future of observability in this AI era thank you it was nice being on your show here and thanks everyone for tuning back in and I hope you join us for the next one