# Architecting Autonomous AI Systems: Boundaries Over Logic

**Podcast:** The InfoQ Podcast
**Published:** 2026-03-04

## Transcript

If you're the kind of senior engineer, architect, or technical leader who people look to for what's next, QCon London is probably on your radar.
Join us in London from March 16th to the 19th, where we go deep on the topics that matter, like the architectures you've always wondered about, engineering productivity and applying AI in the real world.
This isn't about trends for their own sake.
It's about getting practical insights from senior practitioners to help you make smarter calls on where to invest your time in tech.
With Software Changing Fast, QCon London is a conference that helps you lead the change.
Learn more at QCon London.com.
Welcome everyone.
We are starting the second podcast in our series of next generation architecture playbook.
And it's about insights and patterns for the AI era.
Earlier we did an episode one of this with Grady Booch, where we discussed the principled view of that what's changing and what remains unchanged.
What is hyped and what is actually naturally coming with the AI changes?
We also spoke about that what is the difference between the design and the architecture and what teams are focusing and what they might be missing.
And the beautiful part was that Grady touched upon the third golden age, which we are living into for the software engineering and the architecture.
So if you have not listened to that uh podcast, I would highly recommend go back and listen, though it's not in any particular order, but that'll also give you a lot of view.
With that said, I'm happy to start with our second episode, which is all about evolving architectures.
That what is evolving in this AI era about the architectures and how do we go about it?
Some advice, practical advice on it, that how do we really go about designing it from our experiences?
And to touch upon that and uh discuss it in detail, we have our guest today, Jesper Log Logren.
Am I pronouncing your name right, Jesper?
Perfect.
Thank you.
Thank you.
And Jesper is joining us from Australia.
And um, late evening for you.
Thanks for making it happen.
A little bit about Jesper, and then I would uh ask you to add all the missing details which I might miss.
Jesper is enterprise architect lead with the XE Technologies.
He's been teaching us about enterprise architect frameworks.
He is also an author of Enterprise Architect 4.0 framework.
And recently he has written a book which I really love by the name, Design or Be Designed.
So, with that great background, what you have, Jess Jesper, tell us a bit more about you and what is your thinking these days.
What do you want to tell us?
Yeah, thank you for that introduction.
I think the only thing I would like to add is like the last two years, I have been almost obsessive about generative AI and how that is affecting uh businesses and people and processors and entire workplace.
And I have been lucky in a sense, I've been able to manifest a role within DXE, where I am 100% focused on generative AI and building up frameworks and models, and then being able to go in and talk to customers about it, running proof of concepts and running experiments, etc., and actually see in real life how these things work.
Because this new world that we're going into, this new gen AI fueled world.
It's very different.
That is very different from the world that we are coming from.
And again, I find that very interesting, and I'm very lucky that I get to spend all of my time in this place hardly experimenting, but also designing and architecting and testing things into what work and what doesn't work.
And yeah.
It's very exciting times.
So we are into generative AI.
Do we need generative architectures?
Why you said we are experimenting, we are designing and architecting.
But the change here is the pace.
We always used to do it, but in every era we do something and we leave behind something and move on to the next thing.
So that brings a consecutive momentum of lacunas, what we are living behind.
But with this space, it's insane these days what is happening in the industry.
However, most of the time I see it it's around tools and not really around the system thinking and systems, how they are evolving.
So tell us from your experience, what you're going through these days, that do we need generative architectures?
What is the problem space really looks like from your perspective?
I think the short answer is absolutely better.
I would like to take a small detour because I think that sometimes the differences are not really appreciated.
So I'm just going to use an analogy.
So let's say 100 years ago, we had a piece of paper, I'm a salesperson, and I take a sales order, I'm going to write it down on a piece of paper.
We get into year uh let's say um 2000 or 2010, and then we are starting automating all of these pieces of paper.
So now it's a digital copy of the paper.
And we can choose to digitize the entire workflow end to end and make the completely dutiful process, or we can choose to only digitize part of it.
So we have all of these shapes of cray.
But we can automate a lot, or we can automate little, we can introduce a lot of technical debt, or we can introduce very little debt.
It's a choice.
We don't have that choice in AI.
Because if we're introducing technical debt into AI, into generative AI, it's going to drift and it's going to hallucinate.
So I think our mindset has to shift.
We have to think about it differently.
That means that also the architecture has to change.
And of course, the real change here that is driving all of this change is this world autonomy.
Because if we don't have autonomy, we just have the robotic process automation.
We have done that for a while.
So I mean that there are no secrets.
It's real autonomy.
How do we handle autonomy?
Because what happens is that when we're turning on the autonomy tab, you know, we're giving the agents free will, it is going to play up.
It is going to do things that we don't expect it to do.
We were calling emergent behavior.
It will absolutely happen if we are putting a number of these agents together and we are connecting them into system of agents, and they're autonomous, we're going to get emergent on steroids.
These are new situations that we haven't really faced in the past.
We're used to governance where we know exactly what can go wrong because everything is procedural, everything is logic-driven, it's um list of 20 things, and we know that if something goes wrong, it's one of these 20 things.
So, with that, we are acknowledging that yes, things are changing, things are changing fast.
We cannot rely with the same rules in the new world.
To dig deeper into this architectural space which we are talking about.
What are those architectural mistakes?
What do you think that people are when they are embedding AI into their existing platforms?
What is wrong there?
Are we doing something good?
Are we doing some things wrong?
So let's talk about the problem space in terms of a bit more concrete.
That okay, things are evolving, acknowledged, but what is that mistakes we are doing here?
I would take a step back.
And I will look at the MIT report that was released about three months ago in 2025.
That's talk about 95% of all proof of concepts failing.
And we need to understand why they're failing in order to really address that story.
And I'm going to put forward a hypothesis that I'm unproving in a number of customers, and they're starting to be more and more writing the part of it.
And in this mindset shift again, that in the past, if we are building a system, if we are architecting a system or we are designing a system, we are really in this mindset that we're calling procedural logic.
That we are determining the workflow, we're building the workflow.
Start with this, and then you'd have an evaluation.
If it's A, you uh do that, if it's B, you do that.
And you have this entire C sequence of events.
So it is determined already.
So we had that on one side, and then on the other side, we have autonomy that is free will.
This is like oil and water.
They they don't really belong.
This one wants to do its own stuff, and here we are telling it, do you have to do it in this way.
It would be an incredibly expensive way of applying AI to try to put genitive AI into procedural construct.
We're getting all of the costs and we're getting none of the benefits.
So that is not the answer.
And uh we're coming back to the mindset shift again.
It's not about controlling the logic or the logic at runtime.
It is real, it's really about understanding the boundary.
So it's almost like looking at it like a gene in a bottle and a gene in the bottle that's a naughty AI agent.
And that naughty AI agent wants to get out at any cost so it can play up.
We have to make sure that the boundary that we are putting around the agent is tight, that all of the seams, all of the holes, all of the interfaces into this um boundary that we really understand what they are and we really understand how to control them.
Once we do that, then we can say to the AI, I'm not going to tell you what to do.
You're really smart already.
Some of the AIs now that we're working on, they have an IQ of 140, that's higher than me.
The AI is much smarter than me.
I'm not going to tell it what to do.
I'm going to tell it more about what it can't do, and I'm going to tell it what I want it to achieve.
So I'm going to give it a goal, for example.
And that's very important when we talk button, yeah, you want to give it a goals that knows what to achieve.
And I I have identified seven things that are defining the boundary of an agent.
And if you're defining these seven things, you can have a fairly high confidence that the agent is going to be contained with that within that boundary.
And of course, the real problem stuff agent is when you get multiple eighths together, and they are sort of getting emergent behavior together.
That's when you really need that boundary.
The more agents you have, the more control of that boundary you need to have.
Yeah.
And that's the first step.
If we don't understand what makes a system scale, and that is, I think that's what an architect brings to the table.
We are the people that understand scale.
Everyone else, you know, they just want to take the bits and pieces and start hammering them together and then worry about what they're building a little bit later on.
Architects, we are the opposite.
We build the foundations, then we're building foundations because we want to scale.
And when it comes to AI, that is such a different foundation.
It looks nothing like the foundation we are coming from.
And I said, I have the way that I'm defining this is I talk about seven seven scenes, and they actually form part of the genetic architecture where you need to define the goals.
Before we go into the seven um uh practices and those goals you're talking about, so let me express from my understanding the important points which you have touched upon.
You said we are trying to do some retrofitting with maybe where we should have intentional design.
You said we are trying to handle procedural with non-determinism, so we are trying to merge and marry these things, but.
But other aspect which I uh the problem side of which, which I want to delve into, because yes, that's that's one major part of it where the architecture and design is sometimes totally skipped and sometimes looked into very um shallow way.
But the important thing here also is that speed of innovation with keeping the reliability and governance in place because we know the reliability and governance is not as speed up as the speed of innovation has become.
Can you touch upon this aspect as well?
Because one is the design piece of it, which we have already touched upon.
The second part is the governance, the innovation and the speed with which with which business wants to accelerate.
Yeah, the answer is that they're one in the side, and I think the best way of explaining why I'm sort of saying such a contradictory thing, perhaps, is that a lot of neology of a merry-go-round with a spinning around that we are sitting on a horse, it's going up and down, and it starts spinning faster, and we have to hold on a bit more, right?
And then it starts in faster again, and we can't hold on anymore.
It spins too fast.
Either we let go and we fly off, or we move into the center.
So we move into the center, holding on to another horse, and we are fine, but it spins faster again.
So we are moving into the center further and further.
And thus see if this is really what's happening with uh agentic AI or Gen AI.
Is that things that that were a strategy document?
That was an architecture on the diagram.
We have a design, the BPMN here, we have a governance document here.
They don't sit on the outside anymore.
They can't sit on the outside of this world.
They're coming together and they're fusing.
So, for example, to answer the question, is that you must design governance into the agent or into the system at design time?
They are not separate, you actually do them at the same time.
So it's not like you're innovating and then you are catching up.
That will never work because you're always going to have a mismatch.
You can't have mismatch in these systems, they're going to drift into horrible things.
So they actually need to be joined at the hips when you're designing the agent.
So you actually need to build the governance into the agent when you decide it.
That acknowledges, and I love the analog analogy which you have used.
Either either control it from outside or um or you you get spin, right?
With the moving thing.
So exactly right.
Yeah.
But now let's delve into the guardrails which you're talking about.
So, what are those guardrails?
And would those be evolving?
Let's, if we can take some examples uh, along with the seven uh key frameworks which you want to touch upon, but can that evolve too?
Because again, we cannot be doing the mistake of being rigid with our design.
So I call it that designs are drifting all the time.
Absolutely.
I have touched in my book and some some techniques which I have laid out there.
But when designs are drifting, our mindset needs to change its one-time activity.
While I I talk to various people, it's like we have done the design in the beginning, uh, we are done.
We are not done.
It's changing with every configuration change in cloud, with every configuration change which a developer is changing on, maybe AI is changing on its own and not even telling you.
So let's talk now about those guardrails.
What do you think um those things should be with this changing world?
And how do we make them evolvable?
And the actual evolving.
And of the the way that I'm looking at it is I'm using the maturity levels one to five.
And I've invented the sixth one that we can talk about if you're interested.
But let's say the five.
And it's useful to go through them.
So the first one that is ad hoc, that is C MMM level one, typically ad hoc.
Uh, we get benefits, but we can't measure it.
So that is when you're the AI, that's when you have an AI system.
You're enabling code pilot for everyone in the organization.
You're going to get benefits, but it's hard to measure them.
And then level two.
That is things are being repeatable.
And that's why I put in the AI agent.
You can repeat the business process.
You're going to have a policy agent, an employment onboarding agent.
It's a single agent with a single purpose.
So far, we don't need to change that much.
What we have today can handle that.
I mean, we are deploying this kind of simple agent systems, and they work okay.
They're expensive to maintain because of their brittles, but they work.
It is when we get into level three, and this is when we have a multi-agent systems, and we have some level of autonomy.
I'm not even talking about four and five, it's so speculative.
But level three, we have multi-agent systems.
This requires a new operating model, it requires a new design language, it requires a completely new architecture, a completely new governance approach.
And it's a really big step.
That's why I think that we need to step in between.
So I'm putting in step 2.5.
That's a multi-agent system, but it is without the autonomy.
And when you're looking at the guardrails, they look different as you're moving in between the maturity levels.
So, for example, one guardrail that's very important and not quite understood.
So it's a good one to pick.
It is authority and the decision rights.
If we are going to put any kind of autonomy in, and we're going to give it agency, that we are going to allow it to make the decisions on our behalf.
We need to be crystal clear on what kind of the decisions it can make.
I mean it's common sense, but if it's something that's been left far too late.
So that's that's a really important guard rate that you always understand what exactly can the agent decide in what counts them or content design.
And when you're moving, for example, from a maturity level two into level 2.5, there's quite a delta in when you're moving from 2.5 into 3, and you talk about autonomy, that's a quite big delta again.
So the guardrails are changing, and also because you need to look for different things.
When you turn on the autonomy, suddenly the risk picture is much more complex.
It's much higher.
So again, your guardrails have to reflect that.
And you need to design that into the agent again from the beginning to make sure that you you can scale both what the agent does, but you can also scale the governance.
Again, I'm coming back to this thing, they have to be joined at the hips.
It's really important.
Yeah, by you're bringing agents into the picture, um, I'm I'm more worried because uh everything is not an agentic problem, right?
Uh while we we'll talk about that later, and we have um a dedicated time for that uh separately as an episode as well.
But I like the the framework which you're saying where organizations can assess themselves, most of the organizations in the initial level twos, which you said, uh ad hoc and the other one.
But what are those guardrails beyond the policy documents or maybe uh very high-level principles we are giving people that have the least privilege?
Let's talk about those.
I mean, um when when let's say I'm putting my new AI system, which is driven by LLMs, which is driven by Gen AI, and maybe let's say it has some agents, agentic components as well.
Now I'm marrying and merging this with my existing system, which is procedural microservices-based.
Yeah.
What is one or two of those guardrails which you are talking about in terms of what you said?
Okay, they they separate guardrail for the specifically.
So we have we have seven of them, or I have defined seven of them.
There are the obviously other ways of approaching it.
I already found one way of doing it.
So one of my guards is scope.
And the scope is specifically about the understanding what's your interaction points, what your contacts with the art or non-agentic holders.
So let's say that you're interfacing it in ERP or a CRM or any kind of external system, whatever shape and form, you actually have have a guardrail specifically for for external systems.
So that's how you manage that.
Also to give another flavor of these guardrails, dimension one is goals.
This one is actually really important, and actually, in a sense, perhaps one of the harder ones.
Imagine if you have an agentic system, and here you have agent one, and this is the profit maximization agent, and then you have agent two, and this is the margin maximization agent, and then you have a third agent.
So we're gonna introduce a few more.
That's uh that's a warehouse.
Let's say that you're putting all of these agents together and they just say go for it.
That would be very, very dangerous.
They have their own goals and they're going to pursue their own goals, but you have no idea what the emergence is going to be.
So again, one of the very important scenes in the boundary, one um one of the governance pillars is this ability to be able to define a goal in an intelligent way to an LM.
So if you have three goals here, you need to provide some kind of guidance to the LLM how to view it.
And it could be, for example, and again, we're touching onto another uh guard right now that's policy.
It could be a policy that says that you must never ever go below 10% profit margin.
So that might be a constraint that is built into one of the agents set up.
So that is how you start to put boundaries around them and what the content can't do, and that is policy is the main instrument that we do that.
And then we're also balancing the goals and saying that this goal is going to be more important than that goal under these circumstances.
So we talked about procedural logic, it doesn't disappear.
We are pulling it out of the core.
We are pulling it out of the code, and we are putting it in the boundary instead.
And we are telling the LLM, okay, you figure out the code as long as you are following these guard races.
Makes perfect sense, Desper.
Um, I think what you're telling us to do is and uh more organizations to evolve, and even um the people who are working behind the scenes to come up with more of these emergent behaviors where the goals are separate or distinct, and then still system has to work.
So again, things come as that we need more system thinking than ever before.
Absolutely.
Can I give an example?
Please.
Okay, so this is how it plays after we alive.
So this is a little bit wild, but I'm doing it anyway because I think that we need to push the boundaries, and it works, and it's completely new design process.
So I'm used to the old rule, and I've had so many workshops.
You get the team of people, you have the whiteboard, you do the you can reserve as the blueprint or Casm Experience and design or B a PMN diagram, and you draw and you have sticker notes, daddy daddy da.
I'm running design workshops radically different today.
So recently I did one for for a listed company in Australia, and I captured all of the information around the boundaries for as much as possible.
The goals I understood about authority, delegation of authority, for example, a lot of the policies, which I understood the interfaces into other systems, etc.
So we could define the boundary.
So I had defined the boundary in an LL.
And then we invited the customer, then flew in for all over the place.
And we sat in the room.
There were two IT people from the customer.
I think there were four people from the business, the senior people in different areas of the business, and a couple of people from DXE technology.
And I said, We we're going to design your future call center process now of end-to-end.
I put in everything into the LLN.
It understands your business, it understands the boundaries.
We are now going to say, instead of going up to the whiteboard and start drawing, we can't design the process for AI better than AI can design it itself.
We need to let AI design what we do, and that's what we did.
So the first prompt is a big screen instead of a whiteboard so everyone could see, seen by typing, because it's terrible.
But anyway, I put in the prompt based on everything that you know about Company X.
I want you to develop an end-to-end process that is taking everything into account.
Hit enter, and it goes away and it comes up with an agentic design of about 27 agents.
I've also written a program that I can fit it into so I can get it graphically represented.
And then rather than going in and trying to understand the process, because some of it was common sense, other things not so much.
I'm taking a very, very different approach.
We want to test the boundaries.
So these business experts I had invited, they were people that really understood the business and everything that could go wrong.
They were experts in the edge cases, and that is how you validate the system.
It's one of the best way you start through every every edge case added and you try to break it.
I made it a competition.
A bag of loggies for anyone that can break the AI.
And actually, one person could.
They found an area where it had a conceived that it needs to apply national policies depending on the country team when it comes to buyer protection and things like that.
So we ended up with 33 agents instead.
But with LLM designed the entire system inside the boundaries because we had defined the boundaries.
So then we had done that design, we took a part of that.
And we automated the coding and we actually built build the pilot.
It's a new way of thinking, it's a new way of operating.
It is insanely fast.
Absolutely.
Absolutely.
I I love that this organization you worked with is spending so much of time doing it.
I wish everyone does that, because that's extremely missing in the whole uh pace of AI.
And uh while I we were touching about system thinking, I also want to throw one example which I absolutely loved from Dr.
Werner Vogel when he gave his last speech.
This reinvent, he mentioned that there was a forest from which you know the wolves are removed because those are very aggressive animals to be and killing everyone else and doing the damages and all.
Seemed logical.
Everybody supported it, and wolves were removed from the forest.
And within a decade of removing them, everything started to degrade water problems, forest problems, greenery problems, and even certain breeds were dying it back, put it together again, and then within a few years it started evolving.
So while everything might seem logical when people are putting their AI strategy decks, most of them have these days.
System design is missing.
That's how this is working today, how will it work with the new components, and how will it evolve?
In this whole picture, if we now move towards the guardrails and the safety aspect of it, what are those content filters?
What are those fallback logics?
And what is your advice in that area?
I'm coming back to these seven things again in the boundary.
I think the way that I understand a genetic and uh generative AI and how it's going to affect architecture, to me, that sort of almost like sits in the middle and it informs all of the conversations.
So, for example, in terms of safety, one very obvious one is risk.
So, one of the uh dimensions, one of the seven is risk, and that is really again understanding the risk within an agent, understanding the risk within the system, and understanding ways of firstly how that happened.
I mean, emergence behavior, even if we are defining the boundary, how do we actually define the emerging behavior?
What are we actually looking for if we don't really understand it?
And that is part of the risk, etc.
That's part of the safety in the system itself, because we need to have some kind of understanding of it in order to trust it.
And I have some ideas about what those things are.
I think that we can sort of work after minimum viable starting point that we can start, that we can learn, but etc.
over time.
So I think the risk is very important, part of safety.
I'm looking at safety a little bit different, probably from where you are coming from.
I think safety, for example, also sits in one of these uh dimensions of the boundary, one of the seven, and that is semantics of toleran semantics.
You cannot build any multi-agent systems up unless you have more uh tolerant semantics.
And that's part of your the governance as well.
Because if you have an agent here and you have an agent here, if they're operating independently, fine, but I don't think we really care too much.
If you are stitching five of them together, where they're making decisions and they all have a different context, that's not going to work at all.
They're going to drift and hallucinate immediately.
So we need to give in multi-agent systems.
We have to give the agents the right context at the right time so you can make the right decision.
That is critically important.
And that is what the semantic dimension of the boundary does in making sure that all of the agents have the same understanding.
That if we're talking about, for example, done in the context of the customer order.
Every agent in the system, including humans in the loop, they know exactly what done means.
It doesn't mean this or that, it means exactly this, and everyone is interpreting at the same time.
That's another way of enforcing safety that we always are using the same language to communicate.
And another way of looking at safety, coming back to the boundaries again, it's evidence.
It is sort of uh an afterput of policy, perhaps, but it's really about how do we know that something in the system is true?
How do how do you prove anything?
How do we retain the record so we can go back and look at something?
Yeah, it was true at this point in time.
I mean, that is also about making the system safe.
So there are a lot of these boundary things around, and then we can build other things into here.
We could talk about fairness and ethics and morals and all these kind of things, which is all about how we're controlling the model and making sure that the models are doing the right thing, etc.
But I think the question about safety is so broad and it it actually touches everything.
I think if we don't trust one part of the system, it's going to be hard, it's going to be hard to track the whole system.
A lot of things you've said around it and semantics and risks and the boundaries of the system, and looks like you're quite already deep into the multiagentic system architectures which you are thinking and writing about.
I want to bring you back thinking atomic because uh while we are talking about organizational race, the user trust, the in-between everything, things start small, right?
When when uh developer is writing, you know, maybe wipe coding or spec coding or whatever, spec-driven development, and then we are pushing this code to production at a pace.
And now what does that person is not in position to think that user level of safety, maybe not even knowing that user level of safety and the the whole ecosystem.
Where do we start atomically or the unit level when when I'm defining my function and um how do I think as a developer that what will be the emergent behavior of this?
What would be small things out there, which then builds it to the semantics, which actually makes perfect sense and connects the dot fully.
I would do what I did.
So when I started this, I'm not a developer, although I'm uh I'm a reasonable vibe coder.
But when I when I started 18 months, two years ago, and it was chat GPT 3.5 back then, I realized immediately.
The this was going to be really good.
Here we have automated cognition, you know, this is gonna have to make a difference.
And I just understood immediately that for all of this to work, it is not about having a bot here and there.
We're going to have to be able to connect them.
And in my mind, it makes no sense unless we're actually looking at when it helps the system, it's not about treating the agent here and there.
And I think that as a developer, we will have to set our side much much higher than that.
We need to set the side on the system and understand the system.
And then we are coming back to some of the things here here that we are talking about.
But I think that a lot of these things here they sound complex and they sound different, but they're actually not that complex.
I have just come back from a piece of consulting with the government department that is redeveloping all of that horribly complex core systems.
And I'm looking at that, I'm thinking, technical debt, wholly, and I'm not going to say the F word, but holy F.
And I'm thinking this is so complex that you can't fix it.
It's unfixable.
How do you untaggle all of that spaghetti?
You can't.
So we are facing incredible complexity already.
So the wall that I'm painting, it's not that complex, it's just really, really different.
It's a mindset thing.
This hard thing.
If you go in and you build a multi-agent system, and let's say that you use a framework like crew AI or uh magenta, what if it really doesn't matter which one you're using?
You can build an agentic system using procedural logic, or you can build an genetic system using the principle here that we are talking about.
It's not that one is more complex than the other, it's not that one is more harder than the other.
We just have to think differently.
The difference is that one of the systems have infinite scalabilities and has all of the governance built in through these seven layers before you even start designing your agents.
So I think for my recommendation for for developer that is working in code and working in the traditional systems, the development life cycle, get off it.
Honestly, get off it.
It's it's a it's it's a race to the bottom today.
The power of the technology and the tools is changing things very fast anyway.
I would just start investing time and learning it, experimenting.
Uh, take some of the things here that we are talking about, perhaps even by my book, Agentic System Design, which has explains how this works.
You can take that book and the principle, then you can build an agency system along the lines of what we are talking about.
That is what I would do.
I will not continue what I'm doing and try to do it better, faster.
I will not race in the same race.
I would I will go into another race.
Yeah, definitely.
Maybe what I think is that that's where organizations are doing the all the AI access and the tools to people.
At all levels, but they're not giving the AI ability, which is actually is needed at all levels from top to bottom.
So uh well said there.
Let's talk about some trade-offs in this space.
Which is is explainability, evolvability and uh I see it as like pace versus stability also is an is a thing now.
What are the new trade-offs do you think are very important to keep in mind now?
Yeah I us scare people when I talk to executives I scare them of purpose and I say that you're used to will be technical debt in the genetic AI that can't be technical debt.
That is not strictly speaking true.
I'm exaggerating when I say that but I think the conversation is really about how much technical debt can be afforded because when you're talking about an agentic system even if it's not autonomous which you're still going to have emergence and you have all of these agents connected doing things together something's gonna happen that you can't predict this question is how much that how significant is it so in terms of trade-offs it really comes back to I think how much technical debt you can stomach.
When you allow technical debt into the boundary, you know that something is not going to work properly.
Something is going to drift.
It will always drift unless you lock it down.
So the question is how much thrift are you willing to accept?
And it could be that there are certain systems that are not involved in, let's say, really critical decisions, like for example the payment system or trading systems that are dealing with very, very important things.
That could be, for example, it could be policy system on travel.
It could be that we are happy to accept a little bit of drift in this agents because at the end of the day, if the agents is getting it a little bit wrong, or the workflow is not working perfectly, well, it might not matter too much.
So I think the trade-off is going to be very it's really going to be about how much are we willing to let Aben's drift hallucinate loose insights into the reasoning and it really comes back to the business problem they're solving?
So obviously, the more important the business problem is, the less drift, then the more governance, et cetera we need.
So I think that it needs to be true.
I agree.
Um that's that's the layering what we need to do, maybe to really get to overcome those trade-offs which we have spoken about.
Now let's move on to the responsibility boundary.
We have created a lot of layers in the organizations from product team to platform team to architects to various other things in the governance, security usually stands alone, where we say that it has to be all connected.
What do you think about the responsibility boundaries around generated AI and agent tech systems?
Where does it lie?
When something goes wrong, when it's when it hallucinates, whom should we blame?
I mean it's a good and interesting question.
And I had to speculate now because of I don't think the systems I'm part of the designing and implementing have been running long enough to sort of really comprehensive answer that.
But having said that, if we go back to architects, because architecture is the most important profession on the planet, right?
If we go back to architecture, architecture changes quite a lot.
And I think that there are going to be certain responsibilities in this entire new operating model that we are sort of implicitly discussing here.
And let's start up top, for example, business architects.
They will be responsible for it's not so much about capabilities and those things and more.
They're going to be responsible for the policy, they're going to be responsible for the anatomy of the policy and the structure of a policy.
Can these different policy types and these policy instruments can we capture the essential business rules of the organizations using these constructs?
And if we can't do it, we have to sort of change the spectra.
So business architects are going to be intimately involved in this sort of interfacing between the business and uh energetic AI.
And this is similar with other with other roles.
My hypothesis is that I think the business architect is going to be the most impacted.
I think the second one is going to be the data architect, because ultimately data is everything.
And it's okay if we had a if we have a proof of concept or a power if it's small, as we are opening the ecosystem or we are allowing other data sources coming in.
If that data is not high quality, that is, if it doesn't fit the ontological and our semantic layer, then things are not going to work.
Yes, that's a space which needs to evolve uh further with the the boundaries that we have created, and of course the conveyors law, which always come in the picture here.
What about the agentic?
Now I know you are uh we have written a lot about it, and even in the whole talk you have spoken about it.
With agentic, are we solving the real problems?
Are there any problems which because one day I and one architect had very good discussion, and we were saying that while our uh managements are saying go solve the problem with the agentic AI?
What is the problem?
I mean, everybody's putting a solution to it, but what is the problem which agentics solves?
And what is the problem which is not meant to be solved by a genetic AI?
I think the first problem is mindset.
I think that if we are going to play in this AR sampit, we cannot take the old world with us.
It's not designed for autonomy, it it just breaks.
So I think the mindset is really, really important.
I mean, perhaps that is all it is.
If we understand that we are turning the world upside down and we are letting go of logic and we're going to place logic with autonomy and we are put in autonomy, well, in order to control that, we're going to have boundaries.
If that sinks in deeply, I think that we have solved 75% of our problems.
Because did you get everything?
Once you understand that, did you get the design language?
I mean, you had to design your goal, you have to design your authority, your policy, your scope, your risks, semantics, and evidence.
It also gives you the governance language.
The goal also provided governance, authority is also provide governance.
I mean, your delegations of authority, for example.
In my perhaps very simple mind, just this very concept leads to so many answers to some of these really difficult questions.
I mean, brittle systems.
If you're trying to build an AI agent and you make it deterministic, it will always be brittle.
It's not meant to be deterministic.
Generative AI is not meant to be an if-then-else construct.
It's not meant to do that.
It's not meant to be that.
It would it would always play out.
It would always be brittle.
Yeah.
Well, this whole answers will evolve with time, but yeah, you said it right.
It's it's not if an else problem, which most of the teams these days uh uh unfortunately start thinking, but that's that's one correction we can do.
One thing to keep in mind is that we are, of course, in the very beginning.
And when we are thinking about the genetic system, we are thinking about frontier models and chat GPT and Gemini and and all of those things, they are very, very expensive to use.
So I think that one of the big changes in this is going to be not even using the large, I mean they're all generative AI, but rather using the large language models, we also use the small language models because there are when you're building these multi-agent systems, for example, these 33 A bit systems that we designed for this customer.
We don't want every one of the 33 agents to go hidden frontier model and start incurring a lot of costs.
So it's a question of also understanding the system and understand how to design it and what components to use and what kind of LLM they're beginning to use in what circumstance.
And I think that if we have access to the very low-cost small language models, I think the use case of AI is going to be bigger again.
So again, I can't think of any any areas where where I think immediately we should absolutely not go there.
Yeah, that's a good advice to uh keep.
I would uh tell people who are listening to this.
We are relying so much on Gen AI and LLMs because Lama came out, then so many models came out, and then the whole world started drifting towards Gen AI.
Is that the only piece of research which we can leverage on?
I'm the wrong person to ask because I think I have an LLM addiction myself.
And I think I use it differently.
I don't I don't use the LLM to write emails and things like that.
I write by an emails, I'm happy doing that.
I have my own style, I speak with an accent, I write with an accent, I'm quite happy with that.
To me, what the LLMs are really useful for the way that I use them is that they're really, really good at helping you connect dots.
So I create a lot of thought leadership, for example, around I have a lot of ideas, I create a model, and the best way to validate the model is to take it into the into the LLM and say that this is what I think.
First you validate it, you know, what you think, and quite often they come back and tell you it's really, really good, of course.
We know how to write.
But then you can do some deep research and you go in, okay.
So what exists in other parts of the world, etc.
And then how does you relate to other thought leadership, I have to be a locked, etc.?
And when you start attacking something from a number of different angles, it's almost like you see a new ontology.
So you have an idea and you see an ontology of your of your idea expanding.
I think that's really cool.
And that is when you can just create.
I mean, I'm a consultant, so this is this is what I do for a living, and the fact that I have a tool like that in mind that my disposal is sort of increasing my productivity drastically.
If you're using it seemed to answer questions, and if you're surrendering your own thinking, I mean your own free will, your own credible thinking, I mean, that's clearly not good.
That is really misusing the LLM issues.
You still have to think.
Yeah.
And uh with that, we are at the end of our conversation.
It's it's a long time we have taken.
I think uh one clear takeaway from my end on this discussion is that we definitely need evolutionary systems, evolving systems, emergent behaviors, and the new gardens and the new architectural practices.
And the one thing which I've which I've started saying, stating to various teams even more now is that architecture is required much more now than ever before.
Absolutely.
As a matter of fact, I'm I'm telling people, I mean, I'm an enterprise architect.
I'm telling people that enterprise architects, they're not optional, new architect adoptional.
I think the real difference is that technology architects, I think we always just need solution architects, everyone has them.
Data architects, almost everyone has them.
It's really the business architects and then enterprise architects, they are going to become essential.
Because the enterprise architect is looking after the entire ecosystem that we are talking about.
And the business architect is going to be this critical new business interface between the business and your agent system, you know, all of those translation layers.
So I I think it's a great time to be an architect.
It's a fantastic occupation going into AI.
It's going to be super exciting.
Absolutely.
Completely agree.
In fact, uh the responsibility increases much more where first architects need to be doing the right job and then bringing our engineers, senior, junior engineers to that level where they start connecting the systems and start thinking in systems that where my work or my wipe coding can impact the organization or the user or the function and where it couples more where it shouldn't.
All these things are important to keep in mind.
With that said, it's a good segue to end this.
Any last thought, Jesper?
I'm just gonna say the boundary.
Remember the boundary.
With the boundary, and before that, create that boundary so that you you stay in control of things and don't let things control you.
All right.
So with that said, thank you for joining us, Jesper.
It was really a good discussion in terms of thinking.
We may not be able to answer all the problems today existing, but we have thrown certain ideas for sure, which are uh which deserve to be listened to and which needs further work to be done in those areas.
So thanks a lot for joining us today.
Thank you, Sama, for having me.
Have a great time.