# Securing AI-Driven Product Development

**Podcast:** Product Momentum Podcast
**Published:** 2026-09-30

## Transcript

As a product person, we're often told, hey, go and use AI.
But maybe before you start typing all of your innermost thoughts in, pause and make sure the tool you're using is actually acceptable to your company.
We have all these great ideas.
We're feeding them into AI, right?
What's happening when that data goes in?
Consumer-grade material.
What's going on behind the scenes?
And why should I be concerned or paying attention to that?
Yeah, great question.
Well, if you put confidential business information in a public AI tool, don't assume it stays private.
It's likely that that data is being used to improve the model and it's leaving the control of your organization.
It doesn't necessarily mean the model is bad.
It just means that the usage tier, the data handling and the privacy posture is not suitable for.
business sense, sensitive business information.
Why does factoring in security early actually make us faster in the long run and not slow us down?
Well, because the data supports it.
Not only does it demonstrate mastery to be able to open discussions about non-functional requirements like security and performance, good architecture, that type of thing.
But the work has actually been done to demonstrate that this is true.
IBM did a study.
a number of years ago but they found that each stage of shifting security leftward in the product development process um improves the quality in ways that decreases the price and increases the speed.
I'm thinking back to the threat modeling training that you've done for the product team, Jonathan.
So we need to be thinking about that in like the initiation phase and then the discovery phase before we get to develop and deploy, before we actually start building something.
It turns out like security information about vulnerabilities and stuff like that, it's so black and white that the models have been trained on this.
What happens is The level of expertise that you're granted by using AI to do security work in the design phase is so good that there's no reason, there's almost no reason not to use it for you guys.
Like it gives you a superpower.
The other thing to think about too is there's two types of security problems in a product, right?
One are engineering problems.
We consider those bugs.
But security design problems can't be fixed as bugs because they're not bugs, right?
They're just part of the feature.
And the name AI model might bring that up as an identified problem with the way the products are being built or an unhappy path, right?
We call it an abuse story.
Product Momentum is excited to host the latest installment of our Topics and Products series, where we bring ITX's in-house experts on the podcast to tackle tough subjects and give a little insight into how ITX delivers user delight and promotes product advocacy.
Today, with our Vice President and Director of Security, Jonathan Coupel, and Developer and Data Engineer, B-Song Tabin Yang, we will be addressing this rain cloud that's kind of like hovering over this new age of product builders and UX engineers.
So now that product and UX roles are being asked to deliver more with AI tools, what are we missing?
What new risks are we taking on without the time or training to handle properly?
What assumptions have we made that we are, like, consciously or not, ignoring?
We're grateful to have Jonathan and B.
Sung here to help us navigate this topic.
And we're going to dive right in.
So, Jonathan, when a product team brings AI into their build process, what's an assumption about that AI tool that they don't realize is putting them at risk?
Yeah, that's a great question.
Our clients make a lot of assumptions about the tooling we start using, and they figure that we're the experts.
So they're going to figure that we are kind of doing the right thing.
There's a couple of things out there that is important to consider.
One is hygiene, right?
The client is assuming that we're using an AI tool in a way that's still protecting their intellectual property.
i.e.
it's not in training mode.
And it's important for us to make sure that we do the diligence to check to make sure that any AI tool that's integrated into our existing tooling has that option activated or even available, right?
Sometimes the vendor says, oh, we've got AI in the product now, and they may not necessarily go to the extra extent of actually letting us know how they've done the build or how they've done the integration.
And since we're actually developing a new product to bring to market, we want to maintain our clients' competitive advantage by making sure that we cross those T's and dot those I's.
I think the other one that's important is they're going to assume that security is baked into the tooling, right?
In the same way that when you buy a house, you assume that the fire department is going to come when it's on fire, which may or may not be a reasonable assumption.
they want to assume that we're doing the security thing.
And that may or may not be the case.
I know AI is kind of dumb and you have to be really clear about how you declare your requirements as you're going into the design process and telling the AI to include security features as part of your build can't be left as an assumption.
It has to be declared.
I think, yeah, you're saying something important there.
And right there, I think there's another piece here, right?
As a product person, right?
We're often told, hey, go and use AI.
But maybe before you start typing all of your innermost thoughts in pause and make sure the tool you're using is actually acceptable to your company, right?
Like maybe don't go use free chat GPT and enter all of your innermost secrets.
Right.
So to that end, you have the song, right?
You have developer in your title.
You are doing this on a day-to-day basis.
Yeah, I love to hear, you know, a concrete example of product team, right?
We have all these great ideas.
We're feeding them into AI, right?
We got our competitor analysis.
From a data standpoint, right, what's happening when that data goes in?
Yeah, consumer-grade material.
What's going on behind the scenes, and why should I be concerned or paying attention to that?
Yeah, great question.
Well, if you put confidential business information in a public AI tool, don't assume it stays private.
It's likely that that data is being used to improve the model and it's leaving the control of your organization.
It doesn't necessarily mean the model is bad.
It just means that the usage tier, the data handling and the privacy posture is not suitable for sensitive business information.
You should be using an enterprise grade AI tool, which offers business-grade privacy guarantees, such as data retention and data isolation, admin controls and logging.
And it doesn't train on the company's customer's data.
And it's built to handle business-sensitive information.
I remember having early conversations with Jonathan about like, what free AI tools can we use?
And the answer was a very clear, none of them.
And if you are, or if any of your team members are, make it stop now.
Right.
So I liked it.
You called that like enterprise grade, you know, enterprise grade, like licenses and models.
Right.
Yeah.
I think there's like an analogy here.
It's like, you know, you're, you're trusting information to someone.
Like, is this someone that you think is going to go blab to everybody else, your secrets, right?
Like, yeah, think of it that way.
It's like, do you actually trust who you're talking to?
Yeah.
And public AI solutions are, you know, mainly for convenience and broad adoption.
But you have to be mindful what type of data that you're putting in there.
Absolutely.
Right.
So, Jonathan, I have one for you.
I think product people, again, you're always told to go fast.
And, you know, having those conversations with compliance and security, oh, you're slowing us down.
You're slowing down delivery.
But hey, but also use ad, go fast.
Why does factoring in security early actually make us faster in the long run and not slow us down?
Well, because the data supports it.
Not only does it demonstrate mastery to be able to open discussions about non-functional requirements like security and performance, good architecture, that type of thing.
But the work has actually been done to demonstrate that this is true.
IBM did a study a number of years ago.
We could probably include a reference to that study in our show notes if you want to do that.
But they found that each stage of shifting security leftward in the product development process improves the quality in ways that decreases the price and increases the speed.
Their estimate was that by addressing a security concern in design makes that particular feature, that security feature, about 30 times cheaper than fixing it in production.
Right.
So every delay that you introduce to wait to deal with security, right?
Like if you push security off into QA, it's probably about 20 times more expensive than simply addressing it in design.
And then once you put it into production, it's even more expensive because then you actually have to take it out of production again.
You either have to roll it back or you have additional costs that are introduced to the client or the client's customers at the end point.
So for the customer experience, yeah, there's an increase in time and design, but it pays back with interest.
I'm thinking back to the threat modeling training that you've done for the product team, Jonathan.
And you've always said that these are the things that need to happen earlier in the process.
So really, even though we're using AI now, nothing's changed, but we're just reinforcing that priority, right?
Like we need to be thinking about threats.
You said shift left.
So we need to be thinking about that in like the initiation phase and in the discovery phase before we get to develop and deploy, before we actually start building something.
Have we thought about the actual threats that we're introducing, right?
Yeah, and the cool thing is like the training data is so clean for security stuff.
Like there's not a lot of bullshit out there on security information about software products, right?
Like there is with like...
information about food or nutrition, right?
If AIs are trained on what's in the world about nutrition, you're going to get rando stuff, right?
Because that's what they're ingesting.
It turns out like security information about vulnerabilities and stuff like that, it's so black and white that the models have been trained on this.
What happens is the level of expertise that you're granted by using AI to To do security work in the design phase is so good that there's no reason, there's almost no reason not to use it for you guys.
Like it gives you a superpower that you don't get from a lot of other types of ideation assistants because just the quality is so high.
I think, you know, as we're talking about, you know, moving the security analysis forward.
Yeah, it comes to something that Sean and I have talked with in other episodes, the idea of the premortem, right?
And I think as product people, sometimes we're so focused in a premortem on how is a thing going to fail from a usability standpoint or from a business value standpoint, but we should absolutely be baking in how is it could have failed from a security standpoint, right?
Because it might be usable.
Business values would go down if you've got a huge security vulnerability baked into it that you're fixing, you know, at the end.
you know, 100x cost.
Right.
The other thing to think about, too, is there's two types of security problems in a product, right?
One are engineering problems.
We consider those bugs.
But security design problems can't be fixed as bugs because they're not bugs, right?
They're just part of the feature.
Like, if you are building a social networking system, you need to think about the possibility of somebody stalking their ex.
in the product.
That's not a bug, right?
But it's still a part of how you secure the product to make it safer for your users, right?
And the name AI model might bring that up as an identified problem with the way the products are being built or an unhappy path, right?
We call it an abuse story or a misuse story.
You're saying you are actually...
building AI tools, right?
So you are using AI to build tools and you are building AI, you actually building AI products right now.
From your seat, what changes in how you actually use AI day to day once security is a part of your process instead of an afterthought?
Yeah.
So when security becomes part of AI from the beginning, the work changes at the level of everyday engineering judgment.
We stop asking only, does this work?
And start asking, is this safe?
Is it bounded?
Is it audible?
Is it appropriate for the data and actions that are involved?
So that changes everything from the data that we expose to the model, whether we use a hosted or a self-managed system, how we protect against prompt injection, how much autonomy we give to agents, what we log, and where we introduce human approvals in the process.
So security isn't just a governance layer.
It's it materially improves how you build because it enforces clear system boundaries, better failure modes and more trustworthy products.
More trustworthy products is what we are actually I think part of like the underlying theme of this episode is more trustworthy products.
Right.
Because we've been given these tools and everybody's very excited about the speed that we can deliver with.
Right.
I think we, you know, we're in town halls with clients or we are, you know, I'm on the road, I'm on the road meeting new clients.
And like, eventually you get back to like, Hey, actually the real goal here is delivery speed.
Right.
We want you to use these tools because we want to deliver faster.
Not because like we're looking for new and innovative ways to create customer delight.
Right.
Or to like find like new markets to enter or anything like that.
It's just like ship faster.
Right.
So we want.
to accomplish that, but we want to still create trustworthy products.
All right, Jonathan, I kind of talked about, I just kind of went into, you know, when we are even working with brand new clients, they're expecting us to use AI to build products.
So we're saying, we can essentially say that using AI to build products is table stakes now, whether it's actually writing your code for you or whether it's doing some of your initial research or whether it's just doing your documentation, it is a part of your process.
Everybody's doing it.
So from a security standpoint, what's the best practice that you think is getting skipped as teams adopt AI really quickly?
Yeah, that's a tough one.
I think the use of the AI might lead us to not be skeptical of the happy path because we're assuming that the AI is taking that into consideration, right?
I mentioned abuse or misuse cases as a formal way of declaring that, but we're assuming that the AI is taking into account.
I remember seeing another study.
They suggested that by just adding the word securely to the end of a prompt, improved the quality of the code immensely in terms of adding some additional secure code when somebody was actually doing the code build.
It's the same thing here with design, right?
Like, how would we build this feature?
No, how would we build it securely and perform it, right?
So just taking the time to remember that we need to call out those parts of the product that are important that nobody thinks about bringing up because they assume that the AI is doing that work for us.
And it's not, I mean, it's an idiot savant, right?
Like, it's only able to do well what we tell it to do.
I love that.
I think you hit upon something that's really important across the board in product development and it's like a knowing knowledge bias, right?
And if you're assuming that something's being secure and you're not explicitly saying it, then like that's going to cascade down.
So I think that's great advice for folks listening, you know, prompting.
Make sure you explicitly say do it securely.
You're going to get a much better output out of it.
Yeah, even as simple as take.
you know, take a security framework or take the compliance rules of the company, feed them in as additional inputs as you're doing the build and say, in addition to taking into account these compliance or security constraints, let's talk about how to build out this feature, right?
So we don't even have to say it out loud.
We can actually document it as like an incoming markup language file that we use when we build everything for the customer, right?
And then it becomes...
an implicit part of the way we do that work for that client.
Right.
So in your prompt, right.
Mention securely, include your company policies as context and also maybe line and don't get me fired as part of this.
I guess I'm right.
I am curious, you know, from your standpoint, I know you're in there day to day.
I've talked about how you're building products.
What's something that you've noticed that teams are no longer doing with their own processes now that AI is part of kind of their day-to-day evolution?
From my perspective, what I've seen reduce is the work that was done before code arrives.
AI makes it so easy to develop complete code unit tests and architectural suggestions and even documentation that...
Teams can go straight from idea to artifact and without spending time on, you know, core software fundamentals, such as identifying edge cases, validating assumptions, and trade-off analysis between, you know, components.
So it's not that the AI output is bad.
It's, you know, teams can quietly skip these steps.
that are used to create engineering judgment.
So, and another thing I've seen is, you know, deeper code reviews reduce.
So like code reviews now are mainly like this little plausible because the code is there.
It's complete.
It has the test.
It has the documentation, but we're not really asking ourselves, you know, does this fit our system?
Does this fit our maintenance standards in our security model?
And, you know, that's the bottleneck shift, like the volume of code that AI can increase.
AI has increased the volume of code output.
But if our review discipline doesn't increase with it, then we get faster production of unexamined work.
Yeah, so I'll put it this way.
AI didn't reduce the need or didn't remove the need for software discipline.
Instead, it increased the penalty for skipping them.
And the bottleneck has shifted from code production to actually verifying the reasoning of the output of the code, verifying the reasoning of the model.
And teams who do that well are teams who kept, you know, the software disciplines while accelerating execution with AI.
I think, yeah, you hit upon something that...
that video kind of keeps me up at night from the AI world standpoint is how easy it is to have AI outputs that look amazing, right?
Everybody is asked to go a million miles an hour and it's so easy to be like, yeah, right?
That output's great.
Let's roll, right?
Yeah.
So you mentioned it from the code standpoint, we need to be more diligent as the outputs are getting better from a product standpoint, right?
Like we talk about how, hey, AI might be giving you decision recommendations.
It's still your job, right, to make the decision.
Jonathan, Bessonio, for either of you, how as lowercase p product developers, can we fight kind of back against that pull to let the AI do the thinking for us?
Well, I mean, one of the simplest things to do is remember that just because you can do something doesn't mean you should, right?
So every time an AI...
recommends a particular design choice, it's up to you and your judgment, right?
And your own experience to decide whether or not that's a good design choice or whether or not you need to reject it and ask it to try again.
Because I've seen a number of times where an AI will provide something that's properly engineered, but a bad way to do something, right?
Like it would work, but why would you do it that way?
Right, so...
I think human judgment and that's where, you know, people still are important and people still matter and people with your experience still matter is that bringing that judgment to bear on the work that this, you know, intelligent intern, so to speak, as is bringing to the table.
Yeah, I certainly agree more.
Based on you mentioned something kind of like my last thought before I want to get some takeaways, you mentioned edge cases and in your last answer and I.
I had the privilege of working with like the king of finding edge cases at ITX and that's John Rotes, right?
If anybody could find an edge case, it was him.
But the reason why he's able to do those things is not just because of his technical aptitude and his product skill, but it's because he knows, took the time to know the product so well and the customers, right?
And in this game, you know, whoever knows their customers better wins, right?
And it's also able.
to readily identify edge cases.
And I don't think that we can trust AI just to be able to figure out everything that could possibly go wrong.
That's still got to be on us, right?
Yeah.
All right.
So this was an amazing episode.
Thank you so much for being here.
I've got a couple of takeaways.
So one was from one of the original answers, just kind of like know the guardrails that are actually around your AI integrations.
Never make assumptions about security.
Free tools are not for work, right?
If you are using free tools for work because they work better than maybe the tool that's been provided or you're not able to have or you haven't been authorized to use AI tools yet, it's now time to have that uncomfortable conversation and use your product skill to actually make the case for why you want to use these tools.
Right.
I kind of had this thought while we were while we were talking.
This conversation isn't really like inspire fear is really to like use these new superpowers responsibly, especially like in our game.
people that build custom software and in our consultancy, like our reputation is our product, right?
So we have to use these tools responsibly.
Otherwise, our business doesn't survive.
So Jonathan, this is based on a quote that you provided.
I liked the idea of thinking about security issues as a feature that you introduced.
Like you are directly responsible for introducing a security flaw as a capability to a bad actor.
So I thought that was a really cool way to think about it.
And then finding a way to regularly introduce security language into your prompts.
And then lastly, the last takeaway was you are still accountable for all product and design decisions, regardless of who built the code or who actually introduced them into your product.
So this was an amazing episode.
Thank you so much for being here.
I had a really good time and I'm looking forward to, we're going to, we've got a couple of...
episodes in this series planned sort of around using AI to build products.
So we're excited to dig deeper into this topic area.
Thanks so much for being here today.
Thanks for having me.
It's a ride to be here.