# AI Engineering Strategies For Modern Product Builders

**Podcast:** All Things Product with Teresa and Petra
**Published:** 2026-05-19

## Transcript

Hi, folks.
This is All Things Product with Petra Wille.
And Teresa Kortz.
And we're so happy you're here.
Teresa, in one of our last episodes, we discussed that nowadays everybody could become a product builder and code more and work more on the actual product.
But maybe not everybody wants to.
But rumors that I heard say that you are a full-time engineer now.
So maybe you can share a bit more about that journey with us.
Yeah, it kind of happened accidentally slowly over the past year.
But I would say I spend 90% of my time.
Well, I still write a lot.
So maybe like 60-40 is probably more accurate.
I probably spend...
60% of my time literally doing engineering work and probably 40% of my time writing.
And engineering work is that in terms of really working on a product?
Yeah.
So it started with the interview coach.
I've written and shared a ton about me just building my first AI product and learning.
And it just grew from there.
I'm trying to think of like...
how it happened.
I built my first AI product starting in March of 2025.
And then I kind of just kept tinkering and playing, but nothing that was really real.
Like I started playing with my own personal productivity workflows.
That's why I started nerding out on Cloud Code.
But then I started talking to Vistaly and Vistaly makes Opportunity Solution Tree software.
Their founders have been part of my community for a long time.
So I've gotten to see like how they grow, have they've grown and gotten to know them really well.
And they were like, Teresa, you've been doing all this experimenting.
Maybe we could integrate the interview coach into Vistaly.
And so we did that.
That was like a huge step up because I was like, oh, this little toy that I built for my course is now going to be in a production product.
I better learn like real engineering skills like error handling.
But you and I know that you already had them from your past.
No, I did not.
I actually did not.
I had been a front-end engineer in the past, but that's very different from rock-solid engineering in a production environment, make it always work.
And so I literally had to learn, how do you do good error handling?
How do you do automated testing?
When I was a front-end engineer, nobody was doing automated testing.
That wasn't even a thing yet.
I didn't know anything about like a CICD process.
I still barely know something about a CICD process, but I'm learning.
I had to learn about security.
I had to learn about like efficiency.
Like Claude's great at writing code, but it's not very great at writing like code that's going to perform at scale.
That's an area where I've had to like really question it and be like, what if we had 40,000 records?
What if we had 400,000 records?
What if we had 4 million records?
And usually it will design for the like 4,000 records.
And I'll be like, okay, well that doesn't cut it.
Anyway, so it started with the interview coach.
Like I talked to this company, I created this partnership and then I immediately like scared myself.
I was like, am I even capable of doing this?
And I learned a lot and it turned out the interview coach was integrated.
It was fine.
And then we started talking about, I started running experiments about could I do AI generated interview snapshots?
Could I do AI generated opportunity solution trees?
And in the meantime, that was like for Vista Lead.
And then in the meantime, for my courses, I started to experiment with like, well, what else could I add to my courses?
And I built a business fundamentals coach for our new course, business fundamentals.
I recently built an outcome coach.
So you tell it your outcome and it gives you feedback on your outcome.
And I just realized like I am now like most of the time engineer.
And I didn't.
What will you be doing?
The question is, are you running out of ideas and stuff to put into code real quick?
Or is there a long backlog of things that you still would love to work on?
I'm not running out of ideas.
So I'm building on two paths.
So there's the Vistaly path.
So I now have a partnership with Vistaly.
They basically license AI services from me.
Right now, those services are interview snapshot generation and opportunity solution tree generation.
But we're also working on an AI interviewer.
We're working on, I can't share the longer term plans, but there's some like right behind that is a really big idea that I'm very excited about that I'll be able to share eventually.
And that's like the Vistaly path.
And what I love about the Vistaly path is Vistaly, everybody keeps asking me like, why don't you release skills to help people do this?
Well, I love Cloud Code and I love the personal productivity part of Cloud Code, but Cloud Code isn't multiplayer.
So I believe discovery is a team sport and you need a tool that allows everybody on your team to add their interview notes and to add the thoughts to the snapshot and to see the snapshot and to add comments to the tree and to contribute to the tree.
And so like it has to be in a collaborative interface and that's what Vistaly builds.
So the reason why I have my partnership with Vistaly is like, I don't want to build all that stuff.
I don't really want to be a software company.
And so our relationship, I'm almost set up like an AI researcher.
I figure out what the AI can do.
You're the innovation lab to some extent.
Say that again?
You're the innovation lab.
Exactly.
And then I license it to them.
And that wasn't really great because I don't have to worry about SOC 2 compliance.
They already have that.
I don't have to worry about like data compliance with GDPR and the EU.
They already have that.
I just get to like.
play with what's possible, build the AI service around it, and then they deploy in their environment that's got all the compliance going on.
So that works really well.
Then I have this second thread going where I have this really grand vision.
So historically, I've run a training company.
I talked in a previous episode about how I cut five of our seven programs.
Sunsetting.
You were sunsetting them.
And basically, I'm trying to cannibalize myself.
So I think the future of training is you'll have an AI agent that will give you just-in-time training as you need it on the job.
And so I'm building basically a Teresa bot that has all of the skills it needs to be your discovery coach.
But one that is not only based on a 40-minute podcast episode.
No, not one that says ChatGPT, act like Teresa.
One that has access to literally everything I've ever written.
everything I've ever recorded.
That's its foundation, but that's not enough.
That's great.
That's like going to Google and searching what has Teresa said about this.
The reason why I built an outcome coach and a business fundamentals coach and an interview coach, and eventually there'll be an assumption testing coach and an opportunity mapping coach, right?
The reason why I'm building all of these, these are going to be tools for Teresa bot.
So that when you go to Teresa bot and you say, blah, blah, blah, I'm working on this thing.
I have this hard problem.
it knows which coach to invoke to then coach you.
I love it.
And so this is a whole different, so I have like two paths and I'm working on them in parallel, but it's all engineering.
Like it's all really, it's all AI engineering.
It's all really learning context, engineering, prompt writing, orchestration, evals, observability.
And how do you find good enough resources for you to be easy to learn all these new skills?
Well, I started a podcast called Just Now Possible.
I know, I know.
And it's like my cheat code because I hear a team talk about how they solved a hard problem.
And in my brain, I'm like, oh, I can apply that to my work this way.
So that's the first thing is I created like the best cheat code if I found a way to interview some really good teams about how they're building AI.
What's fun about AI engineering right now is that it's like the 90s with the web.
Everybody's learning it together.
There's a lot of people blogging.
There's a lot of YouTube videos.
There's a lot of podcasts.
Slop and a lot of slop.
There is.
So you got to pick your sources for sure.
But honestly, the way that I'm learning is by trying to do it.
And then I'm letting Claude teach me.
So I'll give an example.
I built the foundation for Teresa Bot.
Maybe like...
two months ago.
And I had never done anything with RAG, which is retrieval augmented generation.
I'd never done anything with an embeddings database.
I had no idea like what's the right way to give my agent access to all of my content.
I knew those things existed.
I've been talking to teams about those things.
Through Just Now Possible, I've learned like what's hard about embeddings, how do you have to do it well.
So like I wasn't a total beginner.
but I had never done it myself.
And so what I did was I just started a conversation with Claude and I actually published this conversation on Product Talk.
So if you go to producttalk.org, I have an article about vibe coding best practices.
And this example of how I built Teresa Bot, the full transcript of my back and forth with Claude is in that blog post.
Oh my God.
And I literally just started with like, here's an idea that I have.
Help me think about how I can make this real.
And we just went back and forth and designed an architecture for it, designed like the first mini experiment, which is let's just put 20 blog posts in it and see how good the search is.
And I built the first version of Teresa Bot, which is basically just an embeddings database with a rag step in like two days.
And I launched it inside my Slack community and got real people testing it within like 48 hours.
Just quickly, folks, can you hear how Teresa is still following best practice approaches and still having conversations with real users?
We are not skipping that step.
Not even when we're developing AI products.
I just wanted to get everyone out.
Yeah, it's actually, you know, last summer I took Hamill and Shreya's class AI emails.
I was barely like, I was like a total novice on AI products.
But you know what I love about that class?
I actually think Hamill and Shreya think about AI products the same way I think about discovery.
They've done a really good job of identifying what are the like basic skills or basic habits that are universal and then how you apply them changes a lot of different ways.
And so I feel like I got...
a really good foundation.
The other thing I learned from that class, and this is because I have gotten to know Hamill and we've had a lot of side camp conversations.
I think that I am a data scientist at heart and just didn't know it.
So like, if I think about discovery, like what makes your discovery well is diving deep.
Like what makes your discovery good is diving deep in the messy qualitative data and diving deep in the quantitative data and figuring out like, where's the signal in this noise?
And if you think about AI products, people that just write prompts and release something don't realize this yet.
But if you want your AI product to be good, you have to log traces.
You have to look at your data.
That's Hamill's mantra.
And you have to really dig in and not just like tweak your prompt a hundred times, but dig in and understand what's actually going wrong here.
So I'll give an example.
Yeah.
And then trust your brain with the Eureka moments.
So you need.
Yeah, exactly.
And so like with Teresa bought, I.
to like test it even before I put it in front of users, I generated like 100 questions I thought users would ask and they actually came from my Slack community.
So they were 100 real questions people have asked me in the past.
And then I ran Teresa against those questions and then I looked at what were the errors it made and I like systematically identified the errors and then used that to iterate on the prompt to change the orchestration, to change the context, to improve it before it even saw users.
And this process to me is so analogous to good synthesis and discovery that like, yeah, I'm pretty much a most of the time AI engineer right now, but I don't feel like the work I'm doing is different from the work I've done in the past.
In the past, I was working with interview transcripts and behavioral analytics and support tickets and sales conversations.
And now I'm working with traces.
like AI traces.
But I feel like the process is exactly the same.
And so I think like maybe what I didn't know is that maybe product managers, if you want to be good at discovery, we need to like build some basic data science skills.
And so that's been kind of fun to like uncover like, oh, maybe I always was a data scientist.
Yeah.
Yeah.
I can only second it.
The moment that I learned more about data science, all of my discovery work became so different, way easier.
And innovation was really happening because I was better equipped to surround myself with all the data in forms that really helped me to see whatever we need to build for the users.
So that actually was a data scientist made all the difference here.
So I agree more of that.
I want to come back to something you said earlier, and this is something I'm seeing more on the internet.
So you said, but you've been an engineer before.
And so I want to talk about this because I think some people are using this as like, yeah, but Teresa's different.
I can't do that.
I want to talk about how I was an engineer before.
Yeah, please.
So I have an undergraduate degree from Stanford.
Stanford has a like best in class computer science program.
I did take computer science classes at Stanford.
That alone makes you different.
But, but here's what I took.
I took.
two beginner classes.
Ah.
Right?
I didn't take four years of computer science.
I took two beginning coding classes and the rest of the computer science classes I took were all the math theory side of computer science.
They were not programming related.
Interesting.
So I didn't build...
engineering skills because I wasn't a computer science major.
I was a symbolic systems major.
So the goal of the classes that I had to take were to introduce me to this perspective on the same ideas the rest of the program covered.
So I didn't come out as a college student, an expert in engineering.
What I came out of college as was a designer because my focus was on human computer interaction.
So I had a little bit of technical skills, but I mostly came out as a designer.
The problem is I graduated in 1999.
There were very few companies hiring designers.
What they were hiring was front-end engineers.
And I had learned on my own time, not at Stanford, on my own time, because the internet was a new thing.
I had learned HTML.
CSS was starting to become a thing.
I was learning CSS.
The first company that I worked at had its own proprietary template language, which by the way, the first three companies I worked at had this because this was before front end frameworks.
Like we had to roll our own.
And so when I say I was an engineer, I only did front end template language coding, HTML and CSS.
I never touched a database.
And the whole event back then was like taking a file, your FTP.
And then you put the file on the side.
It was basically taking things live process.
I've never worked in a front-end framework.
I've never touched a database.
I've never ran a SQL query.
Like people think like, oh, it's because Teresa has been an engineer.
Like I can't explain to you how much of these engineering skills I did not have.
That's why I was asking.
So how much of the skills that you already had, how many of the things are familiar?
Because my kind of engineering background is slightly more engineering background, maybe, because I've really studied.
It was still really front-end heavy, what we did back then.
PHP was around, so that's what we use a lot.
And we developed, we did a bit of compiler building and stuff like that.
So I looked into some...
backend stuff basically but that was ages ago and so many things have changed ever since sometimes it helps when ai does stuff these days that i am like maybe i should ask if claude has done this particular step or if he's so it helps me with debugging sometimes but it's not that it's a transferable skill Here's what I will share.
I'm not going to undervalue what I got from my Stanford computer science classes because here's what I did learn.
Even in the very beginner classes, they instill from day one this concept of elegant code.
It's all about how do you take a hard problem, decompose it, create code that's maintainable, that's reusable.
And so like...
I do think I have engineering skills that I learned from my undergraduate degree.
Like, and I learned this from multiple perspectives.
So I didn't just get that from my computer science classes.
I also had to take philosophy classes where I learned how to deconstruct an argument in a very logical way.
Right.
And so I'm not poo-pooing my education.
I actually think the reason why I can learn these skills is because of that foundation.
But I just want to be really clear.
Like I didn't work.
I had a lot of gaps.
And the reason why I was able to do what I've been doing is because Claude can fill those gaps.
Yeah.
You never stop learning.
I think this is what both of us do.
We never stop learning through our entire career.
We always, whenever we feel slightly comfortable, we go look for our next project or our next.
Yeah.
thing that we want to learn or get better at.
And we're quite good in learning new things and picking up new skills.
And I think that is what is a super helpful skill right now.
So whoever's listening, practice to get better at something.
I think this is what helps you to learn something quicker and not being kind of, yeah, thrown off by something coming up like AI.
there's a couple things.
I think that like it's a learning by doing, not learning by reading.
And I read a lot.
Like, don't get me wrong.
I read a lot.
But like, what I love about LLMs now is you can just be like, I want to do this thing.
I have no idea how.
Set me on the right path.
And then you get started and then you tackle another thing you don't know how to do and you can just do that.
And so I want to tie this back to our last episode.
We talked about the myth of product builders and not everybody has to be a product builder.
I'm not sharing my experience because I think everybody needs to become an AI engineer.
I actually wanted to share this because I've been on this path where after I broke my ankle, I decided I was just going to do whatever was of most interest to me.
And I've been pulling this thread.
And it turns out the more I pull the thread, the more of an AI engineer I become.
And that's a little bit surprising to me.
Like I love product.
I love design.
I loved being a coach.
I loved teaching.
And I'm just finding myself wanting to be an AI engineer all day, every day.
And it was literally just finding, like not being afraid of what I don't know and just following the thing that like is fun for me.
Yeah, where passion led you.
Yeah.
And so it's not, the takeaway isn't, oh, everybody should be an AI engineer.
I think the takeaway is, One of the really powerful things of this new technology is it helps us learn things we don't know how to do.
It helps us create things we don't know how to create.
It helps us build skills in areas we wish we had always had.
I always had engineering insecurities.
To work as a front-end engineer and to know nothing about databases, like every job I had, I thought I was going to be discovered as a phony.
Like I always had these limiting beliefs about engineering.
And it turns out I don't anymore.
Like I know anything that I don't know how to do, Claude will teach me how to do.
And Claude is infinitely patient and I have no problem shamelessly asking a million questions.
And like I said, for TeresaBot, I put the whole conversation on the web.
So if you want to see like the things I had to ask about, like I don't really, I know what embeddings are, but I don't know how an embeddings database works.
it's fine to not know those things and you can still push forward.
So I think like whatever it is you're excited about, it doesn't have to be AI engineering.
I think the takeaway is you can just keep pulling on this thread.
And for me, that's just super fun.
And you might be surprised by where you land.
I love that message.
Thank you so much, Teresa.
Amazing.
Thanks for sharing your learnings.
Thanks, Petra.