# Scaling Production AI in Essential Industries

**Podcast:** a16z Podcast
**Published:** 2026-10-18

## Transcript

I did not expect that, but even after coming from shrink theory, I would say the complexity of the problem of working now is probably higher now than it was before.
An AI system that works 90% of the time might look impressive in a research paper, but when you're handling millions of conversations about people's housing and healthcare, that remaining 10% becomes an enormous problem.
In this episode, Elise AI Head of Applied Research Mario Martone joins MTS to explain what it takes to move AI from the lab into essential industries.
Mario spent 15 years in theoretical physics, studying problems like black holes and string theory.
Now he applies that same research mindset to production AI, where his team has to decide when to use an LLM, when traditional machine learning is actually better, and how to evaluate systems operating across millions of real-world interactions.
They get into Elise AI's voice stack, its work fine-tuning open-source models, and how the company has automated workflows that can take someone from looking for an apartment to touring it and eventually moving in.
And for Mario, the intellectual challenge hasn't gotten easier.
He argues that some of the problems his team is tackling in production AI may be even more complex than the ones he encountered in theoretical physics.
Hello, everyone, and welcome back to MTS.
Today, we are joined today by...
Doctor, doctor, because you have two PhDs, so you get a double doctor, Mario Martone.
He is the head of applied research at Elise AI, a company building conversational AI for essential industries like housing and healthcare.
And he has a background in theoretical physics, physicist.
He now leads a team applying the latest machine learning advancements to real world problems.
So Mario, welcome to MTS.
Thank you.
That's great.
Yeah, welcome to the show.
to ask a little bit about your background because you started off being a professor and theoretical physicist, and now you're actually applying all of that learning into applied AI.
And I was just sharing, we were having a little conversation right before we went live about why it's so important to use these skills and apply them, especially to essential industries like housing and healthcare, probably arguably some of the most essential industries.
And so I wanted to just start off learning a little bit about you.
Why that changed?
Why Elise AI?
And what did you take with you?
Right.
Well, I think I agree with you that it's a big gap.
So just to be precise, I mostly worked for 15 years on trying to understand the universe, black holes, string theory.
Things are very, very kind of different from what I do daily on Elise AI.
But I think there is something that actually is quite important, right?
Like we are living in this moment where AI is moving so fast.
And it's really unclear.
what the future will hold.
And so there are extremely open questions.
And so the similarity that I can see, while obviously, you know, working in string theory, things take forever, it's much slower than the pace.
And here it's like, bum, bum, bum, every day there's like a change.
But the easy question of like, the people who are able to like really think deeply about the problems can be very creative, can constantly kind of come up with new ideas, have a huge edge because AI is like a new field every day.
So that's the thing that really...
I think brought me here and allowed me to really have a big impact.
And tell me a little bit about the space.
One of the big things, I mean, I'm a little bit biased because we just had Nina Song on last week at the New York Stock Exchange.
We're talking all about essential industries and why it's so important to actually use this intelligence and get it diffused inside these industries.
Just having stronger models on their own inside labs doesn't necessarily mean the impacts of that intelligence actually gets diffused into the real world, into actual industries.
And in fact, you know, for me, it was like, you know, what one...
an alternative I could have done because I was kind of interested in going to industry is to, for example, try to join one of these like Frontier Lab, right?
But the research that you do at Frontier Lab, it's kind of obviously different from theoretical physics, but it's much more alike.
Like you're working on this big problems.
But one thing I really wanted is to have an impact in the real world.
And I think Elise is like such a unique company in that respect.
Because, you know, you can have these LLMs that are getting growing capabilities, but we are facing every day with the real challenges of what it means for having like consequential decisions.
So we actually, the decisions that the LLMs make for Elise, for our products, they're not just like some random things that appears on the screen, but like drive impacts on people's life.
And that could be very positive or very negative.
So that's the thing that really excites me.
And I think it makes Elise kind of such a unique space to be in.
Completely.
I mean, it excites me too.
I was talking, what's unique about your role, so you do applied research at Elise AI, I think right now, Elise AI's tech stack, it includes predicting renewal risks and it also includes managing voice latency as well.
And so because you're at this layer where you're the head of the applied research angle, I wanted you to just tell me a little bit more about this tech stack and what specifically is really hard about these engineering problems.
Right.
Many, many things are very hard.
I would say that you touched upon some of the best, you know, the kind of the best examples.
Voice is, we were very early on on voice.
We've been in it for three years.
We definitely, and that's one of the things I love about Mina and Tony and their vision.
We definitely started when it was way too early.
So we were trying to make it work when the technology wasn't there.
But voice is extremely difficult.
You're talking about doing tens, hundreds of operations in a span of like a second and a half.
And you're also touching upon another thing.
Like we really need to make sure that we understand people's behavior, right?
In a very precise way.
So that's another thing that is unique about the lease is that when you touched upon predicting renewals, we have a mixture of cutting edge AI in the kind of more LLMs frontier base.
But we also use a lot of traditional machine learning because we have a ton of data.
And so for that, LLMs are not really the answer.
And so we have really a full spectrum of tools that we're using.
And we're using it on full force.
So, for example, the predictions of the old that we're using really elaborate kind of neural network that we built in the house, but they have nothing to do with LLMs.
So for me, as a head of applied research, it's kind of a privileged position because I don't have to be like kind of tunnel vision on one thing.
I can really see the advancement of the entire field of AI across the board.
Wait, tell me a little bit more about this advancement you're seeing, especially because I think people try to throw LLMs at all these problems and not...
all of these problems are fully probabilistic.
You need some deterministic angles.
You need a mix of all these different things.
And so you get to have this bird's eye view of just like what's happening at a macro level where the entire field of AI and research is going, and then you get to apply it actually at the company.
So tell me a little bit more about that.
Right.
And then, you know, not only that, but like one of the major things is to make decisions, right?
Or like, okay, we have a problem.
Like this is a problem, renewing, you know, producing a probability for a person that wants to renew their lease.
Should we throw an LLM?
Should we throw like a deep neural network?
Should we throw like a very simple machine learning model?
And I think that's kind of the decision that me and other people have to make early on, right?
And so here, it's like many people who have joined very recently, they just do have one solution for everything, LLMs, right?
Which it's, I understand the feeling because like you, this LLMs can generate great answers and it seems they're incredibly smart.
But then when you kind of like dig a little bit deeper in the surface, Like you see that some of the crap is like not really good.
And there are certain problems like, for example, having, as you mentioned, determinism is not there.
Like trying to read the same behavior given the input, LLMs fail.
And probabilities is also something that the LLMs have really hard time to elaborate.
So I need to kind of like look at a problem, try to understand two things.
Like one, what's the one the most impactful?
And then two, because we also have to ship at a very fast pace, okay, what's the solutions that can really get us almost as close as possible to the optimal outcome, but in the fastest way possible?
And then we have to double down on that.
Can you give me an example about this actual method that you're using within your team?
So you're able to identify what is wrong with kind of the way that maybe traditional products are trying to build these things, especially when there's high risk here.
You can't get it wrong.
Like even if it's 99% accurate, that's not...
good enough.
It has to be 99.999.
And they joke that every nine that you add is another hill to climb.
It's kind of funny because like, you know, I still read some papers from academia, and there you see really the difference.
Like, when they're talking about reliability, they talk, okay, if the LLM can take it up to 90%, and then we say, 90%?
That means for like 60 million phone calls per month, per year, it means like an enormous amount of failures that is totally unbearable.
So you can really see the difference between people who are like thinking about this problem theoretically and people who actually...
pay the consequences of those mistakes, us, right?
We get inundated of like complaints and also like the impact on people's life.
And so what we did, which I think is quite exceptional, and to me at least is what really engages me and really keeps me going, is to build a mixture of expertise, right?
So we have at least, we have top.
academic talent, you know, including like myself, professors of top universities, as well as like top developers.
So like you have a team that has all eyes and so they can understand deeply, you know, from coding to like kind of more academic questions.
And we're trying to use all these tools to build evals, to understand evals is kind of the answer to your questions.
Like, can we, we have these millions of conversations every day.
We make decisions.
Can we understand at scale?
the impact of those decisions.
And then once you start doing that and you're doing it systematically, kind of like the results speak back to you.
So you really understand, you really see that difference.
If you just like try it at small scale, like five or six conversations, all look quite good.
But at scale, when you get to the millions.
then you see that different methods diverge significantly.
Yeah, no, it's pretty interesting.
It's almost like you're applying the scientific method to product design and how you actually build the product.
I 100% agree with you.
And I think, you know, I'm a strong believer that the scientific method is kind of one of the best ideas that the human accounts have ever generated.
I want to talk to you a little bit too about Elisa's AI voice model as well, because you specifically have, you're able to manage voice latency.
I know you guys were working on voice and communication way before.
Even some of the frontier labs were working on that.
But one of the big issues with it is this latency issue because it's not just a software problem.
It's also hardware.
You have all these considerations, especially you guys.
I don't remember the exact number off the top of my head, but in terms of the customers that you have who are interacting, it's like hundreds of millions.
We have about 5 million phone calls a month.
So about 60 million a year, which is pretty hard.
Exactly.
And for those, you can't have this weird latency between how...
voice is actually interacting with these consumers.
And so how do you think about this problem?
I'd love to get like into your head, into your teams.
It's like, how do you think about this problem at Elise AI?
How do you break it down?
How do you figure out what's the first thing you guys should work on to tackle it?
Right.
So again, unfortunately, this is like, so there are two parameters here, right?
There is one, you know, knowing exactly the current stack, like you know exactly what the current stack, I built it mostly largely myself.
So I have a really unique vision.
And another huge advantage that I have to say, we built it.
entirely on our own.
So we have full understanding of every single part of it.
So you have the full stack.
You understand where...
the bottlenecks are today.
What are the steps that take the longest?
And then you are the outward phasing perspective where you have a lot of different providers, different changes, like fine-tuning open sources or newer speech-to-text, text-to-speech models.
So you have a gist of possible solutions and you try to match them with the current stack.
Basically, the idea is, okay, if we obviously have something that is really bad on our current stock and it is a new solution that you can apply, then you go in full force.
Otherwise, you're trying to constantly understanding and prioritizing, okay, given the solutions that the market has built, this piece of our stock can be improved.
And I'm going to say that another thing that we did to speed up that process is that we built a way to experiment.
So we have a very closed loop.
where something new happens, you know, somebody on Slack messaged me and said, dude, like I saw this post on X.
I don't really have to believe it.
I don't even look at the benchmark.
The thing we do is like, okay, let's set it up.
Let's run an experiment internally.
So we can actually see a performance of every model on our own internal benchmarks versus the other ones that you kind of don't trust it.
Wait, that's so cool.
So it comes in on Slack and then it automatically, is this all automated or do you come in?
We do have some Ubuntu loop, but we're working on animating it actually fully.
Wow, that's so cool.
And so tell me a little bit about this research background that you have.
You're, I mean, a professor at some of the top, in London, right?
One of the top universities.
And so tell me a little bit what you took from that entire framework for how you do research, especially theoretical physics research.
And then how does it actually apply, especially when...
with your team?
How do you actually use the same methods internally?
Yeah, because we are able to, so again, the two things is, one, obviously there is a gain of having some sort of technical strength of mathematics and just understanding math and it helps.
And programming, those things are actually useful.
But also, you understand what it's like to come up with a new idea.
You know, like theoretical physics at the cutting edge of...
or kind of humanity in terms of like reasoning and thinking.
So coming up with new ideas, it's not easy.
Like, how do you get an insight that really changed things?
Like, if it was easy, everybody would have it.
And so like, there is a kind of a process.
I think this is very useful to do it and then we bring it back, right?
So for example, you know, we brought this to fine-tune, right?
There was like a big, one of the things that we invested on heavily in the last...
in months or so, is to try open source models.
And so the thing was, okay, now we have the possibility that we try really fine tuning on our end.
So how do we do it?
Where should we do it?
What are the new things that we should really try?
And because we have the background of being able to generate ideas independently, and you know that you've tackled one of the hardest questions that are out there, you're not scared of like tackling yourself.
And so then we set a group of researchers and we started heavily tackling that and we have been able to actually get incredible results with.
training our own models and delivering it to production.
That's very exciting.
Well, thank you for sharing that.
I'm curious as well, you posted on LinkedIn, this other post the other day, and it was talking about brilliant research backgrounds with minimal real-world impact.
And this is also what we were talking about right before we went live.
Exactly.
And so I think what's really cool, especially as you mentioned, and I feel your energy right now, just getting so excited about applying all this scientific rigor, but to real-world and essential industries like housing and healthcare.
And so what are you doing?
What are some of the hardest problems you say your team at Elyse AI right now?
What are some of the hardest problems you guys are working on?
Right.
So I would say I would name in two, right?
And so one is, for example, LLMs are moving fast, right?
So now one of the very legitimate question is...
six months from now, what problems will be solvable with LLMs that are not solvable now?
And then push it to a year ago or a year from now and so forth.
So like constantly prioritizing what are the things that we could solve with LLMs and we could automate.
And again, we're not talking about math problem.
We're talking about real world iterations.
So like you have to be sure that, oh.
There is something that is really hard that is a real huge frictions for like millions of apartments and now we can actually iterate.
So like this is one of the things that we're constantly spending time on and it's very and depends kind of it's a high stake decisions because moving early and moving fast allows us to be in a much better position six months from now.
And then the other thing that I think is extremely difficult is that again, unless it's growing fast, but we're still 600, you know, like a few hundred people in the company.
And we operate at a scale where we automate millions, which we, 1 million phone cards a day, hundreds of thousands of emails and SMSs.
So we really have developed loops which are fully automated, completely automated.
And now this is wonderful.
great, it's glorious.
But on the other hand, you need to really build some evaluations so that you know, like we have tons of conversations that go on without our inputs.
And these impact, again, these are housing and healthcare.
These are not like some random thing.
These actually impact people's life.
There's consequential changes that happen if we make a mistake.
building those evaluations that we are able to really gather with few people, with 10 people, like they're able to like get an understanding every day how the conversations are going.
That's a really, really hard task, which we have largely cracked, to be honest.
Give me some specific examples there.
So what's an example of something that you fully automated this entire loop from?
Oh, basically, in the entirety of voice calls is out of it.
For example, like...
The two main loops that we've automated, one is leasing people that are interested in apartments.
They basically can do the entirety of starting from looking at a website and deciding that they are interested in apartments to moving in.
It could be kind of fully, literally fully automated.
For example, we have, you can book a tour, then the AI sends you the code, you can go to the house, then you open the house with the code, you interact with the AI, the AI explaining you the house.
So like you're doing the whole thing on your own, right?
The other thing is resonant for resonant, like calling and having issues.
There, there's still a gap that we've not been able to automate because we need robotics.
to fully automate that because, you know, some of the fixes require actual human labor.
And so LLMs can't really do that.
But we have automated a large chunk of that because we can now, you can now, you know, submit a work order to tell you, like, fix this thing.
The AI is capable to understand whether or not it requires a fix immediately.
Then escalates it, finds a vendor, calls up a person.
And then the last piece is a person, actual physical person, coming into your house and fixing it.
Maybe we'll get to like automate the whole thing in a year too.
Wow, that's pretty interesting.
So I want to know a little bit about what's next for you guys.
One of these big, this big conversation I was having with Mina is around this vertical versus horizontal scaling.
I think what's unique about Elise AI is vertically down the stack, especially when it came to housing, because that's where you guys started.
fully went through every single problem, especially, except, I guess, the robotics physically going into your home.
But you'll get there.
And now you're applying it more horizontally.
So now also going really deep down in health care.
What's next?
Are you thinking of another industry or going deeper on this vertical angle?
Yeah.
So we definitely...
Right now we have a handful.
I mean, healthcare, housing, it's like 40% of the GDP in the US.
So, you know, we have a rule to grow a little bit.
And especially we do find that our best way to...
build AI that matters, right?
Is because, you know, one of the things that's unique about Elyse is that we have these incredible things.
Like we have theoretical physicists like myself who have spent 15 years thinking about, you know, looking at the stars and thinking about black holes.
And combined with, they've acquired, combined with like very deep domain understanding and business knowledge, right?
So we really do have this unique.
mixture of skills.
And so we want to continuously move that way.
So we want to be driven by what is next big bet based on what our clients say.
So one of the things, for example, that we are now facing very pretty heavily is to really cut down the cost of running affordable housing.
This seems like a massive problem because we are finding that Affordable housing is a hugely complex, yet extremely impactful and relevant for especially the component of the population that has the least means, right?
The ones that is most vulnerable.
And so we really want to be able to make them more affordable.
And it's extremely complicated, right?
It's basically not automated at all.
The amount of tech depth that is currently in the industry is nearly non-existent.
And so we are really aggressively trying to, and it's a massive component of the entirety of real estate.
So that would be massively impactful if we could bring down the cost of football housing significantly so that will impact consumers immediately and make their lives much better.
I know, that's pretty interesting.
Some of the hardest problems attract the top talent.
And one last thing I wanted to ask you before we go, I know your team is hiring and I wanted to just better understand what you're looking for.
You said earlier that you have a mixture of people with many different backgrounds who have a mixture of skills.
You come from the academia world.
I just wanted to better understand, like, what are you looking for when it comes to how you grow your team?
So the reasons why we are mostly hiring, especially on the technical side in San Francisco and New York, is because We want the best.
I mean, we're pretty high standards.
And the two main things, we want best engineers.
There's a ton of engineering that needs to be done, and we are so short on that.
And in particular, in San Francisco, our second hub is growing fantastically well, but we need more.
We just can't grow hard enough.
And then secondly, we love, in particular me, obviously, researchers.
And the background there is people who...
have a very strong research background, but they really have this fire in them to wanting to use their ideas, their depth of thought in actual impact.
So we want, these are kind of very difficult, kind of unicorns to find.
People who have exceptional academic background, but yet they are really interested in driving that into real world.
And so researchers with a clear focus on impact people's life and make that better.
That's exciting.
All right.
Well, thank you so much for joining us today, Mario.
This is such a fascinating conversation.
I think what's really interesting getting to hear also, this is my second conversation right now at the least, and getting to hear from your perspective, also Minna's perspective, but just the different background and approach that you have coming from that theoretical physics background and then applying rich academia to some of the toughest problems we have now in AI.
And I want to say to the people out there, like, I did not expect that.
But even after coming from string theory, I would say the complexity of the problem of working now is probably higher now than it was before.
Even coming from string theory.
It's pretty high bar.
You hear it here first.
Well, Mario, thank you so much for joining us today on MTS.
Thanks for listening to this episode of the A16Z podcast.
If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review, and share it with your friends and family.
For more episodes, go to YouTube, Apple Podcasts, and Spotify.
Follow us on X at A16Z and subscribe to our Substack at a16z.substack.com.
Thanks again for listening, and I'll see you in the next episode.
This information is for educational purposes only and is not a recommendation to buy, hold, or sell any investment or financial product.
This podcast has been produced by a third party and may include paid promotional advertisements, other company references, and individuals unaffiliated with A16Z.
Such advertisements, companies, and individuals are not endorsed by AH Capital Management LLC, A16Z, or any of its affiliates.
Information is from sources deemed reliable on the date of publication, but A16Z does not guarantee its accuracy.