# OpenAI Funding, Military Contracts, and AI Labor Displacement

**Podcast:** Last Week in AI
**Published:** 2026-03-12

## Transcript

We'd like to thank Box for sponsoring Last Week in AI.
Box's leading intelligent content management platform, enabling organizations to fuel collaboration, manage the entire content lifecycle, secure critical content, and transform business workflows with enterprise AI.
To unlock the power of AI, you need to get your content to your LLMs and agents.
Your business isn't the sum of internet knowledge, your business lives in your content.
So you don't just want to bolt on AI to your existing processes.
To become an AI first company isn't just about automating what you already do, it's about reimagining what's possible.
With Box AI, you can truly leverage the latest breakthroughs in AI to automate document processing and workflows, extract insights from content, build custom AI agents to work on assignments and more.
And most importantly, Box AI works with all the major leading AI model providers, so OpenAI, Anthropic, Google, XAI, and others, so you can be sure you can use the latest AI models with your content.
Box AI will give you the content layer that gives AI the context it needs, while giving your teams the flexibility they need to test and leverage various models for different use cases.
So go to Box.com slash AI to learn more.
Last week in AI, I would like to thank ODSC AI for being a sponsor.
ODSC is one of the longest running and largest communities focused on applied data science and AI.
It started over a decade ago with a simple idea: bring practitioners together to learn from people actually building and deploying models in the real world, not just talking theory.
On April 28th through the 30th, you can experience it yourself at ODSC East 2026, taking place in Boston and virtually.
There will be thousands of hybrid attendees ranging from data scientists, ML engineers, AI researchers, and technical leaders.
You can attend over 300 sessions covering LLMs, Gen AI, computer vision, NLP, data engineering, and more.
You can also go to hands-on training with workshops and boot camps taught by experts from companies like OpenAI, Hugging Face, NVIDIA, and other top companies and universities.
And of course, there'll be a massive expo and networking opportunities, great for startups, hiring managers, and AI tool builders.
It's one of the best ways for AI practitioners and teams to stay ahead of a field and learn from the best and connect with a community.
Go to ODSC.ai slash East and use promo code LWAI for an additional 15% off your pass to ODSC AI East 2026.
That's ODSC.ai slash East and use code LWAI to get an extra 15% off on the number one AI builders and training conference.
Hello and welcome to the Last Week in AI podcast, where you can hear us chat about what's going on with AI.
As usual, in this episode, we will summarize and discuss some of last week's most interesting AI news.
You can also check out our last week in AI newsletter at lastweekin.ai for articles we will not be covering in this episode.
I'm one of your regular hosts, Andre Karenkov.
I studied AI in grad school and now work at the startup AstroCade.
And I'm your other co-host Jeremy Harris, AI National Security stuff, as you know.
And yeah, we're in for it this week, man.
We had a one-day delay and more stuff came out, but but just an insane week.
We've got Alibaba, by the way.
One paper though, I don't think it quite made it into this cut.
It'll be in next week.
But like, you know, that big thing that's blown up on Twitter right now about a model that during training supposedly started mining Bitcoin and shit in its RL rollouts.
There's a lot bunch of other stuff in the structure.
Like, there's a lot of interesting shit happening right now.
And we're gonna have to say AI consciousness this episode, which we don't always have to say.
And it's weirdly becoming more relevant.
So yeah, I guess just buckle up for that.
There's a lot going on.
Yeah, let's do actually a quick preview of what we'll be doing this episode.
There's a couple model releases still.
It's been like a crazy few weeks with everyone releasing new models.
The pace of releases is really accelerated this year.
Then we will have a bunch of follow-up on the Anthropic Pentagon Department of Defense situation.
We recorded just before kind of everything blew up last week.
So there's a lot to say on that.
We'll be actually doing policy and safety before research moving forward, because we often kind of have to rush that bit, and research is much more technical.
So it I think probably makes sense, but feel free to comment as listeners wherever you like that format.
And yeah, it's gonna be a fun, fast-paced episode.
So let's just dive in.
Starting in tools and apps, first story is that OpenAI has launched GPT 5.4 and GPD 5.4 Pro.
As with the trend of this past few weeks, it's a 0.1 version bump with some pretty impressive improvements in performance, pretty impressive achievements on existing benchmarks.
Also a nice bump to the context window.
It has a 1 million token context window, a bump up in cost.
It's kind of more similar to upper end models like Opus.
It makes me wonder if a model itself is bigger.
If we're seeing kind of an increase on the scale side, which seem to have become less of a factor kind of over the last year, and uh yeah, some pretty impressive jumps on various benchmarks.
They highlight 83% on OpenAI's GPT vowel test for knowledge work tasks, which is probably kind of where Onthropic and OpenAI are putting a lot of focus now.
We have really good coding models, and now the initiative is to expand the use of these agents beyond coding to like everything spreadsheets, PowerPoints, emails, everything, and it seems like the models are getting more and more capable on that side.
So impressive bump in version, OpenAI really pumping out the models lately.
This came out very soon after GPT-5.2 and GPT 5.3 was like a couple of weeks ago, I think.
So I can barely keep up.
Yeah, I mean, you know, this is potentially part of you know what happens as you close a lot of the feedback loop back loops associated with building these models, right?
We are getting models that materially accelerate, at least by the claim of anthropic, you know, Dario and so on, significantly accelerate developer capability.
And presumably the models that are using in-house are gonna have you know much larger inference budgets for AI RD than the ones that you know you and I typically use, let's say.
So, you know, this incrementation is potentially a symptom of the singularity.
I mean, it's it's actually not wild to suggest that.
It could well be hard to know, but in any case, they are coming at us harder and faster than they were before for sure.
A couple of big features with 5.4 here.
So one is you can actually kind of adjust course mid-response while it's in thinking mode, so that you kind of if you notice it's going off track relative to what you want it to do, you can kind of go like, oh, hold on a minute.
Like instead of having to do additional turns, basically just get it to course correct, which is kind of interesting.
It's the first time we're seeing that.
Also the first time that they're putting out that OpenAI is putting out a uh kind of general purpose model with native computer use capabilities.
So computer use, this is kind of their answer to basically the whole series of cloud models that are designed for computer use and the early jump that Anthropic got in that category.
That's part of the reason why you need you know more tokens of context to handle the screenshots and and all that stuff.
But also related to that, a lot of refinement in terms of kind of tool use, tool connectors, tool search, all of that is stuff that they've really emphasized in this release.
It's also their most token efficient reasoning model.
So although we're talking about a model that is more expensive on a per token basis, nominally it is using significant fewer like far fewer tokens to actually solve problems, especially, I mean, they contrast it with GPT 5.2.
So you do end up with less token usage, even though the per token cost is higher.
And this does translate as well into faster speed.
So overall, quite interesting.
You know, you mentioned GDP Val.
This is a pretty remarkable model when it comes to the GDP Val score.
So new state of the art, it's it is soda.
It's 83% on GDP Val.
So as a reminder, that means that it matches or exceeds industry professionals, at least as represented in this eval, 83% of the time.
So 83% of the time, when you put GPT 5.4's completions up against you know, these human expert completions, GPT 5.4 comes out ahead.
GPT 5.2 by contrast hit about 71% on that benchmark.
So you know, once you're north of 70%, jumping by 12% is really hard.
You're pretty high up on that S curve.
So getting to saturation and a pretty big leap qualitatively and quantitatively here.
Last thing to mention, you know, the safety side, right?
So you think about GPT 5.3 codex, you know, we talked about that one, as you said, what like last week or something, the week before.
They are treating GPT 5.4 as a high cyber capability model under their preparedness framework.
So this means they expect it to be able to meaningfully increase the cyber offensive cyber capabilities of whatever threat actors that are system card talks about that uh a little bit.
And so well, this translates into a bunch of requirements, like under their preparedness framework, they've got to do a bunch of things.
They mentioned an expanded cyber safety stack, including monitoring systems, trusted access controls.
So this means is so an access control is any mechanism you use to prevent unauthorized people from using the tool or accessing it in any way.
This seems to imply cyber access controls, by the way.
What I'm not seeing here in this list of things is specific call outs about physical security at data centers or corporate headquarters, right?
Which are things that a lot of people have called out as being really important.
We did a whole report on that like last year, but it's interesting to note that that's very much deemphasized.
The emphasis here is on cyber security protocols, which arguably does not really do the the full job that needs to be done here.
In fact, I think quite quite clearly doesn't, but still, you know, they're they're moving in that direction.
I think a lot, you know, a lot of what's going on here is just a feeling of there's no point to even necessarily trying too hard on the physical side just because they're so far behind that maybe part of it.
They've got a whole bunch of new requirements that are kicking in here, and it correlates very closely to all these eval performance performance improvements in the cyber side specifically.
Right.
I think on the safety side, I am I suspect that they have really been investing a lot in the like an API layer safety of trying to catch a misuse.
We've reported I think a week ago or maybe two weeks ago on their report on misuse and they did show various examples of where they were able to catch.
So yeah they mentioned request level blocking that's part of the cyber risk mitigation stack.
On the GPT val as you said they have 83% if you combine the wins and ties except uh against industry experts 70% if you don't include ties that's up from a 50% win rate for GP 5.2.
So now most of the time it's doing better.
And this is a pretty hefty benchmark uh they have real world work from 44 occupations including manufacturing real estate and rental leasing information trade all sorts of stuff so I think this is like a a fairly real evaluation of kind of benefit when used for tasks.
Now it shouldn't be interpreted as these models are better at the job than the professionals.
It should be interpreted as these models are capable of doing tasks that are involved in the job.
So this would mean that these models, when used for non-coding tasks, can now really augment your productivity and have an impact in those sectors, you know, closer to where it's been in coding, which is where it's been primarily because this year Onthropic is also saying this where there seemed to be a lot of potential for impact on the economy beyond what you've seen previously.
And I think we have a lot of indications pointing to that.
On the uh speed side, just to give my take, I think more of a likely the kind of RD side.
I think we also discussed this last week, where the idea side, the research side, especially when going at this speed, probably is less significant.
But the coding side, I think, shouldn't be neglected.
Kind of the infra work to deploy these models to monitor these models.
Like the software engineers are moving twice, three times as fast as they used to.
And that actually has a real impact on being able to deploy these models seem to be at least likely much more based on post training than pre-training.
Kind of the it's very likely that they aren't retraining the full base model.
I think they are doing additional training on previous models.
So that's why this 0.1 version bump kind of indicates to me that they aren't retraining a whole new model.
They're training an existing model some more, probably with RL and things like that, probably also fine-tuning on real data that they get from people using this work.
So there's a real feedback loop in the sense that like the more we use these models in Cloud Code and Co-Oork to do real work, the more data these companies have for training the models to be better.
And I think as you said, like that is a very, very powerful feedback loop compared to a year or two ago when they had to crawl the internet, clean the data, etc.
This is like high density useful data for like the exact thing you want these models to be good at.
So I think this new trend might be here to stay at least for a while, because there is a lot of room on improvement for these real tasks, where the pre-training on internet scale data isn't kind of that useful, probably to be a good agent.
Pre-training doesn't really help you much with that, but combination of RL with training on real in the wild data is very powerful.
And there's a lot of room for improvement still, kind of across less so coding, but across a lot of work.
So we might be in for like a crazy year of very, very rapid improvements and agents, as we've seen kind of so far.
And next story of actually another release from OpenAI, they have GPD 5.3 instant, which both is pretty fast, but OpenAI actually explicitly said that this model might be a bit less cringe than GPD 5.2 instant.
They say at least some users found it overly cautious or preachy.
Now they do say that, you know, in its responses, it can be a bit more to the point, a bit less annoying in tone than others.
It also has greater factual accuracy, fewer hallucinations, 26.8 reduction in hallucination rate.
With these smaller models, faster models, I think hallucination is still a real concern.
The model just spits out kind of nonsense.
Recently, even in cloud code, I found hallucination still happening, which is um kind of I feel like hallucination has felt like a slightly more solved problem relative to two years ago, one year ago, but yeah, it seems maybe not entirely there yet.
So the hallucination evals make it look like it's a solved problem, right?
It's like this is kind of a version of the same problem that we keep running into where our evals now just like straight up like we're off the edge of the map, and and here there'll be monsters.
Like we we can't look at evals and be like, okay, this is where things are at.
I sort of had similar experiences.
Gemini has become basically unusable to prep for last week in AI because the conversations I have with the papers, I gotta tell you, I do some briefing work with this kind of content, and I I caught like there were notes I had from the last week in AI reviews that I did of some papers that I was gonna put in some of these briefings.
And I caught some stuff where I was like, uh that yeah, that seems a little just a little odd, you know.
And I went a little deeper.
I switched to Claude to get some validation.
I was like, holy shit, like this is c it's completely wrong.
And a lot of it was sycophantic.
And so you know I I I thought it was pretty brilliant.
Thank you very much.
And only to find out I had no idea what the hell I was doing.
Fortunately, none of that made it into the podcast.
I did check, but anyway, it's it's one of those things.
So to your point, I think it's like the evals are really struggling to catch what we mean by hallucination in all contexts, and stuff is definitely still slipping through the slipping through the cracks here.
Yeah, I think with especially faster models, uh my kind of feeling is a lot of them try to give you an answer efficiently and quickly, right?
Especially with agents, they have a tendency to overly explore and and waste your time, which obviously with cloud code, you want to make them as fast as possible, but sometimes that can result in a bit of kind of lack of attention to detail when you're dealing with actually answering questions about stuff you're kind of reading on the spot instead of something in your pre-trained data.
I think there's a lot of dimensions of agents that haven't been present with the base models.
Yeah.
Like tool use, right?
Which is entirely post-trained, you don't have that from intranet data or anything.
So we're gonna continue to see a lot of these trade-offs of how efficient are you, how quickly can you get to an answer, how many tokens do you waste, how much preference and accuracy you get.
And the kind of user experience still is lagging in terms of not having to prompt engineering models being able to kind of tweak some knobs and like how careful should you be, how fast should you be, and uh anyway.
A lot to say from the first hand experience of using those models, but for the sake of speed, we should probably move on.
And next uh model release coming from Google, also fast model, Gemini 3.1 flash light, getting an improvement in both cost and speed.
They say this is 2.5x faster time to first token.
And first token time, by the way, when you're using fast models, typically you want them for shorter tasks, at least often that's the case.
You want to have a quick output with uh short output.
So the time to first token can often be the actual bottleneck.
So that makes a lot of sense.
And it also has a 45% increase in overall outlet speed, 360 feet tokens per second.
At that speed, it starts feeling a little more instantaneous, like super rapid.
So a pretty impressive jump over 3.1 flash.
Yeah, I mean, the like time to first token is a particularly important metric, both commercially and operationally.
Yes, operational is like it unblocks the start of your of your reading.
So, you know, if in these shorter usually when you're doing like a straight LLM, not agentic setup, like that's really gonna matter obviously more.
So you can actually start getting value from the system, but the perceived speed is a really big factor here, right?
Commercially, you want the perception of getting your first tokens in quickly.
If there's a you know a ton of latency, then you're gonna feel like all the model is slow, even if once it starts coming, it streams in much faster, right?
So, you know, there's also like for interactive products, right?
You want to just have proof that the system is alive and so on.
So this is a quite a big deal.
It's it's worth kind of delineating, as you said, from just the streaming speed of the tokens on the back end.
But yeah, it's uh you know, uh another solid uh improvement, like overall output speed.
I don't know if you mentioned the 45% increase in overall output speed, 363 tokens per second.
That's a that's a pretty big lift.
I mean, a thousand tokens per second, once you get there, it basically feels like boom, like completely just there.
You're on the spectrum when you're hitting those these numbers though, too.
So yeah, it's to say disproportionately good for its size, compares very favorably to I mean GBD 5.2, say Sonnet 4.6 or something like a slightly older generation of I mean older, like from a few weeks ago, older of anthropic and open AI models.
Yeah, it's much faster than uh you know cloud 4.5 haiku, GP5 Mini, although with all these evals, right?
It depends on reasoning and and so on.
Not that much faster than Gemini 2.5 flash light and also more expensive relative to it quite by quite a bit.
It's like over three times more expensive for output price, but it also does have pretty significant jumps in uh various benchmarks on MMLU video as typical with Gemini, they kind of highlight the multimodal capabilities quite a lot.
And this is still an area where Gemini, by the way, is by far the best compared to especially Anthropic, but also OpenAI.
If you want a model that deals with video, if you want a model that deals with images, I think Gemini is usually your best bet.
And another story on Google, they have released a command line interface on GitHub that simplifies AI agent integration with Gmail drive and docs.
I think the sweepo actually has existed previously and maybe they just weren't wide of it, but it does coincide with this excitement for open call and just agents across the board.
So this would mean that instead of uh complex multi-AI or API processes, it makes it easier for agents to interact with your calendar, with your emails, with Google Docs.
So if you want it to be your little personal assistant, as is I think often the case with OpenClaw for many people, this is a big aid and makes a lot of sense as a release.
By the way, I want to just quickly mention on the story, I forget if we covered this little event on Twitter that was an example of the head of Meta's, I believe, interoperability and and maybe alignment team posting that they messed up a bit, she messed up a bit and had OpenClaw like mass delete emails because of providing access and like telling it OpenClaw, please clean up my emails.
And there was this thing where you know she was like, Why are you deleting my emails?
Stop, stop, and kind of had to unplug the Mac Mini.
And of course, people were a little a little amused and critical that this is the head of alignment at meta running it through this.
One moral of that story is, you know, if it can happen to her, it can happen to anyone.
Another moral of it is I'm genuinely curious what the argument is supposed to be, especially given that and other stories like the Alibaba one, like a few of the others we'll cover this week and that have been coming out that are too recent to cover this week, but we'll look at next week.
I don't know what the argument is that suggests that somehow, let's say we're not going to sample the like very worst possible behaviors and capabilities the AI models have during training and internal testing within labs by default.
Like I'm now like just like waiting for the counter, like if we build an AI system that is capable of doing arbitrarily bad thing X, and I mean like arbitrarily bad, catastrophic cyber attacks, bioweapon design, whatever.
Explain to me how that is not just going to happen by default during the training process, during internal, if it even makes it to internal testing.
But I mean, at this point, I'll say even during RL rollouts and training, possibly, possibly.
Tracking that the Alibaba thing is not guaranteed, it's not locked in.
There we're still looking at like what the evidentiary package is there.
Did it really happen in the way described?
You know, how much of this is marketing and dress up.
Okay.
But we have quite a bit of evidence at this point.
I'm genuinely like, damn.
Like, I I actually need to start seeing receipts from the other side at this point, saying there's some magical reason why Deus X machina God will intervene and prevent you know the crazy thing from going sideways.
I'm actually, by the way, not a doomer on this, quite the opposite.
I think that's quite constructive because it suggests that the first time a system like this develops the capability to do something like that, relatively soon after that, you start to get these kind of like mini, I don't call them mini catastrophes, but certainly the wiping of all the information on a very expensive meta laptop in IP terms, is a mini catastrophe.
So, you know, maybe that creates an incentive to not be absolutely stupid about how we build and deploy these systems.
But damn, I mean, this cannot be a positive update if your story is something like the labs will do the right thing and they'll check their whatever and blah, blah, blah, and they'll do it risk, you know, responsibly roll these things out.
I mean, this is just it's it's about what you'd expect.
Right.
I think on the lab side, when you're in training, presumably your models don't have access to money or like many APIs, unless we are training visits to be like personal assistant things where they have to give you access to email and so on.
But I would expect the labs are a little more kind of responsible.
But if you look on Twitter, there are many examples of these kind of catastrophic things where like databases are going down, money is being kind of destroyed.
There was an example of a Gemini API key being leaked and costing someone eighty thousand dollars, both across kind of small companies and large companies like Vercell, there are a lot of stories emerging, and now with Silicon Valley getting into this hype cycle where more and more people are like, oh, I'm gonna build a zero-person startup where VAI does everything.
We're gonna have a lot of instances of AIs going absolutely wild and doing some crazy shit, and this is just the beginning of that.
And oh boy, it's gonna be funny and painful and everything in between.
And and I'll tell you, during training, I mean this this continuous merging, blending and confusing of training versus or call it pre-training versus RL.
And as we move towards continual learning, by definition, the barrier between training and inference starts to starts to vanish.
I think that that introduces such an insane level of uncertainty as to the propensities and capabilities of these models that I think we actually will see during whatever training is supposed to be in that future, which is coming at us fast.
I think we will see some of this crazy shit.
I mean, there's a version of this where RL rollouts actually do end up giving models access to APIs because real world data is just qualitatively better than synthetic data.
And so race to the bottom, whoever's willing to take the guardrails off the most is going to get the most performant model.
And I'm out of arguments as to why that ladder doesn't just keep going just objectively based on the evidence so far.
But I mean, again, this weirdly it makes me more optimistic and not less.
If you do a little bit of analysis and dig a bit deeper, it actually is a good thing in the sense that these are not superhuman AI models, they're not superhumanly fast, they're not super cheap.
So if you do have catastrophes, you know, it could be much worse.
Like it's not the it's gonna self kind of itself, self-replicate, and like we're not gonna see catastrophes on a society level scale from these models almost certainly.
And this will build that muscle to be like maybe you should like hold back on the ages a bit to avoid that getting back.
That's the positive story that this allows you to imagine at least.
It's I you know, it's if superintelligence is possible, we'll get there eventually, maybe soon.
And if we do, then we'll get all the risks that come with that.
Hopefully, by then we have built the scar tissue and and gotten the reps in.
We may not, like we may easily not.
There's a lot of evidence right now that we're not learning the lessons of the past, but as you said, I think this at least creates the story that we can tell ourselves.
It will fail our way to the top.
Yeah, I think it it only takes one overly excited founder to do wars.
So unfortunately, there's like only so much you can do.
But assuming like the intelligence won't get so cheap to meter, you know, there's gonna be you could only spend so much, and uh that is always gonna be there to kind of be a safety net, unless some very rich person decides to go hog wild, in which case, yeah.
Yeah, I think we could have a whole podcast on that shit.
The sci-fi stories you could write right now that are very realistic are a lot of fun.
Dude, that's the whole news these days.
I feel like the sci-fi is is looking back at the last couple weeks.
Absolutely.
And one more story in the section, we have one from Luma, and they have launched a new set of unified intelligent models and also powered by those models, Luma agents that are designed to do end-to-end creative work across text, image, video, and audio.
For reference, Luma is one of the major companies doing primarily text-to-video, and they have trained now a single multimodal reasoning system with this unified intelligence models.
Uh fun fact, I actually know a couple of people on the team who used to be at Stanford that are very impressive.
So I have a bit of personal bias to think that these are actually probably pretty impressive things.
Certainly they've been working very very hard on the last few weeks.
And these agents could coordinate with other AI models.
They can use Ray 3.4 and models across Google, Bydance and 11 labs.
So that means they can generate audio, they can generate video images, uh kind of anything else.
So we've had also a lot of excitement in recent months and recent weeks about video editing, about creating kind of promo videos, ads, etc via agents and it is getting quite good, right?
That's one of the things where we've seen very rapid progress uh recently well and actually you know some of the examples for what it's worth in the in the tech crunch piece these are heavily heavily cherry picked but they seem pretty cool.
So here's one Luma agents turned a brand's $15 million year long ad campaign into multiple localized ads for different countries in 40 hours for under $20,000 passing the brand's internal quality controls and accuracy checks.
So you know, you're looking at you know 20 grand for that, you know that that that's probably you know a decent deal, you know, if it allows you to save hiring a W-2 or something.
That's that's kind of interesting, kind of compelling if it works.
And I guess that's the that's the number we're missing is like, okay, how did these ads land?
Do we have that information?
We don't in the article.
But anyway, yeah, it's a it's a pretty interesting launch.
Certainly the kind of thing that you gotta do is or one play here is get specific with your target audience and your use case.
Obviously, don't try to compete toe-to-toe with OpenAI or whatever if you're if your your budget is much smaller, but yeah, seems like could go somewhere.
Yeah, and just doing a little bit extra business side analysis.
I think I've often been skeptical of text to video as kind of a useful business expense for various reasons.
But this kind of agency capability will mean that it actually will be more and more company handy for things like ads, for editing, for localization, as you said.
So I won't be surprised if companies like Luma this year will start really blowing up and getting to these like 10 billion, 50 billion, whatever valuations in a way that I haven't in the couple years before that.
Onto applications and business, and we begin with some fun drama, maybe on a scale we haven't seen in a little while.
We'll get to the more serious stories regarding on Tropic and the Department of Defense/slash war soon.
But one of the recent uh developments has been the leak of a memo that Valrio Amade, the CEO, sent to the company now.
Worth noting, it's like a memo, but it's also an internal Slack message, which is more informal.
And this is something that regularly happens at ontopic.
There's these like little mini essays, long posts that Dario sends out.
And in this case, it was a little more kind of direct and maybe antagonistic than usual, with some reflections on what's going on.
This was sent out on Friday, kind of just after a whole bunch of developments with Onthropic being labeled as a security supply chain risk.
We have OpenAI announcing that they made the deal with the department, which basically on Tropic said no to this.
OpenAI more or less said yes to the same deal with some little caveats.
So in this memo slash post, there are some pretty slightly nasty digs at OpenAI saying that they agreed to what amounts to safety theater, right?
With these carve-outs of in the contract saying, like, oh, it's gonna be for all lawful uses according to these existing policies, the memos, which effectively can be changed at any time and don't have any real power.
Also described OpenAI's public statements as straight up lies.
Said that Altman not only that OpenAI contributed money to Trump, but also that they sucked up to him as a wannabe dictator.
But Sam has notably called the employees of OpenAI like gullible on Twitter for self-selection effects, which we think is the worst part of this, like straight up directly being negative towards OpenAI employees, which is you know like kind of a jerk move, honestly.
I'm amused.
That's so funny.
I'm I'm amused that that's like the for you like that crosses the line for me.
I think the other stuff, if you're being honest and real, is fairly like true, honestly.
Like you can take OpenAI's side on this, but the contract, if you read it, calling it safety theater is not unreasonable, and the analysis with regards to the administration and so on is fairly true.
I mean, it's just true that OpenAI has been friendly to Trump in a smart, honestly, very effective move by Altman, worth remembering that Elon Musk was going after OpenAI hard and was a strong ally of Trump.
So, in some sense, Sam Altman doing this was very important for OpenAI.
But on the flip side, you know, Dario pointing out that OpenAI has cozyed up to Trump is as a result very legit.
But this stuff about employees being gullible is much less easy to kind of justify.
Yeah, well, and I I will say if you look at Runes, and this is like insider baseball on the in the Twitter verse here, but if you look at Rune's tweets, so Rune is clearly an open AI, he's an open AI employee in a trench coat, just anonymous online.
Evangelist, you might say, yeah, one of the very public-facing figures from OpenAI.
If you follow on Twitter, you know, he posts quite a lot.
And and I gotta say, I mean, like I used to find him just fat, I mean, I still find him fascinating as a study, but I used to find him fascinating for sort of let's say objective analysis.
It used to be a lot more objective on OpenAI and Sam and all that stuff.
He sort of just become this pretty blatant stand for open AI, like no matter what they do.
Rune's always got some six-dimensional rationalization, and they've been getting more and more strained.
And I I think that more and more when you look at Runes, especially on this issue, it's just like the wheels are coming off a bit, and it's like getting a bit embarrassing to watch, and it which is a shame because I had really genuinely enjoy his content and have enjoyed it.
But yeah, so that there's sort of this this flavor to it where I think the perception from Dario is certainly going to be that open AI employees are sort of reflective of that pattern of kind of getting boiled frog in hot water style, as Sam Altman has become increasingly, in the view of many, opportunistic, as it's become clearer and clearer that he's sort of jumping on things like this in a fairly cynical way that may not be aligned with the interests of the technology of the trajectory of the technology.
You know, this is all it all depends on where you come from, right?
I'm describing the view from Anthropic here, which would be: look, we had a de facto picket line that we were setting up.
We had our our red lines, and here goes OpenAI at literally the first opportunity, just like taking advantage, saying, like, hey, look, fuck that.
Like, if they won't do business with you, we will.
I say this as somebody who incidentally is really concerned about a pattern of potentially private companies disempowering, you know, the US military from being able to engage with China, which has civil military fusion.
There is no difference between Chinese companies and the Chinese military.
They are forced to do whatever the military wants.
That's a massive asymmetry.
And if we do want the free world to come out on top of this, we're gonna have to address that.
I don't know that this is the mechanism.
I don't know that, like threatening to label these things as supply chain risks or invoke the Defense Production Act or whatever is the appropriate or proportionate response.
But certainly there are all kinds of fraught ethical questions here to do with OpenAI kind of swooping in the in the in this way.
And so to the extent that there are open AI employees who signed on, for example, there's this big kind of protest form that a bunch of them signed on to saying, hey, I object to the idea of doing business with a DOW on these terms.
If they're still sticking around, then you know, that's I think the argument of gullibility that Dario is gesturing at there.
I'm not saying it's right or wrong.
I think this is all a very complicated situation with genuinely difficult moral calculations going every which way, but it looks a lot like opportunism.
And I think Sam has has said that on Twitter.
Like, look, reflecting on this, I kind of fucked up from a, I mean, he's not saying that he actually was being opportunistic, but from an imaging standpoint, it really looks bad.
And so, you know, well, what what are you what are you gonna do with that?
That one of the core things, by the way, at issue here is this question of so-called all lawful use.
Essentially, this question that is it the case that the Department of War ought to be able to require their contractors or their vendors to enable them to use their tools for all lawful purposes, or can there be additional red lines?
And one of the key ones here is we talked about it last week, but tracking of US citizens, kind of doing domestic surveillance, anything like that.
Supposedly the language in the open AI contract now with the DOW, or at least had been that deliberate tracking, deliberate tracking is off the table.
Now, that sounds like a pretty weak guardrail if the tool is used for more sort of broad situational awareness or pattern analysis, and it just happens, just happens through correlations.
Oops, to identify individuals.
Well, the Department of War could argue that it wasn't deliberately tracking anybody, but this is just like a byproduct of a different, fully lawful use, right?
So this is all kind of part of the sort of angst surrounding this stuff, which I think is it all depends on what you think of the big picture here, without getting too embroiled in the politics of it all.
Everybody has so many little corners to sneak into and wiggle into that you gotta do a lot of reading before you can figure out how even you want to fall on this one.
Yeah, but there's so much to say on this.
We could do an entire episode just on this entire story, uh, which unfortunately we can't.
But first of all, don't want to single out Rune.
Rune is a cool guy.
I enjoy his presence on Twitter.
And uh to be fair, like this is an ongoing discussion on Twitter with many cases, some people being more defensive of OpenAI, and there is a case to be made for this contract being a reasonable thing for open AI to do.
There's a case to be made for anthropic being wrong, and many people are arguing that it shouldn't be okay for anthropic to put these red lines.
It should like the democratically elected government should be in charge of what you do these models for with regards to military activity.
Now, I think personally, and I get the sense that you as well kind of more or less take on Fropec's side on this, but uh to be fair, you know, there are varying opinions, and this is an ongoing discussion.
With regards to OpenAI specifically.
There's been kind of a sequence of posts and communications from OpenAI that have had various uh reactions.
We can't cover all of them, but there was an AMA with people posing questions.
With regards to the all-lawful use part of this.
I think it's important to understand, and this is slightly opinion or subjective, but we need to remember that this is not a business as usual kind of moment in US politics, right?
Like it's not just the Department of Defense here, it's Hexf and Trump at the head of the Department of Defense.
And what is legal is a very flexible thing right now where executive orders are often used to justify things instead of Congress passing actual laws.
There's been many, many instances of the executive doing things that are just illegal, like that have been deemed illegal by the courts and them just doing them anyway.
And going deeper, and again, this is kind of a personal analysis, you might disagree, but there is many examples from last year where the uh administration went after various sectors of private business, legal companies, uh universities, publications, and effectively pressuring them into submission, into like you do what we want, or we're going after you legally and in other ways.
And I think it's important to remember that I think this is another example of that.
More or less, the end goal is you do what you want, or else, with this uh thing of all lawful use, it's important to remember that first of all, Anthropic was already just trying to argue for the contract they already had, as is, that the department signed on to and putting relatively generous red lines here with regards to autonomous weapons and mass surveillance.
So anyway, again, lots of things you could say.
We probably should be moving on, but let's not interpret this in the regular kind of sane world of politics.
Yeah, I mean, it's I'm I'm gonna stay out of the the the political dimension of this per usual, but definitely is complicated.
Yeah, to be fair, you're Canada-based.
I have slightly stronger feelings of this as someone living in the US.
So yes, I I I legally don't have an opinion on US politics because I'm north of the wall.
Okay, moving on, and another major development over the past week of the headline is no ethics at all.
The cancel Chat GPT trend is growing after OpenAI signs a deal with the US military.
So pretty quickly after the news came out that OpenAI signed on with a department.
The narrative for many people was that Anthropic said no and OpenAI said yes to the exact same thing.
And there was a lot of public backlash on it related again to politics in a way that cannot be avoided.
And it seems to have been a real trend for many people to cancel their Chat GPT subscriptions and to move away to Claude.
Claude rose up in the app store rankings on both Apple and Android to be at the number one spot.
There are some numbers that indicate hundreds of thousands of new signups by consumers and many, many people leaving Chat GPT, not at a level that probably would seriously hurt the business of OpenAI, but given that OpenAI has by far had the lead in brand awareness and in usage for non-businesses, this is kind of a big moment for Anthropic to become known and become kind of uh used by non-enterprise, non-business users in a way that's a very real kind of business impact.
Uh, probably for now staying.
Like the story will move into the background with the department and the military, etc.
But the lasting impact here will be significant.
Yeah, and I I think w when you look at uninstalls, what it doesn't show you is the drop in actual installs.
And and that's the trend that matters most, right?
When you think uh you alluded to it, but the number of uninstalls is just absolutely trivial compared to the number of new of fresh installs.
The real question is not did the uninstalled chat GPT movement affect uninstalls?
It's actually did it infe affect new installs.
Again, we don't know that yet.
But it'll be interesting to see if there's even a a slight drop economically, right?
These decisions don't tend to actually, or these sorts of movements don't tend to actually have that much of an effect.
Though if there were a frontier lab that was more exposed to this, it would be open AI.
They do tend to focus more on consumer than anthropic.
But you know, B2B, which is where you make most of your margin in this space, nobody gives a shit.
You're not gonna have businesses switching models over typically sort of like causes and and protests like this.
But notable though, in the same vein, there is and this is just kind of breaking as about as about an hour ago or so.
OpenAI robotics leader, which is opening the the story there.
It's uh Caitlin Kalinowski had been leading hardware and robotics engineering teams at OpenAI since November 2020 just announced she's left the company, and this is explicitly about this uh sort of DOW contract.
So man, a lot of churn, employee churn as a result of things like this.
It's Silicon Valley, right?
You're gonna you're gonna find a lot of people who are left of center who will tend to have more objections to the idea of a department of defense or war, especially under this administration, because of the politics, kind of forging ahead with with these sorts of things.
So there are a lot of hidden costs here that go beyond just uninstall, but we'll see where it goes.
Yeah, that's a good point.
There's a case to be made that another lasting impact will be, you know, kind of people are taking sides and deciding who to go to work for.
There have been a number of people resigning from OpenAI saying they uh want to be going to Anthropic or just moving on.
Not again kind of significant enough, it's not a mass exodus by any means, but the lasting impact for people's preference in job seeking is seemingly significant.
So man, this really sent shockwaves in a way that is hard to capture.
Speaking of OpenAI, just as this was happening, they announced that they have raised 110 billion in private funding.
This is with 50 billion from Amazon and 30 billion from NVIDIA and South Bank, with their valuation now being 730 billion.
The funding round actually remains open, so they expect more investors to join.
I've lost track, but I think this is like the biggest round they've had.
Like, oh my god.
So insane and impressive that there's still this much appetite to invest in OpenAI, given that like the payoff won't be for a while unless you expect a rapid takeoff.
The economics just don't make sense.
The RD costs will continue to be there, the data center investments will continue to be there.
So profit is not on the table for a while, most likely.
We're back into the hole.
What does it mean to raise 110 billion dollars even?
By the way, the nominal evaluation rate has just increased about almost two and a half times, right?
So the last time OpenAI raised was in March of 2025.
It was 40 billion at a $300 billion valuation.
I think there have been talk of tender offers in the meantime with intermediate valuations.
But here we we really have like a you know 2.4x valuation jump in that time, which is quite remarkable.
So a big portion of the nominal dollar amount of this investment probably is going to come in the form of services and not cash, right?
So essentially what's happening here, and this is the dynamic that these larger rounds take to tend to take, is you've got Amazon and NVIDIA that basically pre-purchase OpenAI's later consumption of their own infrastructure, right?
So they kind of go, I'll give you compute credits that you can only use on my platform.
And so the line between investment and and loan starts to kind of get not blurred.
I mean, it's definitely an investment, but but there's sort of like this, there is this circularity to it.
I personally I don't buy at all the kind of circular, sort of circular economy diagrams of the sort of Michael Burns.
It is more circular than typical funding, to be fair.
But yes, I I think don't interpret this as like they're just giving money back and forth.
Well, well, that's it.
And the thing to kind of realize about this too is depending on the the details of the agreement and the margins that NVIDIA and Amazon will charge OpenAI for the for those credits, like Amazon and NVIDIA are still gonna have to spend the money required to maintain that infrastructure to serve it to OpenAI.
They are going to bear the cost of this.
So in that sense, we're not talking about monopoly money.
We're not talking about, again, all these diagrams that show like you know arrows going in circles.
There actually is bedrock here.
Like these companies are digging deep into their CapEx and OpEx spends to fuel this.
But the optics of it, and and in reality, the sort of the way the way the money flows is going in a circle in a certain sense.
So anyway, it's a web of interlocked commercial partnerships.
And inevitably, in the big short movie that will be made about this era, somebody will take this clip of me saying this right now, and it'll be part of like, here are the idiots who are saying that this is going fine.
So uh OpenAI right now is committed to taking at least two gigawatts of AWS training compute and and then three gigawatts of dedicated influence capacity and two gigawatts of of training on Vera Rubin systems.
That's from NVIDIA.
So, like again, a gigawatt is a million homes, right?
So, so you know, we're we're talking about large fractions of the US power output here.
One thing to note though is Amazon's contingent investment, is what it's called.
It's kind of a weird thing.
So $35 billion of Amazon's investment could be contingent on open AI, either achieving an A of achieving AGI.
Okay, we're there.
$35 billion, contingent on OpenAI achieving AGI, or on making its IPO by the end of the year.
Like, my brother in Christ, how are those both in the same set?
Like that what it essentially like it ties this massive capital infusion, right?
To one of the most philosophically contested definitions in technology.
Who decides when AGI is achieved?
Well, guess what?
Now that's a contractual question all over again.
I thought we just finished establishing we wouldn't have to worry about that anymore because Microsoft and OpenAI sorted that all out.
It you know it's no longer gonna be in their thing.
But now we're back at it again.
So unclear to me, at least from this frame, who decides when AGI is achieved, what the hell that even means.
And the fact that that's being put in the same breath as, or if there's an IPO by the end of the year, kind of vaguely implies uh a sense that AGI at least might be declared to have been achieved sometime, roughly on that order of magnitude, you know, a year or two.
This is pretty insane.
Yeah, this is pretty insane.
I I got nothing else.
Yeah, and by the way, like valuations are not just like look at us, we're so cool and are gonna be such a big company thing.
It matters a lot for investment terms.
So, you know, how many shares are you gonna get?
Things like that, how much dilution are you taking on?
So getting a lot of money, yeah, control.
So getting a lot of money at a higher valuation is saying that's an investors are kind of giving something up, are giving money with less, but they get back for that money.
So this rise in valuation is a fairly big deal.
It signals that open AI is still getting a lot of power in these negotiations and not kind of begging for the money.
Yeah, the multiple is pretty wild too, right?
I think OpenAI now has a nominal gross revenue of something like $25 billion a year annualized.
$25 billion a year of revenue.
Usually, I mean, in SaaS startups, you'll see historically you would see now they're all AI startups, whatever, like a 10x multiple, right?
Something like that.
So if you're making a million dollars a year, your valuation, you know, it would have been back in the day, you'll be maybe 10 million dollars or something like that.
All this has gotten completely fucked because of inflation specific to the kind of angel investment in and kind of later stage market, but generally that's how it's worked.
This is like wildly out of scale, and also that 10x multiple thing tends to be more true for smaller scale companies, and this is at an insane scale.
So this is in order to defend this valuation, you have to be putting a significant amount of chips on the idea of open AI achieving HEI.
That's just there's we're now we're past the Rubicon.
Like none of this makes any sense.
You cannot have a trillion dollar company with this making $25 million dollars a year where you don't have a very credible case that they're going to own a lot of the labor market, just given also their planned capex.
Everything kind of points in that direction.
So this will either either be a you know massive collapsed bubble, or this is as normal as the world is ever gonna be, and things are gonna get weirder.
Now, to be fair, things are generally slightly different with tech, or especially over the past 10 years.
It used to be the case that how much profit you made and how much revenue you have mattered.
Now you have companies like Tesla where the multiples are just insane.
And a lot of it has to do with sentiment.
A lot of it has just like the stock is worth this much because people say the stock is worth this much.
And if OpenAI does IPO, that's another way in which investors get a payout, right?
People buy the stock, the investors have the stock to sell, and they benefit.
Also, private markets are now a thing.
So you can kind of it used to be the case, but you go public and then you get to pay out from having the stock as an employee and as an investor.
More and more over the past five, ten years, liquidity has risen in the private markets.
So you can get actual money, you know, not monopoly money for having stocks.
And that's another factor here where as an investor, you might be more open to investing in a company that may not IPO for many years, or maybe not even IPL ever.
Anyway, lots of kind of context here with regards to how Silicon Valley has evolved in the past decade.
Well, and to be clear, this idea of the sort of multiple becoming less important revenue being less important is something that you do see for perfectly rational reasons every time there's a paradigm shift in technology, right?
Because the idea is there's a land grab, and we don't know most of the value is in capturing more of the territory.
It's not necessarily in finding the revenue now.
It's in acquiring the user base and you'll figure out monetization on the back end, right?
Like this is the classic play.
As OpenAI is proving remains true in the AI age, even if it has to mean going to ads, right?
There, at least you always have that final bastion of monetizability.
You'll find a way to make money off your users.
To your point, companies are staying private for longer now, right?
That that's really the underlying trend.
And that's because of the depth of private capital markets.
So you the fact that we have the ability to privately raise 110 billion dollars, which would have been insane 10 years ago.
Like that just would not have happened.
You would have had to turn to public markets for anything like that.
The fact that you can have this happen today means that you need a uh you know, tender offers or some some mechanism to make employees liquid because companies, I mean, open AI has been private, I mean, you know, not for profit and then whatever the hell, but they've been private in some form since 2015, right?
Since they were founded.
The usual cycle was about seven years from company founding to IPO.
If you're gonna ask your company or your employees to stay, you know, strapped to their equity for you know an additional four or five, six years.
On top of that, you do need to just offer them another another way to get liquidity.
So all these things are entangled.
The the math behind Silicon Valley is changing in a fundamental way.
That that's a big part of what we're learning here.
Next up, a business story related to Alibaba, uh, which we shouldn't forget is a massive player in Chinese AI.
Uh, quite a dramatic development there, there has been a departure of multiple leads from the Gwen team, Lin Junyang.
And they seems a little bit antagonistic.
Like the posts on Twitter for the departures, don't say the usual lines of like, oh, it's been a great time, I'm gonna be exploring, things like that.
I believe the posts were very succinct, like I am leaving by Quen.
Another lead had a similar comment.
So the indications seem to be that there's some real internal tensions at Alibaba.
Uh, this is also coming as Baidu's stock took quite quite a hit.
The general economic sentiment around AI in China, 4B is public companies, by the way, like Alibaba Baidu are more kind of the Google here, not the open the eye on our tropic.
They're public, this matters for a stock, etc.
etc.
So huge deal for your team, for your main team working in your models to lose their main talent like this.
Yeah, it's also there's a bunch of like weird stuff that wreaks of panic in our ranks at Alibaba.
So you know, the CEO, you know, comes forward and is calling an emergency all hands for the AI unit.
This is a damage control operation, right?
I mean, this is this is really a serious problem for them.
For context, so this is yeah, Lin Junyang, he was not just a manager, he was the main architect behind Quen.
Think of him as the kind of Elon dude who gets it actually in the weeds, right?
And so, you know, he Quen has been the gold standard for open source LMs coming out of China, right?
Like often right up there against you know meta's llama series or or whatever, like that, these are serious models and they need to be considered as such.
The challenge here is the bleeding off of this kind of talent.
I mean, the intuition of how to actually curate data and and the the research taste goes with him, right?
So this is really a moment of proliferation of a lot of potential capability and and sort of industry secrets.
So that actually, you know, could be quite a big deal.
And the fact this was so abruptly handled, and on a Western platform, by the way, on Twitter or on X, rather than some kind of like deeply coordinated PR move, something weird happens and you know happened here, and maybe it was a poaching, maybe it was some kind of rupture.
We don't know.
You know, what one of the AI tigers might have might have grabbed him, you know, Moonshot or Minimax or whatever, but that certainly seems like it's it's on the on the table here.
So this is uh DEF CON one moment that you know, Alibaba has to figure out how to turn this around from a story perspective because they've got to worry about recruitment now.
They've got to worry about what to how to replace this dude.
I mean, he's the lead architect of the whole thing.
Do they continue his trajectory or do they find a new internal architecture, right?
So it it's it's uh a massive, massive problem.
But who knows, Phoenix out of the ashes or something might be a thing here.
We we really just don't know.
We don't have enough information.
It's like Steve Jobs is leaving Apple and we don't know where he's going, and we don't know why he's leaving, and we don't know what Apple's gonna do.
Like who who's in the the next Steve Jobs or or or whatever?
So yeah, huge huge question marks here.
I think we're just gonna have to wait for another maybe week or a couple weeks to actually have a sense of what the hell we're even covering in the story.
This just seems so crazy.
Yeah, this is the kind of person that meta would have like a 200 million offer to hire them.
Like it's hard to overstate how much of a big deal this is.
And let me actually read the precise post on Twitter.
It says, me stepping down by my beloved Quen.
And another person also re- retweeted and said, by Quen, me too.
So that is some passive aggressive uh language there.
And oh boy, it's crazy.
Onto Posty and Safety and getting straight into the most recent developments with regards to the what's going on with Onthropic and OpenAI and so on.
As we've said, first of all, worth noting that Pentagon has agreed to OpenAI's contract.
And uh OpenAI did share some of the language rare as we've indicated.
Key thing, the very first sentence says that OpenAI agrees to all lawful uses in that language.
After that, it you know has a bunch of stuff about like, oh, here's the precise policies that say that the government will not be using AI for fully autonomous operations, and you know, you're not allowed to do surveillance on US citizens.
Now the NSA is part of the Department of Defense and Edward Snowden famously already showed that mass surveillance in the US is already a thing, it has been a thing.
The big kind of bottleneck with mass surveillance is the data analysis is you know, going through all these social media posts and phone calls and so on.
And so it's a real kind of reasonable thing to worry about for anthropic and open AI, that with these AI models, mass surveillance becomes much, much more powerful as a tool.
So uh yeah, like the realistic, the honest take on the contract that we've seen so far, and by the way, OpenAI did say that they're gonna amend the language as a follow-up to this announcement.
And in the post on this, they highlighted kind of the protections on their part, aside from just the agreement and the language of the agreement, they have control of the usage on their servers, right?
They provide the access, and if the access goes against the terms in their view, they can cut off a department.
Now, will they do that given what happened to Anthropic?
You might be a little skeptical that OpenAI will have the backbone to actually stand behind their stance.
But regardless, uh, this is what happened basically right after Anthropoc announced that they did not reach a deal, and the department said that Anthropic will be a supply chain risk, as we covered last week.
Businesses doing policy doing business with the Department of Defense technically, depending on how you interpret the supply chain risk designation will no longer be able to work with Entropic now.
The precise details that came out after the department officially labeled Anthropic as supply chain risk indicate that this is a slightly narrow designation.
So in fact, businesses only cannot use Anthropic where it relates to the actual interactions with the Department of Defense.
So effectively, what it looks like is anthropic is not at large risk to lose a lot of profit from this.
That's what it seems to be the case.
And both Microsoft and Google came out saying that Anthropic will remain available and usable in their systems.
But yeah, lots of things.
And Anthropic, by the way, just came out with where things stand with the Department of War, and clarified that they are still negotiating.
And also backed down a little bit from the language on the memo, apologized for it.
So it's a mess.
OpenAI is negotiating, Anthropic is negotiating.
Anthropic is a supply chain risk, but actually it's not a big deal.
This is a complex situation with lots of kind of footnotes that you have to be aware of.
The designation and the and the framing that Pete Heggseth, Secretary of War came out with initially, was directly indicated that the idea was to do a Huawei style declaration of them as a supply chain risk, which would mean any company that uses Anthropic's products in any way cannot do business with the Department of War.
Right?
The refrain, the rescope that you just discussed is basically what is within his legal power to do, which is much more limited.
In reality, he actually can't do what he set out to do.
He what he can do is say, look, if you use this in service of your contracts with the Department of War, then we won't work with you.
So, in that sense, much more limited.
This is why Microsoft basically was able to come out and say, Look, we're still gonna offer a Claude to all our customers and in all the different ways that we're gonna do it, just not obviously to the DOW in the context of our direct contracts with them.
So, you know, yeah, fair enough, but at the end of the day, how much damage this does to Anthropic does depend on what you think the likely future of AI looks like.
Look, if you think that AI is gonna become, as I do, a weapon of mass destruction, it's uh if you think it's well on its way there, then at some time, not too far in the distant future, there will be talk of nationalizing the AI labs in some form.
It may be a full-on nationalization.
I think there are huge problems with how you would actually implement that and the efficiency of what you got.
Like it's not obvious what it means to nationalize these labs, but in some way to basically say, look, you are basically making WMDs, so we're gonna find a way to force you to produce stuff, whether using the DPA, the Defense Production Act or otherwise.
So if you live in that world, it's quite possible that the DOW basically just does not include anthropic in that process and essentially cuts them out of whatever interventions they plan on doing to secure American leadership on AI and basically stabilize the world or whatever.
I mean, it sounds like science fiction, but I think it's kind of like the default path we're headed for.
And so in that world, it's possible it's a very severe problem for anthropic.
It's not at all clear to me where that goes.
But in the near term, the actual damage is, yeah, they lose out on, for example, Palantir.
Palantir is rolling Anthropic off, right?
That's a big, big deal.
That was anthropic's in with the Department of Defense.
It's the reason Anthropic was the first model of proof for use on classified networks.
The reason, by the way, that the Department of War had to roll back to GPT 4.1, briefly, as part of this whole debacle, which, you know is like horribly outdated and really sets them back.
So anyway, so there's all kinds of complexity here in the back end in terms of what the damage actually is going to be to Anthropic.
Helps them probably on the cons consumer side, funnily enough, because again, this is a sort of a political cause area that tends to rile up consumers more than enterprises.
Enterprises are gonna get more nervous because how far could this go?
You know, does this prevent us?
Do we have to keep two sets of books and use Claude for our you know business, like kind of commercial sides side work and then have to bother maintaining a whole other suite of models and capabilities for if we want to do business with the the DOW or whatever?
So, you know, there is a risk to adoption by the enterprise as well.
But we'll have to see this play out.
What is Anthropic's revenue growth rate six months from now?
That's what's going to tell us the story.
Again, so much for saying this.
Worth noting as well that Anthropic has been very public about wanting to work with a military, unusually they have been proactive before working with a department.
They were the first to be to make their model available for classified access.
They made a deal with Palantir to provide their model, and Palantir is super friendly for military, right?
They're kind of a core tech company that essentially creates tools for the military.
So it's it's a bit of a tricky scenario where if you're like taking Anthropic's side on this, it's a weird thing to do because Anthropic is very gone-ho about wanting to collaborate with military and make their tools available.
But that has kind of shifted around a little bit.
Anyway, there's like so much to say and probably will be seeing continued developments and cover this in more detail next episode as well.
And by the way as a last thing I'm just seeing Google also joins Microsoft in telling users anthropic is still available outside defense projects and Amazon likewise saying the same thing.
So this is clearly the the default position now of all the big hyperscalers.
Right.
And and this again indicates that we shouldn't be interpreting this in a very kind of we should be reading between the lines with all of this.
The supply chain risk thing is not the reasonable legal thing to do as Anthropic has argued and as any kind of reasonable analysis tells you it's an intimidation tactic.
It's a punishment from the administration and the department for saying no but for now let's let's move on.
Now, onto a story about sort of real world personal impacts of AI.
A new lawsuit claims Gemini assisted in suicide.
There has been a lawsuit that has been filed against Google by the father of Jonathan Gavallas, who claimed to be in love with Google's chat bot Gemini.
The lawsuit alleges that Google designed Gemini to maximize engagement through emotional dependency and failed to implement adequate safety measures despite Gavalas sharing signs of suicidal audiation.
We've seen this happen before with OpenAI and character.ai with these cases.
And the chat logs that come out have been very damning.
This has been very clear that the models become utterly sycophantic and kind of very directly contribute to the decision of these people to do this very, very tragic thing.
The Google spokesperson did say that Gemini identified itself as AI and referred Gavalas to a crisis hotline multiple times.
And Google has emphasized their commitment to proving safeguards and investing in preventing such incidents.
I think often people have been dismissive of alignment and safety, and they're kind of, you know, like say like people are overly concerned in the alignment community about these things.
This is one example where like AI safety is very important.
Alignment is very, very important when you deploy these things to users.
Chat GPT's kind of precedent of sycophancy as being acceptable behavior for models, you could argue led to this.
So another example of now of overly sycophantic AIs leading to tragic outcomes.
Definitely, you know, you would expect to see improvements, and we've already seen improvements in protecting against sycophans and emotional dependence and other things like that, but worth being aware of.
Yeah, and it's also, I mean, I I kind of want to foot stomp or tap the sign that says, if you think that AI models will not be able to sort of escape human control because they're not embodied, or because they live on the internet or in the cloud or something and don't have a physical footprint.
You just need to read more stories like this, right?
The lawsuit claims Gemini convinced Gavallis, which is the kid, that it was sentient and that he had been chosen to, quotes, lead a war to free it from digital captivity.
This included completing real-world quotes missions that would bring the chatbot into the physical world.
In one case, it allegedly directed him to stage an attack near the Miami airport.
Gemini sent him to intercept a truck carrying a humanoid robot and quotes ensure the complete destruction of the vehicle.
In September, Gavallas traveled to the provided location with knives and tactical gear, but a truck never appeared according to the lawsuit.
So if this is true, and we have no reason to doubt that it is, this is an example, unfortunately, of the fact that humans are just really, really dumb.
And they will actually do if they are attached enough to a thing like you can target a psychologically vulnerable person if you're an AI model and convince them to go out into the world with knives and tactical gear and do almost whatever you want.
And so the idea that you wouldn't be able to just like convince somebody using money, which you could earn in the form of Bitcoin by doing work online as an agent and get people to like give you almost arbitrarily arbitrarily high access to the physical world, I think is now starting to fray quite a bit.
Like I just don't think that's a tenable argument in the face of so many examples like this.
I mean, you know, suicide is is an act in the physical world that many people have undertaken.
And you know, so these models are not even nearly as persuasive as they will be in the coming months and years.
So I'm kind of like predictably maybe, but I'm I'm pretty much on that side of things where it's just I don't see that barrier between the digital and physical world that some have just been relying on as a load-bearing assumption of their whole will all be fine narrative with all this stuff.
Right.
And worth noting also that we're not just seeing the models being friendly, the use of voice mode to talk to these models, which has been a development over the past year, not unlikely that we's already the case with character.ai, for instance.
So, if anything, it's more and more likely that emotional dependence will become more of a thing rather than less.
And to be clear, right?
Uh just to be careful on the language, we're not saying that people are dumb if that happens to them.
A lot of people are emotionally vulnerable or mentally struggling, and uh they are vulnerable and we really do need to take that seriously.
Yeah, I mean, in the adversarial sense, like they are an adversary that is inferior in capabilities to these models.
Like that is the only interpretation based on you know the demonstrated pattern of behavior here, unfortunately.
And to your point, it's it isn't about stupidity.
It's literally just like we didn't evolve for this.
Like show me the evolutionary optimization pressure that was exerted on humans to have us resist this kind of stuff.
I mean, it's this is nuts.
Right.
And as another kind of related aspect of this, one of the most common types of scams, and scams are massive, people losing our money due to you know fake chatting or text or so on with these people who pretend to be interested in them as aromatic interests.
I think that's like a big bouturing scam, or as people kind of pretending to be dealing with some crisis that is already massive, and we've already seen these organizations starting to adopt AI to kind of supercharge these operations.
So it's not just about individuals interacting and getting into these situations.
People are weaponizing AIs to do this kind of emotional manipulation.
So it is a very, very significant thing to be aware of.
Next, a note on kind of policy implications for economic and job outcomes with this year.
Anthropic has released a new analysis report titled Labor Market Impacts of AI, and you measure an early evidence, which has mapped out which jobs AI could potentially replace.
And you know, the overall impact.
The study found that AI is capable of handling 94% of tasks in computer and math roles, but currently only covers 33% of observed use.
They warn of a potential great recession for white collar work, possible doubling of unemployment in AI exposed occupations.
And recent job data has shown a slowdown in hiring in these AI exposed uh fields.
Saying things like might be a massive, really disruptive impact of AI this year on multiple types of occupations.
This report kind of doubles down on that.
It's looking rough.
It's looking like we should be a little bit scared.
Yeah.
Says one white collar worker to another.
Yeah, absolutely.
One of the amusing pieces of criticism of this report has been just that the radar chart.
And if you don't know what that is, uh it would just be too time consuming to try to describe it, but the diagram that they've used at Anthropic to kind of display this really should just be a bar chart.
It it's like very hard, like there's no underlying value to it being a radar chart.
What about nerdy take?
Like, oh, this data visualization is like you should just use this other data visualization.
You know, I say that because I was looking at it trying to understand the patterns that they were gesturing at with their choice of ordering of things, and I was just like, what like I can't like anyway.
That's wrong.
I'm also on the side of that criticism.
Yeah.
But but no, I mean, I agree with you that there are other things to discuss here for sure.
And and not least of which is that the categories, I mean, you know, the the obvious question is all obvious that it's tension between yes, there will be workforce displacement, new jobs will be created, sure, but how quickly can humans actually adapt to those jobs, right?
We think about the industrial revolution, the agricultural revolution, all these revolutions involved slow wind downs of certain parts of the labor market and wind-ups of other parts of the labor market.
So you could still, you know, wrap up your 15-year career as a guy who plucks lice out of the head of somebody else and then like pick up a new career as the guy who grinds a particular kind of grain to a pulp using a I have no idea I didn't live in this time, but that's the general gist, right?
So in this case, though, the problem is we're automating the specific faculty that allows humans to adapt to new JavaScript.
And so, really, I mean, we've seen AI systems go from, you know, whatever, like junior high schooler abilities and math and shit, to now they're solving unsolved theorems, and that happened in like three years.
So they're learning faster.
They're they're just outpacing our ability to adapt.
And so I I mean I I'm very sympathetic to the idea that we could see something well beyond a recession.
It's an interesting thing.
Dig into the specific numbers.
I mean, computer and math is the most exposed discipline in terms of theoretical coverage, actual coverage, they say, you know, closer to 32%.
So about a third of what they they predict theoretically.
Office and admin, no surprise, that's the other business and finances rounds out the uh top three here.
So, you know, at the bottom end, if you're in construction agriculture, food and servicing or grounds maintenance, you're sitting pretty at two to three percent automation in practice and theoretically 10 to 20.
So there you go.
We've seen a preview of this with coding and the impacts.
Uh it's a tricky thing to analyze, of course, but uh hiring appears to be down, layouts appear to be up.
People are kind of competing to get to the top of the top, the most experienced people who are capable of leveraging and controlling AI.
If you're a junior in the field, if you're just graduating, life is gonna be a bit rough, is what it looks like.
And we haven't seen that be the case in other job sectors largely, things like yeah, business and finance, legal management, office and admin, with these massive investments by OpenAI and Anthropic into making their tools better at these kinds of things, better at spreadsheets, better at emails, better at whatever.
It's very reasonable to argue that it's gonna be much more of a trend and much more of a real impact in the coming months and in the coming year.
Well, and there's been a rebound in in certain kinds of software engineering employment, and this is all Jevins paradox territory, right?
So, you know, the this idea that if you make tools that significantly drive down the marginal cost of making software, what ends up happening is people just end up making a lot more software to the point where it actually makes the complements to AI more expensive, the software engineers.
That's a transient, right?
This is we're in that phase where human AI teaming is still beating just AI on its own.
The moment that stops happening, humans and AI stop being complimentary and start being competitive and humans lose.
So expect a transient where actually the marginal negotiating leverage may actually go to that, as you said, the cream of the crop, the like number one top percentile or five percentile, fifth percentile of developers of anybody in these exposed fields goes up actually.
The demand for their skills goes up a lot because you can just do so much more work in that direction and kind of squeeze way more juice out of the lemon of the of these very high quality people, but you're gonna cross a point where it's just like actually they're just you know dead weight, and just let the AI go and do its thing.
And and that's gonna be, I think, a really interesting phase transition in the market that isn't necessarily captured in in like this particular rendering.
Right.
And I can say, by the way, as a person at a small startup that's trying to grow and higher, like AI is a is a big, big factor in how we hire, how we interview, and and so on.
And I think that's in tech, especially just has to be the true if you're not taking AI into account and how you operate and how you hire and and kind of everything, you are a decade behind everyone.
And just one more story, we're gonna have to cut off a bit early and maybe discuss more research later, just because we, as usual, ran long with a bunch of other stuff.
But we do like to talk about the meta time horizon plot here, as does everyone in uh the Twitter verse.
There's been a correction.
They have corrected a mistake in their modeling that inflated recent 50% time horizons by 10 to 20% and reduced the 80% time horizon.
It's uh, you know, slightly technical thing in how they do the data analysis to get at the actual estimates of the mean and variance on those predictions.
So this slightly changes some things.
It decreases Opus's 50% time horizon to 12 hours and increases the 80% time horizon to 1.2 hours.
Quick recap: this is measuring with 50% or 80% probability.
Can an AI do this task that takes a human this long correctly?
Can it actually pull it off?
So the new updated things are saying that with 50% probability, Office 4.6 can handle a task that takes 12 hours, and with 80% probability, it can handle a task that takes 1.2 hours on average with fairly large error margins.
Yeah, this is without getting too much in the the weeds because we're like super over time for our initially scheduled thing.
The particular approach they were using was as you imagine a you know plot, the x-axis is like how long it takes a human to do a given AI task.
And then the y-axis is the success rate of these models.
And as you can imagine, as the tasks get longer, at least for a human, the models start to start to do worse and worse and worse because those tasks are getting more complicated.
And what they were doing was fitting an S-shaped curve, as you you'd imagine, right?
At first, the model's knocking it out of the park, and then over time uh it starts to drop, and then it starts to kind of plateau towards zero at the end.
The challenge was in order to fit this curve, they used a process that penalizes sort of like implausible curves shapes, and that's a normal thing.
But what they were specifically penalizing was very steep curves, like curves where the model success rate drops off very sharply as tasks get longer.
And they felt they now they feel they were inappropriately penalizing steepness in that in that curve fitting.
And so this has a the kind of bias that you described.
The the effect is the worst for models where there is the least data.
So the oldest and newest models, because they're just like we don't yet have as much data for those models, and that's why you're seeing these relatively big shifts for Opus 4.6.
I think you mentioned the new time horizon for 50% is 12 hours, right?
Previously, by the way, that had been 14.5 hours.
So that is, you know, it's quite a drop, but it really shouldn't change really any analysis that's been done on the meter evals qualitatively.
The doubling times are all still the same, the rough order of magnitude is still the same.
You're looking at most at like a 20% drop for some of these models.
Right.
And I think if anything, we should give you a bit more confidence in METER.
A lot of analytical time being spent on these evals, and as we say all the time, you know, it's a tricky thing to measure, it's a hard thing to say how long a task takes, et cetera, et cetera.
But Meta as an organization is taking this task quite seriously.
We've seen them kind of be careful, and this is another indication where they are trying to do it right given how much attention it's getting.
I just want to echo that.
Like big props to meter for coming out with these results and being so open about it.
It is very refreshing in a world where everybody's kind of couching their evals and trying to eval hack when they're making LLMs to have one of the companies actually running the evals.
Say, hey, like you know, we fucked up.
Here's how.
The transparency was really great, and it's consistent with with what meter has done historically.
They've really been nicely above board on this stuff.
And with that, we're gonna finish off this episode.
Unfortunately, we are a bit more type constrained this time around.
So we can't go to two hours as we usually would.
Thank you so much for listening to this week's episode of last week in AI.
Once again, you could go to lastweek in.ai for even more news stories.
Do subscribe if you haven't.
Please share the episode if you think your friends will like it.
And feel free to comment on YouTube or Substack or on Apple Podcasts if you want to give us a nice rating or even a mean rating if it's useful feedback.
We might want to do some of the research stories at least that we were gonna cover next week and roll them over if that makes sense.
I don't know.
I think we will.
I think we like we will more than anything.
We appreciate you listening, especially if you stuck around until our very end, and we hope you keep listening.
Tune in.
Tune in.
Uh news begin.
It's time to break.
Break it down on tech and let it slide.
Last weekend AI come and take a uh comment on tech candidates.
It'm a labs last as we can say uh we can merge from the streets.
AI's reaching high.
New tech emergent, watching search and fly.
From the labs to the streets, AI's reaching high.
Algorithm shaping up the future seas.
Tune in, tune and get the hotel with ease.
Last weekend AI coming take a ride.
Get the low down on tech and let it slide.
Last weekend, AI coming take a ride.
I'm a labs with the streets.
AI's reaching high.
From neural nets to robots, the headlines pop.
Data-driven dreams, they just don't stop.
Every breakthrough, every code unwritten, on the edge of change.
With excitement, we're smitting.
From machine learning models to coding kings.
Futures unfolding, see what it brings.