# Boltz Bio Democratizes AI Protein Design

**Podcast:** Latent Space: The AI Engineer Podcast
**Published:** 2026-02-12

## Transcript

Actually, we only trained the big model once.
That's how much compute we had.
We could only train it once.
And so, like while the model was training, we were like finding bugs left and right.
Yeah.
A lot of them that I wrote.
And like I would I remember like us like sort of like you know, doing like surgery in the middle, like stopping the run, making the fix, like relaunching.
And um yeah, we never actually went back to the start.
We just like kept training it with like the bug fixes along the way.
Uh which was uh impossible to reproduce now.
Yeah, yeah, no.
That model is like has gone through such a curriculum that you know learned some weird stuff.
Uh but uh yeah, somehow by miracle it worked out.
It's a pleasure to have with us today, Gabriela Corso and Jeremy Volvand.
They are they recently founded Volts, a company trying to democracize and bring art structure prediction in biology to you know the masses.
Uh they were both uh recent PhD grads from MIT and have been working on all sorts of foundational papers in like generative uh biology.
Um anyway, uh pleasure to have you here.
Thanks for coming.
Thank you.
Uh I guess we're maybe what, six years post-Alpha Fold 2 right now, which was like kind of a big moment.
Is that right?
I think was it 2021?
So yeah, on going on five years.
Five years, five years, yeah.
Yeah, so maybe for the audience, like let's go back to that moment in time and explain like what was this big moment and why was it interesting?
Why was everyone so excited?
And I think you two were probably quite excited.
So why were you personally excited?
I would start on kind of why that was interesting, kind of, you know, from a scientific standpoint.
So, well, Alpha Fold, so maybe first as a kind of introduction for uh the ones in the audience are not structured biologists.
So the idea of structural biology is that you know, we want to try to understand how you know proteins and other molecules uh take shape inside our cells and you know how they interact.
And structural biology is sort of this beautiful discipline where we are somehow able to understand this minuscule structure atomic details using uh these incredibly um complex methods like you know X-ray crystallography.
And you know, the the dream has always been of a computational biology.
Can we understand kind of the structures without having to you know resolve this crystal, you know, shoot X-rays and so on?
And so Alpha Fold was a real breakthrough in this problem of protein folding, which is trying to understand the structure of a single uh protein.
And to me, it was exciting across kind of many dimensions.
One I was computer scientist, I was working a lot on machine learning, and I saw kind of the impact that kind of the work similar, somewhat similar to what I was doing could have on like a long-standing scientific problem.
And on the second perspective, from a more you know, personal side, the seeing kind of the structures coming out of these models where you know you see kind of this beautiful, you know, creation of life is something that was was very inspiring to me.
And so that was kind of one of the things that led me to start uh working on uh structural biology and in particular with machine learning.
Were you a structural biologist before AlphaFold came out?
I mean did you you did machine learning but it was not in structural biology.
So that actually shifted your career quite dramatically.
Yeah very dramatically I was I was working on some pretty kind of theoretical methodological things and I was starting to see kind of you know some of the challenges in you know kind of doing somewhat theoretical or methodological work and you know seeing kind of the potential impact of you know um doing excellent you know alpha fold was really a machine learning breakthrough but you know an applied machine learning and so that led me to uh want to start working in applied ML.
Our our group at the time was um working a lot like small molecules already and I think AlphaFold is kind of what triggered I think this shift to like working on on biologics.
And at the time I think it like open as many questions you know as it answered in a sense like we the immediate follow-ups were okay like can we do this on other things than proteins can we do you know interactions of small molecules with proteins nucleic acid with proteins.
Can we model more complex protein systems?
And I think, yeah, very rapidly, I think after alpha fold, people realized I think that there was, you know, machine learning could have a could really yeah sort of target this problem very differently than you know than previous methodologies.
Going back to the AlphaFold 2 moment, like I remember this very well.
I was at NERIFs when I guess the results of this famous competition came out.
So you want to be talk about CASP and like what it is and why it was so interesting and exciting.
I think every so every couple of years, I think the goal has always been to you know find protein structures that are a little bit different from what's known.
So CASP over the years has like you know put in a lot of effort to like gather structures from you know academic groups and uh even industry groups to try to create sort of a test set that would be difficult for uh different methods.
And CASP uh 14 was when uh Alpha Fold 2 really blew everything out of the water.
The improvement was so large over you know the previous previous method and also over the previous competitions.
And now CASP continues, you know, we've had CAS-15, we have CAT 16, and you know, sort of what's happened now is that it's really expanding to also all these other modalities, like I was mentioning like protein small molecule, nucleic acid.
And but the goal remains to like you know, really challenge the models, like how well do these models generalize?
And you know, we've seen in some of the latest gasped competitions, like while we're become really, really good at proteins, basically monomeric proteins, um, you know, other modalities still remain pretty difficult.
So it's really essential, you know, in the field that there are like these efforts to gather um you know benchmarks that that are challenging, so keeps us in line, you know, about what the models can do or not.
Yeah, yeah.
It's interesting you say that, like in some sense, CAS, you know, at CAST 14, a problem was solved and like pretty comprehensively, right?
But at the same time, it was really only the beginning.
So you can explain like what was the specific problem you would argue was solved, and then like you know, what is remaining, which is probably quite open.
I think I think we'll we'll steer away from the term solved because we have many friends in the community who get pretty upset at that word.
And I think you know, fairly so.
But the problem that was you know, that a lot of progress was made on was the ability to predict the structure of single chain proteins.
So proteins can like be composed of many chains, and single-chain proteins are you know just a single sequence of amino acids.
And uh one of the reasons that we've been able to make such progress is also because we take a lot of uh hints from evolution.
So the way the models work is that you know they sort of decode a lot of hints that comes from evolutionary landscapes.
So if you have like you know, some protein in an animal, and you go find the uh similar protein across like you know different organisms, uh, you might find different mutations in them.
And as it turns out, if you uh take a lot of these sequences together and you analyze them, you see that some positions in the sequence tend to evolve at the same time as other positions of the sequence, sort of this like correlation between different positions.
And um, in it turns out that that is uh typically a hint that these two positions are close in three dimension.
So part of the, you know, part of the breakthrough has been like our ability to also decode that very, very effectively.
But what it implies also is that in absence of that coevolutionary landscape, the models don't quite perform as well.
And so, you know, I think when that information is available, maybe one could say, you know, the the problem is like somewhat solved from the perspective of structure prediction.
When it isn't, it's it's much more challenging.
And I think it's also worth also differentiating the sometime we confound a little bit structure prediction and folding.
Folding is the more complex process of actually understanding like how it goes from like this disordered state into like a structured like state.
And that I don't think we've made that much progress on, but the idea of like, yeah, going straight to the answer, uh, we've become uh pretty good at.
So there's this protein that is like just a long chain and it folds up.
Yeah.
And and so we're good at getting from that long chain in whatever form it was originally to the thing, but we don't know how it necessarily gets to that state.
And there might be intermediate states that it's in sometimes that we're not aware of.
That's right.
And that relates also to like, you know, our general ability to model like the different, you know, proteins are not static, they move, they take different uh shapes based on their energy states.
And I think we are also not that good at understanding the different states that the protein can be in and at what frequency, what probability.
So I think the two problems are quite related in some ways.
Still a lot to solve, but I think it was very surprising at the time, you know, that even with these evolutionary hints that we were able to, you know, to make such such dramatic progress.
So I want to ask why does the intermediate states matter?
But first, I kind of want to understand why do we care what proteins are shaped like.
Yeah, I mean, the proteins are kind of the machines of uh our body, you know.
The way that all the processes that we have in our cells, you know, work is typically through proteins, sometimes other molecules sort of intermediate interactions, and through that interactions, we have all sorts of cell functions.
And so when we try to understand, you know, a lot of biological how our body works, how disease work, we often try to boil it down to okay, what is going right in case of you know our normal biological function and what is going wrong in case of the disease state.
And we boil it down to kind of you know proteins and kind of other molecules and their interaction.
And so when we we try predicting the structure of proteins, it's critical to you know have an understanding of kind of those those interactions.
It's a bit like seeing the difference between having kind of a list of parts that you would put it in a car and seeing kind of the car in its final form.
You know, seeing the car really helps you understand what it does.
On the other hand, kind of going to your question of, you know, why do we care about you know how the protein folds or you know how the car is made, to some extent is that you know, sometimes when it something goes wrong, you know, there are you know cases of you know proteins misfolding in some diseases and so on.
If we don't understand this folding process, we don't really know how to uh intervene.
There's this nice line in the um, I think it's in the fold two manuscript where they sort of discuss also like why we even hopeful that we can target the problem in the first place, and then this this notion that like well, for proteins that fold the folding process is almost instantaneous, which is a strong like you know, signal that like, yeah, like we should we we might be able to predict that this very like constrained uh thing that that the protein does so quickly.
And of course that's not the case for you know for all proteins, and there's a lot of like really interesting mechanisms in the cells.
But yeah, I remember reading that and thought, yeah, that's somewhat of an insightful point.
I think one of the interesting things about the protein folding problem is that it used to be actually studied, and part of the reason why people thought it was impossible, it used to be studies as kind of a like a classical example of like an MP problem.
Uh, like there are so many different, you know, type of you know shapes that you know this amino acids could take.
And so this grows combinatorially with the size of the sequence.
And so there used to be kind of a lot of actually kind of more theoretical computer science thinking about and studying problem protein folding as an MP problem.
And so it was very surprising also from that perspective, kind of seeing machine learning so clear there is some you know signal in those sequences through evolution, but also through kind of other things that you know, us as humans we're probably not really able to uh to understand, but that this models have learned.
Yeah, so Andrew White, we were talking to him a few weeks ago, and he said that he was following the development of this and that there were actually ASICs that were developed just to solve this problem.
So in that there were many, many, many millions of computational hours spent trying to solve this problem before Alpha Fold.
And just to be clear, one thing that you mentioned was that there's this kind of coevolution of mutations, and that you see this again and again in different species.
So explain why does that give us a good hint that they're close by to each other.
Yeah.
Um, like think of it this way that, you know, if I have you know some amino acid that mutates, it's gonna impact everything around it, right?
In three dimensions.
And so it's almost like the protein through several probably random mutations in evolution, like, you know, ends up sort of figuring out that this other amino acid needs to change as well for the structure to be conserved.
Uh so this whole principle is that the structure is probably largely conserved, you know, because there's this function associated with it.
And so it's really sort of like different positions compensating for for each other.
I see.
Those hints in aggregate give us a a lot of information about what is close to each other, and then you can start to look at what kinds of folds are possible given the structure, and then what is the end state, and therefore you can make a lot of inferences about what the actual f total shape is.
Yeah, that's right.
It's almost like you know, you have this big like three-dimensional valley, you know, where you're sort of trying to find like these like low energy states, and there's so much to search through that's almost overwhelming.
But these hints, they sort of maybe put you in an area of the space that's already like kind of close to the solution, maybe not quite there yet.
And and there's always this question of like how much physics are these models learning, you know, versus like just pure like statistics.
And like I think one of the things, at least I believe, is that once you're in that sort of approximate area of the solution space, then the models have like some understanding, you know, of how to get you to like, you know, the lower energy, uh low energy state.
And so maybe you have some some light understanding of physics, but maybe not quite enough, you know, to know how to like navigate the whole space well.
Okay.
So we need to give it these hints to the case.
So you can get it into the right valley and then it finds the minimum or something.
Yeah.
One interesting explanation about our AlphaFold 3 works that I think it's quite insightful, of course, doesn't cover kind of the entirety of what Alpha Fold does.
That is um then gonna borrow from uh Sergei of Chinikov at MIT.
So he sees kind of Alpha Fold.
And the interesting thing about AlphaFold is it's got this very peculiar architecture that we have since you know used.
And this architecture operates on this you know pairwise contacts between amino acids.
And so the idea is that probably the MSA gives you this first hint about what potential amino acids are close to each other.
MSA is multiple multiple sequence alignment, exactly.
This evolutionary exactly this evolutionary information.
Yeah, and you know, from this evolutionary information about potential contacts, then it's almost as if the model is sort of running some kind of you know dystero algorithm where it's sort of decoding, okay, these have to be closed.
Okay, then if this is a closed and this is connected to this, then this has to be somewhat close.
And so you decode uh this that becomes basically a pairwise kind of distance matrix, and then from this rough pairwise distance matrix, you decode kind of the actual potential structure.
Interesting.
So there's kind of two different things going on in the the kind of coarse grain and then the fine-grain optimization.
Interesting.
Yeah.
Very cool.
Yeah, you mentioned Alpha Fold III.
So maybe it's a good time to move on to that.
So yeah, AlphaFlow 2 came out and it was like, I think fairly groundbreaking for this field.
Everyone got very excited.
A few years later, Alpha Fold 2 came out.
And maybe for some more history, like what were the advancements in Alpha Fold 3?
And then I think maybe we'll, after that, we'll talk a bit about the uh sort of how it connects to bolts.
But anyway.
Yeah, so after Alpha Fold 2 came out, you know, Jeremy and I got into the field and with many others, you know, the clear problem that, you know, uh was you know obvious after that was okay, now we can do individual chains.
Can we do interactions?
Interaction different proteins, proteins with small molecules, proteins with other other molecules.
And so why are interactions important?
Interactions are important because to some extent that's kind of the way that you know these machines that you know, these proteins have a function.
You know, the function comes by the way that they interact with other proteins and other uh molecules.
Actually, in the first place, you know, the individual machines are often, as Jeremy was mentioning, not made of a single chain, but they're made of the multiple chains, and then these multiple chains interact with other molecules to give uh the function to uh those.
And on the other hand, you know, when we try to intervene of these interactions, think about like a disease, think about like a biosensor or many other ways we are trying to design a molecules or proteins that interact in a particular way with what we would call a target protein or target.
You know, this problem after AlphaVolt 2, you know, became clear, kind of one of the biggest problems in the field to solve.
Many groups, including kind of ours and others, you know, started making some kind of contributions to this problem of trying to model these interactions.
And AlphaVolt 3 was, you know, was a significant advancement on the problem of modeling interactions.
And one of the interesting things that they were able to do while some of the rest of the field that really tried to try to model different interactions separately, you know, how protein interacts with small molecules, how protein interacts with other proteins, how RNA or DNA have to structure, they put everything together and train very large models with a lot of advances, including kind of changing kind of some of the key architectural choices, and managed to get a single model that was able to set a new state-of-the-art performance across all of these different kind of modalities, whether that was protein small molecules, that's critical to developing kind of new drugs, uh, protein protein, understanding interactions of proteins with RNA and DNAs and so on.
Just uh to satisfy the AI engineers and the audience, what were some of the key architectural and data changes that made that possible?
Yeah, so one critical one that was not necessarily just unique to Alpha World 3, but there were actually a few other teams, including ours in the field that proposed this was moving from modeling structure prediction as a regression problem.
So where there is a single answer and you're trying to shoot for that answer to a generative modeling problem where you have a posterior distribution of possible structures and you're trying to sample this distribution.
And this achieves two things.
One is starts to allow us to try to model more dynamic systems.
As we said, you know, some of these structures can actually take multiple structures.
And so you can now model that through kind of modeling the entire distribution.
But on the second hand, from more kind of core modeling questions, when you move from regression problem to a generative modeling problem, you are really tackling the way that you think about uncertainty in the model in a different way.
So if you think about, you know, I'm undecided between different answers.
What's gonna happen in a regression model is that you know, I'm gonna try to make an average of those different kind of answers that I had in mind.
When you have a generative model, what you're gonna do is you know sample all these different answers and then maybe use a separate models to analyze those different answers and pick out the best.
So that was kind of one of the critical improvements.
The other improvement is that they significantly simplified to some extent the architecture, especially of the final model that takes kind of those pairwise representations and turns them into an actual structure, and that is uh now looks a lot more like a more traditional transformer than you know, like a very specialized equivariant architecture that it was uh in AlphaFold 3.
So this is a bitter lesson a little bit?
There is some aspect of a bitter lesson, but the interesting thing is that it's very far from you know being like a simple transformer.
This field is one of the argue very few fields in applied machine learning where we still have kind of architecture that are very specialized, and you know, there are many people that have tried to replace these architectures with you know simple transformers, and you know, there is a lot of debate in the field, but I think kind of the most of the consensus is that you know the performance that we get from the specialized architecture is vastly superior than what we get through a single transformer.
Another interesting thing that I think on the staying on the modeling machine learning side, which I think it's somewhat counterintuitive, seeing some of the other kind of uh fields and applications is that scaling asn really worked kind of the same uh in this field.
Now, you know, models like Alpha Fold 2 and AlphaFold 3 are you know still very large models, but at the same time, they in terms of parameters, they're actually not very big.
They are definitely below a billion parameters.
You know, if you hear these days in LLM space, you know, a model with less than a billion parameters, you'd think can't do anything.
But on the other hand, when you look at the computational cost of running these models, they are actually a lot more expensive than uh it is to run a language models because as Jeremy was saying, we go from instead of having sort of quadratic operations to now a cubic operation.
And so it's interesting how right now in the field, and and this is maybe related to you know having kind of less data or you know, needing more inactive biases, but we have this ratio of you know amount of computation to parameters that is much much higher than in other in other places.
If I recall Alpha Fold 2 was like what 70 million parameters, something like that.
Yeah, it's it's uh something like that.
It's quite yeah, it's quite small around 100 or so.
Yeah.
These decisions of triangle layers and like these for Alpha Fold 2, this like interesting equivariant architecture, like really were priors that it baked in a lot of the physics of the system.
And also co-evolution data is I think people have argued that it's kind of like almost like a database lookup of some sorts.
It also sort of so that provides in some sense more parameters as well.
Yeah, I mean it's uh it's more definitely the amount of like you know, pure like compute flops, yeah, is is very high, and it's almost like more, yeah, more almost more like reasoning-based, maybe than like more just like information extraction.
You know, I think one of the things that the part of the reason the LLMs are so large isn't just because of their reasoning capability, but also because of like like the sheer quantity of information that they store.
And I think here there's a little bit less of that, you know, and I think it's more about like you know, decoding this input rather than maybe like memorizing as much of it.
So is there like a loop in the architecture that allows it to compute more for per parameter?
Like how does that work?
Part of it is just you know exclusively this fact that instead of you know having operations that operate on the single chain, they operate on the pairwise.
And so you instead of having like quadratic number of up uh kind of interactions, you have a cubic number of interactions.
And so that on its own, you know, leads you to have you know smaller kind of representation sizes, but more representation that leads to more flops but fewer parameters.
On the other hand, you know, there is actually also this idea of you know, there's somewhat similar to to reasoning where you recycle kind of this operation so from alpha fold two, but also kind of alpha fold three.
They have this interesting framework where you know you start we as we were discussing, kind of the input to the model is sort of like this initial understanding of the interactions, either from the evolution of the multiple sequence, but also potentially from what we call templates that are basically database lookup of similar structures.
And so how the model works is that you know it decodes this and tries to understand a good, you know, potential rough structure of the pairwise interaction.
And then what you can do is basically do this recycling where you feed this understanding back to the input of the model and then try to decode it again.
And people do this three or four times, and you know, in some cases, you know, I've even tried to do it uh tens of times.
And so you can see it as a very, very early version of kind of uh reasoning uh or you know, trying to uh to get yeah.
So you you know, uh Alpha Fold 2, really cool, Alpha Fold 3, really cool.
Um, but Alpha Fold 3 came with a catch.
And I think this catch was important for the development of you know, bolts and so on.
So yeah, the catch was that it was an amazing paper, nature uh paper, but unfortunately they uh decided not to release the model.
Uh you know, Alpha Fold 2 uh was open source, and since then was was used, I think the reported numbers is you know, more than a million scientists.
AlphaFold III for you know commercial reasons that you know um did minus since spin-off isomorphic lab.
This is now trying to become sort of like a new pharmaceutical company, uh, had decided to keep this model internal and and only use it internally.
And now uh both, you know, we were in the field and you know building on top of models like AlphaFold.
And so now we no longer had you know, kind of the base starting point uh to build on top.
But even more importantly, everyone in both kind of academic research and in industry no longer had access to these incredible models that you know was you know really useful to try to understand biologists, but also try to develop new therapeutics.
I decided that to take the matter in our own hands and decided to kind of try to obtain a model that was of similar accuracy.
And so largely also using a lot of you know the uh information that was in the Alpha 3 manuscript, we went ahead and built Boltz One, which was the first fully open source kind of model to approach the level of accuracy of Alpha Fold 3.
And you know, along the way, and and you know, uh we can talk about it more, but you know, we realized that it was probably too ambitions to have, you know, to see this as uh an academic project, and you know, there are a lot of things that were kind of missing.
And so we decided to also start uh a public benefit company to push kind of this mission of you know democratizing access to these models that we started with uh Bolts One.
Quick interjection.
I mean, I remember this, it was actually shocking how fast you got Boltz one out.
Like it was just like two or three months, right?
I think we started in late May and it came in November, if I remember correctly, so slightly longer, but yeah, yeah, it was relatively quick.
I mean, for what it's worth, like, you know, we were working on some of the some similar ideas at the time.
I think like we, you know, for example, this idea of like having a diffusion model on top of um this, like, more this pairwise strong was something that we were we were exploring independently.
Now, when the paper came out, it was like really clear, like especially for example, on the data pipelines.
There was like so much that we were like not really doing, and so there was a lot to like catch up on.
But we were already in a place, I think, where we had you know some experience working in you know with with the data and working with this type of models, and I think that put us already in like a good place to you know to produce it quickly.
And you know, and I would I would even say like I think we could have done it quicker.
The problem was like for a while we didn't really have the compute, and so we couldn't really train the model, and actually we only trained the big model once.
Uh that's how much compute we had.
We could only train it once.
And so, like while the model was training, we were like finding bugs left and right.
Yeah, uh, a lot of them that I wrote.
And like I would I remember like us like sort of like you know, doing like surgery in the middle, like stopping the run, making the fix, like relaunching, and yeah, we never actually went back to the start, we just like kept training it with like the bug fixes along the way, uh, which was uh impossible to reproduce now.
Yeah, yeah, yeah.
No, that model is like has gone through such a curriculum that you know it's learned some weird stuff.
Uh but uh yeah, somehow by miracle it worked out.
The other uh funny thing is that the way that we were training uh most of that model was through uh a cluster from the Department of Energy, but that's sort of like a shared cluster that many groups use, and so we were basically training the model for two days, and then it would go back to the queue and stay a week in the queue.
So it was it was it was pretty painful.
And so we actually kind of towards the end with Evan, the CEO of Genesis, and basically I was telling him a bit a bit about the project, and you know, kind of telling him about this frustration with the compute.
And so likely, you know, he offered to kind of help.
And so we uh we got the the help from Genesis to you know finish up the the model, otherwise it probably would have taken a couple of extra weeks of weekends.
Yeah, yeah.
Boltz one, how did that compare to Alpha Fold three?
And then there's some progression from there.
Yeah, so I would say kind of the bolts one, but also kind of these other kind of set of models that came um around the same time, were kind of approaching, were a big leap from you know, kind of the previous kind of open source models, and you know, kind of uh really kind of approaching the level of Alpha Fold III.
But we'd stay still say that you know, even to this day there are you know some specific instances where Alpha Fold III works better.
I think one common examples is antibody antigen prediction, where you know, Alpha Fold 3 still seems to have an edge in in many situations.
Obviously, these are somewhat different models.
They are, you know, you run them, you obtain different results.
So it's it's not always the case that one model is better than the other, but kind of in aggregate we still, especially at the time.
So Alpha Fold Free is you know still having a bit of an edge.
We should talk about this more when we talk about both gen, but like how do you know one is one model is better than the other?
Like you so you I make a prediction, you make a prediction, like how do you know?
Yeah, so easily, you know, the the great thing about kind of structure prediction, and you know, once we're gonna go into the design uh space of designing new small molecules and new proteins, this becomes a lot more complex.
But a great thing about structure prediction is that a bit uh like you know, CASP was doing, basically the way that you can evaluate them is that you know you train uh the model on a structure that was you know released across the field up until a certain time.
And you know, one of the things that we didn't talk about that was really critical in all this development is the uh PDB, which is the protein data bank, is this common resources, basically common database where every biologist can uh publish their structures, and so we can you know train on you know all the structures that we put in the PDB until a certain date, and then we basically look for recent structures, okay, which structures look pretty different from anything that was published before, because we really want to try to understand generalization.
And then on this new structure, we evaluate all these different models.
And so you just know when Alpha Fold was three was trained, you know, when you're you intentionally trained to the same data or something like that.
Exactly.
Right.
Yeah.
And so this is kind of the way that you can somewhat easily kind of compare these models.
Obviously, that assumes that you know the training.
You've always been very passionate about validation.
I remember like diff doc, and then there was like diff doc L in DocGen.
You've thought very carefully about this in the past.
Like actually, I think Docgen is like a really funny story that I think I don't know.
I don't know if you want to talk about that.
It's an interesting like Yeah, I think one of the amazing things about putting things open source is that we get a ton of feedback from the field.
And you know, sometimes we get kind of great feedback of people really liking the model.
But honestly, most of the times, uh, you know, to be honest, that's also maybe the most useful feedback is you know, people sharing about where it doesn't work.
And so, you know, at the end of the day, it's critical.
And this is you know also something, you know, across other fields of machine learning.
It's always critical to set to do progress in machine learning and set clear benchmarks.
And as you know, you start doing progress of certain benchmarks, then you know you need to improve the benchmarks and make them harder and harder.
And this is kind of the progression of you know how the field operates.
And so, you know, the example of of uh doc gen was, you know, we published this initial uh model called Divdoc in my first year of PhD, which was sort of like you know, one of the early uh models to try to predict bio kind of interactions between protein small molecules, um that we bought a year after uh Alpha Fold 2 was published.
And now on the one end, you know, on these benchmarks that we were using at the time, uh Div Doc was was doing uh really well kind of you know uh outperforming kind of some of the traditional physics-based methods.
But on the other hand, you know, when we started, you know, kind of giving these uh tools to kind of many uh biologists, and uh one example was uh that we collaborated with was the group of Nick Politzi at Harvard.
Uh we noticed started noticing that there was this clear pattern where for proteins that were very different from the ones that we're trained on, the models was struggling.
And so, you know, that seemed clear that you know this is probably kind of where we should you know put our focus on.
And so we first developed, you know, with uh Nick and his group a new benchmark, and then you know, went after and said, okay, what can we change and kind of about the current architecture to improve this uh pattern of generalization?
And this is the same that you know we're still doing today, you know, uh, kind of where does the model not work, you know, and then you know, once we have that benchmark, you know, let's try to uh throw everything we uh any ideas that we have of the problem.
And there's a lot of like healthy skepticism in the field, which I think you know is is is great.
And I think you know it's very clear that there's a ton of things the models don't really work well on.
But I think one thing that's probably you know undeniable is just like the pace of how the pace of progress, you know, and how how much better we're getting, you know, every year.
And so I think if you you know, if you assume you know any constant you know rate of progress moving forward, I think things are gonna look pretty cool in some point in the future.
ChatGPT was only three years ago.
Yeah, I mean it's wild, right?
Like, yeah, yeah.
Yeah, it's one of those things, like even being in the field, you don't see it coming, you know, and like I think, yeah, hopefully we'll you know we'll we'll continue to have as much progress we've had the past few years.
So this is maybe an aside, but I I'm really curious you get this great feedback from the from the community, right?
By being open source.
My question is partly like, okay, yeah, if you open source and everyone can copy what you did, but it's also maybe balancing priorities, right?
Where you like all my customers are saying I want this, there's all these problems with the model, yeah, yeah, but my customers don't care, right?
So, like, how do you how do you think about that?
Yeah, so I would say a couple of things.
One is you know, part of uh our goal with Bolts, and you know, this is also kind of established as kind of the mission of the public benefit company that we started, is to democratize the access to these tools.
But one of the reasons why we realized that Bolts needed to be a company, it couldn't just be an academic project, is that putting a model on GitHub is definitely not enough to get you know chemists and biologists across you know, both academia, biotech, and and pharma to use your model to uh in their therapeutic programs.
And so a lot of what we think about, you know, at Bols beyond kind of the just the models is thinking about all the layers that come on top of the models to get you know from you know those models to something that can really enable scientists in the industry.
And so that goes into building kind of the right kind of uh workflows that take in kind of, for example, the data and try to answer kind of directly that those problems that you know the chemists and the biologists are asking, and then also kind of building the infrastructure.
And so this to say that you know, even with models fully open, you know, we see a ton of potential for you know products in the space.
And the critical part about a product is that even you know, for example, with an open source model, you know, running the model is not free.
You know, as we were saying, these are a pretty expensive model, and especially, and maybe we'll get into this, you know, these days we're seeing kind of a pretty dramatic inference time scaling of these models where you know the more you run them, the better the results are.
But there, but there you know, you start getting into a point that compute and compute cost becomes a critical factor.
And so putting a lot of work into building the right kind of infrastructure, building the optimizations and so on, really allows us to provide you know a much better service potentially to the open source models.
That to say, you know, even though you know with a product we can provide a much better service, I do still think and we will continue to put a lot of our models open source because the critical kind of role of think of open source models is you know helping kind of the community progress on the research and you know, from which we we all benefit.
And so, you know, we'll continue to on the one hand, you know, put some of our kind of base models open source so that the field can build on top of it.
And you know, as we discussed earlier, we learn a ton from you know the way that the field uses and builds on top of our models, but then you know, try to build a product that gives the best experience possible to scientists so that you know, like a chemist or a biologist doesn't need to you know spin off uh HPU and you know set up you know our open source model in a particular way, but can just you know a bit like you know, I even though I am a computer scientist, machine learning scientist, I don't necessarily take an open source LLM and try to kind of spin it off.
But you know, I just maybe open the ChatGPT app or a clot code and just use it as an amazing product.
Uh, we kind of want to give the same experience to scientists from the world.
I heard a good analogy yesterday that a surgeon doesn't want the hospital to design a scalpel, right?
So just buy the scalpel.
You wouldn't believe like the the number of people, even like in my uh short time, you know, between uh fault three coming out and and the end of the PhD, like the number of people that would like reach out just for like us to like run up a fold three for them you know or things like that just because like you know bolts in our case you know just because it's like not that easy you know to do that you know if you're not a computational person and I think like part of the goal here is also that you know we continue to obviously build an interface with computational folks but that the you know the models are also accessible to like a larger broader audience and and then that comes from like you know good interfaces and stuff like that.
I think one like really interesting thing about bolts is that with the release of it you didn't just release a model but you created a community.
Yeah.
Did that community it grew very quickly did that surprise you and like what is then the evolution of the community and how is that fed into bolts company if you look at its growth it's it's like very much like when we release a new model it's like there's a big big jump.
But yeah it's I mean it's been great you know we have a Slack community that has like thousands of people on it and it's actually like self sustaining now which is like the really nice part because you know it's it's almost overwhelming, I think, you know, to be able to like answer everyone's questions and help.
It's really difficult, you know, with the the few people that we were.
But it ended up that like you know, people would answer each other's questions and like sort of like you know, help one another.
And so the the the Slack, you know, has been like kind of yeah, self self-sustaining, and that's been that's been really cool to see.
Um, and um, you know, that's that's for like the Slack bar, but then also obviously on GitHub as well.
We've had like a nice, nice community.
Um, you know, I think we also aspire to be even more active on it, you know, than we've been in the past six months, it's been like a bit challenging, you know, for us.
But um, yeah, the community has been has been really great.
And you know, there's a lot of papers also that have come out with like new evolutions on top of bolts.
And it's surprised us to some degree because like there's a lot of models out there.
And I think like, you know, sort of people converging on that was was really cool.
And you know, I think it speaks also, I think, to the importance of like, you know, when when you put code out, like to try to put a lot of emphasis and like making it like as easy to use as possible and something we thought a lot about when we released the the code base.
You know, it's far from perfect, but you know.
Do you think that that was one of the factors that caused your community to grow is just the focus on easy to use, make it accessible?
I think so, yeah.
And we we've we've heard it from a few people over the over the years now.
And um, you know, and some people still think it should be a lot nicer.
And they're and they're right.
Uh and they're right.
But um, yeah, I think it was, you know, at the time maybe a little bit easier than than other things.
The other I think part uh I think led to to the community, and to some extent, I think, you know, like the uh somewhat the trust in the community and kind of what we what we put out is the fact that you know it's not really being kind of you know one model, but and maybe we'll talk about it, you know, after bolts one, you know, there were maybe another couple of models kind of released, you know, uh, or open source kind of soon after.
We kind of continued kind of that open source journey, or at least bolts two, where we were not only improving kind of the structure prediction, but also starting to do affinity predictions, understanding kind of the strength of the interactions between these different models, which is this critical component, critical property that you often want to optimize uh in discovery programs, and then you know, more recently also kind of putting design uh model.
And so we've sort of been building this suite of of models that come together, interact with one another, where you know, kind of there is almost an expectation that you know we we take very at heart of you know, always having kind of you know, across kind of the entire suite of different tasks, the best or across the best model uh out there, so that it's sort of like our open source tool can be kind of the go-to model for everybody in the in the industry.
I really want to talk about BoltsGen, but before that, one last question in this direction.
Was there anything about the community which surprised you?
Were there any like if someone was doing something and you're like, why would you do that?
That's crazy, or that's actually genius.
And I never would have thought about that.
I mean, we've had many contributions.
I think like some of the interesting ones, like, I mean, we had you know, this one individual who like wrote like a complex GPU kernel, you know, for part of the architecture.
On a piece of the funny thing is like that piece of the architecture had been there since Alpha Fold 2.
And I don't know why it took bolts for this, you know, for this person to you know to decide to do it, but that was like a really great contribution.
We've had a bunch of others, like you know, people figuring out like ways to you know hack the model to do cyclic peptides, like you know, there's I don't know if there's any other interesting.
And this was you know something that initially was uh proposed as you know as a message in the Slack channel by Timothl was basically he was you know there are some cases, especially, for example, we discussed you know antibody antigen interactions where the models don't necessarily kind of uh get the right answer.
What he noticed is that you know the models were somewhat stuck into predicting kind of the the antibody to interact with a part of the antigen that was incorrect.
And so he basically ran the experiments in this model.
You can condition, basically you can give hints.
And so he basically gave you know random hints uh to the model, basically, okay.
You should buy into this residue, uh, you should buy into the first residue, or you should buy into the 11th residue, or you should buy to the 21st residue, or you know, basically every 10 residues scanning the entire antigen.
And residues are the the the amino acids amino so the first amino acids, the 11th amino acid, and so on.
So it's sort of like doing a scan and then you know, conditioning the model to predict all of them, and then looking at the confidence of the model in each of those cases and taking the top.
And so it's sort of like a very somewhat crude way of doing kind of inference time search, but surprisingly, you know, for antibody antigen friction it actually kind of helped quite a bit.
And so there's some you know interesting ideas that you know as a um obviously as kind of developing the model you say kind of you know wow this is why would the model you know be so dumb but you know it's it's very interesting and and that you know leads you to also kind of you know start thinking about okay how do why can I do this you know not with this brute force but you know in a in a smarter way and so we've also done a lot of work on that direction.
And that speaks to like the you know the power of scoring we're seeing that although I'm sure we'll talk about it more when we talk about Bulls gem but um you know our ability to like take a structure and determine that that structure is like good you know like somewhat accurate whether that's a single chain or like an interaction is a really powerful way of improving you know the the models like sort of like you know if you can sample a ton and you assume that like you know if you sample enough you're likely to have like you know the good structure then it really just becomes a ranking problem.
And you know now we're you know part of the inference time scaling that Gabriel was talking about is is very much that.
It's like, you know, the more we sample, the more we like, you know, the ranking model ends up finding something it really likes.
Um, and so I think our ability to get better at ranking, I think is also what's going to enable sort of the next, you know, next big big breakthroughs.
Interesting.
But I guess there's a my understanding, there's a diffusion model and you generate some stuff, and then you I guess it's just what you said, right?
Then you rank it using a score, and then you finally um and so like can you talk about those different parts?
Yeah.
So, first of all, like the one of the critical kind of you know, beliefs that we had, you know, also when we started working on Boltz1 was sort of like the structure prediction models are somewhat, you know, R-field version of some foundation models, you know, learning about kind of how proteins and other molecules interact, and then we can leverage that learning to do all sorts of other things.
And so with Boltz two, we leverage that learning to do things affinity predictions.
So understanding kind of, you know, if I give you this protein, these small molecules, how tightly is the interaction.
Uh for Boltzchain, what we did was taking kind of that kind of foundation models and then fine-tune it to predict kind of entire new proteins.
And so the way basically that that works is sort of like instead of for the protein that you're designing, instead of fitting in an actual sequence, you feed in a set of blank tokens and you train the models to you know predict both the structure of kind of that protein and with the structure also what the different amino acids of that proteins are.
And so basically the way that uh Boltzhain operates is that you feed uh this a target, a protein that you may want to kind of bind to, or you know, another DNA RNA, and then you feed the high-level kind of design specification of you know what you want your new protein uh to be.
For example, it could be like an antibody with a particular framework, could be a peptide, could be many other things.
And that's with natural language or within.
And that's you know, basically, you know, prompting, and we have kind of this sort of like spec that you you specify, and you know, you feed kind of this this spec to the model, and then the model translates into you know a set of you know tokens, a set of conditioning to the model, a set of you know blank tokens, and then you know, basically the codes as part of the diffusion models decodes a new structure and a new sequence uh for your your protein, and you know, basically, and then we take that and as Jeremy was saying, you know, trying to score it, uh you know, how good of you know, a binder it is to the original target that you can you're using basically bolts to predict the folding and the affinity to that molecule.
So and then that is your that kind of gives you a score.
Exactly.
So you you use this model to predict the structure, and then you do two things.
One is that you predict the structure and with something like bolts two, and then you basically compare that structure with what the model uh predicted, what bolts chain predicted.
And this is sort of like in the field calls consistency.
It's basically you want to make sure that you know the structure that you're predicting is actually what you're trying to design, and that gives you a much better confidence that you know that's a good design.
And so that's the first filtering.
And the second filtering that we did uh as part of kind of the the Boltz-Chun pipeline uh that was released is that we look at the confidence that the model has in the structure.
Now, unfortunately, kind of going to your your question of you know predicting affinity, unfortunately, confidence is not a very good predictor of affinity, and so one of the things that we've actually done a ton of progress, you know, since uh released Boltzchain, and uh kind of we have some uh new results that we are gonna kind of announce soon is kind of you know the ability to get much better uh heap rates when instead of you know trying to rely on confidence of the model, we are actually directly trying to predict the affinity of that interaction.
Okay, just backing up a minute, so your diffusion model actually predicts not only the protein sequence but also the folding of it, exactly.
And actually, kind of the way uh one of the big different things that we did compared to other models in the space, and you know, there were some papers that already kind of done this before, but we uh really scaled it up, was you know, basically somewhat merging kind of the structure prediction and the sequence prediction into almost the same task.
And so the way that Bolt Strength works is that you are basically the only thing that you're doing is predicting the structure.
So the only sort of like supervision is we give you a supervision on the structure.
But because the structure is atomic, and you know, the different amino acids have a different atomic composition.
Basically, from the way that you place the atoms, we also understand not only kind of the structure that you wanted, but also the identity of the amino acid that you know the models believed was there.
And so we've basically instead of you know having these two supervision signals, you know, one discrete, one continuous, that somewhat you know, don't interact well together.
We sort of like build kind of like an encoding of you know sequences in structures that allows us to basically use exactly the same supervision signal that we are using to bolts too that you know uh you know, largely similar to what Alpha Volfree uh proposed, which is very scalable, and we we can use that to design new proteins.
Interesting.
Maybe a quick shout-out to Hannes uh stuck on our team who like did all this work.
Um yeah, yeah, it was that was a really cool idea.
I mean, like looking at the paper, and there's this like encoding or you just uh add a bunch of, I guess, kind of yeah atoms, which can be anything, and then they get sort of rearranged and then basically plopped on top of each other, so that and then that encodes what the amino acid is, and there's sort of like a unique way of doing this.
That was that was like such a really such a cool, fun idea.
I think that idea was had existed before.
Yeah, there were a couple of papers that had proposed this and and Anis really took it to um to the large scale.
In the paper, a lot of the paper for both gen is dedicated to actually the validation of the model.
In my opinion, and all the people we basically talk about feel that this sort of like in the wet lab or whatever the appropriate, you know, sort of val like in real-world validation is the whole problem, almost, or not the whole problem, but a big giant part of the problem.
So can you talk a little bit about the highlights from there that really because it's to me, uh the results are impressive both from uh the perspective of the you know the model and also just the effort that went into the validation by a large team?
First of all, I I think I should try saying is that both when we were at MIT and Tommy Syakola's and working at Barcelay's lab, as well as at Boltz, uh, you know, we are not uh we were not a biolab and you know, we're not a therapeutic company.
And so to some extent, you know, we were first uh forced to you know look outside of you know our group, our um team to do the experimental validation.
One of the things that really Anis in the team pioneer was the idea, okay.
Can we go not only to you know, maybe a specific group and you know, trying to find a specific system and you know, maybe overfit a bit to that system and and trying to validate, but how can we test these models across a very wide variety of different settings so that you know anyone in uh in the field and you know, printing design is you know, such a kind of wide task with all sorts of different applications from therapeutic to you know biosensors and many others that you know, so can we get a validation that is kind of goes across uh many different tasks?
And so he basically put together, you know, I think it was something like you know, 25 different you know, academic and industry labs that committed to you know testing uh some of the designs from the model, and some of this testing is still ongoing, and you know, giving uh results kind of uh back to us in exchange for you know, hopefully getting some you know new gray sequences for uh their task.
And he was able to you know coordinate this you know very wide uh set of you know uh scientists.
And uh already in the paper, I think we shared results from I think uh eight to ten different labs, uh kind of showing results from you know designing peptides, uh designing uh to target, you know, ordered proteins, peptides uh targeting disorder proteins, we showed results, you know, of uh designing proteins that bind to small molecules, we showed results of you know designing nanobodies and across a wide variety of different targets.
And so that sort of like gave to the paper a lot of you know validation to the model a lot of validation that was kind of uh white uh and so those would be therapeutics for those animals, or are they relevant to humans as well?
They're relevant to humans as well.
Uh obviously you need to do some work into quote unquote humanizing them, making sure that you know they have the right characteristic to so they're not uh toxic to humans and so on.
There are um some approved medicine in the market that are nanobodies.
There's a general pattern, I think, in like in trying to design things that are smaller, you know, like it's easier to manufacture.
At the same time, like that comes with like potentially other challenges, like maybe a little bit less selectivity than like if you have something that has like more hands, you know.
But the yeah, there's this big desire to you know try to yeah, design mini proteins, nanobodies, small peptides, you know, that just are just great drug modalities.
Okay.
I think we were left off.
We were talking about validation in the lab.
And I was very excited about seeing like all the diverse validations that you've done.
Can you go into some more detail about them?
Yeah.
Specific ones.
Yeah, the nanobody one, I think we did what was it, 15 targets?
Is that correct?
14.
14 targets testing.
So we typically the way this works is like we uh make a lot of designs, all right, on the order of like tens of thousands, and then we like rank them and we pick like the top N.
Uh in this case, n was 15, right?
For each target.
And then we like measure sort of like the success rates, both on like how many targets we were able to get a binder for, and then also like more generally, like out of all of the you know, binders that we designed, how many actually proved to be uh good binders.
Some of the other ones I think involved like, yeah, like uh we had a cool one where there was a small molecule and we designed a protein that you know binds to it.
That has a lot of like interesting applications, you know.
For example, like Gabri mentioned like biosensing and things like that, uh, which is pretty cool.
We had a disordered protein, I think you mentioned also.
And yeah, I think some of maybe some those were some of the the highlights.
Yeah, so I would say that the way that we structure kind of some of those validations uh was on the one end, we have validations across a whole set of different problems that you know the biologists that we were working uh with came to us with.
So we were trying to, for example, in some of the experiments design peptides that would uh target RAC C, which is a target that is involved in metabolism.
And we had you know a number of other applications where we were trying to design you know, peptides or other modalities against some other therapeutic relevant targets.
We designed some um proteins to bind small molecules, and then some of the other testing that we did was really trying to get like a more broader sense of how does the model work, especially when tested, you know, on somewhat generalization.
So one of the things that you know we we found with the field was that a lot of the validation, especially outside of the validation that was done on specific problems, was done on targets that have a lot of you know known interactions in the training data.
And so it's a bit always a bit hard to understand, you know, how much are these models really just regurgitating kind of what they've seen or trying to imitate what they've seen in the training data versus you know, really be able to design uh new proteins.
And so one of the experiments that we did was to uh take nine targets from the PDB, filtering to things where there is no known interaction uh in in the PDB.
So basically the model has never seen kind of this particular protein bound or a similar protein bound to another protein.
So there is no way that the model from its training set can sort of like say, okay, I'm just going to uh kind of tweak something, tweak something and just imitate this particular kind of interaction.
And so we we took design proteins, we worked with uh adaptive CRO and basically tested you know 15 mini proteins and 15 nanobodies against each one of them.
And the very cool thing that we saw was that on two-thirds of those targets, we were able to from these 15 designs get uh nanomolar uh binders, nanomolar, roughly speaking, just a measure of you know how strongly kind of the interaction is, uh roughly speaking, kind of like a nanomolar binder is approximately the kind of binding strength of binding that you need for a therapeutic.
Okay, yeah.
So maybe switching uh directions a bit, Boltz Lab was just announced um this week or was it last week?
Yeah, yeah.
This is like your first, I guess, product, if if that's the if you want to call it that.
Can you talk about what Boltzlab is and um yeah, you know, what you hope that people take away from this?
Yeah, you know, as we mentioned, like I think at the very beginning, is the goal with the product has been to you know address what the models don't on their own.
And there's largely sort of two categories there.
I'll split it in three.
The first one, it's one thing to predict, you know, a single interaction, for example, like a single structure.
It's another to like, you know, very effectively search a space, uh, design space to produce something of value.
What we found, like sort of building up this product is that there's a lot of steps involved, you know, in that that we sort of need to like, you know, accompany the user through.
You know, one of those steps, for example, is like, you know, the the creation of the target itself, you know, how do we make sure that the model has like a good enough understanding of the target so we can like design something and there's all sorts of tricks, you know, that you can do to improve like a particular, you know, structure prediction.
And so that's sort of like you know, the first stage.
And then there's like this stage of like, you know, designing and searching the space efficiently.
You know, for something like Bulls gen, for example, like you, you know, you you design many things and then you rank them.
For example, for small molecule, the process is a little bit more complicated.
We actually need to also make sure that the molecules are synthesizable.
And so the way we do that is that, you know, we have a generative model that learns to use like appropriate building blocks such that you know it can design within a space that we know is like synthesizable.
And so there's like, you know, this whole pipeline really of different models involved in being able to design a molecule.
And so that's been sort of like the first thing.
We call them agents.
We have a protein agent and we have a small molecule design agents.
And that's really like at the core of like what powers you know the BOLS lab platform.
So these agents are they like a language model wrapper or they're just like your models and you're just calling them agents because they they they sort of perform a function on behalf of the phone.
They're more of like uh, you know, a recipe, if you wish.
And I think uh we use that term sort of because of, you know, sort of the complex pipelining and automation, you know, that goes into like all this plumbing.
So that's the first part of the product.
The second part is the infrastructure.
You know, we need to be able to do this at very large scale for any one, you know, group that's doing a design campaign.
Let's say you're designing, you know, I'd say a hundred thousand possible candidates, right, to find the good one.
That is you know a very large amount of compute.
Uh, you know, for small molecules, it's on the order of like a few seconds per design.
For proteins, it can be a bit longer.
And so, you know, ideally you want to do that in parallel, otherwise it's gonna take you weeks.
And so, you know, we've put a lot of effort into like, you know, our ability to have a GPU fleet that allows any one user, you know, to be able to do this kind of like large parallel search.
So you're amortizing the cost over your users, basically.
Exactly.
And you know, to some degree, like it's whether you use 10,000 GPUs for like, you know, uh a minute is the same cost as using, you know, uh one GPUs for God knows how long, right?
So you might as well try to paralyze if you can.
So, you know, a lot of work has gone has gone into that, making it very robust, you know, so we can have like a lot of people on the platform doing that at the same time.
And the third one is is the interface.
And the interface comes in in two shapes.
One is in form of an API, and that's you know, really suited for companies that want to integrate, you know, these pipelines, these agents directly in existing, you know, workflows that they have or like existing user interfaces that they have.
And we're already like partnering with you know a few distributors, you know, that are gonna integrate our API.
And then the second part is new user interface.
And you know, we we've put a lot of thoughts also into that.
And this is when I I mentioned earlier, you know, this idea of like broadening the audience.
That's kind of what the the user interface is about.
And we've built a lot of interesting features in it, you know, for example, for collaboration, you know, when you have like potentially multiple medicinal chemists are going to the results and trying to pick out, okay, like what are the molecules that we're going to go and test in the lab.
It's powerful for them to be able to, you know, for example, each provide their own ranking and then do consensus building.
And so there's a lot of features around launching these large jobs, but also around like collaborating on analyzing the results that we try to solve, you know, with with that part of the platform.
And so Boltz Lab is sort of combination of these three objectives into like one, you know, sort of cohesive platform.
Who is this accessible to?
Everyone.
You do need to request access today.
We're still like, you know, sort of ramping up the usage, but anyone can request access.
If you are an academic in particular, we uh you know, we provide you know a fair amount of free credit so you can like play with the platform.
If you are a startup or a biotech, you may also reach out and we'll typically like actually hop on a call just to like understand what you're trying to do and also provide a lot of free credit to get started.
And uh of course, also with larger companies, we can deploy these this platform in a more like secure environment.
And so that's like more like custom you know deals that we make, you know, with with the partners.
You know, that and that's I sort of at the ethos of Boltz.
I think this idea of like uh servicing everyone and not necessarily like going after just you know the the really large enterprises, and that starts from the open source, but it's also you know main a key key design principle of of the product itself.
Yeah.
One thing I was thinking about with regards to infrastructure, like in the LLM space, you know, the cost of a token has gone down by I think a factor of a thousand or so over the last three years, right?
Yeah, yeah.
And is it possible that like essentially you can exploit economies of scale and infrastructure that you can make it cheaper to run these things yourself than for any person to roll their own screen?
100%.
Yeah.
I mean, we're already there, you know, like running bolts on our platform, especially on in a large screen is like considerably cheaper than it would probably take anyone to put the open source model out there and run it.
And and on top of the infrastructure, like one of the things that we've been working on is accelerating the models.
So, you know, our our small molecule screening pipeline is 10x faster on Boltz Lab than it is in the open source, you know, and that's that's also part of like you know, building building a product, you know, of something that that scales really well.
And we really wanted to get to a point where like you know, we could keep prices uh very low in a way that it would be a no no-brainer, you know, to to use bolts through through our platform.
How do you think about validation of your like agentix systems?
Because you know, as you're saying earlier, like we're alpha fold style models are really good at let's say monomeric, you know, proteins where you have you know co-evolution data.
But now suddenly the whole point of this is to design something which doesn't have right, you know, co-evolution data, something which is really novel.
So now you're basically leaving the the domain that you thought was you know that you you know you were good at.
So like how do you validate that?
Yeah, I like Gabriel Complete, but there's um there's obviously uh you know a ton of computational metrics that we rely on, but those are only take you so far.
You really gotta go to the lab, you know, and and test, you know, okay, with this method A and this method B, how much better are we?
You know, how much better is my uh my hit rate, how stronger are my binders also?
It's not just about hit rate, it's also about how good the binders are.
And there's really like no way, nowhere around that.
I think we're you know, we've really ramped up the amount of experimental validation that we do so that we like really track progress, you know, as scientifically sound, you know, as as possible.
Yeah, no, I think you know, one thing that is unique about using maybe companies like us is that because we're not working on like maybe a couple of therapeutic pipelines where you know our validation would be focused on those.
We when we do an experimental validation, we try to test it across tens of targets.
And so that on the one end, we can get a much more statistically significant results and really allows us to make progress from the methodological side without being you know steered by you know overfitting on any one particular system.
And of course, we choose, you know, we always try to choose uh targets and problems are sort of like at the frontier of what's possible today.
So, you know, you don't want something too easy, you don't want something too hard, otherwise you're not gonna see progress.
And so, you know, this is a somewhat evolving set of targets.
We talked earlier about the targets that we looked at with with Boltrend.
Now we are even trying kind of you know, even harder targets, both for us for molecules and proteins.
And so we try to keep ourselves on the on the boundary uh of what's possible.
So do you have like infrastructure, or is this like you just have a lot of different partnerships with academic labs and you're just gonna keep pushing on these and driving these?
We do partially this through academic labs, but more and more we do this through uh CROs, just because of you know, to some extent, it's also we need kind of replicability, often kind of you know going after the same targets, you know, multiple times, and you know, to see the the progress from you know one month to the next.
And speed and speed speed of execution.
Yeah.
So what happens if you start getting a bunch of like really strong binders against therapeutic targets?
What do you do?
Um in open source, like yeah.
I mean, you know, I mean, we're when we say we have no interest in making drugs, we're serious, like you know.
Uh I mean when it when it was with the academic labs, basically the you know, I was they keep it, they do whatever they want with it.
And with the CROs so far, yeah, we've been we've been very yeah, releasing that.
You know, I I will also say, and and I think this is a bit been a bit of the issue that I have with with some of kind of the things I've been said in the field, is when we say that we design new proteins, or we say that we design new molecules, you know, go and bind these particular targets.
We should be very clear, you know, these are not drugs, you know, these are not things that are ready to be put into a human, and there is still a lot of development that goes with it.
And so this is this is kind of to us, you know, we see ourselves as you know building tools for scientists, you know.
At the end of the day, you know, it really relies on the scientist having a great therapeutic hypothesis and then pushing through kind of all the stages of development.
And you know, we try to build tools that can accompany them uh in that journey.
We it's not like a magic box where you know uh you can just turn it and get FDA approved drugs.
FDA approved drugs, FDA approved drugs.
Um but actually that brings up an interesting question that I have I've been wondering about is do you guys see yourself staying in this for lack of a better way of saying it layer?
Or do you think that you'll start to either on a physical sense looking at different layers of the virtual cell, so to speak, or also, you know, so there's the like the development process, it goes, you know, sort of like design pre-clinical, clinical approval, and thinking about in improving the performance throughout that process based on the designs, is that a direction that you guys are pushing?
Yeah, so one of the things that's Jeremy said, you know, we are not a therapeutic company, and we want to kind of stay not to be a therapeutic company, always be at the service of you know all the different you know, companies, including therapeutic companies that we serve.
And you know, that to some extent does mean, you know, that we need to try to, you know, go deeper and deeper and getting these models better and better.
One of the things that we are doing across many other uh in the field is, you know, now that we are ready, they're starting to be good both for small molecules and for proteins to design kind of binders, design relatively tight binders, is starting to look at all these other properties, you know, the call developabilities or at me that you know we care about when developing a drug and trying, can we design them from uh from a get-go?
The thing about those properties in some of them, you know, you need to, you know, start having an understanding of the cell.
And so that's on the one end, kind of why we need that understanding.
But also, you know, the way that we also think about all different and complex diseases is that these models and these tools that we're building have a good understanding of kind of you know biomolecular interactions and kind of their interactions.
Now, at the same time, every disease is often kind of unique and every therapeutic hypothesis is unique.
And so you maybe want to have something that needs to uh hit the particular, you know, let's say target in a virus in a particular way, but you don't maybe know exactly what uh way you want to do.
And so maybe in the first set of designs, you're gonna try to target different epitopes in different ways, and then you're gonna test them in the lab, maybe directly in vivo, and you're gonna see which ones work and which one don't.
And so then you need to bring those results back into the models, and then the models can start to have a more uh wider understanding, you know, not just of the biophysical of the uh antibodies interacting with that target, but also how that is shaped within the entire cell.
And so, first of all, you know, that means on the one end that we need, you know, kind of these loops, and this is also partially how we we design the platform to be, but that also means that we also need to start understanding more and more kind of higher level things, and you know, I wouldn't say that we're working in any way on like a virtual cell like uh others are, but we're definitely thinking kind of very deeply about kind of you know, how does you know, kind of the way that we target um certain proteins interfere, interact with you know, maybe pathways that are existing in the cell.
One question that has come up is you talk a lot about user interface and so on, and I think this is really important.
But like my experience with dealing with medicinal chemists, when you give them machine learning models, is they are the most superstitious, skeptical, like pseudo-religious people I've ever talked to when it comes to doing science.
So sorry for the municipal chemist listening.
Yeah, they're they're amazing.
Like they're absolutely so I've worked with some spectacular medicinal chemists who just pull magic out of their hat again and again, and I have no idea how they do it.
But when you bring them a machine learning model, it is sometimes quite tricky to get them to deal with it.
How has your interaction been with this?
And how have you thought about like building Bulslab to work with the skeptics?
One of the grave value unlocks for us and for a product has been when we brought to the team uh mission chemist.
His name is uh Jeffrey.
So I think kind of like on the one end, you know, day one, you know, he obviously had a lot of opinions on kind of a lot of the ways that we should uh uh change, you know, both kind of the way that the agents worked, the way that the platform worked.
Uh, but it's been really amazing, kind of, you know, once also we started kind of shaping kind of the platform uh in a better way with with this feedback, how we went from you know to some extent, you know, uh fair skepticism to him, you know, actually using a lot more compute than any of our uh computational uh folks in the team, you know, um at times that you know he's you know running, you know, he has all these sort of uh hypothesesis, okay.
Maybe I can hit this protein this particular way, I can hit it in the other way.
Actually, let me look at for this particular molecular space, uh, let me try to optimize for this particular interactions.
So he ends up, you know, running several screens in parallel, you know, using hundreds of GPUs, you know, on his own.
And you know, so this has been you know pretty incredible to see kind of how, you know, maybe the way that I was more thinking about a problem, which is okay, you're just trying to design a binder, uh small molecule to a particular protein.
The way that he thinks about it is you know, much more deeply and you know, trying all these different things, these different hypotheses.
And then, you know, once he gets the results from the model, he doesn't just you know take the top 15, uh, but he really kind of looks over and you know, kind of tries to understand you know the different things, and then when we uh select you know, maybe some designs to bring forth, you know, he has you know something where you know both the models understand that something is good, but all himself as well.
And that's why we also built kind of the platform to be uh an interface for you know this kind of uh this kind of chemist and you know also like a collaborative experience.
I think at the end of the day, like you know, for people to be convinced, you have to show them something that they didn't think was possible.
And until you have that aha moment, you know, I think the skepticism will remain.
But then when you know, every once in a while I think there's like a result that like really surprises people, and then it's like, oh wow, okay, this is actually I can do something with this.
So you just get in their hands, have them try it out, and they'll be convinced.
Yeah, or like at maybe once the lab results come back.
Or their their friend, yeah, or maybe one of their colleagues is convinced.
And yeah, I think it it takes going to the lab, I think, at some point.
There's no avoiding that, you know, as beautiful as the platform can be, as nice as the molecules might look, you know, that the model predicted.
I think what really convinces people is like, you know, hits.
Yeah.
Yeah.
You see the results.
Exactly.
Yeah.
Cool.
Thank you for you know taking the time to chat with us.
Yeah.
Is there anything that you would like your audience to know?
I mean, first of all, you know, uh, we're just getting started, you know, uh continuing to build a team.
And so uh definitely always looking for uh great folks, both on you know, kind of you know software side, you know, machine learning side, but also scientists uh to join the team and help us, you know, um shape on the infrastructure side too.
Indeed.
If you if you think that if you want a new challenge, because this is not just next token prediction, this is really a new engineering challenge that hasn't been under the if you if no matter you know how much experience you have with you know biologists and chemistry, if you want to come you know help us in a shape what you know biology and chemistry hopefully will look like in five, ten years, um, we'd love to hear from you.
And so um go to BoltStop Bio and you know, come join the team.
Cool.
Thank you.
Awesome.
Thank you so much so much.
Thank you.