Tag: ai models

Illustration of a desk with a figure from a recent paper about Deep-learning-based gene perturbation

When Complexity Misleads

On August 9, 2025

In this week’s AI ∩ Bio paper, we look at a multi-dataset benchmark where deep learning models trained to predict single-cell perturbation outcomes did not outperform two deliberately simple baselines: “assume nothing changes” and “predict the additive combination of single perturbations.” Performance improved when models were reduced to just their internal embeddings and a linear decoder—no deep layers, just a direct projection from embedding space to gene expression. In other words, the models had learned biologically meaningful representations during training, but using those representations to make predictions through a complex decoder often made things worse.

This result may not surprise those familiar with traditional machine learning approaches. As the bias–variance tradeoff reminds us: when signal is weak and noise is high, simpler models often generalize better and overfit less, leading them to outperform more complex alternatives.

This study revealed both conditions:

The signal from genetic perturbations is often modest or additive.
The noise in single-cell gene expression is high, due to dropout, amplification bias, and biological variability.

If your model must detect subtle shifts in expression, but your measurement system is saturated with noise, then even an optimal model will struggle to generate meaningful predictions. And a large model may do worse, because it fits spurious correlations that simpler models ignore.

AI ∩ Bio

Before adding more architecture, it’s worth asking:

Have we clearly defined our outcome?
Do we understand the system we are working with?
Can our data realistically support the question we’re asking?

Otherwise, we risk building a computational Rube Goldberg machine: impressive, fragile, and ultimately unclear.

Illustration of a desk with a figure from a recent paper about AlphaFold3 Evaluations

Testing the Physics Beneath the Predictions Beyond RMSD: What AlphaFold3 Really Understands

By Kristin Gleitsman

On July 27, 2025

In AI ∩ Bio

This week’s paper is personal for me.

Dan Herschlag was my postdoc advisor. Dan taught me how to think mechanistically, how to test assumptions, and how to pursue scientific clarity with unflinching rigor.

That legacy is all over this paper. And is incredibly needed in this era of AI hype where data volume often sidelines careful model-driven science.

Herschlag et al. show that while AlphaFold3 predicts protein structures with backbone-level structural precision, it struggles to capture the physical rules that govern biological function.

But this paper isn’t a takedown: it’s a roadmap for how to go further.

What They Did

Rather than rely on RMSD alone, the team evaluated AlphaFold2 and AlphaFold3 against:

Energetic rules: bond torsions, hydrogen bonds, and van der Waals contacts
Experimental ensembles: from multi-temperature crystallography
Model confidence: comparing pLDDT to physical plausibility

What They Found

~30% of side-chain interactions deviated from experimental observations, often with incorrect partners or implausible geometries
High-confidence predictions (>90 pLDDT) still showed strained or physically invalid conformations
Energetically-favorable conformations could increase RMSD and be penalized by the model
AlphaFold3 missed ~85% of conformational variability seen in real experimental ensembles

Key Insight: Physics ≠ Proximity

AlphaFold can recapitulate Ramachandran and Lennard-Jones-like patterns. But that doesn’t mean it understands physical constraints.
To improve these tools, we need evaluation metrics grounded in molecular energetics, not just geometry.

AI ∩ Bio

Takeaway for Scientists

This paper is a reminder that progress requires more than prettier predictions. It demands models that reflect the physics that drive biology.

Illustration of a desk with a figure from a recent paper about binding affinity predictions

Beyond Binding: Rethinking Drug Design in the Age of AI and Structural Biology

By Kristin Gleitsman

On July 23, 2025

In AI ∩ Bio

In molecular design, we often prioritize what’s measurable over what’s meaningful.

For decades, binding affinity has served as a cornerstone of early-stage drug discovery, not because it captures biological function in full, but because it’s one of the few properties we can quantify systematically and optimize across large libraries.

Now, as AI models generate binders faster than we can validate them, we must ask: What exactly are we optimizing for? And what datasets are we training on?

What we know:

Strong binding doesn’t guarantee efficacy
Residence time and conformational flexibility can matter more than affinity
Cellular context — target expression, pathway crosstalk, and off-target interactions — often dictates outcome in clinical applications

Yet much of the public data — and many AI training sets — still orbit around Kd, IC₅₀, and docking scores. These are abundant and easy to label, but they capture only a narrow slice of pharmacological reality (and we’re not even accounting for the fact that these measurements are highly dependent on the specific conditions- buffer, temperature, etc).

If we train models on what’s easy to measure, we shouldn’t be surprised when they generate molecules that impress in silico — and disappoint in vivo.

The problem isn’t that binding doesn’t matter. It does. The problem is that binding isn’t biology.

AI ∩ Bio

Toward More Meaningful Models

To do better, we’ll need to:

Incorporate multi-parametric data: kinetics, permeability, metabolism, toxicity, immune activation
Train models to include mechanism and uncertainty, not just affinity
Elevate datasets that link structure to systems, not just structure to scores

The best work ahead won’t just generate molecules; it will surface better models about how they work, and where they fail in the journey from discovery to clinic.

Illustration of a desk with a figure from a recent paper about Graphinity

Investigating the volume and diversity of data needed for generalizable antibody–antigen ΔΔG prediction

By Kristin Gleitsman

On July 19, 2025

In AI ∩ Bio

This week’s AI ∩ Bio: Reading the Revolution paper isn’t just a new model or a shiny dataset. It’s a reality check that cuts through superficial benchmark performance to ask harder questions about AI for antibody design.

While many models report high accuracy on test sets that closely resemble their training data, Hummer et al. go further, testing how models perform when challenged with unseen antibody–antigen complexes. This approach led to a rigorous investigation of why models fail, and what it would take to make ΔΔG prediction truly generalizable.

They introduce Graphinity, a graph neural network trained to predict changes in antibody–antigen binding affinity (ΔΔG). They rigorously stress-test the system: varying sequence identity cutoffs, benchmarking against other architectures, simulating noisy labels, and even filtering mutations by evolutionary plausibility.

The findings are clear and sobering: it’s not the model that’s the bottleneck. It’s the data: data size, data diversity, and data alignment with real-world biological utility.

Graphs showing considerations for experimental ΔΔG dataset generation, with respect to ML predictiveness

a,b, Model performance with varying training plus validation dataset size

Scientific Insight

The core contribution isn’t Graphinity’s design (though it performs well). It’s the authors’ commitment to not mistaking narrow wins for general insight. They show that a 0.9 Pearson correlation can be misleading if training and test data are too similar. That scaling dataset size doesn’t help if you’re mutating the same few complexes. And posit that no model—regardless of architecture—can learn to generalize if it’s never exposed to true diversity.

Crucially, they quantify the gap: at least 90,000 experimental ΔΔG values may be needed to reach robust generalization. And they begin to chart how we might get there—by prioritizing diverse substitutions, broad antigen coverage, realistic effect sizes, and evolutionarily grounded mutations—not just alanine scans.

To R&D leaders

This paper demonstrates what it looks like to take biological realism seriously. If you’re evaluating models for therapeutic antibody design, don’t just ask about top-line metrics. Ask how performance changes when similar complexes are removed from the test set. Dig deeply into what signal the model is learning and whether that signal will hold for the next antigen.

AI ∩ Bio

To early-career scientists

This paper helps us see what not fooling yourself looks like. It’s a reminder that model performance only matters when grounded in data that reflects the system’s true complexity. If you want to build models that generalize, start by designing experiments that enable that.

Illustration of a desk with figures from a paper on X-Atlas/Orion

X-Atlas/Orion: Your Model is Only as Good as Your Training Data

By Kristin Gleitsman

On June 27, 2025

In AI ∩ Bio

I hope you will forgive me this week, because this week’s paper isn’t an AI paper per se… it’s a methods and data generation paper, but one that sits squarely at the intersection of AI and biology. While much of the field is focused on building more powerful models, even the best architectures can’t reliably infer causality, dose effects, or subtle regulatory responses from observational data alone. Controlled perturbations are essential for training models that work towards explanation, not just correlation.

In this preprint, the team at Xaira Therapeutics, a next-gen biotech building AI-native drug discovery tools, introduces FiCS Perturb-seq, a scalable, industrialized platform for dose-aware single-cell CRISPR. And the open source dataset they generated using this method, X-Atlas/Orion, is something truly remarkable.

Scientific Insight

The core breakthrough here isn’t a novel discovery, it’s the industrialization of perturbation data generation. FiCS Perturb-seq standardizes the full pipeline: fixation, cryopreservation, FACS enrichment, automation, and high-density single-cell loading. This enables X-Atlas/Orion, a dataset of 8 million dual-sgRNA cells with deep, reproducible transcriptomic profiles—built not for exploration alone, but for training dose-aware, causal foundation models. A key analytic innovation: using sgRNA abundance as a proxy for knockdown strength, enabling dose-dependent transcriptional analyses that move beyond the binary logic of most Perturb-seq studies.

For R&D leaders, this paper is a great example for how you can scale experimental biology to meet the needs of machine learning. It’s not about a single step, it’s about asking the right questions about what data is really needed and designing the system to generate that data at scale: QC before sequencing, consistent cell handling, and deep coverage per perturbation.

AI ∩ Bio

To early-career scientists: this paper highlights the importance of infrastructure for discovery. The authors didn’t invent fixation or dual guides—they made them scalable, automatable, and reliable enough to support next-generation modeling. If you’re working at the interface of wet lab and ML, remember: your model is only as good as your training data. The signal (i.e., what am I really measuring and what can it actually tell me), the structure, and the variability you build into your system will define what the model learns and what it misses.