Rather than reviewing a paper, this week’s post takes a broader view on where we are at with respect to interpretability in genomics ML models.

As the field continues to rigorously interrogate the most recent ML genomics models, it has become clear that it’s incredibly easy to fool ourselves about what these models are learning, what they can predict, and how to do better.

One component of improving on the current state is to get more serious about interpretability. Most interpretability efforts remain retrospective—feature rankings, attention maps, or gradient plots that rationalize outputs but rarely reveal how, or whether, the model’s reasoning aligns with biology.

If we care about mechanism (and we should, because this is how models become more extensible and useful), we need a shift in stance. Interpretability should not be a gloss applied at the end of analysis, it should be part of how models are built, tested, and revised.

Here I posit that there are four questions worth asking of every architecture and dataset to help us move in that direction.

What are we interpreting: mechanisms, predictions, or confounds?

Each target demands a different standard of evidence. Mechanistic interpretability seeks causal structure; predictive interpretability seeks justification; artifact detection seeks bias. Without distinguishing them, we risk mistaking coherence for truth.

What biological hypotheses are encoded in the model architecture?

Every design choice carries an implicit worldview: MLPs flatten dependencies; GNNs canonize known graphs; transformers elevate context as signal. These are not neutral—they shape what the model is capable of discovering, and what it will systematically miss.

Can multimodal data be used to falsify interpretations?

Adding data layers isn’t just about increasing modeling power. Done correctly, an additional modality can act to challenge the others, serving as an independent test of whether the model’s inferences hold up under a different lens.

How can interpretability inform model iteration?

Used well, interpretability is diagnostic. It surfaces blind spots: missing biological priors, unrepresentable hierarchies, or architectural constraints that obscure mechanism. Those failures are invitations to refine both model and experiment.

AI ∩ Bio

Illustration of scientific paper with the title "AI ∩ Bio: Reading the Revolution"

More in this series

  1. ChatNT: The future of biological assistants—or a mirage in a lab coat?
  2. GET: A Foundation Model for Transcription, Still Between Promise and Proof
  3. X-Atlas/Orion: Your Model is Only as Good as Your Training Data
  4. From Better Models to Better Questions: A Pathology AI Rethink
  5. The Model Isn’t the Magic: How Cytoland shows that domain expertise—not just deep learning—is what makes AI in biology work.
  6. Boltz-2: How much can 3D structure really tell us about molecular binding energetics?
  7. Investigating the volume and diversity of data needed for generalizable antibody–antigen ΔΔG prediction
  8. Beyond Binding: Rethinking Drug Design in the Age of AI and Structural Biology
  9. Testing the Physics Beneath the Predictions Beyond RMSD: What AlphaFold3 Really Understands
  10. Hype, Hurdles, and Hepatotoxicity: A Bold Step for AI-Designed Drugs, But Still Miles to Go
  11. When Complexity Misleads
  12. What Are Genomic Transformers Actually Learning?
  13. mRNABench and the Future of AI in Biology: Why Domain Knowledge Wins
  14. OncoGAN Creates Synthetic Cancer Genomes, Opening New Paths for Privacy-Preserving Precision Oncology
  15. Readable Rules, Testable Models: A New Grammar for Virtual Cells
  16. Multimodal CustOmics: Fusing Pathology Images and Tumor Genomics for Next-Gen Cancer Diagnostics
  17. Beyond Perturbation Simulations: PDGrapher Shows a Faster Way to Identify Actionable Targets
  18. Can AI design epigenetic anti-aging strategies?
  19. What happens when physicians use GPT-4 for diagnosis
  20. Can generative AI predict emergent phenomena?
  21. DeepSomatic and the question of how AI learns from itself
  22. Toward Mechanism-Centric Interpretability in Genomic Machine Learning
  23. From AlphaEvolve to DeepEvolve: What We’re Learning About Machine-Led Scientific Discovery
  24. Kosmos and the Culture of Discovery
  25. From Embeddings to Insight
  26. From Prediction to Explanation: How BIOREASON Reframes Genomic AI as a Reasoning Problem
  27. AI Models Need Better Truth—Platinum Pedigree Shows How
  28. From Evolutionary Intolerance to Clinical Insight: What popEVE Teaches Us About Missense Variants
  29. Interpretable Latent Spaces, Messy Biology: What AUTOENCODIX Teaches Us About Autoencoders in the Wild
  30. The Gene Ontology Knowledgebase in 2026

Why it matters

Interpretability is not a transparency feature; it’s a scientific claim about correspondence between computation and biology.
And like any scientific claim, it must be testable, falsifiable, and revised in light of evidence.