Appendix D: Case Studies
These short applied cases support the interpretation of life sciences AI claims. Each case maps a published result or capability claim to the three-tier evidence framework (Demonstrated, Theoretical, Beyond Current Capabilities) and the validation discipline the handbook applies elsewhere in detail. Cross-references to the relevant chapters follow each case.
Case 1: Structure Prediction as a Research Input
A team without an experimental structure uses AlphaFold DB to generate hypotheses about a protein domain. The correct use is hypothesis generation, followed by confidence review (pLDDT distribution, PAE matrix), domain inspection, and assay planning. The high-pLDDT regions are usable for docking and mutagenesis design; the low-pLDDT regions are predictions of disorder and should be treated as such.
The failure mode this case avoids: treating the structure visualization as if it were a measured structure. AlphaFold confidence metrics are decision aids; reading the visualization without reading the confidence is misuse.
References: AlphaFold Protein Structure Database, 2026; Jumper et al., 2021. See Protein Structure Prediction chapter.
Case 2: Protein Binder Design
A protein engineering group uses RFdiffusion for backbone proposals and ProteinMPNN for sequence design. The design is not a success until the prespecified experimental function and relevant biophysical properties support it. In-silico confidence and biophysical filters can reduce the candidate set before bench testing, but they are model outputs rather than measurements.
Evidence tier placement: backbone generation and inverse-folding sequence design are Demonstrated. Function on a specific target is Theoretical until the assay confirms it.
References: Watson et al., 2023; Dauparas et al., 2022; Dauparas et al., 2025. See Protein Design and Engineering chapter.
Case 3: Docking Benchmark Failure
A docking method looks strong by RMSD but generates physically implausible poses. PoseBusters shows why chemical validity checks belong beside geometric metrics: bond lengths, stereochemistry, severe clashes, ring geometry. Many low-RMSD deep-learning docking poses fail at least one basic validity check.
The lesson generalizes: a single metric can hide invalid outputs in many AI domains. Validity filtering belongs in the evaluation itself.
References: Buttenschoen et al., 2024. See Small Molecule Generation and ADMET, and Translational Evidence and Failure Modes chapters.
Case 4: Single-Cell Perturbation Forecasting
A group uses GEARS to rank perturbations for a follow-up experiment. The result supports prioritization, not general causal certainty. The Nature Methods evaluation by Ahlmann-Eltze and colleagues (2025) shows that for many perturbation-prediction tasks, PCA plus linear regression matches or beats deep methods. The right discipline is to compare against a strong simple baseline matched to the endpoint before attributing value to model complexity.
Evidence tier placement: GEARS-style perturbation prediction is Demonstrated for bounded settings (in-distribution gene pairs); generalization to novel-context settings is Theoretical or weaker.
References: Roohani et al., 2024; Ahlmann-Eltze et al., 2025. See Perturbation Prediction and Virtual Cells chapter.
Case 5: Closed-Loop Experimentation in Chemistry
A self-driving lab chooses the next experiment based on prior measurements. Coscientist uses GPT-4 as a planner with cloud laboratory platforms and optimizes palladium-catalyzed cross-coupling reactions. The system needs protocol versioning, instrument logs, objective functions, and stopping rules. Counting loop iterations is the right discipline: a platform that runs once is automation, not autonomy.
Evidence tier placement: closed-loop optimization for bounded chemistry tasks is Demonstrated. Open-ended autonomous biology beyond bounded settings is Beyond Current Capabilities.
References: Boiko et al., 2023; Burger et al., 2020. See Self-Driving Laboratories chapter.
Case 6: Multi-Agent Biology Research
Virtual Lab (Stanford) coordinates a principal investigator agent and specialist agents, with high-level feedback from a human researcher, to design 92 SARS-CoV-2 nanobodies. Two designs showed improved binding to the JN.1 or KP.3 variant while retaining strong binding to the ancestral spike protein in experimental validation. The contribution is the multi-agent workflow with experimental follow-through, not the absolute binding numbers.
Evidence tier placement: multi-agent biology research with wet-lab validation for one target is Demonstrated. Generalization across biology classes is Theoretical.
References: Swanson et al., 2025. See Agentic Science Workflows chapter.
Case 7: A-Lab and the Novelty-Claim Discipline
A-Lab reported 41 new inorganic compounds, from a set of 58 targets, autonomously produced over 17 days of continuous operation. A PRX Energy critique from solid-state chemists argued that the novelty claim did not hold because many entries were misclassified or were known phases. The episode is a clear case study of how autonomous-discovery novelty claims can outrun validation: throughput is not discovery.
Lesson: automated characterization can be wrong in characteristic ways; independent expert review is part of validation; novelty claims should pass scrutiny from the relevant subdiscipline before being treated as discoveries.
References: Szymanski et al., 2023; Leeman et al., 2024. See Self-Driving Laboratories chapter.
Case 8: AI-Discovered Drug in Phase IIa
Rentosertib (formerly ISM001-055) is a generative-AI-discovered TNIK inhibitor for idiopathic pulmonary fibrosis. Nature Biotechnology reported the discovery program and Phase 1 work; Nature Medicine reported a 71-participant, 12-week randomized, double-blind, placebo-controlled Phase 2a trial. The primary endpoint was safety and tolerability; forced vital capacity was a secondary endpoint, and the trial was not designed to establish clinical efficacy. A 52-week Phase 3 trial began dosing in China in September 2026 (NCT07687459).
Evidence tier placement: an AI-discovered candidate reaching Phase 2a is Demonstrated for this specific development path. Clinical success at registration scale is Theoretical for this specific candidate and Beyond Current Capabilities as a general AI-discovered-drug claim.
References: Ren et al., 2025; Xu et al., 2025. See Small Molecule Generation and ADMET, Translational Evidence and Failure Modes chapters.
Case 9: Variant Interpretation Inside the ACMG Framework
A clinical laboratory adds AlphaMissense outputs to its variant-interpretation pipeline. The correct use is as PP3 (computational evidence of pathogenicity) or BP4 (computational evidence of benignity) input alongside population data, functional studies, and segregation. The score is not a classification; the framework is.
Lesson: variant predictors are evidence inputs into an interpretation framework, not substitutes for it. Laboratories should select calibrated evidence thresholds and validate implementation for the intended context rather than transfer a model’s research categories directly into clinical classification.
References: Cheng et al., 2023; Richards et al., 2015. See Variant Effect Prediction chapter.
Case 10: Single-Cell Foundation Model in Practice
A research group uses scGPT for cell-type annotation and considers extending it to perturbation prediction. For annotation, the model is Demonstrated. For perturbation, the published evidence is contested by Ahlmann-Eltze and colleagues (2025): deep methods do not consistently beat PCA plus linear regression. The right discipline is to use scGPT for what it is demonstrated to do (representation, integration, atlas search) and run a linear baseline before adopting it for perturbation prediction.
References: Cui et al., 2024; Ahlmann-Eltze et al., 2025. See Single-Cell Foundation Models chapter.
Case 11: AlphaFold 3 Restricted Release and Open-Source Response
AlphaFold 3 was initially released through a server in May 2024. DeepMind released inference code and weights for academic, noncommercial use in November 2024 (Abramson et al., 2024). Boltz and Chai are separately developed models with different licences. Programs needing on-premise or commercial use must compare current licences, benchmark scope, compute requirements, and validation rather than treating these systems as interchangeable.
Lesson: licensing and code availability are first-class evaluation criteria. A model that cannot run in the required environment is not usable for the program.
References: Abramson et al., 2024; Abramson et al., 2024, addendum; Wohlwend et al., 2024, preprint; Chai Discovery et al., 2024, preprint. See Protein Structure Prediction, Reproducibility and Open Science chapters.
Case 12: Agentic Research Planning Without Lab Authorisation
A research agent drafts hypotheses and suggests protocols. The output requires source verification, human review, and authorization before any biological action. Permissions should be separated: read, analysis, procurement, and laboratory execution are distinct authorizations. Read access is not risk-free because an agent can expose sensitive data, follow malicious retrieved instructions, or move information into an unauthorized context. Laboratory execution creates additional physical risk and requires explicit human authorization.
Lesson: agentic systems are useful as research tools when tasks are bounded, sources are checked, and lab actions require explicit human authorization. Treating fluent agent output as validated science is the failure mode.
References: M. Bran et al., 2024; ARPA-H IGoR, 2026. See Agentic Science Workflows, Information Hazards in Capability Research chapters.