Small Molecule Generation and ADMET

Published

July 7, 2026

Small molecule discovery has multiple AI layers stacked on top of decades of cheminformatics: generative chemistry that proposes candidates, structure-based scoring that ranks them against targets, property predictors that triage on absorption-distribution-metabolism-excretion-toxicity, and benchmark infrastructure that lets the field tell which methods actually transfer. The generative step gets the most attention and is the smallest part of the value chain. The value is in validation, in physical and chemical plausibility filtering, and in choosing tasks where AI shifts a real stage gate. Insilico Medicine’s ISM001-055 (rentosertib) is the clearest current example of an AI-assisted candidate with peer-reviewed randomized Phase 2a evidence in a real indication; no AI-discovered small molecule has yet been approved.

Learning Objectives

Use this chapter to:

  • Generate, rank, dock, and optimize small molecules while keeping chemistry, ADMET, and experimental validation in view.
  • The valuable unit is not a generated structure; it is a compound series with route feasibility, binding, selectivity, developability, and biological evidence.

Prerequisites: AI for the Life Sciences for the chapter reference and utility standard; Evaluation Principles for Life Sciences AI for the validity-and-split discipline.

Summary: Generate, rank, dock, and optimize small molecules while keeping chemistry, ADMET, and experimental validation in view. Generative chemistry and property prediction help triage campaigns, but docking artifacts and clinical attrition still dominate translation risk.

Key point: The valuable unit is not a generated structure; it is a compound series with route feasibility, binding, selectivity, developability, and biological evidence. Open question: whether in-silico gains survive route feasibility, binding assays, selectivity, ADMET, and clinical attrition.

Bottom line: Small-molecule AI connects target biology to chemical biology, ADMET, drug repurposing, clinical translation, and evidence failure modes.

Field Guide

What is this field trying to solve? Generate, rank, dock, and optimize small molecules while keeping chemistry, ADMET, and experimental validation in view.

What is the core idea? The valuable unit is not a generated structure; it is a compound series with route feasibility, binding, selectivity, developability, and biological evidence.

What is the current state of the field? Generative chemistry and property prediction help triage campaigns, but docking artifacts and clinical attrition still dominate translation risk.

What do we know, and what remains open? Known reference points include REINVENT, DiffDock, EquiBind, Pocket2Mol, Chemprop, ADMET-AI, ADMETlab, MoleculeNet, Therapeutics Data Commons, PoseBusters, ChEMBL, PubChem, and BindingDB. What remains open is whether in-silico gains survive route feasibility, binding assays, selectivity, ADMET, and clinical attrition.

Why does this matter? Small-molecule AI connects target biology to chemical biology, ADMET, drug repurposing, clinical translation, and evidence failure modes.

Introduction

Small molecule discovery is the longest-running AI application in life sciences and the one where the gap between in-silico performance and clinical translation is widest. The generative-chemistry literature is large (REINVENT, JT-VAE, GraphAF, MolGAN, JANUS, REINVENT 4) and demonstrates goal-directed generation against in-silico objectives. The structure-based docking literature (Vina, DiffDock, EquiBind, Pocket2Mol) produces candidate poses with increasingly favourable benchmark RMSDs. The ADMET-prediction literature (Chemprop, ADMET-AI, ADMETlab) produces methods that triage candidate libraries.

Clinical translation evidence is narrower. Insilico Medicine’s ISM001-055, an AI-discovered TNIK inhibitor for idiopathic pulmonary fibrosis, reached randomized Phase 2a testing with peer-reviewed Nature Medicine evidence after the discovery and Phase I narrative was published in Nature Biotechnology (Ren et al., 2025; Xu et al., 2025). Recursion has multiple AI-discovered candidates in clinical trials (REC-617, REC-4881). Insitro and others have partnerships and preclinical activity but limited clinical readouts. No AI-discovered small molecule has yet been approved by FDA or EMA. AI is moving real candidates into trials, but Phase III evidence, regulatory approval, and post-marketing safety remain the drug-discovery gates.

This chapter applies the evidence framework layer by layer. The discipline matters because every layer has methods that work well in published benchmarks and fail at the gates after them.

What is demonstrated?

Generative chemistry: methods that produce candidate libraries

The REINVENT lineage is the most-cited generative chemistry framework. REINVENT 1.0 (Olivecrona et al., 2017) applied reinforcement learning to SMILES generation against a scoring function. REINVENT 2.0 (Blaschke et al., 2020) refined the architecture and multi-objective scoring. REINVENT 4 (Loeffler et al., 2024) is the current open-source release. The cumulative track record is real: REINVENT-derived candidates have appeared in patent literature and in preclinical programmes.

Graph-based generative approaches include JT-VAE (Jin et al., 2018, preprint) and JANUS (Nigam et al., 2022), which generate graph-structured molecules rather than SMILES strings. Transformer-based methods include Chemformer (Irwin et al., 2022) and the foundation-style ChemBERTa-2 (Ahmad et al., 2022, preprint).

The shared honest finding: generative chemistry produces large candidate sets; the value depends on the property-prediction layer, the synthetic-accessibility filter, and the assay layer that follows. The generative method is the smallest determinant of programme success.

Generative benchmarks separate different questions. GuacaMol evaluates distribution learning, validity, novelty, and goal-directed optimisation tasks (Brown et al., 2019). MOSES standardises molecular generation datasets and metrics such as validity, uniqueness, novelty, scaffold similarity, fragment similarity, and distributional distance (Polykovskiy et al., 2020). These benchmarks are useful for method comparison, but neither establishes target engagement, synthetic route feasibility, or clinical value.

Structure-based docking and pose prediction

Physics-based docking (Vina (Trott and Olson, 2010), Smina (Koes et al., 2013)) remains the operational baseline. Deep learning docking includes:

PoseBusters provides the main validity check for this layer. It showed that low-RMSD poses from DiffDock and similar deep methods routinely fail basic physical and chemical validity checks (bond lengths, stereochemistry, ring geometry, severe clashes) (Buttenschoen et al., 2024). The rule for evaluators is that RMSD without validity filtering is not the right metric.

Property prediction and ADMET

Chemprop (Yang et al., 2019) is the canonical message-passing neural network for molecular property prediction and remains the baseline against which newer methods are measured. ADMET-AI (Swanson et al., 2024) is a Chemprop-derived ADMET platform designed for large-scale library triage. ADMETlab 3.0 (Fu et al., 2024) is a multi-endpoint ADMET platform with published performance figures.

The shared rule: property predictors are useful for screening triage at large scale. They are not substitutes for experimental ADMET, and clinical PK/PD remains the validation that matters.

Benchmarks: MoleculeNet and the Therapeutics Data Commons

MoleculeNet (Wu et al., 2018) standardised scaffold-split benchmarks for molecular property prediction across solubility, toxicity, protein-binding, and quantum-mechanical tasks. The Therapeutics Data Commons (Huang et al., 2022) extended this principle to a broad multi-task benchmark covering small molecules, antibodies, vaccines, gene editing, and clinical trials, with biology-aware splits (scaffold, time, target) and a leaderboard that gives outside observers a comparable view.

For prospective medicinal chemistry, the split matters as much as the model. Time-split validation estimates whether a model trained on older chemistry helps with later chemistry rather than memorising contemporaneous analogues (Sheridan, 2013). Synthetic-accessibility scoring is also part of validity, not a postscript; the classical SAscore formalised fragment and complexity penalties for drug-like molecules (Ertl and Schuffenhauer, 2009). A generator that improves a property score while worsening synthesizability has not improved a program.

The operational lesson: published numbers without scaffold or time splits are inflated. Any AI drug discovery claim should be triangulated against MoleculeNet, TDC, or PoseBusters before it earns trust.

Clinical Translation: ISM001-055 and AI-Discovered Drug Candidates

Insilico Medicine’s ISM001-055 (rentosertib) is the furthest-advanced AI-discovered small molecule in real clinical trials. The compound is a TNIK inhibitor for idiopathic pulmonary fibrosis. The Nature Biotechnology paper covers the discovery and Phase I narrative (Ren et al., 2025). The randomized Phase 2a trial was subsequently published in Nature Medicine, with safety/tolerability and exploratory forced vital capacity signals in IPF (Xu et al., 2025). Phase 2a is meaningful clinical evidence; it is not registration-grade evidence and does not establish approval-level benefit-risk.

Recursion has multiple clinical AI-derived candidates (REC-617 CDK7 inhibitor, REC-4881) with company-reported pipeline updates. Insitro has expanded partnerships including a 2025 Eli Lilly small-molecule collaboration. None has reached Phase III readout with AI-discovered candidates.

Evidence anchor summary

Evidence Anchor What It Supports Practical Constraint
REINVENT lineage Goal-directed molecule generation against an in-silico objective Value depends on the property predictor and assay layer that follow
DiffDock, EquiBind, Pocket2Mol Fast pose proposals over a binding site PoseBusters validity required; RMSD alone is insufficient
Chemprop, ADMET-AI, ADMETlab 3.0 ADMET triage of large libraries Not a substitute for experimental ADMET or clinical PK
MoleculeNet Scaffold-split molecular ML benchmarks Datasets age; absolute numbers shift over years
GuacaMol / MOSES Generative-molecule benchmark discipline Benchmark success does not imply assay success
Time-split validation Prospective analogue of future medicinal chemistry Requires dated chemistry and clear leakage controls
Synthetic-accessibility scoring Early manufacturability triage A score is weaker than a real route
Therapeutics Data Commons Multi-task therapeutic discovery benchmark Continuous evaluation; submission discipline varies
PoseBusters Validity checks for docking poses Reframes the docking-quality conversation; raw RMSD is no longer the metric
ISM001-055 / rentosertib AI-discovered TNIK inhibitor with randomized Phase 2a evidence Phase 2a is not registration; pivotal evidence and approval are still absent

What is theoretical?

Several capabilities are plausible but not yet established.

Foundation models for chemistry. Chemformer, ChemBERTa-2, and similar pretrained backbones extend the SMILES-language-model idea to molecular property prediction. Whether this changes performance at scale (vs. task-specific Chemprop-style models) is contested; the published wins are modest and task-dependent.

End-to-end AI drug discovery pipelines. Several companies (Insilico Medicine, Recursion, Iktos, Atomwise, Schrodinger) operate end-to-end pipelines that integrate generation, structure-based scoring, ADMET prediction, and candidate triage. The Insilico ISM001-055 path is the clearest current example of such a pipeline delivering a Phase IIa candidate. Whether this generalizes beyond target-specific success is not yet established.

Active-learning closed-loop drug discovery. Coupling generative chemistry with automated chemical synthesis and assay (covered in Self-Driving Laboratories) is feasible at limited scale. The combination of automation, generation, and learning at production drug-discovery scale is plausible but not yet deployed at the scale of a real medicinal-chemistry programme.

Pose-aware affinity prediction at production quality. Boltz-2 (Passaro et al., 2025, preprint) added binding-affinity prediction to AF3-class structure prediction. The published evidence suggests modest but real improvements; production-quality affinity prediction for real drug discovery decisions is not yet established.

LLM-orchestrated medicinal chemistry workflows. ChemCrow (M. Bran et al., 2024) and related tools wrap LLMs around chemistry-specific tools (RDKit, retro-route planners, property predictors). Production deployment in real medicinal-chemistry programmes is emerging; the evidence for net acceleration vs. expert manual work is preliminary.

What is beyond current capability?

A few framing claims are not supported by current evidence.

AI replaces medicinal chemistry. It does not. The medicinal-chemistry intuition that integrates structure, mechanism, synthetic strategy, and developability across decades of experience is not encoded in any current pipeline. AI changes the throughput of candidate proposal and triage; the decisions that gate a program remain human.

AI-discovered drugs are reliably better than human-designed drugs. No evidence exists for this claim. AI-discovered candidates that have reached clinic show similar attrition profiles to traditionally discovered ones at the same stage; the AI advantage, if it is one, is in cycle time and breadth of exploration rather than in clinical success rate.

Generic ADMET predictors replace experimental ADMET. They do not. Predictors are useful for triage; they are not substitutes for the in-vitro and in-vivo studies that regulators require for IND-enabling packages.

Docking scores predict binding affinity in vitro. The correlation is weak, and weaker still for novel scaffolds. Even with PoseBusters-style validity filtering, docking score is a triage signal, not a binding-affinity prediction.

What would make this more promising?

Small-molecule AI becomes more promising when the result survives the next chemistry or biology gate, not when the in-silico score improves.

Claim Evidence that raises or lowers confidence
“The model generated useful molecules” Validity, novelty, synthetic accessibility, route feasibility, and denominators for proposed, filtered, made, and tested compounds
“The docking result is credible” PoseBusters-style physical validity, binding assay data, selectivity data, and comparison with standard docking baselines
“The ADMET model is useful” Scaffold or time-split validation, calibration on the relevant chemical series, and experimental ADMET follow-up
“The candidate is AI-discovered” Target engagement, pharmacology, PK/PD, safety package, clinical stage, and the specific decision changed by AI
“The platform improves discovery” Prospective comparison against a standard workflow with cycle time, hit rate, cost, and attrition denominators

The claim should advance one gate at a time: generated library, made compound, assay-active hit, optimized lead, development candidate, clinical signal, pivotal evidence.

What should researchers, biotech teams, funders, and program leaders do with this?

For researchers and program leaders evaluating AI claims in small molecule discovery:

  • Demand scaffold or time splits. Random-split numbers are inflated. The MoleculeNet and TDC conventions are the right reference. If a paper reports only random splits, treat the headline number as preliminary.
  • Apply PoseBusters-style validity to any docking claim. Low-RMSD poses are not sufficient evidence. Bond-length, stereochemistry, clash, and ring-geometry checks belong in the evaluation.
  • Read the property-prediction stack honestly. Chemprop is the baseline; an ADMET predictor that does not beat Chemprop on the relevant TDC task is not the right tool.
  • Separate generation benchmarks from program evidence. GuacaMol and MOSES are method benchmarks. Target engagement, route feasibility, assay potency, and ADMET still require program-specific evidence.
  • Triangulate against the Therapeutics Data Commons. Vendor claims are weaker than TDC-leaderboard performance. If the vendor has not submitted, ask why.
  • Read clinical news with stage-appropriate evidence weight. Phase I is safety; Phase 2a is signal; Phase 2b and III are evidence; approval is the bar. Insilico Medicine’s Phase 2a publication is meaningful, not registration. Recursion’s pipeline is not yet pivotal evidence.
  • Use synthetic-accessibility filters as part of the generative stack. A candidate that scores well in silico and cannot be made in eight steps is not a candidate. SAscore and retro-route planners are part of the triage.
  • Document the full pipeline. Generative method, version, training set, scoring function, splitting policy, post-hoc filters, synthetic route, assay used. Reproducibility in AI drug discovery lives in the documentation, not in the model name.
  • Watch for vendor-reported metrics in marketing. Company press releases compress nuance. The peer-reviewed publications (Ren et al., 2025; Xu et al., 2025) are the right references for ISM001-055, not the press release.