Information Hazards in Capability Research
Capability research can create knowledge that helps science and also lowers barriers to misuse. The practical task is responsible research communication, not broad policy writing. The standard is not secrecy by default. The standard is deliberate disclosure that preserves enough detail for scientific verification while avoiding unnecessary operational detail that raises misuse risk. The 2024 NIH and HHS DURC policy and similar frameworks provide institutional context; researcher judgment, IBC review, and staged release mechanisms provide the practical infrastructure (NIH Office of Science Policy, 2024; NSABB, 2023).
Use this chapter to:
- Help researchers publish useful AI-biology capability work without lowering barriers to misuse or hiding legitimate evidence.
- The question is not secrecy by default; it is whether methods, details, or artifacts meaningfully change harmful capability access.
Prerequisites: none. This chapter is the dual-use companion to the technical chapters.
Summary: Help researchers publish useful AI-biology capability work without lowering barriers to misuse or hiding legitimate evidence. Publication judgment, disclosure review, screening, and governance practices exist, but AI-biology capability release norms are still forming.
Key point: The question is not secrecy by default; it is whether methods, details, or artifacts meaningfully change harmful capability access. Open question: how to preserve verification while reducing operational misuse risk.
Bottom line: Information hazards connect synthetic biology, autonomous labs, agents, sequence design, biosecurity, and open science.
What is this field trying to solve? Help researchers publish useful AI-biology capability work without lowering barriers to misuse or hiding legitimate evidence.
What is the core idea? The question is not secrecy by default; it is whether methods, details, or artifacts meaningfully change harmful capability access.
What is the current state of the field? Publication judgment, disclosure review, screening, and governance practices exist, but AI-biology capability release norms are still forming.
What do we know, and what remains open? Known reference points include DURC and PEPP frameworks, DNA screening practices, biosecurity reviews, model release policies, information-hazard literature, and institutional review processes. What remains open is how to preserve verification while reducing operational misuse risk.
Why does this matter? Information hazards connect synthetic biology, autonomous labs, agents, sequence design, biosecurity, and open science.
Introduction
AI-biology work now includes protein design, genome modelling, lab automation, and agentic workflows. ARPA-H IGoR’s inclusion of agentic systems and laboratory automation shows why researcher practice needs explicit review gates (ARPA-H IGoR, 2026). The 2024 NIH and HHS DURC policy provides institutional framing for the kinds of research that warrant explicit dual-use review (NIH Office of Science Policy, 2024). The broader NSABB oversight report recommended a more integrated biosecurity framework for future science (NSABB, 2023).
Recent Science and Nature Biotechnology commentary adds the AI-biosecurity frame: governance should be proportional to capability, access, and downstream experimental enablement, and safeguards should be built into generative biology tools rather than treated as an external afterthought (Bloomfield et al., 2024; Baker and Church, 2024; Wang et al., 2025). Researcher practice combines disclosure judgement, IBC engagement, and staged-release mechanisms.
What is demonstrated?
Demonstrated capability includes models that design proteins, model biomolecular interactions, generate sequences, and help plan experiments in bounded settings. RFdiffusion, AlphaFold 3, Evo, and self-driving laboratory systems supply concrete examples of capability progress (Watson et al., 2023; Abramson et al., 2024; Nguyen et al., 2024). The AlphaFold 3 initial restricted-release decision, the AlphaProteo arXiv-without-code release, and the open-source response (Boltz, Chai) collectively illustrate the live community debate over capability disclosure.
| Evidence Anchor | What It Supports | Practical Constraint |
|---|---|---|
| RFdiffusion and Evo | Capability gains in design and sequence modelling | Publication detail should match legitimate verification needs |
| AlphaFold 3 restricted release | A case study in capability disclosure debate | Open-source response (Boltz, Chai) emerged within months |
| ARPA-H IGoR | Agentic and automated research infrastructure | Human authorisation and protocol standards matter |
| 2024 NIH/HHS DURC policy | Updated framework for dual-use review | Institutional engagement required |
| NSABB 2023 report | Biosecurity oversight framework for future science | Advisory report, not a substitute for institutional review |
| IGSC screening protocol | Sequence and customer screening expectations for synthetic DNA providers | Voluntary consortium protocol; coverage depends on provider participation |
| NIST safe-proxy TEVV | Risk evaluation without testing sequences of concern | Proxy systems still require careful assay design |
| Bridge2AI | Ethical AI-ready data framing | Data and model release need governance |
| Bloomfield / Baker and Church / Wang | AI-biosecurity governance and built-in safeguards | Commentary and policy framing; implementation still institutional |
What is theoretical?
Theoretical capability includes publication norms that preserve reproducibility while reducing operational misuse. This requires journal policies, institutional review, and field-specific norms rather than ad hoc redaction. Controlled-access mechanisms (trusted-reviewer access to model weights and code) are emerging in adjacent fields and could be extended to AI-biology capability work. Cross-institutional review boards for AI-biology DURC are an active discussion.
DNA synthesis screening is an important example of why publication judgment and infrastructure have to be considered together. The IGSC protocol describes screening of synthetic DNA orders and customers, but voluntary screening cannot cover every provider or every future sequence-design workflow (IGSC, 2024). AI-biology papers should avoid normalising “screening bypass” as a technical achievement; when screening limitations matter scientifically, report the governance conclusion without operationalising evasion.
Built-in safeguards are part of tool design. Generative biology systems should log high-risk requests, screen generated sequences where appropriate, separate analysis permissions from procurement or lab-execution permissions, and document escalation paths for dual-use outputs. This does not replace institutional review. It reduces the chance that safety is added only after a capability has already been distributed (Wang et al., 2025).
Evaluation practice is beginning to catch up. NIST’s safe-proxy TEVV study used benign proteins as stand-ins for sequences of concern, preserving the governance question while avoiding direct testing of dangerous designs. The result was deliberately calibrated: current AI-assisted protein design systems could produce predicted structural similarity, but did not reliably preserve activity while evading screening (Ikonomova et al., 2025). This is the right pattern for information-hazard work: test the capability boundary without publishing operational recipes for misuse.
What is beyond current capability?
Beyond current capabilities includes perfect separation of beneficial and harmful uses at publication time. Dual-use judgement remains contextual and imperfect. No framework reliably distinguishes all benign uses from all harmful uses of any given capability disclosure; the practical aim is reasoned judgement supported by institutional review, not algorithmic certainty.
What would make this more promising?
Release practice becomes more promising when decisions can show both preserved verification and reduced operational misuse risk. Stronger practice would include documented review gates, controlled access for sensitive artifacts, and post-release monitoring that detects whether safeguards worked as intended.
What should researchers, biotech teams, funders, and program leaders do with this?
- Perform a disclosure review before releasing code, model weights, protocols, or datasets that materially increase biological capability.
- Keep reproducibility artefacts available to trusted reviewers when public release is not appropriate.
- Avoid step-by-step operational details when high-level scientific reporting is sufficient.
- Document the reason for any redaction or staged release.
- Prefer built-in safeguards (logging, screening, permission separation, and escalation paths) for generative biology tools.
- Engage IBC for biological materials work; engage DURC institutional review when the work falls within the 2024 NIH/HHS framework.
- Stay current with NIH H5N1 ePPP policy, NSF research security framing, and funder-specific dual-use requirements.