Information Hazards in Capability Research
Capability research can create knowledge that helps science and also lowers barriers to misuse. The standard is deliberate disclosure that preserves enough detail for scientific verification while avoiding unnecessary operational detail that raises misuse risk. On July 20, 2026, the United States issued the USG Policy for Stopping High-Risk Life Sciences Research. It prohibits federal support for defined dangerous gain-of-function research and specified international research of concern, and requires agencies to issue implementation guidance. The policy states that purely computational research, including models, software, and in-silico design of novel biological agents, is not prohibited unless it involves an entity of concern. It applies when a proposal includes plans to create or modify an agent that would meet its dangerous gain-of-function definitions. All provisions of Executive Order 14292 remain in effect while agency implementation is pending. NIH states that activities paused under the 2025 notices remain paused until NIH-specific requirements are established (NIH NOT-OD-26-101). Applicability and review routes should be confirmed with the institution and funder rather than inferred from an older DURC label.
Use this chapter to:
- Help researchers publish useful AI-biology capability work without lowering barriers to misuse or hiding legitimate evidence.
- The question is not secrecy by default; it is whether methods, details, or artifacts meaningfully change harmful capability access.
Prerequisites: none. This chapter is the dual-use companion to the technical chapters.
Summary: Help researchers publish useful AI-biology capability work without lowering barriers to misuse or hiding legitimate evidence. Publication judgment, disclosure review, screening, and governance practices exist, but AI-biology capability release norms are still forming.
Key point: The question is not secrecy by default; it is whether methods, details, or artifacts meaningfully change harmful capability access. Open question: how to preserve verification while reducing operational misuse risk.
Bottom line: Information hazards connect synthetic biology, autonomous labs, agents, sequence design, biosecurity, and open science.
What is this field trying to solve? Help researchers publish useful AI-biology capability work without lowering barriers to misuse or hiding legitimate evidence.
What is the core idea? The question is not secrecy by default; it is whether methods, details, or artifacts meaningfully change harmful capability access.
What is the current state of the field? Publication judgment, disclosure review, screening, and governance practices exist, but AI-biology capability release norms are still forming.
What do we know, and what remains open? Known reference points include DURC and PEPP frameworks, DNA screening practices, biosecurity reviews, model release policies, information-hazard literature, and institutional review processes. What remains open is how to preserve verification while reducing operational misuse risk.
Why does this matter? Information hazards connect synthetic biology, autonomous labs, agents, sequence design, biosecurity, and open science.
Introduction
AI-biology work now includes protein design, genome modelling, lab automation, and agentic workflows. ARPA-H IGoR’s inclusion of agentic systems and laboratory automation shows why researcher practice needs explicit review gates (ARPA-H IGoR, 2026). The July 2026 USG policy now provides the controlling federal funding framework for defined dangerous gain-of-function research and international research of concern. NIH reports a 120-day window for agency implementation guidance and a 90-day window to establish a single independent third-party review body; previously paused activities remain paused pending NIH-specific requirements (NIH NOT-OD-26-101).
Recent Science and Nature Biotechnology commentary adds the AI-biosecurity frame: governance should be proportional to capability, access, and downstream experimental enablement, and safeguards should be built into generative biology tools rather than treated as an external afterthought (Bloomfield et al., 2024; Baker and Church, 2024; Wang et al., 2025). Researcher practice combines disclosure judgement, IBC engagement, and staged-release mechanisms.
Demonstrated capability
Demonstrated capability includes models that design proteins, model biomolecular interactions, generate sequences, and help plan experiments in bounded settings. RFdiffusion, AlphaFold 3, Evo, and self-driving laboratory systems supply concrete examples of capability progress (Watson et al., 2023; Abramson et al., 2024; Nguyen et al., 2024). The AlphaFold 3 initial restricted-release decision, the AlphaProteo arXiv-without-code release, and the open-source response (Boltz, Chai) collectively illustrate the live community debate over capability disclosure.
| Evidence Anchor | What It Supports | Practical Constraint |
|---|---|---|
| RFdiffusion and Evo | Capability gains in design and sequence modelling | Publication detail should match legitimate verification needs |
| AlphaFold 3 restricted release | A case study in capability disclosure debate | Open-source response (Boltz, Chai) emerged within months |
| ARPA-H IGoR | Agentic and automated research infrastructure | Human authorisation and protocol standards matter |
| NIST CAISI agent red-teaming | Tool-use and agent-hijacking risk under adversarial testing | Results concern agent security, not biological misuse capability (NIST, 2026) |
| 2026 USG Policy for Stopping High-Risk Life Sciences Research | Federal funding prohibitions and risk-based oversight for defined high-risk research | Purely computational work is not prohibited unless it involves an entity of concern; planned creation or modification of a covered agent can bring in-silico work into scope |
| NSABB 2023 report | Proposed biosecurity oversight framework for future science | Advisory report, not a substitute for current policy or institutional review |
| IGSC screening protocol | Sequence and customer screening expectations for synthetic DNA providers | Voluntary consortium protocol; coverage depends on provider participation |
| NIST safe-proxy TEVV | Risk evaluation without testing sequences of concern | Proxy systems still require careful assay design |
| Bridge2AI | Ethical AI-ready data framing | Data and model release need governance |
| Bloomfield / Baker and Church / Wang | AI-biosecurity governance and built-in safeguards | Commentary and policy framing; implementation still institutional |
Theoretical capability
Theoretical capability includes publication norms that preserve reproducibility while reducing operational misuse. This requires journal policies, institutional review, and field-specific norms rather than ad hoc redaction. Controlled-access mechanisms, including trusted-reviewer access to model weights and code, could be extended to AI-biology capability work.
DNA synthesis screening is an important example of why publication judgment and infrastructure have to be considered together. The IGSC protocol describes screening of synthetic DNA orders and customers, but voluntary screening cannot cover every provider or every future sequence-design workflow (IGSC, 2024). AI-biology papers should avoid normalising “screening bypass” as a technical achievement; when screening limitations matter scientifically, report the governance conclusion without operationalising evasion.
Built-in safeguards are part of tool design. Generative biology systems should log high-risk requests, screen generated sequences where appropriate, separate analysis permissions from procurement or lab-execution permissions, and document escalation paths for dual-use outputs. This does not replace institutional review. It reduces the chance that safety is added only after a capability has already been distributed (Wang et al., 2025).
Evaluation practice is beginning to catch up. NIST’s safe-proxy TEVV study used benign proteins as stand-ins for sequences of concern, preserving the governance question while avoiding direct testing of dangerous designs. The result was deliberately calibrated: current AI-assisted protein design systems could produce predicted structural similarity, but did not reliably preserve activity while evading screening (Ikonomova et al., 2025). The 2026 International AI Safety Report likewise distinguishes benchmark and model evidence from real-world biological outcomes and notes that material and operational barriers remain difficult to assess (International AI Safety Report, 2026). This is the right pattern for information-hazard work: test the capability boundary without publishing operational recipes for misuse.
Beyond current capability
Beyond current capabilities includes perfect separation of beneficial and harmful uses at publication time. Dual-use judgement remains contextual and imperfect. No framework reliably distinguishes all benign uses from all harmful uses of any given capability disclosure; the practical aim is reasoned judgement supported by institutional review, not algorithmic certainty.
Evidence that would change the assessment
Release practice becomes more promising when decisions can show both preserved verification and reduced operational misuse risk. Stronger practice would include documented review gates, controlled access for sensitive artifacts, and post-release monitoring that detects whether safeguards worked as intended.
Implications for research and program decisions
- Perform a disclosure review before releasing code, model weights, protocols, or datasets that materially increase biological capability.
- Keep reproducibility artefacts available to trusted reviewers when public release is not appropriate.
- Avoid step-by-step operational details when high-level scientific reporting is sufficient.
- Document the reason for any redaction or staged release.
- Record the exact model version, system configuration, tools, access tier, evaluator resources, and test date. Do not generalize a benchmark result to a materially different deployment without new evidence.
- Prefer built-in safeguards (logging, screening, permission separation, and escalation paths) for generative biology tools.
- Consult institutional biosafety, research-security, legal, or funder review when the activity falls within their remit. Not every use of biological material requires the same review route.
- Check the July 2026 USG policy, subsequent agency implementation guidance, institutional requirements, and funder-specific restrictions before starting or changing covered work.