Neuroscience AI and Brain Foundation Models
Brain data connect molecular identity, cellular physiology, circuits, behavior, and disease. Neural recordings, connectomics, brain atlases, imaging, and neural decoding are not one problem. Each modality measures a different layer of the nervous system, and each requires a different validation design. The central question is not whether a model is “brain-like.” The central question is what neural signal it predicts, in which organism, under which task, and whether the held-out evidence matches the claim.
Use this chapter to:
- Model neural activity, brain structure, behavior, atlases, connectomics, and decoding tasks without overstating cognition or clinical meaning.
- Recording modality, species, task, stimulus, behavioral state, and anatomical reference determine what a brain model actually represents.
What is this field trying to solve? Model neural activity, brain structure, behavior, atlases, connectomics, and decoding tasks without overstating cognition or clinical meaning.
What is the core idea? Recording modality, species, task, stimulus, behavioral state, and anatomical reference determine what a brain model actually represents.
What is the current state of the field? Neural activity prediction, latent dynamics, atlases, and bounded decoding are advancing; general brain models and unconstrained thought reading are not demonstrated.
What do we know, and what remains open? Known reference points include Allen Brain Observatory, DANDI, BICAN, MICrONS, LFADS, CEBRA, BrainLM, POYO, connectomics datasets, and neural-decoding studies. What remains open is whether models transfer across animals, sessions, tasks, recording platforms, and clinical contexts.
Why does this matter? Neuroscience connects organismal biology to cells, imaging, aging, behavior, virtual organisms, devices, and therapeutic translation.
Introduction
Brain data are heterogeneous by design. Calcium imaging measures optical signals tied to neural activity. Electrophysiology measures spikes, local field potentials, or cortical surface activity at different spatial and temporal scales. Functional MRI measures blood-oxygen-level-dependent activity indirectly. Connectomics reconstructs cellular wiring from imaging data. Brain atlases organize anatomy and cell types. Behavior links activity to an organism-level output. A model trained on one of these layers should not be read as if it models all of them.
The recent signal for this field is that foundation-model language has moved from analogy into peer-reviewed neuroscience. Wang and colleagues trained a foundation model of neural activity from mouse visual cortex recordings and tested response prediction for new stimulus types (Wang et al., 2025). The result matters because it shows that neural response models can benefit from scale, pretraining, and transfer. It does not show that a model understands vision in the human sense, predicts cognition broadly, or replaces experiments.
Connectomics provides a second signal. The MICrONS functional connectomics dataset links dense calcium imaging from visual areas of an awake mouse with large-scale electron microscopy reconstruction, creating an unusually rich structure-function resource (MICrONS Consortium, 2025). Subsequent analysis used the same resource to study wiring rules in mouse visual cortex (Ding et al., 2025). These are high-value datasets for AI because they do not merely provide labels. They tie activity, connectivity, anatomy, and stimulus context together.
Human neural decoding supplies a third signal, but it requires strict interpretation. Semantic reconstruction from fMRI can recover coarse language meaning under cooperative, trained conditions (Tang et al., 2023). Intracortical and surface-recording speech neuroprostheses can decode attempted communication in research participants with severe paralysis (Willett et al., 2023; Metzger et al., 2023). These studies are important research interfaces. They should not be collapsed into claims about general thought reading, consumer monitoring, or clinical deployment without the corresponding validation.
What Counts as Neuroscience AI?
Neuroscience AI covers models that analyze, predict, decode, reconstruct, or organize nervous-system data. The chapter uses five practical categories.
Neural activity models predict spikes, calcium responses, local field potentials, fMRI time series, or latent neural dynamics. LFADS inferred single-trial neural population dynamics from spiking data and helped establish the modern deep-learning vocabulary for neural dynamics (Pandarinath et al., 2018). CEBRA uses contrastive learning to learn latent embeddings jointly from neural and behavioral data (Schneider et al., 2023). BrainLM applies self-supervised masked prediction to large fMRI collections as a foundation-model approach to brain activity recordings (Ortega Caro et al., 2024).
Brain imaging models analyze fMRI, MRI, microscopy, calcium imaging, or electrophysiology-derived features. In this chapter, brain imaging models are research tools unless clinical validation is present. Clinical neuroimaging AI belongs to a separate medical-device and clinical-evidence discussion.
Connectomics models segment neurons, trace processes, predict synaptic connectivity, align structure with function, or analyze wiring rules. Their outputs depend on reconstruction quality, sample preparation, segmentation accuracy, and the biological context of the tissue.
Brain atlas models integrate cell identity, anatomy, spatial coordinates, and cross-species mappings. The Allen Mouse Brain Common Coordinate Framework is a key reference atlas for aligning mouse brain data (Wang et al., 2020). The BRAIN Initiative Cell Atlas Network extends atlas work toward human, non-human primate, and mouse reference resources through an official NIH program (BICAN, NIH BRAIN Initiative).
Neural decoding systems infer a stimulus, behavior, intention, semantic content, or communication output from neural data. Decoding is not one task. Decoding hand velocity from motor cortex, continuous language from fMRI, and attempted speech from ECoG have different privacy, consent, error, and validation requirements.
What is demonstrated?
Neural Activity Prediction in Defined Systems
The strongest demonstrated capability is prediction of neural responses in bounded experimental systems. The Wang et al. visual-cortex model is important because it trained on large-scale activity recordings across mouse visual cortex and evaluated response prediction for new stimulus types (Wang et al., 2025). Its credibility comes from measured neural responses and explicit held-out conditions.
The correct reading is narrow and useful. The model addresses visual-cortex activity in mice under defined stimulus and recording conditions. It does not demonstrate a complete model of mouse behavior, whole-brain dynamics, or human cognition. The same standard should apply to every neural activity model: what stimulus or behavior was held out, which animals were held out, whether neurons or sessions were held out, and whether the model beats simpler baselines.
Large-scale public datasets make this work possible. The Allen Brain Observatory Visual Coding dataset provided standardized physiology from mouse visual cortex and helped normalize open neural-response resources (de Vries et al., 2020). DANDI is the BRAIN Initiative archive for neurophysiology data, including electrophysiology, optophysiology, behavioral time series, and immunostaining images (DANDI Archive). These infrastructure projects matter because brain models are rarely reproducible without raw signals, metadata, stimulus timing, behavioral context, and preprocessing details.
Latent Dynamics and Neural Population Models
Neural population models are demonstrated as analysis tools. LFADS showed that deep generative models can infer latent dynamics from single-trial spiking activity and improve interpretation of motor-cortical recordings (Pandarinath et al., 2018). CEBRA showed that contrastive embedding can integrate behavior and neural data into consistent latent spaces across modalities and tasks (Schneider et al., 2023). POYO-1 frames population decoding as a scalable multi-session problem and demonstrates transfer across neural recordings where neuron identity does not carry over between sessions (Azabou et al., 2023).
The practical value is not that these methods “solve” neural coding. They reduce high-dimensional recordings into representations that support decoding, hypothesis testing, and experimental comparison. The risk is overinterpreting a latent axis as a biological mechanism. Latent dimensions are statistical objects until linked to perturbation, behavior, anatomy, or reproducible physiology.
Connectomics and Structure-Function Resources
Connectomics is now an AI-relevant data layer because reconstruction scale and functional pairing have improved. MICrONS combines functional calcium imaging with dense connectomic reconstruction across multiple mouse visual areas (MICrONS Consortium, 2025). Ding and colleagues used this structure-function resource to analyze wiring rules in visual cortex (Ding et al., 2025).
Connectomics gives models explicit anatomical structure, tests whether wiring improves functional prediction, and exposes where structure-function mappings remain incomplete.
The failure boundary is equally important. Connectomics datasets are expensive, sparse in species coverage, and bounded by brain region and tissue processing. Segmentation errors, missing synapses, sample distortion, and incomplete cell-type annotation can propagate into downstream models. A connectome is a measurement artifact plus biological reality, not a perfect wiring diagram.
Brain Atlases and Cell-Type Grounding
Brain atlases make neuroscience AI more biological. A model that knows only voxel coordinates or electrode channels is less interpretable than one linked to region, layer, cell type, developmental context, and species. The Allen Mouse Brain Common Coordinate Framework is a reference for mouse-brain alignment and spatial comparison (Wang et al., 2020). BICAN provides the official NIH framework for reference cell atlases across human, non-human primate, and mouse brain resources (BICAN, NIH BRAIN Initiative).
Atlas-linked models can support cell-type annotation, region alignment, cross-dataset comparison, and hypothesis generation. They do not remove biological heterogeneity. Brain-region names differ across atlases. Cell-type taxonomies evolve. Human tissue differs from mouse tissue. Disease, age, postmortem interval, surgical context, and dissociation method can alter measured features. Atlas grounding should make these assumptions visible, not hide them.
Neural Decoding as Research Interface
Neural decoding is demonstrated in several research settings. Tang and colleagues reconstructed continuous language meaning from fMRI, with important cooperation and training constraints (Tang et al., 2023). Willett and colleagues decoded attempted speech from intracortical recordings in a high-performance speech neuroprosthesis study (Willett et al., 2023). Metzger and colleagues decoded speech and avatar-control outputs from high-density cortical surface recordings (Metzger et al., 2023).
The evidence is real, but the interpretation must stay bounded. These systems are trained and evaluated in specific participants, tasks, recording modalities, and consent contexts. They are not general-purpose tools for inferring private mental content. A decoded sentence, semantic representation, or movement intention is a model output under a task protocol. It is not an unconstrained observation of thought.
What is theoretical?
Cross-Modal Brain Foundation Models
Theoretical capability includes models that transfer across fMRI, electrophysiology, calcium imaging, behavior, anatomy, and cell identity. This is plausible because each modality observes part of the same biological system. It is difficult because the modalities differ in time scale, spatial resolution, measurement noise, and experimental protocol.
BrainLM is an example of the fMRI path: pretrain on large activity recordings, then fine-tune or use representations for downstream tasks (Ortega Caro et al., 2024). POYO-1 is an example of the neural population path: learn across recordings where individual neuron identity differs across sessions (Azabou et al., 2023). The theoretical next step is integration across modalities, not simply larger models within one modality.
The hard validation question is whether transfer reflects biology or dataset regularities. Site, scanner, electrode array, stimulation protocol, animal strain, task design, and preprocessing can all become shortcuts. Cross-modal transfer must be evaluated with held-out laboratories, held-out tasks, held-out subjects or animals, and held-out disease states.
In-Silico Neuroscience and Virtual Circuits
Virtual circuits are plausible in bounded settings. A model may predict how a visual-cortex population responds to a new stimulus or how a motor-cortical population relates to movement. Connectomics can constrain circuit models by adding anatomical structure. Atlases can constrain models by adding cell identity and region context.
The field is still far from a general executable brain. The useful near-term ambition is narrower: build models that predict a defined neural response well enough to guide the next experiment. A virtual circuit earns credibility when it nominates a perturbation, stimulation pattern, or recording condition, and the experiment confirms or falsifies the prediction.
AI-Guided Experiment Selection
Neuroscience experiments are expensive. AI can help select stimuli, perturbations, recording sites, stimulation protocols, or imaging fields. The strongest use case is active learning: model uncertainty identifies the next measurement that will most improve the representation.
This is theoretical for many brain domains because the loop requires tight integration between model, instrument, protocol, and data capture. It also requires recording failures. If a model only records successful predictions, it becomes a confirmation machine. Failed stimulus predictions, bad segmentations, poor decodes, and non-transferable sessions are the evidence that makes the next model better.
What is beyond current capability?
General Human Brain Model
A general model of the human brain remains beyond current capabilities. Such a model would need to predict cognition, behavior, disease progression, treatment response, development, aging, and individual context across modalities. Current systems remain bounded by species, recording modality, task, tissue context, and available ground truth.
This boundary matters because “brain foundation model” can sound more general than the evidence supports. A useful foundation model is still a bounded representation learner. It does not become a theory of mind by pretraining on more recordings.
Unconstrained Thought Reading
Unconstrained thought reading is beyond current capabilities. Neural decoding studies require training data, cooperation, task constraints, or invasive recordings. They decode aspects of stimulus processing, intended movement, attempted speech, or semantic representation under a protocol. They do not provide direct access to private subjective experience.
The privacy implications are still real. Even bounded decoders should be handled with explicit consent, data minimization, participant review where appropriate, security controls, and careful communication. The ethical standard should rise before the technical capability is generalized, not afterward.
Clinical Diagnosis from Research Embeddings
Research embeddings should not be treated as clinical diagnostic tools. A model trained to represent fMRI time series, predict visual responses, or decode speech has not demonstrated clinical validity unless it has been evaluated for the clinical context of use. Disease heterogeneity, comorbidity, medication, scanner protocol, clinical workflow, and regulatory requirements are separate evidence layers.
What would make this more promising?
Stronger neuroscience AI claims require stronger held-out biology. A system moves from useful research tool to credible program infrastructure only after it survives the biological transfer it is supposed to support.
For brain foundation models, the case becomes more promising if a system trained on one set of animals, tasks, laboratories, and recording platforms accurately estimates neural activity in genuinely held-out animals, stimulus families, tasks, or laboratories while beating strong encoding and linear baselines. The key is not larger pretraining by itself. The key is transfer that remains measurable when shortcuts are removed.
For connectomics, the case becomes more promising if structure improves functional prediction in prospective or held-out settings, and if segmentation uncertainty is propagated into the downstream analysis. A wiring diagram that makes a falsifiable prediction about circuit response is stronger than a visually impressive reconstruction alone.
For neural decoding, the case becomes more promising if systems generalize across sessions and participants under consented protocols, report failure modes transparently, and maintain performance without task-specific leakage. A decoder that works only after extensive participant-specific training remains useful, but it supports a narrower claim than a portable decoder.
For clinical or translational claims, the case becomes more promising only after context-of-use validation. A brain imaging model or neural embedding would need external clinical cohorts, site variation, prespecified endpoints, comparator performance, calibration, and safety analysis before it should be discussed as a decision-support tool.
What should researchers, biotech teams, funders, and program leaders do with this?
Use neuroscience AI to improve measurement, representation, and experiment selection before using it to make biological or clinical claims. The practical posture is to define the decision first, then select the method and validation plan.
For a research group, the useful next steps are concrete. Build datasets around raw-signal provenance, stimulus timing, behavioral annotations, atlas versions, and recording metadata. Use held-out animals, subjects, sessions, stimuli, and sites when the scientific question is transfer. Preserve failed decodes and failed response predictions because they define the boundary of the model.
For a program leader, the evaluation questions should be operational:
- Which decision changes because of this neuroscience tool?
- What is the weakest split under which the model still looks good?
- Does it beat a simple encoding model, GLM, PCA model, or linear decoder?
- Which biological contexts break it?
- What consent, privacy, and data-governance rules apply if human neural data are involved?
- What experiment would falsify the claim within one budget cycle?
The safest deployment path is staged. Start with analysis support and quality control. Move to experimental prioritization only after held-out validation. Treat clinical, diagnostic, neurostimulation, or patient-facing use as a separate evidence program.
Current Models, Datasets, and Benchmarks
| Resource | Type | What It Supports | Evidence Boundary |
|---|---|---|---|
| Foundation model of mouse visual cortex | Neural activity foundation model | Predicting visual-cortex responses to new stimulus types | Mouse visual cortex, defined stimulus and recording context (Wang et al., 2025) |
| BrainLM | fMRI foundation model | Self-supervised representation learning from fMRI recordings | Conference evidence; external cohort validation still task-specific (Ortega Caro et al., 2024) |
| POYO-1 | Neural population decoding framework | Multi-session neural decoding and transfer | Motor-region nonhuman-primate recordings; project and conference evidence (Azabou et al., 2023) |
| CEBRA | Neural-behavioral embedding | Joint analysis of behavior and neural activity | Latent representation, not causal mechanism (Schneider et al., 2023) |
| LFADS | Latent dynamics model | Single-trial neural population dynamics | Strong analysis tool, with interpretation tied to task and dataset (Pandarinath et al., 2018) |
| MICrONS | Functional connectomics dataset | Structure-function analysis in mouse visual cortex | Bounded by mouse visual areas and reconstruction assumptions (MICrONS Consortium, 2025) |
| Allen Brain Observatory | Open visual physiology resource | Visual-cortex activity modeling and benchmarks | Mouse visual system under standardized stimulus protocols (de Vries et al., 2020) |
| Allen Mouse Brain CCF | Reference atlas | Spatial alignment of mouse brain data | Mouse atlas; taxonomies and mappings require version control (Wang et al., 2020) |
| DANDI Archive | Official data archive | Neurophysiology sharing and reuse | Data quality depends on depositor metadata and NWB/BIDS discipline (DANDI Archive) |
| BICAN | Official brain cell atlas program | Human, non-human primate, and mouse reference atlases | Atlas generation and harmonization remain active (BICAN, NIH BRAIN Initiative) |
| Brain-Score | Benchmark framework | Comparing artificial models with neuroscience and psychology experiments | Benchmark choice defines the brain-like claim (Brain-Score) |
Evidence Standards
Neuroscience AI claims need biology-aware validation. The following questions are the minimum.
Signal: What is measured? Spikes, calcium fluorescence, fMRI, ECoG, EEG, structural MRI, electron microscopy, cell identity, behavior, or a derived feature?
Biological context: Which species, strain, sex, age, brain region, layer, cell type, disease state, task, and preparation?
Held-out unit: What was withheld from training? Random time points, trials, neurons, sessions, animals, subjects, stimuli, tasks, laboratories, scanners, electrodes, disease states, or species?
Baseline: Does the model beat simple comparators such as linear regression, ridge regression, GLMs, state-space models, PCA, canonical encoding models, or task-specific decoders?
Mechanism: Is the model predicting a measurement, organizing a representation, or making a causal claim? Causal claims need perturbation or intervention evidence.
Portability: Does performance survive new animals, subjects, sites, scanners, arrays, stimulus sets, preprocessing pipelines, and analysis teams?
Consent and governance: For human neural data, what consent covers secondary use, decoding, sharing, reidentification risk, and participant control?
Failure Modes
| Failure Mode | What It Looks Like | Why It Matters |
|---|---|---|
| Stimulus leakage | Strong response prediction on familiar stimulus families | The model may memorize stimulus-response structure rather than generalize |
| Session leakage | Train and test share animal, array, scanner, or recording day patterns | Performance does not transfer to new biological contexts |
| Atlas mismatch | A region or cell-type label is treated as stable across atlases | Biological interpretation drifts when taxonomies differ |
| Segmentation error | Connectomic reconstruction appears precise | A tracing or synapse-detection error becomes a wiring claim |
| Latent-axis overinterpretation | A dimension is named as memory, attention, or disease | Statistical representation is mistaken for mechanism |
| Decoder overclaim | A bounded task decoder is described as thought reading | Privacy and capability are overstated |
| Clinical extrapolation | Research embeddings are discussed as diagnostic tools | Clinical validity and regulatory evidence are missing |
| Animal-to-human transfer | Mouse circuit evidence is framed as human brain evidence | Species and task differences are hidden |
Practical Validation Checklist
Use this checklist before accepting a neuroscience AI claim.
- Name the modality and raw signal, not only the model family.
- Name the organism, subject group, brain region, task, and recording platform.
- Identify the held-out unit. Prefer held-out animal, subject, task, stimulus class, session, laboratory, or site over random trial splits when the claim is about transfer.
- Run simple baselines. A neural network that does not beat a well-specified encoding model or linear decoder has not earned a stronger claim.
- Report uncertainty and failure modes by condition, not only average performance.
- Preserve raw-to-processed provenance: raw data, preprocessing, model version, atlas version, stimulus timing, behavioral annotations, and exclusion criteria.
- Evaluate privacy risk for human neural data before public release or reuse.
- Keep clinical language out unless the model has clinical validation for a defined context of use.
- Treat connectomics, atlas, and imaging outputs as measurements with error, not ground truth without qualification.
- Require prospective experimental confirmation before treating a model as a guide for intervention, stimulation, or causal mechanism.
FAQ
Is a brain foundation model the same as a general artificial brain?
No. A brain foundation model is a pretrained representation for a defined brain data modality or task family. It may transfer better than a single-task approach, but its claim remains bounded by the data and validation design.
Can neural decoding read thoughts?
Current evidence does not support unconstrained thought reading. Neural decoders can infer specific signals under task, training, consent, and recording constraints. The strongest studies are careful about cooperation and experimental context.
Are BCIs covered here as clinical tools?
No. This chapter treats BCIs as research interfaces for neural decoding and communication studies. Clinical guidance, device approval, implantation decisions, and patient counseling require separate clinical and regulatory evidence.
What makes connectomics useful for AI?
Connectomics gives AI models structural context. It can link activity to wiring, constrain circuit hypotheses, and expose structure-function relationships. Its limitations are reconstruction error, scale, species, and tissue context.
What is the best first question to ask about a neuroscience AI model?
Ask what was held out. A random split answers a weaker question than a held-out animal, subject, stimulus family, recording session, laboratory, or disease state.
How should brain imaging foundation models be interpreted?
They should be interpreted as representation tools unless clinical validation is present. A useful fMRI or MRI embedding supports research analysis without becoming a diagnostic system.