Getting Started - NeuroMaestro

This page will get you from zero to a running pipeline job in about 5 minutes.

Prerequisites¶

Python 3.10+
Access to an HPC cluster with SLURM
Neuroimaging software installed on the cluster (fMRIPrep, XCP-D, AFNI, FSL, MRIQC, QSIPrep as needed)
Singularity containers for containerized tools

Installation¶

1. Install the package on your HPC system:

pip install neuromaestro

To work from source instead, clone the repository and install it in development mode:

git clone https://github.com/GeorgiaCenterforDevelopmentalScience/NeuroMaestro.git
cd NeuroMaestro
pip install -e '.[dev]'

2. Verify the installation:

neuromaestro --help
neuromaestro-gui --help

Minimal Quickstart (5 minutes)¶

The fastest way to verify everything works is a dry-run: it runs the pre-flight checks, prints the task dependency plan, and writes the job wrapper scripts without submitting any jobs.

Step 1: Initialise your config directory¶

Run neuromaestro init once to create a config directory pre-populated with template files:

neuromaestro init /scratch/my_study

This creates:

/scratch/my_study/
├── config/                    ← pass this to --config-dir
│   ├── config.yaml
│   ├── hpc_config.yaml
│   ├── project_config/
│   │   ├── template_config.yaml   (start from this one)
│   │   └── (the other bundled configs)
│   └── results_check/
│       ├── template_checks.yaml
│       └── (the other bundled checks)
└── scripts/
    └── (template .sh scripts)

All bundled config and check files are copied, not just the template. Copy template_config.yaml to {your_project}_config.yaml and template_checks.yaml to {your_project}_checks.yaml, then edit those. The rest are reference bundles and are described in Dev & Test Config Modes. One of them, ds027, is a complete published run you can reproduce from public data (see Reproduce the Published Example).

You then pass --config-dir /scratch/my_study/config to every neuromaestro command. The pipeline uses four config files split into two tiers:

Important

Skip --config-dir by exporting an environment variable. If you always work with the same config directory, add this line to your ~/.bashrc once:

export NEUROMAESTRO_CONFIG_DIR=/scratch/my_study/config

After that, all neuromaestro commands will pick it up automatically and you can omit --config-dir. You can still override it any time by passing --config-dir explicitly.

Required — create one per project:

File	Location inside `--config-dir`	What you do
`{project}_config.yaml`	`project_config/`	Per-project settings: HPC paths, module versions, and task parameters. This is the main file you create for each new study.
`{project}_checks.yaml`	`results_check/`	Output file checks used by `--resume` and `check-outputs`. Required for resume and output verification to work.

Shared — review once, rarely touch after that:

File	Location inside `--config-dir`	What you do
`hpc_config.yaml`	root	Scheduler settings, cluster-wide resource profiles, and job submission flag templates. Review when setting up on a new cluster.
`config.yaml`	root	Global task definitions shared across all projects: each task’s script, upstream task, output folder, and the name of the resource profile it uses. Ships with defaults for the standard task set. Only edit this when adding a new task type.

The most common setup task is editing {project}_config.yaml. Use the GUI to generate a template:

neuromaestro-gui

Go to Project Config → Project Config tab → Generate Template, fill in your cluster paths, module names, and task parameters. See the Project Config Guide for a full annotated example.

Step 2: Detect your subjects¶

# Standard BIDS dataset (folders named sub-001, sub-002, ...)
neuromaestro detect-subjects /data/BIDS --prefix "sub-"

# No prefix (folders named 001, 002, ...)
neuromaestro detect-subjects /data/raw --prefix ""

Returns subject IDs with the prefix stripped; pass these bare IDs to --subjects in all subsequent commands. See Complete Pipeline Walkthrough for how the prefix mechanism works.

Step 3: Dry-run a single subject¶

neuromaestro run \
  --subjects 001 \
  --input /data/raw \
  --output /data/processed \
  --work /data/work \
  --config-dir /scratch/my_study/config \
  --project my_study \
  --session 01 \
  --prep unzip_recon \
  --dry-run

Full pipeline (all stages, multiple subjects):

neuromaestro run \
  --subjects 001,002,003 \
  --input /data/raw \
  --output /data/processed \
  --work /data/work \
  --config-dir /scratch/my_study/config \
  --project my_study \
  --session 01 \
  --prep unzip_recon \
  --intermed volume \
  --staged-prep cards,kidvid \
  --bids-prep rest,dwi --bids-post rest,dwi \
  --mriqc all \
  --dry-run

With --dry-run, no jobs are queued. The output includes the DAG execution plan, which lists each task with the tasks it waits for. For the single-subject command above it reads:

DAG execution plan:
  unzip <- (no dependencies)
  recon <- unzip

The wrapper script for each task is written to /data/work/my_study/log/wrapper/ and shows the module loads, exported variables, and submission command. If the plan looks right, remove --dry-run to submit for real. See run command reference for all available flags.

Step 4: Monitor your jobs¶

squeue -u $USER      # SLURM queue
neuromaestro-gui     # Job Monitor tab, after Sync Database; open the printed token link

Next Steps¶

Complete Pipeline Walkthrough covers all seven stages with architecture context. For config details, see the Project Config Guide and Pipeline Tasks Reference. The GUI Reference walks through Analysis Control, job monitoring, and report generation. Full CLI flag documentation is in the CLI Reference.