All pipelines

Batch processing

Run one analysis with the same settings on a folder of files and get one summary table, a log and, for LDF, the trial files.

What each pipeline measures

What each pipeline measures

  • LDF: the LDF Process steps (decimate, filter, cut trials around each onset), then the response features of the mean trial (onset 0 s, baseline = the pre-onset part): peak latency and amplitude, onset delay, FWHM, AUC, rise and decay time, integral.
  • LFP: ERP per channel as in LFP Analysis (onsets on the mean-subtracted stimulus). N1 = minimum in the N1 window, P2 = maximum in the P2 window; latency in ms after the stimulus, amplitude relative to the pre-stimulus mean, multiplied by Amplitude scale (1e6: V to µV). With Compute CSD (≥ 3 channels, in the listed order, top to bottom): the CSD minimum in the N1 window per channel (AmpUnit / mm²) and the sink channel of the file. Spacing (µm) 0 uses the spacing saved in each file (lfp_spacing_um); a file that gives none gets its ERP rows without a CSD instead of failing (its Checks say so).
  • MUA: spike sorting per channel as in MUA Analysis. The random generator is set to Random seed before every channel, so the same file always gives the same clusters, whatever its position in the list. Per channel: spikes in units, units (good / rejected by the quality check: SNR < 2 or > 2% ISIs below the refractory period), mean rate and rate per unit, mean SNR, worst ISI violation and, with a stimulus, the rate in the response window vs the baseline window.
  • EEG: one file per participant, raw or cleaned, through the steps of EEG Analysis in the same order: bad channels (the file's own, those typed in Bad channels that the file has and, with Also suggested bad channels, the flat or very noisy ones) → zero-phase filters (high-pass, low-pass, notch) → re-reference (average, mastoids or typed channels) → trials around the events of a continuous recording (Events and names, e.g. S 1 = Standard) → rejection by peak-to-peak or absolute amplitude → ERP per condition (baseline subtracted) → mean or peak amplitude in the window at the chosen channels. One row per condition: trials kept and rejected, value (µV), latency (ms, peak only), peak on the window edge, channels used, bad channels, reference and sampling rate.
  • Imaging: per ROI the mean brightness and ΔF/F (F0 = mean of the first baseline frames) with its peak and peak time; with a line, the vessel diameter (FWHM; Robust diameter ignores red blood cells) as mean, min and max. No smoothing or normalisation is applied; Motion correction registers every frame onto the mean image first.
  • Response features: the nine features of Signal Characterization, for the mean of the series in each file or for every series.

Good to know

  • Channels are the numbers saved in the file (lfp_channels / mua_channels), otherwise the row numbers. Empty = all channels.
  • Vector fields take numbers separated by spaces, e.g. 5 50 for a window or 45 70 75 70 for a line.
  • The same pipelines run from scripts: R = Batch.run('ldf', folder, params, outFolder); Batch.defaults('ldf') lists the settings.

A file that fails does not stop the batch: it gets a red row with the reason, and the other files are processed. Cancel stops before the next file and still writes the summary of the files done. Rows are coloured by status: green ok, orange warning (some channels failed), red error, grey skipped.

Walkthrough on the demo batches

Inputs and outputs

In

  • LDF: cropped .mat files from LDF Extract (stim, LDF, t, Fs)
  • LFP: .mat files from Extract Ephys (lfp_data, stim_data, lfp_fs, stim_fs; t_lfp, lfp_channels and lfp_spacing_um optional: Spacing 0 uses each file's lfp_spacing_um)
  • MUA: .mat files from Extract Ephys (mua_data, mua_fs; t_mua, mua_channels, stim_data + t_stim optional)
  • EEG: one file per participant, as EEG Analysis reads it: BrainVision .vhdr, EDF / BDF, EEGLAB .set, FieldTrip .mat, XDF; raw (continuous) or already cut into trials
  • Imaging: .mat with stack (or frames), optional timeVec / t and roiMask / roiMasks, or a multi-frame TIFF
  • Response features: any file Signal Characterization reads (segmentedLDF + segmentedTime, lfp_data + t_lfp, t + y, t + LDF)

Out

  • <name>_summary.csv and <name>_summary.mat (table summary + struct batch with the pipeline, settings, files, statuses, messages, log and date; Export… in the window saves the same two variables): columns File, Status, Message, then the pipeline's results
  • LDF, LFP, MUA and EEG: a Checks column: the checks of the matching window (LDF Process, LFP Analysis, MUA Analysis, EEG Analysis) in one cell (OK, 1 to check: Topic: … or 2 warnings: Topic: …, the first of the most serious kind): per file for LDF, LFP and EEG (the same on every row of the file), per channel for MUA; the log line of each file ends with the same. The checks do not change the Status: read them before using the numbers
  • <name>_log.txt: date, settings and one line per file (result or error)
  • LDF: trials/<file>_segments.mat per recording (segmentedLDF, segmentedTime, Fs), ready for LDF Average

From scripts

The same pipelines run without the window, which is useful for many sessions or for a methods section:

p = Batch.defaults('ldf');          % the settings and their defaults
p.threshold = 2.5;
R = Batch.run('ldf', folder, p, outFolder);

Pipelines: 'ldf', 'erp', 'mua', 'eeg', 'roi', 'features'. The MUA pipeline sets the random generator to the Random seed setting before every channel, so the same file always gives the same clusters, and leaves MATLAB's global random stream as it was.

Demo expectations

Try demo batch writes a few files per pipeline, each with slightly different known answers (generator: core/demo/demoBatch.m, described on the demo data page).

  • Data: Try demo batch writes synthetic files for the chosen pipeline, each with slightly different known answers, and fills the settings.
  • LDF: 4 cropped recordings (200 s, 7 stimuli of 5 s). What you should get: 7 onsets and 6 trials per file (the last stimulus is too close to the end); peak latency ~3, 3.5, 4 and 4.5 s and peak amplitude ~20, 25, 30 and 35 PU (within ~2 PU); one trial file per recording in the trials folder. The Checks column of each file says 1 to check: Trials (that left-out stimulus) and nothing else.
  • LFP: 3 recordings, 8 channels 100 µm apart. What you should get: 8 rows per file; the N1 is largest and the CSD sink (SinkChannel) is on channel 3, 4 and 5, with the N1 at ~12, 15 and 18 ms (roughly −100 to −140 µV on the sink channel). The Checks column says OK for every file.
  • MUA: the demo MUA recording and a copy recorded at twice the gain, channel 4. What you should get: 2–3 units and the same spike count in both files (sorting does not depend on the gain); the evoked rate (5–55 ms after each stimulus) is several times the baseline rate. The Checks column has no warnings; Signal-to-noise may be a Check when a weak cluster (unit 3 seen from channel 5) is kept.
  • EEG: the 3 raw recordings of the EEG demo (BrainVision, 500 Hz, 32 channels against FCz, a noisy T7, blinks in 8 trials each). The demo marks T7 bad, filters 0.1–30 Hz, re-references to the average, cuts −200 to 800 ms around S 1 = Standard, S 2 = Target, S 3 = Novel, rejects above 100 µV peak-to-peak and measures the mean amplitude 300–400 ms at Pz. What you should get: 3 rows per file, 62 trials per file with exactly the 8 blink trials rejected, and the P300 Target (about 5.6 to 7.7 µV) > Novel (about 3.4 to 4.7 µV) > Standard (about 0.7 to 1.9 µV) in every file: the same numbers as EEG Analysis with the same settings. The Checks column has no warnings; Trials per condition is to check (Target and Novel keep fewer than 20 trials, enough for the P300).
  • Imaging: 3 stacks with a cell (roiMask) and a vessel crossed by the line 25 50 63 50. What you should get: peak ΔF/F ~0.5, 1.0 and 1.5 at ~6.7 s (the second of two transients, which rides on the tail of the first) and mean vessel diameter ~10, 12 and 14 px.
  • Response features: 4 trial files. What you should get: peak latency ~3, 3.5, 4, 4.5 s and peak amplitude ~20, 25, 30, 35 PU (one row per file); Series = Each series gives one row per trial (32 rows).

Troubleshooting: Batch processing in the guide.