Module 14: Computer Vision for EM

Lesson Flow

Learn

Goals and Concepts

Start with the capability target and concept set for this module.

Practice

Studio Activity

Apply the ideas in a guided activity tied to realistic outputs.

Check

Assessment Rubric

Use the rubric to verify competency and identify improvement targets.

Teach

Slides and Worksheets

Open the teaching deck, worksheet, and editable slide source.

Interactive Lab

Practice in short loops: checkpoint quiz, microtask decision, and competency progress tracking.

Checkpoint Quiz

Q1. Which output most clearly demonstrates module competency? A measurable artifact linked to an explicit method A general reflection about interest in neuroscience A list of future goals without evidence

Competency is shown through measurable, method-linked evidence.

Q2. What should always accompany a technical claim in this curriculum? A limitation or uncertainty statement A motivational quote A citation count

Every claim should include boundaries and uncertainty.

Q3. What is the best next step after identifying a gap in understanding? Define a concrete practice task and verification criterion Skip to a harder module Ignore the gap if progress is slow

Progress improves when gaps become explicit practice targets.

Microtask Decision

Choose the action that best improves scientific reliability.

Record method, version, and decision rationale before sharing results Share results quickly and document details later Rely on memory for why decisions were made

Progress Tracker

State is saved locally in your browser for this module.

Read capability target
Complete studio activity
Review assessment rubric
Pass checkpoint quiz
Complete microtask
Complete annotation challenge

0% complete

Annotation Challenge

Click the hotspot with the strongest evidence for the requested feature.

Selected hotspot: none

Capability target

Design and evaluate a CV pipeline for EM imagery that is fit for a specific connectomics task and explicitly bounded by known failure modes.

Why this module matters

CV is central to modern connectomics throughput. Without rigorous validation and domain-aware error analysis, CV outputs can silently corrupt reconstruction and inference.

Concept set

1) Task-model fit

Technical: detection, segmentation, denoising, and classification tasks require different objective functions and architectures.
Plain language: pick models for the job, not by popularity.
Misconception guardrail: one model can solve all EM tasks equally well.

2) Error taxonomy over headline metrics

Technical: splits/merges, boundary drift, and artifact sensitivity are often more informative than aggregate accuracy.
Plain language: understand exactly how models fail.
Misconception guardrail: high benchmark score implies safe downstream use.

3) Validation as release gate

Technical: model release should require acceptance criteria tied to biological impact and reproducibility checks.
Plain language: do not ship CV outputs without clear go/no-go rules.
Misconception guardrail: visual plausibility is sufficient validation.

Core workflow

Define EM task and acceptable error envelope.
Select baseline and candidate CV approaches.
Run evaluation using biologically relevant metrics.
Perform failure-case review on ambiguous regions.
Publish model card with limitations and intended use.

60-minute tutorial run-of-show

00:00-08:00 task framing + exemplar failure modes.
08:00-20:00 choose metrics tied to downstream biology.
20:00-34:00 evaluate baseline vs candidate model.
34:00-46:00 error taxonomy and triage discussion.
46:00-56:00 model card drafting.
56:00-60:00 competency check.

Studio activity

Scenario: Compare two segmentation-support CV models for an EM subvolume.

Outputs

metric table with biological interpretation,
failure-case log,
model-card limitation statement.

Assessment rubric

Minimum pass: clear task-model rationale, biologically relevant metrics, explicit limitations.
Strong performance: robust failure analysis and operational release criteria.
Failure modes: metric-only reasoning, weak split design, no deployment boundaries.

Key architectures for EM connectomics

U-Net (Ronneberger et al. 2015)

Encoder-decoder architecture with skip connections. The encoder downsamples the image to extract features; the decoder upsamples to produce pixel-level predictions; skip connections preserve fine-grained spatial detail. Originally designed for biomedical image segmentation. In connectomics, 3D U-Nets predict boundary/affinity maps at each voxel.

Why it works for EM: EM images have consistent texture and contrast patterns. The encoder learns to detect membranes, vesicles, and other structures; the decoder produces a per-voxel prediction map.

Flood-Filling Networks (Januszewski et al. 2018)

An iterative approach: a CNN predicts whether each neighboring voxel belongs to the same object as the current seed, and the segment “grows” outward. FFNs produce instance segmentation directly (each neuron gets a unique ID) without the separate watershed + agglomeration step.

When to use: FFNs are computationally expensive but produce high-quality segmentation with fewer post-processing stages. Used in FlyWire and other Google-based reconstructions.

Affinity prediction + watershed + agglomeration

The standard two-stage pipeline: (1) A 3D CNN predicts pairwise affinity between neighboring voxels (probability they belong to the same segment). (2) Watershed transform produces an over-segmentation of millions of supervoxels. (3) Agglomeration merges supervoxels based on affinity scores at boundaries.

When to use: More modular and parallelizable than FFN. Standard in academic pipelines (Funke et al. 2019).

Data augmentation for EM

Training data is expensive (manual annotation). Augmentation expands the effective training set:

Elastic deformations: simulate tissue warping
Intensity variations: simulate staining differences
Missing section simulation: randomly drop sections during training so the model learns to handle gaps
Rotation/flipping: standard geometric augmentations
Artifact injection: add synthetic knife chatter or charging patterns

Content library references

Reconstruction pipeline — End-to-end segmentation architecture
Metrics and QA — VI, ERL, boundary F1 for evaluation
Artifact taxonomy — What CV models must handle
Error taxonomy — Merge/split/boundary errors from CV

Teaching resources

References

Ronneberger O et al. (2015) “U-Net: Convolutional Networks for Biomedical Image Segmentation.” MICCAI 2015.
Januszewski M et al. (2018) “High-precision automated reconstruction of neurons with flood-filling networks.” Nature Methods 15(8):605-610.
Funke J et al. (2019) “Large scale image segmentation with structured loss.” IEEE TPAMI 41(7):1669-1680.
Lee K et al. (2019) “Superhuman accuracy on the SNEMI3D connectomics challenge.” arXiv:1706.00120.

Quick practice prompt

Document one CV result with one supported use case and one forbidden use case.

Teaching Materials

Slide Deck

Classroom-ready deck links for teaching and delivery.

Open slide deck page

Open rendered HTML deck

Download PowerPoint (.pptx)

Activity Worksheet

Learner worksheet aligned to the studio activity and rubric.

Open worksheet

Slide Source

Marp source file for editing and rendering.

course/decks/marp/modules/module14.marp.md