Module 4 · Core Architectures

Data: Sequences, Images & Imbalance

Before any clever architecture can run, raw DNA letters and raw tactile photos must become numbers — and the dataset itself must be shaped so the model can learn fairly from it. This module covers how both papers encode their data, and how Paper 1 fights the defining problem of rare-disease genetics: almost no examples.

▶

Audio recap

A ~2-minute spoken summary of this module — great for revision on the go.

4.1 Computers need numbers

Recall from Module 1: every model in this course is ultimately a stack of y = Wx + b operations. Matrices multiply vectors. They cannot multiply the letter "G" or a photograph. So the very first design decision in any deep learning project is the encoding question: how do we turn the raw input into vectors without destroying — or inventing — information?

Both papers face it head-on:

Paper 1: DNA is text-like — a string over {A, C, G, T} (Module 0). How does a string become a matrix?
Paper 2: a tactile reading is an image. How does a picture become numbers? (Spoiler: it already is one.)

Why you care Oral examiners love the question "walk me through what the model actually receives as input." If you can say "a 101×4 one-hot matrix" for Paper 1 and "an H×W×3 pixel grid, cut into patches" for Paper 2 — with reasons — you sound like someone who has trained models, not just read about them.

4.2 Encoding DNA: from letters to a 101×4 matrix

Step one in Paper 1's pipeline is integer encoding: each of the five possible symbols — A, C, G, T, and N (an unknown/ambiguous base reported by the sequencer) — is mapped to an integer in 0–4. But integers are only a bookkeeping step, because feeding them directly into a network has a hidden flaw.

The false-ordering trap If A=0, C=1, G=2, T=3, the model sees numbers on a line: it will treat T as "three times bigger" than C, and G as sitting "between" C and T. Is G really less than T? Is C the average of A and G? Of course not — nucleotides are categories with no order and no distances. Raw integer inputs smuggle in relationships that do not exist, and the model will happily learn from the lie.

The fix is one-hot encoding: each symbol becomes a vector with a single 1 and zeros elsewhere. With the four bases (plus N handled as a fifth symbol or an all-zero/uniform row):

A = (1, 0, 0, 0) C = (0, 1, 0, 0) G = (0, 0, 1, 0) T = (0, 0, 0, 1)

Now every base is exactly the same "distance" from every other — no fake ordering. A 101-letter DNA window becomes a 101×4 matrix: 101 positions, 4 channels per position. Squint and you'll notice this is shaped like a one-dimensional image with 4 color channels — which is precisely why 1-D convolutions (Module 5) apply so naturally.

Tiny worked example The sequence ACGT one-hot encodes to the 4×4 matrix with rows (1,0,0,0), (0,1,0,0), (0,0,1,0), (0,0,0,1). The sequence AAGT differs only in row 2 — the encoding preserves exactly where and how sequences differ, nothing more.

Paper 1's window design Each variant is embedded at the center of a 101-bp window: the mutated base plus ±50 bp of flanking context on either side. Centering keeps the model's attention on the mutation neighborhood — the local sequence context that determines a motif's meaning — while 101 bp stays computationally cheap compared to feeding whole genes. All sequences are padded or trimmed to this uniform length so every training example is the same 101×4 shape.

4.3 Images as data

An image needs far less ceremony: it is already a grid of numbers. A color image is an H × W × C array — height H rows, width W columns, and C=3 channels (red, green, blue intensity) per pixel. A 224×224 RGB image is 224×224×3 ≈ 150,000 numbers.

Paper 2's tactile readings are ordinary images in this sense: the camera inside a GelSight, DIGIT, or GelHex sensor outputs an RGB photo of the deforming gel (Module 0). No special encoding needed — the challenge is what the network does with those pixels. Module 5 shows the CNN way (slide small filters over the grid); Module 7 shows the Vision Transformer way (cut the image into patches and treat them like words).

4.4 Embeddings: a first look

One-hot vectors are honest but dumb: sparse (mostly zeros), high-dimensional if the vocabulary is large, and fixed forever — the encoding of A never adapts to the task. The alternative is an embedding: assign each symbol (or word, or image patch) a dense, learned vector, and let training move those vectors around. After training, things that behave similarly end up nearby in the vector space — "rough" near "coarse," similar image patches near each other.

One-hot vs. embedding One-hot: sparse, hand-fixed, all symbols equidistant, dimension = vocabulary size. Embedding: dense, learned by gradient descent, similarity emerges from data, dimension is a design choice. Paper 1's small 5-symbol alphabet makes one-hot perfectly adequate; Paper 2 lives and dies by embeddings — its whole method is aligning tactile embeddings with language embeddings. Module 7 makes this central.

4.5 Class imbalance: when the model ignores the rare

Here is Paper 1's defining data problem. Rare monogenic diseases are, by definition, rare: for some diseases only a handful of pathogenic variants are documented. A naive dataset would contain thousands of examples of common classes and a few dozen of rare ones.

Why is that fatal? Because training minimizes average loss. A model can score 99% accuracy by simply always predicting the majority classes and never predicting a minority class at all — the few extra mistakes barely dent the average. The rare disease class, the one a clinician most needs flagged, becomes invisible.

Three standard remedies — Paper 1 uses all three:

(a) Oversampling / augmentation: manufacture more minority-class examples so the classes see comparable amounts of training signal (Section 4.6).
(b) Class-weighted loss: multiply each example's loss by a weight that is larger for rarer classes, so one minority mistake "hurts" as much as many majority mistakes. Paper 1 uses class-inverse weighting: each class's weight is proportional to 1 / (its frequency) — the rarer the class, the bigger the penalty for getting it wrong.
(c) Stratified splitting / sampling: when carving the data into train/validation/test sets (or batches), preserve each class's proportion in every piece. Paper 1 splits 70/15/15, stratified, with a fixed random seed of 42 — so every split contains every class, and the experiment is exactly reproducible.

Augmentation is not a free lunch Synthetic minority examples can inject artifacts — statistical quirks of the generation procedure that real biology doesn't have — and the model may learn those instead. Paper 1 itself concedes that its synthetic augmentation "may not fully mimic real-world benign variations." Any examiner who has read the paper knows this line; you should too.

4.6 Data augmentation

Data augmentation means applying label-preserving transformations to training examples to multiply the data: change the input in ways that do not change the correct answer. For images, the classics are horizontal flips, small crops, and rotations — a flipped photo of wood is still wood. For DNA, Paper 1 uses two domain-specific transformations:

Reverse complementation

DNA is double-stranded: two chains zipped together, where A always pairs with T and C always pairs with G, and the two strands run in opposite directions. So every sequence has a mirror twin — read the other strand backwards and you get an equally valid representation of the same physical stretch of DNA. To compute it: reverse the sequence, then swap A↔T and C↔G.

Example 5'-ATGC-3' → reverse: CGTA → complement each base: GCAT. So ATGC and GCAT are the same piece of DNA seen from the two strands. A model that labels one "cystic fibrosis" should label the other identically — a perfect label-preserving transformation.

k-mer jittering

k-mer jittering makes small, biologically plausible random nucleotide replacements in the window (a k-mer is just a short subsequence of length k). The intuition: real sequencing has read errors, and real populations carry natural harmless variation, so tiny perturbations around the variant should not flip the diagnosis. Jittering simulates that noise, teaching the model to key on the causal mutation rather than memorizing every flanking letter.

Paper 1's augmentation pipeline For each documented mutation, Paper 1 generates N = 500 augmented examples — up to a 500× boost for the scarcest classes — placing each mutation centrally within randomly generated background sequence and applying reverse complementation and k-mer jittering. The negative class, "Not a Disease," consists of purely random 101-bp sequences. Note the design choice hiding there: random sequences are easy negatives. A real benign variant sits in real genomic context and looks much more like a pathogenic one — a known limitation the paper acknowledges, and Section 4.7's whole story.

4.7 Synthetic data: power and peril

Paper 1's dataset is largely synthetic: real documented mutations, but embedded in artificial random backgrounds, multiplied by artificial transformations, and contrasted against artificial random negatives. That buys control (perfect balance, unlimited examples, exact reproducibility with seed 42) at the price of realism.

Consequences you must be able to reason about aloud:

The "Not a Disease" class is random noise — separating a real human genomic window from random letters is far easier than separating a pathogenic variant from a benign variant in the same gene. The clinically important boundary is the hard one, and it is barely tested.
Near-perfect reported metrics therefore partly measure the easiness of the synthetic task, not clinical readiness.
Augmentation artifacts (Section 4.5's warn box) may inflate apparent robustness.

Exam favorite "How would the results change on real clinical data?" Expected answer: performance would drop — most sharply on distinguishing pathogenic variants from realistic benign variants, because the model never saw hard negatives during training. Say this before the examiner does: acknowledging the synthetic-data ceiling voluntarily reads as maturity, not weakness.

Exam warm-up — say it out loud

Why one-hot encode DNA rather than just feeding A=1, C=2, G=3, T=4?
Explain reverse complementation to a non-biologist in three sentences, with the ATGC → GCAT example.
Paper 1 uses three separate weapons against class imbalance — name them and give one sentence of intuition each.
Why is the "Not a Disease" class design a limitation of Paper 1?