Both papers are, at their core, supervised machine learning systems. This module gives you the full vocabulary of that game: data, labels, loss, gradient descent, splits, overfitting, and the first evaluation metrics.
Classical programming: a human writes explicit rules ("if the DNA letter at position 51 is T, then…"). Machine learning flips this: we show the program examples and let it find the rules itself. Instead of writing the recognizer, we write a procedure that produces a recognizer from data.
In supervised learning — the setting of both papers — the data is a set of pairs (x, y): an input x and its correct answer, the label y. The numbers describing each input are its features. Paper 1's inputs are 101-letter DNA windows labeled with a disease; Paper 2's inputs are tactile images labeled with a material.
Two flavors of supervised learning, split by the type of label:
For a K-class problem, labels are usually written as one-hot vectors: a vector of K entries with a single 1 at the true class and 0 everywhere else. Example: with 4 classes, "class 2" (counting from 0) is encoded as (0, 0, 1, 0). This turns a label into a probability distribution that puts all its mass on the truth — exactly the target that cross-entropy (Module 1.7) compares against.
To improve a model you must first score it. A loss function compresses "how wrong is the model on this example?" into a single non-negative number — 0 would mean perfect, larger means worse. Training is then simply: adjust the weights to make the average loss over the training data smaller.
For multi-class classification the standard loss is cross-entropy, which you met in Module 1.7: with a one-hot target it equals −log ptrue, punishing the model heavily when it assigns the true class a tiny probability (assigning 0.9 costs 0.105; assigning 0.01 costs 4.6).
The loss is a function of all the model's weights. The gradient ∇L points uphill in weight-space (Module 1.4), so we repeatedly step the other way:
Here η (eta) is the learning rate — the step size. Too large, and you overshoot the valley, bouncing around or diverging entirely. Too small, and training crawls, possibly never reaching a good solution in your compute budget. It is the single most important hyperparameter to get right.
Computing ∇L over the entire dataset per step is wasteful, so in practice we use stochastic (mini-batch) gradient descent: each step estimates the gradient from a small random batch of examples. Paper 1 uses a batch size of 32 — each update looks at 32 DNA windows. One full pass through the training set is called an epoch; training runs for many epochs.
We never judge a model on the data it trained on — it may have simply memorized it. So the dataset is split into three disjoint parts:
| Split | Used for | How often it's touched |
|---|---|---|
| Training set | Computing gradients and updating weights | Every step |
| Validation set | Tuning hyperparameters (learning rate, architecture) and deciding when to stop (early stopping, Module 3.6) | Every epoch, but never for gradients |
| Test set | The final, honest estimate of real-world performance | Once, at the very end |
Underfitting: the model is too simple (or under-trained) to capture the real pattern — it performs poorly even on the training data. Overfitting: the model captures the training data too well, memorizing its noise and quirks, and so fails on new data. The goal is neither: it is generalization.
The classic diagnostic: as training proceeds, training loss keeps falling while validation loss bottoms out and starts rising. The model is now improving its memorization of the training set at the expense of everything else.
The fix families (each gets its own treatment later): more data, data augmentation (Module 4 — Paper 1 augments its rare-disease examples heavily), regularization such as dropout and weight decay (Module 3.5), and early stopping (Module 3.6).
Accuracy — the fraction of predictions that are correct — is intuitive but dangerously misleading under class imbalance. If 99% of screened patients are healthy, a "model" that predicts healthy for everyone scores 99% accuracy while detecting zero sick patients. Both papers face imbalance (rare diseases; unevenly sampled materials), so they must report better metrics.
The bookkeeping tool is the confusion matrix. For two classes ("positive" = has the disease):
| Predicted positive | Predicted negative | |
|---|---|---|
| Actually positive | TP (true positive) | FN (false negative — a miss) |
| Actually negative | FP (false positive — a false alarm) | TN (true negative) |
From it:
Precision: of everything I flagged, how much was real? Recall: of everything real, how much did I flag? F1 is their harmonic mean — it is dragged toward the smaller of the two, so a model can't hide a terrible recall behind a great precision.