Supervised Learning

1What Is Supervised LearningIntuition

This section answers: To make a machine “learn” something, what is the most direct approach?

The most direct approach is somewhat similar to using worked examples to teach people to generalize rules: give the model manyinputs and corresponding targets, and make the predictions gradually approach these targets. The recorded target value here is calledlabel; “supervised” means that during training each input has a comparable target signal, not that there is a teacher watching in real time.

Therefore, supervised learning can be understood asusing “input–target” examples to learn a prediction rule. It aims to solve: given a new email not seen during training, how to make a checkable judgment based on past examples. During training, theinputis the email content and the corresponding label,outputis a model that can produce categorical or numerical predictions for new emails.

Its basic process is to first have the model predict the training examples, then compare the predictions with the labels, and then adjust the rule based on the error and repeat. When the model gives a “spam probability of 0.8” for a new email, it means that under the current rule it leans toward the spam category; this does not prove that the label is absolutely correct, nor can it guarantee that it will remain reliable when the source of emails changes.

2Two Types of Tasks: Classification and RegressionIntuitionMath

The labels in the previous section were “spam/normal.” But answers are not always this kind of “choose one of several.” What the answer looks like divides Supervised Learning into two major categories.

Classification and regression describethe two basic forms of Supervised Learning output, used to avoid mixing up categories, probabilities, and continuous numerical values.

Given sample features and training targets, classification outputs a class or probabilities for each class, and regression outputs a continuous numerical value. During training, both first produce predictions, then use a task-appropriate loss to compare the prediction with the target, and finally adjust the model.

3What Does “Learning” Actually Learn?Math

Classification and regression look different, but when abstracted into mathematics, what training does isthe same thing. What is it?

Think of the rule to be learned as aparameterized function f_θ: feed in the input x, and it outputs a prediction f_θ(x). Here θ is the set of parameters that the model needs to learn,n is the total number of training samples,i is the sample index,xᵢ and yᵢ are the input and label of the i-th sample, respectively. Learning means finding a set of parameters θ, which makes the average error on the training samples as small as possible:

L is the loss function, which turns the disagreement between a prediction and the label into a number; for now just understand its role. See 1.2 for specific losses and 1.3 “Gradient Descent” for how parameters are updated.

Here “learning” refers tofinding a set of parameters from the candidates that has a relatively small average loss. The input is the training samples, the current parameters, and the loss rule; the output is the updated parameters and the prediction function determined by them. During training, first use the current parameters to compute predictions, then compute the loss for each sample, then aggregate the batch average and update the parameters; after repeating for many batches, use independent data to judge whether the rule has actually improved.

3.1 Work Through a Binary Classification Batch by Hand

First, unify notation: let y=1 represent spam, and y=0 represent normal email; the model always outputs the same quantity p=P(y=1|x), that is the probability that “this email is spam.” Now consider a small batch with only two emails:

• The first is actually spam (y=1), and the model outputs p=0.80, so the probability it assigned to the true class is q=0.80.
• The second is actually normal mail (y=0), and the model outputs p=0.30; the probability of normal mail is 1-p, so the true class probability q=0.70.

The full form of binary cross-entropy is:

When y=1, the second term becomes 0, and the loss is −ln(p); when y=0, the first term becomes 0, and the loss is −ln(1−p). In other words, both cases can be written uniformly as −ln(q), where q is the probability that the model assigns to the true class.

A decreasing average loss means the model is closer to the labels under the current sample and loss definition, but it does not mean every sample is predicted correctly, nor does it automatically imply that it can generalize. Parameter update methods vary by model; the binary classification mini-batch in this section only explains “how probability becomes loss” and cannot replace the later gradient derivation or test set acceptance.

4Success Criterion: Not Memorization, but GeneralizationIntuitionEngineering

Since training is about “making predictions close to the labels,” does getting all the labels in the training set right mean success?

So the standard practice is to split the data into three parts, each with its own role:

4.1 Metrics Must Correspond to Error Costs

Suppose there are only 100 spam emails among 10,000 emails. A model that classifies all emails as “normal” still has 99% accuracy, but it misses all spam emails. Therefore, acceptance testing cannot only ask “how many overall are correct”; it must also look separately at:

The model outputs probabilities, and finally classifying them as “spam/normal” still requires choosing a threshold. The lower the threshold, the more are blocked and the more false positives; the truly appropriate threshold depends on the business cost of the two types of errors.

Generalization evaluation describeswhether the model remains effective on data that did not participate in training and selection, and it solves the problem that training scores cannot represent real-world performance.

The inputs to evaluation are a frozen model, independent samples and their targets; the outputs are acceptance evidence such as loss, precision, recall, and business slices. The specific procedure: first use the training set to fit parameters, then use the validation set to select the approach, and finally use the test set only after the approach is frozen.

When validation error starts to increase while training error continues to decrease, it indicates the model is more like memorizing training details. But a single high test score is still affected by sample representativeness, temporal changes, and data leakage, and cannot prove reliability in all future scenarios.

5The Cost of Supervision: Labeling BottleneckEngineering

Where does such a direct and effective method fall short? The answer lies in the word “supervision”.

The fuel for supervised learning isdata with target values. Targets may come from manual labeling, or from transaction results, sensor measurements, user behavior, or rule systems. What is truly difficult is not just increasing the number of labels, but obtaining data that is consistent with the real target, covers deployment scenarios, and whose quality can be verified:

• Expensive, slow: large-scale labeling is a huge amount of human effort, and many fields still require experts (medical imaging, law).
• The target may be a proxy: clicks do not necessarily equal liking, and historical approval results may also replicate past biases.
• Noisy and ambiguous: some tasks are difficult for people themselves to label consistently, and automatic logs may also record errors.
• Distributions change: training emails come from the past, while attack methods, users, and language will continue to change.

The “labeling bottleneck” recordsthe limitations of reliable target signals in cost, quality, and coverage, used to determine whether a supervised scheme can support real deployment.

When evaluating, input the labeling source, sampling scope, consistency results, and deployment scenarios; output the usable label scale, noise slices, coverage gaps, and priorities for continued collection.

The approach is not just to count the total number of labels, but to first sample and review labeling consistency, then compare coverage across different populations, times, and sources, and subsequently check whether the labels align with the true business objective. A high consistency rate indicates that the labeling rules are relatively stable, not that the target itself is unbiased; cheap automatic logs do not mean they are trustworthy, while expert review is more accurate but may be expensive and slow.

6It's Not the Only Paradigm: Four Ways of LearningIntuition

If no external target values are provided, what signal can a machine still learn from? Put supervised learning into a bigger map.

This comparison table describeswhere the training signal comes from and what is produced after learning, used to choose an appropriate paradigm when no manual labels are available.

The input is the data available for the task, the feedback method, and the interaction conditions; the output is the choice of a supervised, unsupervised, self-supervised, or reinforcement learning path, along with the corresponding model, structural representation, or policy.

When making a judgment, first check whether there is an external target value; then check whether the data can construct targets on its own; finally, check whether the problem requires delayed rewards obtained through actions. The classification result indicates the main source of supervision, not that an algorithm can only belong to one traditional category; self-supervised learning can be like supervised training in computational form, and RLHF also mixes supervised fine-tuning, reward models, and reinforcement learning, so the boundaries allow combinations.

7In the era of large models, does it still matter?EngineeringSynthesis

Since large models rely on self-supervised pre-training, why still learn supervised learning?

Because it is thecommon language, and it is still everywhere today:

• Turning a base model into an assistant: Instruction fine-tuning (SFT) is supervised learning—"instruction" is the input and "ideal answer" is the label (see the "Fine-tuning" deep-dive page).
• Many downstream tasks: For classification, scoring, and extraction in vertical domains, many are still cheapest and most stable when using labeled data for supervised learning.
• Evaluation philosophy: Evaluation sets with reference answers follow the idea of held-out testing; but evaluating a model on a dataset does not mean the model has undergone supervised training on that dataset.

The role of supervised learning in the era of large models can be understood asusing high-quality target signals to constrain the specific behavior of a general-purpose model. It addresses the problem that a model with broad pre-training capabilities may not necessarily answer according to task requirements.

Inputs can be instructions and ideal answers, domain samples and labels, and the output is a supervised fine-tuned model or a task-specific predictor.

In application, first define the behavior you want the model to exhibit, then collect representative input–target pairs, then minimize the loss between predictions and targets, and validate on an independent slice. An improvement in downstream scores indicates the model is more suitable for the current objective and distribution, but it cannot be inferred that its foundational knowledge has increased comprehensively; when demonstration data is narrow, labels are biased, or deployment tasks change, supervised adaptation can still fail.

8Connecting the entire causal chainSynthesis

From samples paired with inputs and targets to reliable predictions on unseen samples, how is the evidence chain built step by step?

1. Give the model samples of 'input + target value' to let it learn the rule from input to output—this is supervised learning. Target values may be noisy, so data quality is also part of the mechanism.(§1)
2. Discrete answers mean classification; continuous ones mean regression.(§2)
3. Abstractly, training = adjusting parameters to make 'predictions' close to 'labels', i.e., minimizing the loss.(§3)
4. The criterion for success is not memorizing the training set, but being accurate on unseen data (generalization), so we should split into training/validation/test.(§4)
5. It requires target values paired with inputs; sources can be manual labeling or observational records such as transaction outcomes and sensor measurements. The bottleneck lies in the cost, quality, target consistency, and scenario coverage of the supervision signal; it cannot be attributed solely to manual labeling.(§5)
6. When comparing other paradigms, examine the signal source and learning objective separately: unsupervised learning seeks structure from unlabeled data; self-supervised learning constructs targets from the data itself, reducing dependence on manual labels and supporting large-scale pre-training; reinforcement learning learns sequential decision-making from rewards brought by actions. These differences do not mean that all three emerged to circumvent the labeling bottleneck.(§6)
7. In the era of large models, it has not been replaced; instead, it is redirected to calibrate behavior (SFT), adapt to downstream tasks, and perform evaluation.(§7)

9Common MisconceptionsIntuition

This section only disambiguates: which statements sound reasonable but would directly lead to wrong data, training, or acceptance decisions?

10Check whether you really understandSelf-test

The questions move step by step from definitions to real acceptance; each scenario question uses only concepts already established on this page.

1. Where does 'supervision' come from? Why is the label not necessarily equal to objective truth?
2. What is the fundamental difference between classification and regression? Give one example of each.
3. Explain in one sentence: abstractly speaking, what is supervised learning training doing?
4. Why does 'predicting the entire training set correctly' not mean the model is successful? How should it be correctly evaluated?
5. The data bottleneck of supervised learning is not just 'expensive manual labeling.' What other problems does it include?
6. What are the connections and differences between self-supervised and supervised learning, respectively, in terms of computational form and the source of the supervisory signal?
7. In the era of large models, in which stages does supervised learning still play a role?
8. Among 10,000 emails, only 100 are spam. The model predicts all as normal. What is the accuracy? Why does this result not show that the model is effective?
9. You find that duplicate versions of the same email appear in both the training set and the test set. Why does the test score become inflated? How should you re-split the data?
10. Design three acceptance slices for the spam model, and for each slice explain the errors or metrics to observe.

11Concept Dependencies and Extended LearningPath

As page 1 of the official path, which content must be mastered now, and which content only needs to be encountered now and left for later nodes to expand?