Training Loss

Training loss is a score that tells the model how wrong its prediction was. During training, the goal is to reduce this score.

Training Loss Playground

Guess the missing word, watch the loss

The model tries to fill in the blank. Each time you press Start Training, it learns a little and makes a better guess, so the loss meter drops.

The cat sat on the ___

Correct answer: “mat”

Training loss -

Low (good)High (bad)

Attempt 0 of 6

The closer the prediction is to the correct answer, the lower the loss.

Loss Goes Down Over Time

The bars get shorter as it learns

As the model sees more examples, its guesses improve and the loss shrinks. Taller bar = more wrong.

High

Med

Lower

Low

Step 1High loss

Step 2Medium loss

Step 3Lower loss

Step 4Low loss

As the model learns from more examples, its predictions can improve and the loss can become smaller.

What Loss Tells Us

Reading the score

Loss is just a number, but it maps neatly onto how close the guess was.

High Loss

The prediction was far from the expected answer.

Medium Loss

The prediction was somewhat close.

Low Loss

The prediction was close to the expected answer.

Very Low Loss

The model may be learning well, but we still need to test it on new examples.

🔑

Key takeaways

• Training loss tells how wrong the model was.
• Lower loss usually means better predictions.
• Loss helps guide model learning.
• Loss changes during training.
• Low training loss does not always mean the model is perfect.

Summary

Training loss is like feedback for the model. It tells the model how far its prediction was from the expected answer, helping it improve during training.

Previous: Pretraining Next: Epochs