22  Loss Functions Have Tails Too

def squared_loss(r): return r ** 2
def absolute_loss(r): return abs(r)
def huber_loss(r, delta=1.0):
    return 0.5*r**2 if abs(r) <= delta else delta*(abs(r) - 0.5*delta)

Plotting all three loss curves side by side, then their gradients, makes the design choice visible: squared loss’s gradient grows without bound as the residual grows, which is precisely why one bad observation can dominate a squared-error fit.