Numbers & Functions · 5 Exponentials and Logarithms

5.4 The shape of growth

Read the shape of 2x from left to right. Far to the left, the curve hugs down close to zero but never touches it — 2−10 is tiny but still positive. The machine x 2x x3 passes through (0, 1) hugs zero never touches range (0, ∞) explodes the exponential eventually outruns any polynomial Figure 5.2. The exponential’s personality. On the left it hugs zero but never touches (range (0, ∞)); it passes through (0, 1); on the right it explodes — and eventually outruns any polynomial, even x3.

can never output zero or a negative (its range is (0, ∞)). At x = 0 it passes through height 1. To the right it explodes — not “goes up steadily” like a line, it accelerates. This is exponential growth, and it eventually outruns everything. A grain of rice doubled on each square of a chessboard buries the planet. A base between 0 and 1 — like (1 2)x — does the mirror image: it decays.

That’s exponential decay: radioactive atoms, a cooling drink, medicine clearing from your blood. Same machine, base flipped.

5.5 What exponentials are good for (and where they lie)

The friendly face: compound interest. Put $100 in an account paying 10% a year. After one year, $110. But in the second year you earn 10% on the whole $110, gaining $11, not $10. Your interest earns interest. Each year multiplies your money by 1.1, so after x years you have 100 · (1.1)x — an exponential. This is growth feeding on itself, and it’s quietly where the number e comes from historically.

Now the turn: nothing compounds forever. Compound interest says $100 becomes, after 500 years, more money than exists on Earth. Obviously that doesn’t happen. Something always intervenes. The same lie hides in scarier Exponentials and Logarithms x 2x growth (1 2)x decay base > 1 grows; base between 0 and 1 decays same machine, base flipped Figure 5.3. Growth versus decay, mirror images across the vertical axis. Base greater than 1 grows; base between 0 and 1 decays.

exponentials. An epidemic’s first weeks really do grow exponentially — each sick person infects two more. If that continued, within months every human would be infected several times over, which is impossible. So the growth runs out of room: as fewer susceptible people remain, the doubling slows, and finally stops. The curve bends over and levels off.

That lazy-S shape is the logistic curve, and it’s what exponential growth actually looks like in any finite world.

Exponential growth is always a model of the early phase. The real question is never “how fast does it grow?” but “what is the ceiling, and when does it start to bite?” Anyone who promises you exponential growth forever — in money, in followers, in anything — is selling you the first half of an S-curve and hiding the second. You’ll meet this S-curve again in machine learning — and you already have every piece. The exponential decay e−x heads toward zero as x grows. Build this little machine from it: σ(x) = 1 1 + e−x .

When x is large and positive, e−x is nearly zero, so σ(x) is nearly 1. When x is large and negative, e−x is enormous, so σ(x) is nearly 0. At x = 0 we get 1 2. So this machine squashes any input on the whole number line into the range between 0 and 1.

time size ceiling (asymptote) "the model" (pure exponential) exponential start steepest (inflection) plateau "reality" is an S-curve: the ceiling always bites Figure 5.4. The honest correction. A pure exponential (dashed) shoots to infinity; in any finite world reality tracks the logistic S-curve (solid) — explosive start, steep inflection, then a plateau as the ceiling bites. x σ(x) 1 0 (0, 1 2) →0 →1 σ(x) = 1 1 + e−x squashes the whole line into (0, 1) Figure 5.5. The sigmoid σ(x) = 1 1+e−x — the S-curve written as a formula.

It squashes the whole number line into (0, 1), passing through (0, 1 2). This is how an artificial neuron “decides.”

Exponentials and Logarithms This is the sigmoid function, and it is exactly the S-curve. It’s how an artificial neuron decides: it takes a runaway input of any size and squashes it into a bounded answer between 0 (no) and 1 (yes), with 1 2 as “unsure.” The curve that tames an epidemic and the curve that lets a neural network make up its mind are the same curve — and you just built it from e−x, a reciprocal, and a +1. ▶Run it yourself — drag the knobs on growth, decay, and the decision curve. (notebook 5.1 in Colab) Slide the base of bx from above 1 to below 1 and watch growth flip into decay. Then watch a pure exponential race off to infinity while the S-curve bends to a ceiling. Before each slide, predict which way the curve will move — then check yourself.

All 8 chapters of Numbers & Functions · Foundations · All courses