Lesson 2 - Understanding Limits

Welcome to Limits#

This lesson introduces the single most important idea standing between you and calculus: the limit. A limit captures the value a function approaches as its input gets closer and closer to some point, and it works even when the function has no value at that point at all. That last part sounds strange, but it is exactly what makes limits powerful, and it is the reason derivatives, the engine behind almost all model training, are even possible.

By the end of this lesson, you will be able to:

• Explain in plain language what a limit is and why it differs from simply plugging in a value
• Work through the classic example $\lim_{x \to 1} \frac{x^2 - 1}{x - 1} = 2$, where the function is undefined at the point itself
• Approach a limit numerically with NumPy and watch the values converge
• Describe one-sided limits and what it means for a function to be continuous
• Explain why limits are the foundation that derivatives are built on

You should be comfortable with basic Python and NumPy and remember a little high-school algebra (factoring a difference of squares is enough). No prior calculus is needed. Let’s begin.

What Is a Limit?#

Most of the functions you have used so far behave politely. If you want to know the value of $f(x) = x + 1$ at $x = 1$, you plug in 1 and get 2. Done. There is nothing subtle happening: the function has a clear value everywhere.

But not every function is that cooperative. Consider this one:

Try to evaluate it at $x = 1$. The numerator becomes $1 - 1 = 0$, and the denominator becomes $1 - 1 = 0$. You get $\frac{0}{0}$, which is undefined. Division by zero has no answer, so the function simply does not exist at $x = 1$. There is a hole in the curve at that exact spot.

And yet, here is the interesting question. What happens near $x = 1$? If you feed the function values like 0.9, 0.99, 0.999, the results clearly head somewhere specific. They do not blow up or wander; they settle down toward a single number. That number, the value the function is heading toward, is the limit.

A limit is the value a function approaches as its input approaches a particular point. The notation looks like this:

Read aloud, this says: “As $x$ approaches 1, the function $\frac{x^2-1}{x-1}$ approaches 2.” Notice what it does not say. It does not say the function equals 2 at $x = 1$, because the function has no value there. It describes the destination the function is traveling toward, not the function’s value at the point itself.

Seeing the Hole#

Let’s make this concrete with a picture before touching any code. The function $f(x) = \frac{x^2-1}{x-1}$ is undefined only at $x = 1$. Everywhere else it behaves like a simple straight line, as you will prove with algebra in a moment. The chart below plots the function and marks the single missing point.

Look closely at $x = 1$. The line clearly wants to pass through the point $(1, 2)$, but there is an open circle, a hole, marking that the function is not actually defined there. From either side, the curve marches straight toward height 2. The limit fills in the value your eye already expects, even though the function leaves the spot blank.

This is the visual intuition for the entire lesson. A limit answers the question your eye asks automatically: “Where is this curve clearly going?”

Solving the Limit with Algebra#

Before we verify anything numerically, it is worth seeing why the limit equals 2. The trick is to rewrite the function into a form that is no longer $\frac{0}{0}$ at the point of interest.

Start with the numerator, $x^2 - 1$. This is a difference of squares, which factors cleanly:

Substitute that back into the function:

Now you can see the source of the trouble and its cure at the same time. For every value of $x$ except $x = 1$, the factor $(x - 1)$ is not zero, so you are allowed to cancel it from the top and bottom:

The complicated-looking function is, away from the hole, just the line $x + 1$. That is exactly why the graph looked like a straight line. Now the limit is easy, because $x + 1$ is perfectly defined at $x = 1$:

This is the standard strategy for limits that come out to $\frac{0}{0}$: manipulate the expression algebraically until the problematic factor cancels, then substitute the value directly.

Approaching the Limit Numerically#

Algebra gives you the exact answer, but you can also watch the limit happen by feeding the function values that creep toward 1 from both directions. This is where NumPy is handy. There is no dataset to download for this lesson; you only need NumPy.

import numpy as np
# no dataset needed - numpy only

def f(x):
    return (x**2 - 1) / (x - 1)

# Values approaching 1 from the LEFT (smaller than 1)
left = np.array([0.9, 0.99, 0.999, 0.9999])
print("from the left:")
for x in left:
    print(f"  x = {x:<7}  f(x) = {f(x):.4f}")
# Output:
# from the left:
#   x = 0.9      f(x) = 1.9000
#   x = 0.99     f(x) = 1.9900
#   x = 0.999    f(x) = 1.9990
#   x = 0.9999   f(x) = 1.9999

As $x$ climbs toward 1 from below, $f(x)$ climbs toward 2. Now approach from the other side, with values slightly larger than 1.

# Values approaching 1 from the RIGHT (larger than 1)
right = np.array([1.1, 1.01, 1.001, 1.0001])
print("from the right:")
for x in right:
    print(f"  x = {x:<7}  f(x) = {f(x):.4f}")
# Output:
# from the right:
#   x = 1.1      f(x) = 2.1000
#   x = 1.01     f(x) = 2.0100
#   x = 1.001    f(x) = 2.0010
#   x = 1.0001   f(x) = 2.0001

From above, $f(x)$ descends toward 2. Both sides agree on the same destination, even though neither side ever lands on it. Squeezed between 1.9999 and 2.0001, the limit can only be 2, which matches the algebra exactly.

Each of these results is just $x + 1$ in disguise: $f(0.9) = 1.9$, $f(1.1) = 2.1$, and so on, because the function equals $x + 1$ at every input except 1 itself.

Now try the one value that breaks everything, the point of the hole.

print(f(1.0))
# Output: a division-by-zero warning, and the result nan (not a number)

NumPy cannot evaluate the function at exactly $x = 1$. It produces nan (not a number) and warns about dividing by zero, which is precisely the behavior the limit is designed to work around. The function has no value here, but its limit is still a clean, well-defined 2.

One-Sided Limits#

You just approached $x = 1$ from two directions, and that distinction has a name. A one-sided limit describes the value a function approaches as the input comes in from only one side.

• The left-hand limit uses values smaller than the target and is written with a minus sign: $\lim_{x \to 1^-} \frac{x^2 - 1}{x - 1} = 2$
• The right-hand limit uses values larger than the target and is written with a plus sign: $\lim_{x \to 1^+} \frac{x^2 - 1}{x - 1} = 2$

Here is the rule that ties them together: the ordinary (two-sided) limit exists only when both one-sided limits exist and agree. In our example both sides head to 2, so the full limit is 2. If the two sides disagreed, perhaps the function jumped from one height to another at the point, then the two-sided limit would simply not exist.

This is not a technicality you can ignore. Functions with sudden jumps, steps, and breaks appear all the time, and for those, asking “which side?” is the only way to get a sensible answer.

Limits and Continuity#

One-sided limits lead naturally to one of the most useful ideas in calculus: continuity. Informally, a function is continuous at a point if you can draw it through that point without lifting your pen. No holes, no jumps, no gaps.

More precisely, a function $f$ is continuous at a point $a$ when three conditions all hold:

1. $f(a)$ is defined (the function actually has a value there).
2. The limit $\lim_{x \to a} f(x)$ exists (both sides agree on a destination).
3. The two match: $\lim_{x \to a} f(x) = f(a)$.

Run our example through this checklist at $a = 1$. The limit exists and equals 2, so condition 2 passes. But condition 1 fails: $f(1)$ is undefined. Because one condition breaks, the function is not continuous at $x = 1$. That broken point is exactly the hole you saw in the graph.

Interestingly, this is a gentle kind of discontinuity. The limit exists and is finite; the only problem is the missing point. Mathematicians call this a removable discontinuity, because you could “repair” the function simply by defining $f(1) = 2$ and patching the hole. The simpler function $x + 1$, which agrees with our function everywhere except the hole, is continuous everywhere, including at $x = 1$.

Why Limits Underpin Derivatives#

Everything so far has been building toward the reason limits matter for machine learning. The headline is this: a derivative, the slope of a function at a single point, is defined as a limit. You cannot have one without the other.

Here is the problem a derivative solves. The slope between two points on a curve is straightforward: it is the change in height divided by the change in input.

That formula measures the slope of the straight line connecting two points that are a distance $h$ apart. But you do not want the slope across a gap. You want the slope at a single point, the instantaneous steepness right where you are standing. The natural move is to shrink the gap $h$ toward zero, bringing the second point closer and closer to the first.

The trouble is that setting $h = 0$ outright gives you $\frac{f(x) - f(x)}{0} = \frac{0}{0}$, the very same undefined expression you met at the start of this lesson. You cannot plug in zero directly. But you can take a limit as $h$ approaches zero:

This is the definition of the derivative, and it is a limit through and through. It uses the exact same idea you practiced here: approach a forbidden point ($h = 0$) and ask where the expression is heading, rather than what it equals at the point itself. The $\frac{0}{0}$ form is not a dead end; it is an invitation to take a limit.

Because nearly every machine learning model is trained by following the slope of a loss function downhill (a process called gradient descent), and because those slopes are derivatives, and because derivatives are limits, this small idea sits at the foundation of the entire field. You will build the derivative formally in the next lesson, standing on the limits you understand now.

Practice Exercises#

Now it is your turn. Try these before checking the hints.

Exercise 1: Approach a Different Limit#

Consider the function $g(x) = \frac{x^2 - 4}{x - 2}$, which is undefined at $x = 2$. Using NumPy, evaluate it at the values 1.9, 1.99, 2.01, and 2.1, and decide what $\lim_{x \to 2} g(x)$ appears to be.

import numpy as np

def g(x):
    return (x**2 - 4) / (x - 2)

# Your code here

Exercise 2: Cancel the Factor by Hand#

For the same function $g(x) = \frac{x^2 - 4}{x - 2}$, factor the numerator, cancel the common factor, and use direct substitution to find $\lim_{x \to 2} g(x)$ exactly. Confirm it matches your numerical answer from Exercise 1.

# This one is pencil-and-paper. Write out the factoring,
# then verify with one line of NumPy at a value very close to 2.

Exercise 3: Check Continuity#

Decide whether $g(x) = \frac{x^2 - 4}{x - 2}$ is continuous at $x = 2$, and explain which of the three continuity conditions it fails. Then state how you could patch the function to make it continuous there.

import numpy as np

def g(x):
    return (x**2 - 4) / (x - 2)

# Try evaluating exactly at x = 2 and observe what happens.
print(g(2.0))

Summary#

Congratulations! You have met the idea that makes calculus, and therefore most of machine learning, possible. Let’s review what you learned.

Key Concepts#

What a Limit Is

• A limit is the value a function approaches as its input approaches a point
• A limit describes where a function is heading, not necessarily its value at the point
• A function can have a perfectly good limit at a point where it is itself undefined

The Worked Example

• $f(x) = \frac{x^2 - 1}{x - 1}$ is undefined at $x = 1$ because it gives $\frac{0}{0}$
• Factoring $x^2 - 1 = (x-1)(x+1)$ and cancelling shows $f(x) = x + 1$ away from the hole
• Therefore $\lim_{x \to 1} \frac{x^2 - 1}{x - 1} = 2$, confirmed numerically from both sides

One-Sided Limits and Continuity

• A one-sided limit approaches from only the left ($x \to a^-$) or only the right ($x \to a^+$)
• The two-sided limit exists only when both one-sided limits exist and agree
• A function is continuous at a point when it is defined there, its limit exists, and the two match
• A missing point with an existing limit is a removable discontinuity, a hole you could patch

Why It Matters

• The slope between two points is $\frac{f(x+h) - f(x)}{h}$; shrinking $h$ to zero gives $\frac{0}{0}$
• The derivative is defined as the limit $\lim_{h \to 0} \frac{f(x+h) - f(x)}{h}$
• Limits turn the forbidden $\frac{0}{0}$ into a well-defined answer

Why This Matters#

Limits look like an abstract piece of math, but they are the quiet machinery underneath everything you will do with models. Every time a model trains, it nudges its parameters in the direction that reduces a loss, and that direction comes from a slope, a derivative. Derivatives are limits. Without the ability to ask “where is this heading as the gap shrinks to nothing?” there would be no gradient descent, no backpropagation, and no learning.

You also saw a habit worth keeping: pair intuition, algebra, and code. The graph showed you where the function was going, the algebra proved it, and NumPy let you watch it converge. Holding all three views at once is exactly how you will reason about the more elaborate functions ahead, where you cannot always factor by hand and have to trust the rules instead.

Next Steps#

You now understand what a limit is, how to compute one even where a function is undefined, and why limits are the foundation of derivatives. In the next lesson, you will use this idea to define the derivative properly and use it to find the high and low points of a function, the exact skill optimization depends on.

Keep Building Your Skills#

You have taken a real step toward the math that powers machine learning. The limit may feel like a small idea, but it is the hinge on which derivatives, gradients, and optimization all turn. Keep the three-way habit you practiced here, picture it, prove it, and check it in code, and the calculus in the coming lessons will feel far less mysterious. Master where functions are heading, and you are ready to ask how fast they get there.