Lesson 4 - Linear Systems

Welcome to Linear Algebra#

This lesson opens the linear-algebra thread of the course. You will learn what a system of linear equations is, why its solution is the point where the equations meet, and how to find that solution both by hand and with NumPy. Everything starts from a simple, concrete idea: two straight lines on a plane, and the single place where they cross.

By the end of this lesson, you will be able to:

• Recognize a system of linear equations and express it in standard form
• Explain why the solution to a system is the intersection of its equations
• Solve a two-equation system by hand using elimination
• Solve the same system in NumPy with np.linalg.solve
• Describe Gaussian elimination conceptually and the three possible outcomes: one solution, none, or infinitely many

You should be comfortable with basic Python and NumPy and remember how a line is written as $y = mx + b$. No prior linear algebra is needed. Let’s begin.

What Is a Linear System?#

A linear equation is any equation whose variables appear only to the first power, with no squares, no products of variables, and no curves. On a plane with two variables $x$ and $y$, every linear equation draws a straight line. You have already met them in slope-intercept form:

A system of linear equations, also called a linear system, is simply a collection of two or more linear equations that share the same variables. The word simultaneous is sometimes used, and it captures the key idea: you are not solving the equations one at a time, you are looking for values of the variables that satisfy all of them at once.

Here is the system you will work with for the rest of the lesson:

Each line is an equation. Each equation, on its own, has infinitely many solutions, every point along its line. The interesting question is what happens when you ask both equations to hold at the same time.

The Solution Is an Intersection#

A solution to a linear system is a set of variable values that makes every equation true simultaneously. Geometrically, each equation is a line, and a point lies on a line only if its coordinates satisfy that line’s equation. So a point that satisfies both equations must lie on both lines, which means it sits exactly where the two lines cross.

That crossing point is called the point of intersection, and finding it is what “solving the system” means. The figure below plots both equations from our example. The orange line is $2x + y = 5$ and the blue line is $x - y = 1$. They meet at a single point.

Reading the chart, the lines cross at $(2, 1)$. You can confirm this is a genuine solution by substituting $x = 2$ and $y = 1$ back into both equations:

• First equation: $2(2) + 1 = 5$. True.
• Second equation: $2 - 1 = 1$. True.

Both left-hand sides equal their right-hand sides, so $(2, 1)$ satisfies the whole system. That is what makes it the solution.

Writing a System in Standard Form#

To solve systems systematically, it helps to write every equation the same way. Linear algebra prefers the standard form (also called the general form), where the variable terms sit on the left and the constant sits on the right:

Our example is already there:

Written this way, every equation is described by just three numbers: the coefficient of $x$, the coefficient of $y$, and the constant. If an equation arrives in slope-intercept form, you can always rearrange it. For example, $y = 30x + 1000$ becomes $30x - y = -1000$ by moving the $y$ to the left and the constant to the right.

Once every equation is reduced to its coefficients, the whole system can be captured as a grid of numbers, a structure called a matrix, which you will meet properly in a later lesson. For now, the takeaway is simpler: a linear system is fully described by its coefficients and constants, and standard form is how you line them up.

Solving by Elimination#

You can solve a small system by hand. The most reliable manual method is elimination: combine the equations so that one variable cancels, leaving a single equation in one variable that you can solve directly.

Look again at the system:

Notice that the first equation has $+y$ and the second has $-y$. If you add the two equations together, the $y$ terms cancel:

With $x = 2$ in hand, substitute it back into either original equation to find $y$. Using the second equation:

So the solution is $x = 2$, $y = 1$, exactly the intersection you read off the chart. Elimination worked here because the $y$ coefficients were already equal and opposite. In general you first scale one or both equations so that a variable’s coefficients match, then add or subtract to cancel it.

Solving with NumPy#

By hand, elimination is fine for two equations. With three, four, or twenty variables it becomes slow and error-prone. NumPy solves linear systems for you with a single function: np.linalg.solve.

The function expects the system split into two pieces. The first is the coefficient matrix, one row per equation, holding the coefficients of the variables. The second is the right-hand side, the column of constants. For our system,

the coefficient matrix is the grid of coefficients $\begin{bmatrix} 2 & 1 \\ 1 & -1 \end{bmatrix}$ and the constants are $\begin{bmatrix} 5 \\ 1 \end{bmatrix}$.

import numpy as np

# no dataset needed - numpy only

# Coefficients of x and y, one row per equation
A = np.array([
    [2.0,  1.0],   # 2x + 1y
    [1.0, -1.0],   # 1x - 1y
])

# The constants on the right-hand side
b = np.array([5.0, 1.0])

solution = np.linalg.solve(A, b)
print("solution [x, y]:", solution)
# Output: solution [x, y]: [2. 1.]

NumPy returns [2. 1.], meaning $x = 2$ and $y = 1$, the same answer you found by hand and read off the chart. The order of the values matches the order of your columns: the first number is $x$ (the first column of A), the second is $y$.

You can confirm the solution programmatically by plugging it back in. Multiplying the coefficient matrix by the solution should reproduce the right-hand side.

check = A @ solution
print("A @ solution:", check)
# Output: A @ solution: [5. 1.]

The result [5. 1.] matches b exactly, so the solution is correct. The @ operator performs matrix multiplication, which here computes the left-hand side of each equation using the solved values, an operation you will study in depth in a later lesson.

Gaussian Elimination, Conceptually#

The method NumPy uses under the hood is a tidy, mechanical version of the elimination you did by hand, called Gaussian elimination. The idea is to apply simple operations to the equations until the system is so simplified that the answer is obvious.

Only three row operations are allowed, because each one keeps the solution unchanged:

• Swap two equations (the order of equations never matters).
• Multiply an equation by a nonzero number (scaling both sides keeps it true).
• Add a multiple of one equation to another (combining true equations gives another true equation).

These are exactly the moves you already use when solving by hand. Gaussian elimination just applies them in a disciplined order. The goal is to reshape the system so the first equation gives $x$ directly, the second gives $y$ directly, and so on.

Let’s walk our system through it. Stacking the coefficients and constants together gives:

The bar separates the coefficients from the constants. Step 1: divide the first row by 2 so its leading coefficient becomes 1.

Step 2: subtract the new first row from the second row to knock out the leading 1 below it.

Step 3: divide the second row by $-1.5$ so its leading coefficient becomes 1. That row now reads $y = 1$.

Step 4: subtract $0.5$ times the second row from the first row to clear the $0.5$. The first row now reads $x = 2$.

The final grid translates straight back to $x = 2$ and $y = 1$. The solution is now read off directly from the right-hand column. You can reproduce these exact steps in NumPy.

import numpy as np

# Stack coefficients and constants into one augmented grid
M = np.array([
    [2.0,  1.0, 5.0],
    [1.0, -1.0, 1.0],
], dtype=np.float32)

M[0] = M[0] / 2          # Step 1: make the leading coefficient of row 0 a 1
M[1] = M[1] - M[0]       # Step 2: zero out the value below it
M[1] = M[1] / -1.5       # Step 3: make the leading coefficient of row 1 a 1
M[0] = M[0] - 0.5 * M[1] # Step 4: clear the value above row 1's pivot

print(M)
# Output:
# [[ 1.  0.  2.]
#  [-0.  1.  1.]]

The last column, [2, 1], is the solution: $x = 2$, $y = 1$. (The -0. is just a floating-point representation of zero.) This is the same disciplined elimination NumPy runs for you when you call np.linalg.solve, only here you can watch every step.

Three Possible Outcomes#

Not every system behaves as neatly as our example. When you have two equations and two unknowns, exactly one of three things can happen. Understanding all three keeps you from being surprised when NumPy returns an error or an unexpected result.

One Solution#

This is the case you have seen. The two lines have different slopes, so they cross at exactly one point. There is a single $(x, y)$ that satisfies both equations, and np.linalg.solve returns it. Our system $2x + y = 5$ and $x - y = 1$ falls here.

No Solution#

If the two lines are parallel but distinct, they never cross, so no point satisfies both equations at once. The system is called inconsistent. Consider:

The second equation is just the first one’s left side doubled, but the constant does not match (doubling 5 gives 10, not 3). The lines have the same slope and different intercepts, so they run alongside each other forever. NumPy refuses to solve it.

import numpy as np

A = np.array([
    [2.0, 1.0],
    [4.0, 2.0],   # same direction as row 0, scaled by 2
])
b = np.array([5.0, 3.0])

try:
    np.linalg.solve(A, b)
except np.linalg.LinAlgError as e:
    print("No unique solution:", e)
# Output: No unique solution: Singular matrix

Infinitely Many Solutions#

If the two equations describe the exact same line, then every point on that line satisfies both, and there are infinitely many solutions. For example, $2x + y = 5$ and $4x + 2y = 10$ are the same equation written twice (the second is the first scaled by 2, constant included). The “two” lines lie perfectly on top of each other.

A useful single number distinguishes the one-solution case from the other two. The determinant of the coefficient matrix is nonzero exactly when the lines have different directions, which is exactly when a unique solution exists.

import numpy as np

A_unique   = np.array([[2.0, 1.0], [1.0, -1.0]])  # different slopes
A_parallel = np.array([[2.0, 1.0], [4.0, 2.0]])   # same slope

print("unique-system determinant:  ", round(np.linalg.det(A_unique), 1))
print("parallel-system determinant:", round(np.linalg.det(A_parallel), 1))
# Output:
# unique-system determinant:   -3.0
# parallel-system determinant: 0.0

A nonzero determinant (here $-3.0$) signals one clean solution. A determinant of $0.0$ warns that the lines are either parallel or identical, so there is no unique answer. You will explore the determinant and what it measures in a later lesson; for now, treat it as a quick check on whether a system is solvable.

Practice Exercises#

Now it is your turn. Try these before checking the hints. Each uses only NumPy.

Exercise 1: Solve a New System#

Use np.linalg.solve to find $x$ and $y$ for the system below, then verify your answer by substituting it back.

import numpy as np

# Your code here

Exercise 2: Detect an Unsolvable System#

Write code that tries to solve the parallel system $x + 2y = 4$ and $2x + 4y = 9$, and prints a friendly message if NumPy reports it has no unique solution.

import numpy as np

# Your code here

Exercise 3: Use the Determinant as a Solvability Check#

Before solving, write a small function that takes a coefficient matrix and prints whether the system has a unique solution, based on its determinant. Test it on $\begin{bmatrix} 2 & 1 \\ 1 & -1 \end{bmatrix}$ and on $\begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix}$.

import numpy as np

# Your code here

Summary#

Congratulations! You have solved your first linear systems, by hand, with elimination, and with NumPy, and you understand what the solution means geometrically. Let’s review what you learned.

Key Concepts#

Linear Systems

• A linear equation has variables only to the first power and draws a straight line
• A system of linear equations is two or more equations sharing the same variables
• A solution is values that satisfy every equation at once; geometrically it is the point of intersection of the lines

Standard Form

• Standard form writes each equation as $Ax + By = c$, variables on the left, constant on the right
• In this form a system is fully described by its coefficients and constants

Solving Systems

• Elimination combines equations to cancel a variable, leaving one equation in one unknown
• np.linalg.solve(A, b) solves a system given a coefficient matrix A and constants b
• Always verify a solution by substituting it back, or with A @ solution

Gaussian Elimination

• A disciplined version of elimination using three solution-preserving row operations: swap, scale, and add a multiple of one row to another
• The goal is to reshape the system until each variable’s value can be read off directly

Three Outcomes

• One solution: lines have different slopes and cross at a point (determinant nonzero)
• No solution: lines are parallel and distinct (the system is inconsistent)
• Infinitely many solutions: the equations describe the same line
• A nonzero determinant of the coefficient matrix signals a single unique solution

Why This Matters#

Linear systems are the entry point to all of linear algebra, and linear algebra is the language machine learning is written in. When you fit a line to data, solve for regression weights, or take an optimization step, you are, under the hood, finding values that satisfy a system of linear equations. The intersection picture you built here, two lines meeting at a point, is the seed of much richer ideas: vectors as directions in space, matrices as transformations, and high-dimensional solution sets you cannot draw but can still reason about. Master the small case now, and the big cases become a natural extension rather than a leap.

Next Steps#

You can now describe, set up, and solve a linear system, and you know when a unique solution exists. Next you will meet the building block that linear algebra is made of: the vector, a quantity with both direction and magnitude that turns these grids of numbers into geometry.

Keep Building Your Skills#

You have taken your first real step into linear algebra by turning a pair of equations into a single point on a plane. That move, from algebra to geometry and back, is the heart of everything ahead. As you learn vectors, matrices, and transformations in the coming lessons, keep returning to this lesson’s simple picture: equations are lines, and solutions are where they meet. Hold onto that intuition, and the more abstract machinery will always have something concrete to stand on.