What Is Linear Algebra?
Linear algebra is the branch of mathematics that studies vectors, matrices, and the linear relationships between them. A relationship is "linear" when the output changes in direct proportion to the input, with no squares, no curves, and no surprises. The subject gives you three things: objects that hold many numbers at once (vectors and matrices), an operation that transforms them (matrix multiplication), and a language for the space they live in (vector spaces).
Most people meet the pieces one at a time, a matrix in one lesson, a pair of simultaneous equations in another, and never see that they belong to a single subject. Linear algebra is what connects them. In school its first fragments appear as matrices and simultaneous equations (NCERT Class 12 in India, and the high-school standards of the US Common Core), and it becomes a subject in its own right at university, where it turns out to underlie almost everything computational.
What Are The Building Blocks Of Linear Algebra?
Linear algebra is built from three kinds of object, each holding a different amount of information: scalars, vectors, and matrices.
Table: The three core objects of linear algebra.
Object | What it is | Example |
|---|---|---|
Scalar | A single number that scales things | $3$, $-2.5$, $\pi$ |
Vector | An ordered list of numbers; a point or arrow in space | $\mathbf{v} = \begin{bmatrix} 2 \ -1 \end{bmatrix}$ |
Matrix | A rectangular grid of numbers; a table or a transformation | $A = \begin{bmatrix} 1 & 2 \ 3 & 4 \end{bmatrix}$ |
A scalar is just a number. Its job is to stretch or shrink other objects.
A vector is an ordered list of numbers. Geometrically it is an arrow from the origin with a length and a direction, and a velocity, a force, or a single row of a dataset are all vectors. The geometric picture is worked through in vectors, and the dot product is the operation that combines two vectors into one number measuring how much they point the same way.
A matrix is a grid of numbers. It can store a table of data, or it can act as a machine that takes one vector in and sends a different vector out. That second role, the matrix as a transformation, is where linear algebra gets its power.
How Does Linear Algebra Solve Systems Of Equations?
The central problem of linear algebra is solving a system of linear equations, and its cleanest form is a single matrix equation, $A\mathbf{x} = \mathbf{b}$.
Take two equations, where $x$ and $y$ are the unknowns:
$$x + y = 5$$ $$x - y = 1$$
Collect the coefficients into a matrix $A$, the unknowns into a vector $\mathbf{x}$, and the right-hand sides into a vector $\mathbf{b}$:
$$A = \begin{bmatrix} 1 & 1 \ 1 & -1 \end{bmatrix}, \qquad \mathbf{x} = \begin{bmatrix} x \ y \end{bmatrix}, \qquad \mathbf{b} = \begin{bmatrix} 5 \ 1 \end{bmatrix}$$
The entire system collapses to one line:
$$A\mathbf{x} = \mathbf{b}$$
Solve it by elimination, one step per line:
$$\text{add the two equations:} \quad 2x = 6 ;\Rightarrow; x = 3$$ $$\text{substitute back:} \quad 3 + y = 5 ;\Rightarrow; y = 2$$
So the solution is $\mathbf{x} = \begin{bmatrix} 3 \ 2 \end{bmatrix}$.
That one equation $A\mathbf{x} = \mathbf{b}$ scales without changing shape. Whether you have 2 unknowns or 2 million, the form is identical, and the same tools still apply (elimination, and the inverse of a matrix, among others). For the school-level groundwork, see linear equations and systems of equations. The compact form itself is the matrix equation, often solved on paper through an augmented matrix.
Not every system has one neat answer, which is a point beginners miss. A system of linear equations lands in exactly one of three cases.
Table: The three possible outcomes of a linear system.
Outcome | What the lines do | Example |
|---|---|---|
One solution | The lines cross at a single point | $x+y=5,\ \ x-y=1$ |
No solution | The lines are parallel and never meet | $x+y=1,\ \ x+y=3$ |
Infinitely many | Both equations are the same line | $x+y=1,\ \ 2x+2y=2$ |
What Is A Vector Space?
A vector space is any collection of objects you can add together and scale by numbers while staying inside the collection.
The flat plane you draw graphs on is a vector space, written $\mathbb{R}^2$. Ordinary 3D space is another, written $\mathbb{R}^3$.
Add two vectors in the plane and you land on another point in the plane. Scale one by $2$ and you are still in the plane. Those two rules, closed under addition and closed under scaling, are the whole definition.
The idea is powerful because a vector space does not have to be made of arrows. The set of all polynomials forms one. So does the set of all sound waves. Once a collection obeys the two rules, every tool of linear algebra applies to it, which is why one subject can describe geometry, signals, and data with the same vocabulary.
Inside a space, a basis is a smallest set of vectors that can build every other vector by scaling and adding. The count of vectors in a basis is the dimension of the space. The rank of a matrix measures the dimension of the space that its columns actually reach.
What Is A Linear Transformation?
A linear transformation is a function that moves every vector in a space while keeping grid lines straight, evenly spaced, and anchored at the origin. Every such transformation is exactly a matrix, and applying it is matrix multiplication.
Take the matrix that rotates the plane a quarter turn:
$$R = \begin{bmatrix} 0 & -1 \ 1 & 0 \end{bmatrix}$$
Apply it to the vector pointing right, $\mathbf{v} = \begin{bmatrix} 1 \ 0 \end{bmatrix}$:
$$R\mathbf{v} = \begin{bmatrix} 0 & -1 \ 1 & 0 \end{bmatrix}\begin{bmatrix} 1 \ 0 \end{bmatrix} = \begin{bmatrix} 0 \ 1 \end{bmatrix}$$
The arrow that pointed right now points up. A rotation, a reflection, a stretch, or a shear is each a transformation, and multiplying two matrices means doing one transformation and then the other. This is the reason matrix multiplication is defined the way it is, and the reason order matters, since rotating and then stretching lands somewhere different from stretching and then rotating.
What Does A Determinant Tell You?
The determinant of a square matrix is a single number that says how much the matrix's transformation scales area (in 2D) or volume (in 3D).
For a 2 × 2 matrix, it is one subtraction:
$$\det\begin{bmatrix} a & b \ c & d \end{bmatrix} = ad - bc$$
A determinant of $2$ means the transformation doubles every area. A determinant of $1$ preserves area, which a pure rotation does. The value that matters most is $0$.
When the determinant is zero, the transformation squashes space flat, collapsing a 2D region onto a line, and nothing can undo it. That is exactly when the matrix has no inverse, and exactly when $A\mathbf{x} = \mathbf{b}$ stops having a single tidy solution. The determinant, the inverse, and the solving of systems are one connected story, told in full under matrices and determinants and determinants.
What Are Eigenvalues And Eigenvectors?
For a given transformation, an eigenvector is a rare direction that only gets stretched, never knocked off its own line, and its eigenvalue is how much it stretches.
Most vectors change direction when a matrix acts on them. A few hold their line, and for those the matrix behaves like plain scalar multiplication:
$$A\mathbf{v} = \lambda \mathbf{v}$$
Here $\mathbf{v}$ is the eigenvector and $\lambda$ (lambda) is its eigenvalue. Take a matrix that stretches the plane by $2$ across and $3$ up:
$$A = \begin{bmatrix} 2 & 0 \ 0 & 3 \end{bmatrix}$$
The horizontal direction is an eigenvector with eigenvalue $2$, and the vertical direction is an eigenvector with eigenvalue $3$. Finding these directions tells you what a transformation really does underneath all its rotating and mixing, which is why eigenvalues and eigenvectors turn up in the stability of a bridge, the compression of an image, and the ranking behind a search engine.
Why Is Linear Algebra So Foundational?
Linear algebra sits under so many fields for three reasons, and none of them is habit.
It handles many things at once. One vector can hold an entire row of data, and one equation $A\mathbf{x} = \mathbf{b}$ can hold a system with millions of unknowns. Packing quantities into vectors and matrices turns a crowd of numbers into a single object you can reason about.
Linear is the part we can actually solve. Most real systems are nonlinear and awkward. Zoom in close enough, though, and almost anything looks straight, so we approximate hard problems with linear ones. Calculus does this too, one tangent line at a time.
Computers are built for it. A matrix operation breaks into many small, identical multiplications that a graphics card runs in parallel. That match to the hardware is why modern machine learning is, underneath, a tall stack of matrix multiplications.
The subject earns the word "foundational" not because it is old, but because it is the widest bridge in mathematics between a real problem and a computer that can solve it.
Who Invented Linear Algebra?
The pieces of linear algebra were built by different people, in different countries, across two centuries, often for reasons that had nothing to do with each other.
Two other names shaped the subject we use today:
Arthur Cayley (1821–1895, England) turned the matrix from a bookkeeping grid into an algebra, defining how to add, multiply, and invert matrices in his 1858 work on the theory of matrices.
Carl Friedrich Gauss (1777–1855, Germany) gave his name to Gaussian elimination, the systematic row-by-row method for solving a system of linear equations that still runs inside scientific software today.
Where Is Linear Algebra Used In The Real World?
The same handful of ideas, vectors and matrices and their transformations, run a surprising range of technology.
Machine learning: a neural network is layers of matrix multiplication, and training one nudges millions of matrix entries at the same time.
Computer graphics and games: every rotation, zoom, and camera move is a matrix applied to the coordinates of a 3D model.
Engineering: the forces in a bridge, the currents in a circuit, and the airflow over a wing are each written as $A\mathbf{x} = \mathbf{b}$ and solved with linear algebra.
Economics: input-output models track how the output of one industry feeds demand in another, held as one large matrix.
Data and search: compressing an image, recommending a film, and ranking web pages all reduce to eigenvalues and matrix factorisation.
One toolkit runs learning machines, film studios, power grids, and national economies. Mathematics is the shared language beneath fields that look nothing alike.
What Are The Most Common Linear Algebra Mistakes?
Three misconceptions trip up most beginners. They are among the errors documented in published studies of undergraduate linear-algebra students, not just classroom folklore.
Thinking linear algebra is "just matrices."
Where it slips in:
A student learns to add and multiply matrices, gets fluent at the arithmetic, and quietly concludes that the arithmetic is the whole subject.
Don't do this:
Do not treat the matrix as the point. Matrix arithmetic is the tool, not the idea.
The correct way:
Hold on to what the matrix stands for: a vector, a transformation, or a system. The grid of numbers is one way to write down an object that lives in a vector space. The meaning is the object, not the grid.
Confusing a vector with its coordinates.
Where it slips in:
A student sees $\begin{bmatrix} 2 \ 3 \end{bmatrix}$ and believes the vector is those two numbers, full stop.
Don't do this:
Do not tie a vector permanently to one list of numbers. Change the reference frame, meaning the basis, and the same arrow is described by a different pair of coordinates.
The correct way:
Read a vector as an arrow with a fixed length and direction. The coordinates are only its shadow in one chosen set of axes. Rotate the axes and the arrow is unchanged even though its numbers change.
Assuming every system has a unique solution.
Where it slips in:
A student solves $A\mathbf{x} = \mathbf{b}$, finds one answer, and expects that outcome every time.
Don't do this:
Do not assume one and only one solution exists. A linear system can have exactly one solution, none at all, or infinitely many.
The correct way:
Check the geometry first. Two lines can cross once, run parallel, or lie on top of each other, and the determinant tells you which case you are in before you start computing.
Practice Problems On Linear Algebra
Work these before checking the answers. They cover the objects, the operations, and the ideas above.
Solve the system $x + y = 5$ and $x - y = 1$.
(Answer: $x = 3$, $y = 2$.)Compute the determinant of $\begin{bmatrix} 3 & 1 \ 2 & 4 \end{bmatrix}$.
(Answer: $(3)(4) - (1)(2) = 10$.)Find the product $\begin{bmatrix} 1 & 2 \ 3 & 4 \end{bmatrix}\begin{bmatrix} 1 \ 1 \end{bmatrix}$.
(Answer: $\begin{bmatrix} 3 \ 7 \end{bmatrix}$.)How many solutions do $x + y = 2$ and $2x + 2y = 4$ have?
(Answer: infinitely many, since the two equations are the same line.)Scale the vector $\begin{bmatrix} 2 \ -1 \end{bmatrix}$ by the scalar $3$.
(Answer: $\begin{bmatrix} 6 \ -3 \end{bmatrix}$.)Give the eigenvalues of $\begin{bmatrix} 2 & 0 \ 0 & 3 \end{bmatrix}$.
(Answer: $2$ and $3$.)
Where Should You Go Next After Linear Algebra?
The subject opens onto the rest of higher mathematics, and a few doors lead most naturally from here.
Systems of equations. Begin where linear algebra began, with the methods for solving many equations together and reading off whether a solution even exists.
Matrices and determinants. Go deeper on the grid itself, how a determinant decides invertibility, and how a matrix encodes a transformation.
Eigenvalues and eigenvectors. Learn the directions a transformation leaves alone, the single idea behind stability analysis, data compression, and search ranking.
A live Bhanzu trainer teaches these in order, starting from the picture (arrows, grids, and spaces) before the symbols, in the Bhanzu algebra program.
Was this article helpful?
Your feedback helps us write better content