Complex Numbers for AI

Open a paper on Fourier transforms and i is everywhere. Look at transformer positional encodings and you'll find sin and cos pairs at different frequencies: the real and imaginary parts of complex exponentials, written out in trig instead. Read about quantum computing and the whole framework lives in complex vector spaces.

Complex numbers look abstract because the square root of -1 sounds like a trick question. It isn't. It's the natural language of rotations and oscillations, and once that clicks, i stops looking mysterious and starts looking like the right tool for the job. Anything that spins, vibrates, or oscillates is fair game for complex numbers.

Skip this lesson and a few doors stay locked: the Discrete Fourier Transform, the FFT built on top of it, RoPE (Rotary Position Embedding) in modern language models, and the specific frequencies the original Transformer paper chose for its sinusoidal positional encodings. All four rest on the same handful of complex-number facts.

This lesson builds complex arithmetic from scratch, connects it to geometry, and shows exactly where it shows up in machine learning.

What is a complex number?

A complex number has two parts: a real part and an imaginary part.

z = a + bi

where:
  a is the real part
  b is the imaginary part
  i is the imaginary unit, defined by i^2 = -1

That's it. Extend the number line into a plane: real numbers on one axis, imaginary numbers on the other. Every complex number is a point in that plane.

Complex arithmetic

Addition. Add the real parts together, add the imaginary parts together.

(a + bi) + (c + di) = (a + c) + (b + d)i

Example: (3 + 2i) + (1 + 4i) = 4 + 6i

Multiplication. Use the distributive law and remember that i^2 = -1.

(a + bi)(c + di) = ac + adi + bci + bdi^2
                 = ac + adi + bci - bd
                 = (ac - bd) + (ad + bc)i

Example: (3 + 2i)(1 + 4i) = 3 + 12i + 2i + 8i^2
                            = 3 + 14i - 8
                            = -5 + 14i

Conjugate. Flip the sign of the imaginary part.

conjugate of (a + bi) = a - bi

The product of a complex number and its conjugate is always real:

(a + bi)(a - bi) = a^2 + b^2

Division. Multiply numerator and denominator by the conjugate of the denominator.

(a + bi) / (c + di) = (a + bi)(c - di) / (c^2 + d^2)

This eliminates the imaginary part from the denominator, giving you a clean complex number.

The complex plane

The complex plane maps every complex number to a 2D point. The horizontal axis is the real axis, the vertical axis is the imaginary axis.

z = 3 + 2i  corresponds to the point (3, 2)
z = -1 + 0i corresponds to the point (-1, 0) on the real axis
z = 0 + 4i  corresponds to the point (0, 4) on the imaginary axis

A complex number is simultaneously a point and a vector from the origin. This dual interpretation is what makes complex numbers useful for geometry.

Polar form

Any point in the plane can be described by its distance from the origin and its angle from the positive real axis.

z = r * (cos(theta) + i*sin(theta))

where:
  r = |z| = sqrt(a^2 + b^2)     (magnitude, or modulus)
  theta = atan2(b, a)             (phase, or argument)

Rectangular form (a + bi) is good for addition. Polar form (r, theta) is good for multiplication.

Multiplication in polar form. Multiply the magnitudes, add the angles.

z1 = r1 * e^(i*theta1)
z2 = r2 * e^(i*theta2)

z1 * z2 = (r1 * r2) * e^(i*(theta1 + theta2))

This is why complex numbers are perfect for rotations. Multiplying by a complex number with magnitude 1 is a pure rotation.

Euler's formula

Euler's formula is the bridge between complex exponentials and trigonometry:

e^(i*theta) = cos(theta) + i*sin(theta)

This is the most important formula in this lesson. When theta = pi:

e^(i*pi) = cos(pi) + i*sin(pi) = -1 + 0i = -1

Therefore: e^(i*pi) + 1 = 0

Five fundamental constants (e, i, pi, 1, 0) linked in one equation.

Why Euler's formula matters for ML

Euler's formula says that e^(i*theta) traces the unit circle as theta varies: (1, 0) at theta = 0, (0, 1) at theta = pi/2, (-1, 0) at theta = pi, (0, -1) at theta = 3pi/2. A full rotation closes the loop at theta = 2pi.

That's not a coincidence: complex exponentials are rotations, and rotations are everywhere in signal processing and ML.

Connection to 2D rotations

Multiplying the complex number (x + yi) by e^(i*theta) rotates the point (x, y) by angle theta around the origin.

Rotation via complex multiplication:
  (x + yi) * (cos(theta) + i*sin(theta))
  = (x*cos(theta) - y*sin(theta)) + (x*sin(theta) + y*cos(theta))i

Rotation via matrix multiplication:
  [cos(theta)  -sin(theta)] [x]   [x*cos(theta) - y*sin(theta)]
  [sin(theta)   cos(theta)] [y] = [x*sin(theta) + y*cos(theta)]

They produce identical results. Complex multiplication IS 2D rotation. The rotation matrix is just complex multiplication written in matrix notation.

Phasors and rotating signals

A complex exponential e^(iomegat) is a point rotating around the unit circle at angular frequency omega. As t increases, the point traces the circle.

The real part of this rotating point is cos(omegat). The imaginary part is sin(omegat). A sinusoidal signal is the shadow of a rotating complex number.

e^(i*omega*t) = cos(omega*t) + i*sin(omega*t)

Real part:      cos(omega*t)    -- a cosine wave
Imaginary part: sin(omega*t)    -- a sine wave

This is the phasor representation. Instead of tracking a wiggly sine wave, you track a smoothly rotating arrow. Phase shifts become angle offsets. Amplitude changes become magnitude changes. Addition of signals becomes vector addition.

Roots of unity

The N-th roots of unity are N points equally spaced on the unit circle:

w_k = e^(2*pi*i*k/N)    for k = 0, 1, 2, ..., N-1

For N = 4, the roots are: 1, i, -1, -i (the four compass points). For N = 8, you get the four compass points plus the four diagonals.

Roots of unity are the foundation of the Discrete Fourier Transform. The DFT decomposes a signal into components at these N equally-spaced frequencies.

Connection to the DFT

The Discrete Fourier Transform of a signal x[0], x[1], ..., x[N-1] is:

X[k] = sum_{n=0}^{N-1} x[n] * e^(-2*pi*i*k*n/N)

Each X[k] measures how much the signal correlates with the k-th root of unity, a complex sinusoid at frequency k. The DFT breaks a signal into N rotating phasors and reports the amplitude and phase of each one.

Why i is not imaginary

i is a 90-degree rotation operator. Multiply a real number by i once and it swings 90 degrees onto the imaginary axis. Multiply by i again (i^2) and it swings another 90 degrees, landing on the negative real axis. That's why i^2 = -1: not a mystery, just a half-turn built from two quarter-turns.

The word "imaginary" is a historical accident. Descartes coined it dismissively, and the name stuck. But i is no more imaginary than negative numbers were when people first rejected them. Negative numbers answer "what do you subtract 5 from 3 to get?" The imaginary unit answers "what do you square to get -1?" Neither question is nonsense, and neither answer is fictional.

It's the rotation, not the label, that explains why complex numbers are everywhere in engineering. Anything that rotates (electromagnetic waves, quantum states, signal oscillations, positional encodings) is naturally described by complex numbers.

Complex exponentials vs trigonometric functions

Before Euler's formula, engineers wrote signals as Acos(omegat + phi): amplitude A, frequency omega, phase phi. This works but makes arithmetic painful. Adding two cosines with different phases requires trigonometric identities.

With complex exponentials, the same signal is Ae^(i(omega*t + phi)). Adding two signals is just adding two complex numbers. Multiplying (modulating) is just multiplying magnitudes and adding angles. Phase shifts become angle additions. Frequency shifts become multiplications by phasors.

The entire field of signal processing switched to complex exponential notation because the math is cleaner. The "real signal" is always just the real part of the complex representation. The imaginary part is carried along as bookkeeping, making all the algebra work out naturally.

Connection to transformers

Sinusoidal positional encodings (original Transformer paper):

PE(pos, 2i) = sin(pos / 10000^(2i/d))
PE(pos, 2i+1) = cos(pos / 10000^(2i/d))

The sin and cos pairs are the real and imaginary parts of complex exponentials at different frequencies. Each frequency provides a different "resolution" for encoding position. Low frequencies change slowly (coarse position). High frequencies change quickly (fine position). Together they give each position a unique frequency fingerprint.

RoPE (Rotary Position Embedding) takes this further. It explicitly multiplies query and key vectors by complex rotation matrices. The relative position between two tokens becomes a rotation angle. Attention is computed using these rotated vectors, making the model sensitive to relative position through complex multiplication.

Step 1: Complex class

Build a Complex number class that supports arithmetic, magnitude, phase, and conversion between rectangular and polar forms.

import math
 
class Complex:
    def __init__(self, real, imag=0.0):
        self.real = real
        self.imag = imag
 
    def __add__(self, other):
        return Complex(self.real + other.real, self.imag + other.imag)
 
    def __mul__(self, other):
        r = self.real * other.real - self.imag * other.imag
        i = self.real * other.imag + self.imag * other.real
        return Complex(r, i)
 
    def __truediv__(self, other):
        denom = other.real ** 2 + other.imag ** 2
        r = (self.real * other.real + self.imag * other.imag) / denom
        i = (self.imag * other.real - self.real * other.imag) / denom
        return Complex(r, i)
 
    def magnitude(self):
        return math.sqrt(self.real ** 2 + self.imag ** 2)
 
    def phase(self):
        return math.atan2(self.imag, self.real)
 
    def conjugate(self):
        return Complex(self.real, -self.imag)

Step 2: Polar conversion and Euler's formula

Rectangular and polar form both describe the same point, so you need a fast way to move between them. Euler's formula makes that a one-line function:

def to_polar(z):
    return z.magnitude(), z.phase()
 
def from_polar(r, theta):
    return Complex(r * math.cos(theta), r * math.sin(theta))
 
def euler(theta):
    return Complex(math.cos(theta), math.sin(theta))

Verify: euler(theta).magnitude() should always be 1.0. euler(0) should give (1, 0). euler(pi) should give (-1, 0).

Step 3: Rotation

Rotating a point (x, y) by angle theta is one complex multiplication:

point = Complex(3, 4)
rotated = point * euler(math.pi / 4)

The magnitude stays the same. Only the angle changes.

Step 4: DFT from complex arithmetic

The DFT formula from the Concept section translates directly into code: for each frequency k, multiply every sample by the matching root of unity and add up the results.

def dft(signal):
    N = len(signal)
    result = []
    for k in range(N):
        total = Complex(0, 0)
        for n in range(N):
            angle = -2 * math.pi * k * n / N
            total = total + Complex(signal[n], 0) * euler(angle)
        result.append(total)
    return result

This is the O(N^2) DFT. Each output X[k] is the sum of the signal samples multiplied by roots of unity.

Step 5: Inverse DFT

The inverse DFT reconstructs the original signal from its spectrum. The only changes from the forward DFT: flip the sign in the exponent and divide by N.

def idft(spectrum):
    N = len(spectrum)
    result = []
    for n in range(N):
        total = Complex(0, 0)
        for k in range(N):
            angle = 2 * math.pi * k * n / N
            total = total + spectrum[k] * euler(angle)
        result.append(Complex(total.real / N, total.imag / N))
    return result

This gives you perfect reconstruction. Apply DFT, then IDFT, and you get back the original signal to machine precision. No information is lost.

Step 6: Roots of unity

The DFT and IDFT above both call euler() at multiples of 2*pi/N. Those calls are exactly the roots of unity, so it's worth generating them directly instead of recomputing them inline:

def roots_of_unity(N):
    return [euler(2 * math.pi * k / N) for k in range(N)]

Verify two properties:

• Every root has magnitude exactly 1.
• The sum of all N roots is zero (they cancel out by symmetry).

These properties are what make the DFT invertible. The roots of unity form an orthogonal basis for the frequency domain.

Python has built-in complex number support. The literal j represents the imaginary unit.

z = 3 + 2j
w = 1 + 4j
 
print(z + w)
print(z * w)
print(abs(z))
 
import cmath
print(cmath.phase(z))
print(cmath.exp(1j * cmath.pi))

For arrays, numpy handles complex numbers natively:

import numpy as np
 
z = np.array([1+2j, 3+4j, 5+6j])
print(np.abs(z))
print(np.angle(z))
print(np.conj(z))
print(np.real(z))
print(np.imag(z))
 
signal = np.sin(2 * np.pi * 5 * np.linspace(0, 1, 128))
spectrum = np.fft.fft(signal)
freqs = np.fft.fftfreq(128, d=1/128)

Run code/complex_numbers.py. It generates outputs/skill-complex-arithmetic.md, a quick reference for complex-number operations in ML and signal-processing contexts.

1. Complex arithmetic by hand. Compute (2 + 3i) * (4 - i) and verify with the code. Then compute (5 + 2i) / (1 - 3i). Draw both results on the complex plane and check that multiplication rotated and scaled the first number.

2. Rotation sequence. Start with the point (1, 0). Multiply by e^(i*pi/6) twelve times. Verify that you return to (1, 0) after 12 multiplications. Print the coordinates at each step and confirm they trace a regular 12-gon.

3. DFT of a known signal. Create a signal that is the sum of sin(2pi3t) and 0.5sin(2pi7*t) sampled at 32 points. Run your DFT. Verify that the magnitude spectrum has peaks at frequencies 3 and 7, with the peak at 7 being half the height of the peak at 3.

4. Roots of unity visualization. Compute the 8th roots of unity. Verify that they sum to zero. Verify that multiplying any root by the primitive root e^(2pii/8) gives the next root.

5. Rotation matrix equivalence. For 10 random angles and 10 random points, verify that complex multiplication gives the same result as matrix-vector multiplication with the 2x2 rotation matrix. Print the maximum numerical difference.

• Visual Introduction to Euler's Formula - builds geometric intuition without heavy notation
• Su et al.: RoFormer (2021) - the paper introducing Rotary Position Embedding using complex rotations
• Vaswani et al.: Attention Is All You Need (2017) - the original Transformer paper with sinusoidal positional encodings
• 3Blue1Brown: Euler's formula with introductory group theory - visual explanation of why e^(i*pi) = -1
• Needham: Visual Complex Analysis - the best visual treatment of complex numbers, full of geometric insight
• Strang: Introduction to Linear Algebra, Ch. 10 - complex numbers in the context of linear algebra and eigenvalues