Understanding of Fourier Transform

This article distills the intuition I built while studying the Fourier series and the Fourier transform. Starting from the familiar idea of an inner product on vectors, we trace a single thread: inner product of functions → trigonometric series → complex exponential form → Fourier integral → Fourier transform → Discrete Fourier Transform (DFT).

📌

In one line — the Fourier series / transform decomposes a function into a linear combination of mutually orthogonal (independent) sinusoidal basis functions. It is exactly the same logic as expressing a vector in an orthogonal basis. The only difference is that the basis is now an infinite-dimensional set of sinusoids.

1. Orthogonal Functions

The inner product originates with vectors, but we want to extend the same idea to functions.

1) Inner product of vectors

For two vectors \(\vec{u} = u_1 i + u_2 j + u_3 k\) and \(\vec{v} = v_1 i + v_2 j + v_3 k\):

\[ \vec{u}\cdot\vec{v} = |\vec{u}||\vec{v}|\cos\theta = u_1 v_1 + u_2 v_2 + u_3 v_3 = \sum_{i=1}^{3} u_i v_i \]

The case \(\theta = 90^\circ\) (inner product = 0) is called orthogonality (independence).

2) Inner product of functions

For two functions \(f_1, f_2\) on an interval \([a, b]\), the inner product is defined as:

\[ (f_1, f_2) = \int_{a}^{b} f_1(x) f_2(x) \, dx \]

It is harder to picture geometrically, but if the two functions are independent then

\[ (f_1, f_2) = \int_{a}^{b} f_1(x) f_2(x) \, dx = 0 \quad \Rightarrow \quad f_1, f_2 \text{ are } \mathbf{orthogonal} \text{ on } [a,b] \]

Example. Take \(f_1(x) = x^2,\; f_2(x) = x^3\) on \([-1, 1]\): the integral vanishes, so they are orthogonal.

3) Orthogonal sets

A set of orthogonal functions (an infinite set) is defined as follows.

A set of real-valued functions \(\{\phi_0(x), \phi_1(x), \phi_2(x), \dots\}\) is said to be orthogonal on an interval \([a, b]\) if \((\phi_m, \phi_n) = \int_a^b \phi_m(x)\phi_n(x)\,dx = 0\) for \(m \ne n\).

In other words, every pair of distinct functions in an orthogonal set is mutually orthogonal.

4) Norm

One more term we'll lean on is the norm.

\[ \|\vec{u}\| = \sqrt{\vec{u}\cdot\vec{u}} \quad (\text{norm}), \qquad \vec{u}\cdot\vec{u} = \|\vec{u}\|^2 \quad (\text{square norm}) \]

For vectors, the inner product of a vector with itself gives its square norm. The same idea defines the norm of a function:

\[ \|\phi_n(x)\| = \sqrt{\int_a^b \phi_n \cdot \phi_n} \quad (\text{norm}), \qquad \|\phi_n(x)\|^2 = \int_a^b \phi_n \cdot \phi_n \quad (\text{square norm}) \]

2. Vector Analogy — linear combination of functions

If the orthogonal functions are independent, their key feature is linearity. With an orthogonal (infinite) set as the basis, we should be able to superpose them to represent any function.

\[ f(x) = c_0\phi_0(x) + c_1\phi_1(x) + c_2\phi_2(x) + \dots + c_n\phi_n(x) + \dots, \quad n = 0, 1, 2, \dots \quad \cdots (1) \]

But how do we find the coefficients \(c_n\)? Borrow the trick from vectors.

In a 3-D Cartesian system:

\[ \vec{u} = c_1 \vec{u_x} + c_2 \vec{u_y} + c_3 \vec{u_z} \quad [c_i: \text{scalar component of the vector}] \]

If the basis vectors are mutually orthogonal, any vector is a linear combination of them. To find the coefficients \(c_1, c_2, c_3\), take inner products:

\[ \begin{aligned} (\vec{u}, \vec{u}_x) &= c_1\|\vec{u}_x\|^2 + 0 + 0 \quad &\Rightarrow& \quad c_1 = \frac{\vec{u}\cdot\vec{u}_x}{\|\vec{u}_x\|^2}\\ (\vec{u}, \vec{u}_y) &= 0 + c_2\|\vec{u}_y\|^2 + 0 \quad &\Rightarrow& \quad c_2 = \frac{\vec{u}\cdot\vec{u}_y}{\|\vec{u}_y\|^2}\\ (\vec{u}, \vec{u}_z) &= 0 + 0 + c_3\|\vec{u}_z\|^2 \quad &\Rightarrow& \quad c_3 = \frac{\vec{u}\cdot\vec{u}_z}{\|\vec{u}_z\|^2} \end{aligned} \]

Now apply the same logic to (1). For functions, the inner product is continuous, so it becomes an integral (vectors are discrete):

\[ \int_a^b f(x)\phi_m(x)\,dx = c_0 \int_a^b \phi_0(x)\phi_m(x)\,dx + c_1 \int_a^b \phi_1(x)\phi_m(x)\,dx + \dots + c_n \int_a^b \phi_n(x)\phi_m(x)\,dx + \dots \]

By orthogonality every \(m \ne n\) term vanishes, and only the parallel term \(m = n\) survives:

\[ c_n = \frac{\int_a^b f(x)\phi_n(x)\,dx}{\int_a^b \phi_n(x)^2\,dx} \]

3. Trigonometric Series — Fourier Series

According to Zill & Wright, 5th edition, Exercises 12.1 – 12, the following set is mutually orthogonal for every \(x\):

\[ \left\{ 1, \cos\frac{n\pi}{p}x, \sin\frac{n\pi}{p}x \right\}, \quad n = 1, 2, 3 \quad \text{on}\;[-p, p] \]

So we can expand an arbitrary function \(f(x)\) as a linear combination of these basis functions:

\[ f(x) = c_0 + c_1\cos\frac{\pi}{p}x + c_2\sin\frac{\pi}{p}x + c_3\cos\frac{2\pi}{p}x + c_4\sin\frac{2\pi}{p}x + \dots \]

By convention, setting \(c_0 = a_0/2\), \(c_{2n-1} = a_n\), \(c_{2n} = b_n\) gives the cleaner form:

\[ \boxed{\; f(x) = \frac{a_0}{2} + \sum_{n=1}^{\infty}\left( a_n \cos\frac{n\pi}{p}x + b_n \sin\frac{n\pi}{p}x \right) \;} \quad \cdots (2) \]

Finding \(a_0, a_n, b_n\)

Apply the vector analogy: take the inner product of both sides with each basis function.

A) \(a_0\) — inner product of \(f(x)\) with 1 over \([-p, p]\):

\[ \int_{-p}^{p} f(x)\cdot 1\,dx = \frac{a_0}{2}\int_{-p}^{p} 1\,dx + 0 = p\,a_0 \quad \Rightarrow \quad a_0 = \frac{1}{p}\int_{-p}^{p} f(x)\,dx \]

B) \(a_n\) — inner product of \(f(x)\) with \(\cos(m\pi x/p)\):

\[ \int_{-p}^{p} f(x)\cos\frac{m\pi}{p}x\,dx = a_n \int_{-p}^{p} \cos\frac{n\pi}{p}x \cdot \cos\frac{m\pi}{p}x\,dx = p\;\;(m=n) \quad \Rightarrow \quad a_n = \frac{1}{p}\int_{-p}^{p} f(x)\cos\frac{n\pi}{p}x\,dx \]

C) \(b_n\) — inner product of \(f(x)\) with \(\sin(m\pi x/p)\):

\[ b_n = \frac{1}{p}\int_{-p}^{p} f(x)\sin\frac{n\pi}{p}x\,dx \]

Equation (2) is the Fourier series of \(f(x)\); the coefficients from A), B), C) are the Fourier coefficients.

What does the Fourier series actually mean?

Back to the vector analogy — a 3-D vector is a linear combination of basis vectors and can represent any vector in 3-D space:

\[ \vec{u} = c_1 \vec{u}_x + c_2 \vec{u}_y + c_3 \vec{u}_z \quad \longrightarrow \quad \vec{u}\;(\text{every vector in 3-D space}) \]

A function is the same — the point is that the linear combination can represent every \(f(x)\):

\[ f(x) = \frac{a_0}{2} + \sum_{n=1}^{\infty}\left( a_n\cos\frac{n\pi}{p}x + b_n\sin\frac{n\pi}{p}x \right) \;\longrightarrow\; f(x)\;(\text{any function defined on } [-p, p]) \]

Two caveats follow.

Like a Taylor series, a Fourier series is an infinite series, so convergence becomes an issue.
Truncation introduces error, so the accuracy depends on where we stop.

Conditions for convergence

Even if \(f(x)\) is discontinuous, that's fine as long as the discontinuities are finite in number. Outside the discontinuities (piecewise continuous), if \(f\) and \(f'\) exist on \([-p, p]\), convergence is guaranteed even at the discontinuities themselves.

\[ \text{At a point of discontinuity, the Fourier series converges to } \frac{f(x_+) + f(x_-)}{2} \]

4. Expansion in a Fourier Series — a worked example

Expand the following function as a Fourier series.

\[ f(x) = \begin{cases} 0, & -\pi < x < 0 \\ \pi - x, & 0 \le x \le \pi \end{cases} \]

Plug all coefficients into the Fourier series. The result is:

\[ f(x) = \frac{\pi}{4} + \sum_{n=1}^{\infty} \left\{ \frac{1 - (-1)^n}{n^2 \pi}\cos nx + \frac{1}{n}\sin nx \right\} \]

💡

Interpretation. The index \(n\) plays the role that \(i, j, k\) play for vectors — it enumerates basis components. Here \(n\) runs over countably infinitely many basis functions. (In theory, \(f(x)\) is a linear combination of infinitely many sinusoidal basis functions on \([-\pi, \pi]\).)

At the discontinuity \(x = 0\), the convergence condition gives \(\dfrac{f(0+) + f(0-)}{2} = \dfrac{\pi + 0}{2} = \dfrac{\pi}{2}\).

Orthogonality holds only on \([-\pi, \pi]\), so the Fourier expansion is valid only inside that interval.

What happens outside \([-p, p]\)?

The shape on the orthogonal interval simply repeats periodically.

Equation (2) writes an arbitrary \(f(x)\) as a linear combination of orthogonal sinusoids with different fundamental periods. By analogy: a vector in 2-D \((i, j)\) is less expressive than one in 3-D \((i, j, k)\), which is less expressive than one in 4-D \((i, j, k, t)\). Higher dimension means we can represent more.

Among the sinusoids in the linear combination, the one with the longest period sets the overall period of \(f(x)\). For the form in (2), the orthogonality interval \([-p, p]\) of length \(2p\) becomes the period.

\[ \boxed{\; 2p \text{ is the fundamental period of the sum} \;} \]

Sum of an orthogonal set of trigonometric functions

A linear combination of the basis functions above — orthogonal on \((-p, p)\) — converges to an arbitrary function as \(n \to \infty\).

📐

Summary — the Fourier expansion of \(f(x)\) is a \(2p\)-periodic function on \((-p, p)\). \(2p\) is the fundamental period of the (linear-combination) sum.

Reference: in Zill & Wright (5th ed.), page 663, example graphs show that partial sums get closer to \(f(x)\) as \(n\) grows — moving \((a) \to (b) \to (c) \to (d)\) toward the target on \((-\pi, \pi)\). The Fourier series is itself periodic.

5. Complex (exponential) form of the Fourier series

For analyzing periodic signals, the complex form is more convenient. By Euler's formula:

\[ e^{i\varphi} = \cos\varphi + i\sin\varphi, \quad \cos x = \frac{e^{ix} + e^{-ix}}{2}, \quad \sin x = \frac{e^{ix} - e^{-ix}}{2i} \]

Substituting into (2):

\[ f(x) = \frac{a_0}{2} + \sum_{n=1}^{\infty}\left( \tfrac{1}{2}(a_n - i b_n)e^{\,i n\pi x/p} + \tfrac{1}{2}(a_n + i b_n)e^{-i n\pi x/p} \right) = c_0 + \sum_{n=1}^{\infty}\left( c_n e^{\,i n\pi x/p} + c_{-n} e^{-i n\pi x/p} \right) \cdots (3) \]

Each \(c\)-coefficient is computed the same way as before:

\[ c_0 = \frac{1}{2p}\int_{-p}^{p} f(x)\,dx, \quad c_n = \frac{1}{2p}\int_{-p}^{p} f(x)\,e^{-i n\pi x/p}\,dx, \quad c_{-n} = \frac{1}{2p}\int_{-p}^{p} f(x)\,e^{\,i n\pi x/p}\,dx \]

Tidying (3) gives the compact form:

\[ \boxed{\; f(x) = \sum_{n=-\infty}^{\infty} c_n\, e^{\,i n\pi x/p} \;} \quad \cdots (4) \qquad c_n = \frac{1}{2p}\int_{-p}^{p} f(x)\, e^{-i n\pi x/p}\,dx \]

\(n = 0, \pm 1, \pm 2, \pm 3, \dots\). At a discontinuity: \(\dfrac{f(x_+) + f(x_-)}{2}\). \(c_n\) is the per-basis weight; \(e^{\,i n\pi x/p}\) is the basis itself. This is essentially the same linear combination of sinusoids as (2), just compactly written.

6. Wave basics — period, frequency, wavelength

A quick refresher on wave terminology.

Time for one cycle — period \(T = \text{time}/\text{cycle}\)
Cycles per second — frequency \(f = \text{cycle}/\text{time}\)
Distance per cycle — wavelength \(\lambda = \text{distance}/\text{cycle}\)

Wave speed is therefore:

\[ v = \frac{\lambda}{T} = \lambda f \]

Two ways to read it. ① Distance per time — one cycle covers \(\lambda\) in \(T\) seconds. ② Per cycle the distance is \(\lambda\), and there are \(f\) cycles per second.

Fundamental frequency

Back to the Fourier expansion:

\[ f(x) = \frac{a_0}{2} + \sum_{n=1}^{\infty}\left( a_n\cos\frac{n\pi}{p}x + b_n\sin\frac{n\pi}{p}x \right) \qquad f(x) = \sum_{n=-\infty}^{\infty} c_n\, e^{\,i n\pi x/p} \quad \cdots (5) \]

The overall period of \(f(x)\) is set by the longest-period basis sinusoid in the expansion — that is, the one at \(n = 1\).

\[ T(\text{period}) = 2p, \quad p = T/2, \quad \omega = \frac{2\pi(1\;\text{cycle})}{T} \]

This is the fundamental period of the Fourier expansion. Hence:

\[ \cos\frac{n\pi}{p}x \;\Longleftrightarrow\; \cos\frac{n\,2\pi}{T}x = \cos n\omega x \]

\[ \boxed{\; f(x) = \frac{a_0}{2} + \sum_{n=1}^{\infty}(a_n\cos n\omega x + b_n\sin n\omega x), \qquad f(x) = \sum_{n=-\infty}^{\infty} c_n\, e^{\,i n\omega x} \;} \]

⚠️

Common confusion. Isn't the period \(T = 2p/n\)? — If it were written that way, that would describe the period of each individual basis function. The \(T\) and frequency above refer instead to the period and frequency of \(f(x)\) as a whole (the \(n = 1\) case).

The point is to identify \(f(x)\)'s own fundamental frequency and period, then re-express the Fourier expansion using those quantities so they're easy to read off.

If we use the circular convention, one cycle equals \(2\pi\,[\mathrm{rad}]\). The fundamental frequency in that convention is then called the fundamental angular frequency \(\omega\):

\[ \omega = \frac{2\pi(1\;\text{cycle})}{T} \]

* Note: \(2\pi\,[\mathrm{rad}]\) is a dimensionless angle, not a distance. The distance per cycle (wavelength) is a separate concept — don't confuse "cycle" with "distance." Cycle is expressed in terms of distance or period, not equated with them.

7. Frequency spectrum

\(f(x)\) is a sum of many sinusoids (think of them in blue). Plotting the fundamental angular frequency on the \(x\)-axis and the amplitude (coefficient) on the \(y\)-axis gives the frequency spectrum.

🎯

What the frequency spectrum tells us — which sinusoids make up the time-domain function \(f(x)\), and with what amplitudes. The waveform is sinusoidal anyway, so frequency + amplitude is enough to characterize each component.

Aside: roughly, do high-frequency components tend to have small amplitudes?

These ideas are the core meaning of the Fourier series.

Amplitude and phase of \(c_n\)

Strictly, the amplitude is \(|c_n|\). Since \(c_n\) itself is complex (a vector in the complex plane), it can also shift phase:

\[ c_n = a + bi, \quad |c_n| = \sqrt{a^2 + b^2}, \quad \varphi = \tan^{-1}\!\left(\frac{b}{a}\right) \]

\(\varphi\) is the phase shift.

Spectrum examples

Two simple examples.

\(f(x) = e^{-x},\;(-\pi < x < \pi)\)
\(f(x) = \begin{cases} 0, & -1 < x < 0 \\ 1, & 0 \le x < 1 \\ 0, & 1 \le x < 3 \end{cases}\)

Q. Express each function as a complex Fourier series and plot its frequency spectrum across \(n\).

Example 1) \(f(x) = e^{-x},\;(-\pi < x < \pi)\)

\[ c_n = \frac{1}{2\pi}\int_{-\pi}^{\pi} e^{-x}\cdot e^{-i n x}\,dx = \frac{1}{2\pi}\int_{-\pi}^{\pi} e^{-(in+1)x}\,dx = \frac{-1}{2\pi(in+1)}\!\left[ e^{-(in+1)x}\right]_{-\pi}^{\pi} \]

\[ = \frac{-1}{2\pi(in+1)}\!\left[ e^{-i n\pi}\,e^{-\pi} - e^{i n\pi}\,e^{\pi} \right] = \frac{-1}{2\pi(in+1)}(-1)^n[e^{-\pi} - e^{\pi}] = (-1)^n \frac{\sinh\pi}{\pi(in+1)} \]

(using \(e^{\pm i n\pi} = \cos n\pi \pm i\sin n\pi = (-1)^n\))

\[ f(x) = \sum_{n=-\infty}^{\infty} (-1)^n \frac{\sinh\pi}{\pi(in+1)}\, e^{i n\omega x} = \frac{\sinh\pi}{\pi}\sum_{n=-\infty}^{\infty}(-1)^n\frac{1 - in}{n^2+1}\,e^{i n\omega x} \]

\[ c_n = \frac{\sinh\pi}{\pi(n^2+1)}(1-in), \quad |c_n| = \frac{\sinh\pi}{\pi(n^2+1)}\sqrt{n^2+1} = \frac{\sinh\pi}{\pi\sqrt{n^2+1}} \]

Here \(n\omega\) is the fundamental frequency of each component sinusoid, and \(\omega\) is the fundamental frequency of the whole Fourier series. With \(\omega = 2\pi/T = 2\pi/2\pi = 1\), we get \(n\omega = n\).

8. Integral transform method — Fourier integral

Error function, Laplace transform, Fourier transform — all are integral transforms.

So far \(f(x)\) has been a sum of discrete sinusoidal components:

\[ f(x) = \sum_{n=1}^{\infty}: \quad n=1, n=2, n=3, \dots, n\to\infty \]

But the "missing" frequencies between, say, \(n = 1\) and \(n = 2\) can also be components. To account for them we need the continuous idea of integration:

\[ \int_1^{\infty}: \quad n=1 \to \text{continuously varying sinusoids all become components} \to n\to\infty \]

From discrete to continuous

\[ f(x) = \frac{a_0}{2} + \sum_{n=1}^{\infty}\left(a_n\cos\frac{n\pi}{p}x + b_n\sin\frac{n\pi}{p}x\right) \]

\[ a_0 = \frac{1}{p}\int_{-p}^{p} f(x)\,dx, \quad a_n = \frac{1}{p}\int_{-p}^{p} f(x)\cos\frac{n\pi}{p}x\,dx, \quad b_n = \frac{1}{p}\int_{-p}^{p} f(x)\sin\frac{n\pi}{p}x\,dx \]

Plug the coefficients back in:

\[ f(x) = \frac{1}{2p}\int_{-p}^{p} f(t)\,dt + \frac{1}{p}\sum_{n=1}^{\infty}\left( \left(\int_{-p}^{p} f(t)\cos\frac{n\pi}{p}t\,dt\right)\cos\frac{n\pi}{p}x + \left(\int_{-p}^{p} f(t)\sin\frac{n\pi}{p}t\,dt\right)\sin\frac{n\pi}{p}x \right) \]

Why is \(p\) in the denominator? Because the Fourier expansion's period is \(2p\) (its longest fundamental period). To convert the sum into an integral, we need a differential \(d\alpha\) to fall out of \(\pi/p\).

What should that differential be? Since \(n\) controls the frequency of \(\cos, \sin\), mimicking a continuous change of frequency means using:

\[ \frac{(n+1)\pi}{p} = \alpha_{n+1}, \quad \frac{n\pi}{p} = \alpha_n, \quad \frac{\pi}{p} = \Delta\alpha = \alpha_{n+1} - \alpha_n \]

\[ f(x) = \frac{1}{2\pi}\!\left(\int_{-p}^{p} f(t)\,dt\right)\!\Delta\alpha + \frac{1}{\pi}\sum_{n=1}^{\infty}\!\left( \left(\int_{-p}^{p} f(t)\cos\alpha_n t\,dt\right)\cos\alpha_n x + \left(\int_{-p}^{p} f(t)\sin\alpha_n t\,dt\right)\sin\alpha_n x \right)\!\Delta\alpha \]

Stretch the domain to infinity (\(p \to \infty\)) and let the basis become continuous (\(\cos\alpha\,d\alpha\)):

\[ p \to \infty: \quad \Delta\alpha \to 0, \quad \alpha_n = \Delta\alpha\cdot n \quad \Rightarrow \quad \lim_{\Delta\alpha\to 0} \sum_{n=1}^{\infty} F(\alpha_n)\Delta\alpha = \int_0^{\infty} F(\alpha)\,d\alpha \]

\[ \boxed{\; f(x) = \frac{1}{\pi}\int_0^{\infty}\!\!\left( \!\!\left(\int_{-\infty}^{\infty} f(t)\cos(\alpha t)\,dt\right)\cos\alpha x + \left(\int_{-\infty}^{\infty} f(t)\sin(\alpha t)\,dt\right)\sin\alpha x \right) d\alpha \;} \]

This is the Fourier integral of \(f\) on \((-\infty, \infty)\). If \(f\) is even or odd, the form simplifies further.

Even / odd Fourier integral

For an even function:

\[ f(x) = \frac{2}{\pi}\int_0^{\infty} A(\alpha)\cos(\alpha x)\,d\alpha, \quad A(\alpha) = \int_0^{\infty} f(x)\cos(\alpha x)\,dx \quad \cdots (6) \]

For an odd function:

\[ f(x) = \frac{2}{\pi}\int_0^{\infty} B(\alpha)\sin(\alpha x)\,d\alpha, \quad B(\alpha) = \int_0^{\infty} f(x)\sin(\alpha x)\,dx \quad \cdots (7) \]

Complex form of the Fourier integral

The Fourier integral also has a complex form.

\[ f(x) = \frac{1}{\pi}\int_0^{\infty}\!\!\left( \int_{-\infty}^{\infty} f(t)[\cos\alpha t \cos\alpha x + \sin\alpha t \sin\alpha x]\,dt \right) d\alpha = \frac{1}{\pi}\int_0^{\infty}\!\!\left( \int_{-\infty}^{\infty} f(t)\cos[\alpha(t-x)]\,dt \right)\!d\alpha \]

In \(\alpha\), \(\cos\) is even, so we can rewrite:

\[ f(x) = \frac{1}{2\pi}\int_{-\infty}^{\infty}\!\!\left( \int_{-\infty}^{\infty} f(t)[\cos\alpha(t-x) + i\sin\alpha(t-x)]\,dt \right)\!d\alpha = \frac{1}{2\pi}\int_{-\infty}^{\infty}\!\!\left(\int_{-\infty}^{\infty} f(t)\,e^{i\alpha(t-x)}\,dt\right) d\alpha \]

(The \(i\sin\alpha(t-x)\) term integrates to zero, since \(f(t)\) doesn't depend on \(\alpha\).)

🤔

Why does \(\sin, \cos\) integrate over \((0, \infty)\), while the complex form integrates over \((-\infty, \infty)\)? — In (6), (7) the \(\alpha\) inside \(A(\alpha), B(\alpha)\) ends up absorbed into the exponential, leaving \(f(t)\) alone. The resulting expression in \(\alpha\) is even, so extending the range to \((-\infty, \infty)\) is valid.

\[ \boxed{\; f(x) = \frac{1}{2\pi}\int_{-\infty}^{\infty} F(\alpha)\,e^{-i\alpha x}\,d\alpha, \qquad F(\alpha) = \int_{-\infty}^{\infty} f(t)\,e^{i\alpha t}\,dt \;} \]

Complex form. \(e^{-i\alpha x}\) is the sinusoidal basis; \(C(\alpha)\) is the per-basis coefficient (like a vector weight). The integral form invites a more general framing — the integral-transform framework below.

9. Integral transforms & Fourier transform pairs

A transform pair is:

\[ f(x)\;\xrightarrow{\;\text{integral}\;}\;F(\alpha), \qquad F(\alpha)\;\longrightarrow\;f(x) \]

\[ \begin{cases} F(\alpha) = \int_a^b f(x) K(\alpha, x)\,dx & \quad\text{(real domain → } \alpha \text{ domain)}\\ f(x) = \int_a^b F(\alpha) H(\alpha, x)\,d\alpha & \quad (\alpha \text{ domain → real domain)} \end{cases} \]

The pair always works together. The functions \(K(\alpha, x), H(\alpha, x)\) used to switch domains are called kernels. For the Laplace transform, for instance, \(K(s, x) = e^{-sx}\) and \(H(s, x) = e^{sx}/(2\pi i)\).

Fourier transform pair

\[ \boxed{\; \mathcal{F}\{f(x)\} = \int_{-\infty}^{\infty} f(x) K(\alpha, x)\,dx = F(\alpha), \quad K(\alpha, x) = e^{i\alpha x} \;} \]

\[ \boxed{\; \mathcal{F}^{-1}\{F(\alpha)\} = \int_{-\infty}^{\infty} F(\alpha) H(\alpha, x)\,d\alpha = f(x), \quad H(\alpha, x) = \frac{e^{-i\alpha x}}{2\pi} \;} \]

Transforms for even / odd functions

Even function — Fourier cosine transform

\[ \mathcal{F}_c\{f(x)\} = \int_0^{\infty} f(x)\cos\alpha x\,dx = F(\alpha), \qquad \mathcal{F}_c^{-1}\{F(\alpha)\} = \frac{2}{\pi}\int_0^{\infty} F(\alpha)\cos\alpha x\,d\alpha = f(x) \]

Odd function — Fourier sine transform

\[ \mathcal{F}_s\{f(x)\} = \int_0^{\infty} f(x)\sin\alpha x\,dx = F(\alpha), \qquad \mathcal{F}_s^{-1}\{F(\alpha)\} = \frac{2}{\pi}\int_0^{\infty} F(\alpha)\sin\alpha x\,d\alpha = f(x) \]

Like the Laplace transform, the Fourier transform can solve differential equations: differentiation in the real domain pulls a factor onto the transformed function.

\[ \begin{aligned} \mathcal{F}\{f'(x)\} &= -i\alpha F(\alpha)\\ \mathcal{F}\{f''(x)\} &= -\alpha^2 F(\alpha)\\ \mathcal{F}\{f^{(n)}(x)\} &= (-i\alpha)^n F(\alpha)\\ \mathcal{F}_s\{f''(x)\} &= -\alpha^2 F(\alpha) + \alpha f(0)\\ \mathcal{F}_c\{f''(x)\} &= -\alpha^2 F(\alpha) - f'(0) \end{aligned} \]

10. Example — solving the heat equation

Example 1. Solve the heat equation

\[ k\frac{\partial^2 u}{\partial x^2} = \frac{\partial u}{\partial t}, \quad -\infty < x < \infty,\; t > 0 \]

\[ u(x, 0) = f(x), \qquad f(x) = \begin{cases} u_0, & |x| < 1\\ 0, & |x| > 1 \end{cases} \]

Take the Fourier transform of both sides:

\[ \mathcal{F}\!\left\{ k\frac{\partial^2 u(x, t)}{\partial x^2} \right\} = \mathcal{F}\!\left\{\frac{\partial u}{\partial t}\right\} \quad \Longrightarrow \quad -k\alpha^2 U(\alpha, t) = \frac{dU(\alpha, t)}{dt} \]

\[ U(\alpha, t) = c\, e^{-k\alpha^2 t} \]

From the initial condition:

\[ \mathcal{F}\{u(x, 0)\} = \mathcal{F}\{f(x)\} = \int_{-\infty}^{\infty} f(x)\,e^{i\alpha x}\,dx = 0 + \int_{-1}^{1} u_0\,e^{i\alpha x}\,dx + 0 = u_0\frac{e^{i\alpha} - e^{-i\alpha}}{i\alpha} \]

\[ U(\alpha, 0) = u_0\frac{e^{i\alpha} - e^{-i\alpha}}{i\alpha} = c \]

🤔

Question 1 — \(u(x, t)\) depends on both space and time, so why Fourier-transform only in space and leave \(t\) alone? Couldn't we do a 2-D Fourier transform?

A 2-D transform like \(F(\alpha, \beta) = \int\!\int f(x,y)\,e^{-i(\alpha x + \beta y)}\,du\,dv\) is in principle possible, and including \(t\) should still yield a result. But we usually peel \(t\) off separately, for two reasons:

An \(n\)-dimensional Fourier transform that includes \(x, y, z, \dots\) all together (a first-order setup) is not always guaranteed to exist or be useful — \(x, y, z\) appear implicitly in each other, and orthogonality can break (a stability issue: in PDEs, elliptic problems are typically stable, while hyperbolic and parabolic ones can be unstable, depending on the regime).
The PDE is first-order in \(t\), so as in Example 1 it makes physical and mathematical sense to separate \(t\) and solve sequentially.

Reading \(dU(\alpha, t)/dt\) intuitively (in a 2-D setting): the snapshots \(t_1, t_2, t_3\) are continuous moments in time. At each moment, \(k(\partial^2 u/\partial x^2 + \partial^2 u/\partial y^2)\) updates the Fourier-domain representation, and we step forward. In code, you can either store time inside an array or sweep it in an outer loop — either way, the time evolution lives on the \(t \times \alpha\) axis alongside the spatial transform.

Solving a PDE with Fourier means: move to the Fourier domain \((\alpha, \beta)\), solve there for a weak solution, then transform back to the real domain. Next we wrap up with the Discrete Fourier Transform (DFT) — a stepping stone toward spectral methods.

11. Discrete Fourier Transform (DFT)

The continuous form is:

\[ f(x) = \frac{1}{2\pi}\int_{-\infty}^{\infty} F(\alpha)\,e^{-i\alpha x}\,d\alpha, \qquad F(\alpha) = \int_{-\infty}^{\infty} f(t)\,e^{i\alpha t}\,dt \]

To actually solve numerically, this continuous picture must be turned discrete.

Step 1 — bound the domain

Replace the infinite real line with the bounded interval \([0, 2p]\):

\[ f(x) = \sum_{n=-\infty}^{\infty} c_n\, e^{i n\omega x}, \qquad c_n = \frac{1}{2p}\int_0^{2p} f(x)\,e^{-i n\omega x}\,dx \quad \cdots (4) \]

Step 2 — make the basis count countably infinite

Move from an uncountably infinite basis to a countably infinite (integer) one:

\[ f(x) = \int_{-\infty}^{\infty} F(\alpha) H(\alpha, x)\,d\alpha \quad\Longrightarrow\quad f(x) = \sum_{n=-\infty}^{\infty} c_n\,e^{i n\omega x} \quad\Longrightarrow\quad f(x) = \sum_{n=0}^{N-1} c_n\,e^{i n\omega x} \]

In theory the basis is countably infinite; in practice we truncate to \(N\) terms.

Step 3 — discretize \(x\) (sampling rate)

\(x\) is also uncountably infinite over \((0, 2p)\), so it too must be made finite — this is the heart of discretization: cover \(x\) with finitely many sample points.

We can't carry uncountably many points, so we sample. The spacing \(\Delta x = T\) between samples is the sampling rate.

Taking one cycle as \(2\pi\) (angular) and \(N\) samples per cycle:

\[ N\,(\#\text{of samples}) = \frac{2\pi}{T} = \omega, \qquad p = \pi \]

\[ \alpha = \frac{n\pi}{p}, \quad \omega = \frac{2\pi}{2p}, \quad T = \frac{2\pi}{N} \]

The number of basis functions and the number of sample points are independent in principle, but we usually take them equal (it makes the matrix square, which is easier to solve). \(k\) and \(n\) don't strictly have to match.

\[ f^0 = \sum_{k=0}^{N-1} c_k\,e^{i(k)\omega(kT)}, \quad f^1 = \sum_{k=0}^{N-1} c_k\,e^{i(2)\omega(kT)}, \quad \dots \quad f^{N-1} = \sum_{k=0}^{N-1} c_k\,e^{i(n)\omega(kT)} \]

\[ \alpha = \frac{n\pi}{p}, \quad \omega = \frac{2\pi}{2p} \quad \begin{aligned}\alpha&: \text{basis sinusoidal frequency}\\ \omega&: \text{Fourier-expansion frequency}\end{aligned}, \qquad n\alpha = \omega \;(n=1 \text{ gives the longest period}) \]

Matrix form

\[ \begin{pmatrix} f_0 \\ f_1 \\ f_2 \\ \vdots \\ f_{N-1} \end{pmatrix} = \begin{pmatrix} 1 & 1 & 1 & \cdots & 1 \\ 1 & \omega_N & \omega_N^2 & \cdots & \omega_N^{N-1} \\ 1 & \omega_N^2 & \omega_N^4 & \cdots & \omega_N^{2(N-1)} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 1 & \omega_N^{N-1} & \omega_N^{2(N-1)} & \cdots & \omega_N^{(N-1)^2} \end{pmatrix} \begin{pmatrix} c_0 \\ c_1 \\ c_2 \\ \vdots \\ c_{N-1} \end{pmatrix} \]

\[ \omega_N = e^{\,i\,2\pi/N} = \cos\frac{2\pi}{N} + i\sin\frac{2\pi}{N} \]

🧭

Picturing what "solving with Fourier" means — start from a PDE in \(f(x)\), decompose into basis components, find the coefficients \(c_0, c_1, c_2, \dots, c_{N-1}\), and recombine to recover \(f(x)\). In matrix form, equation (4) becomes the linear system shown above.

Summary

Vectors → functions: extending the inner product to an integral carries orthogonality, normalization, and linear combinations straight over to functions.
Fourier series: decompose a \(2p\)-periodic function in the orthogonal basis \(\{1, \cos(n\pi x/p), \sin(n\pi x/p)\}\). Coefficients come from the vector-analogy inner product.
Complex form: Euler's formula collapses the series to a single line \(\sum c_n e^{in\pi x/p}\); \(|c_n|\) is the amplitude and \(\arg c_n\) is the phase.
Frequency spectrum: plot \(n\omega\) on the \(x\)-axis and \(|c_n|\) on the \(y\)-axis to read off, at a glance, which sinusoids make up the function.
Fourier integral / transform: send \(p \to \infty\) so that the sum becomes an integral. With kernel \(K(\alpha, x) = e^{i\alpha x}\) we get the transform pair \(f(x) \leftrightarrow F(\alpha)\).
Solving PDEs: differentiation maps to \((-i\alpha)^n F(\alpha)\), so a PDE becomes an ODE in \(\alpha\)-space — illustrated on the heat equation.
DFT: replace the infinite integral with a finite sum, the uncountable basis with \(N\) discrete bases, and continuous \(x\) with samples at spacing \(T\). The result is an \(N \times N\) Vandermonde system in \(\omega_N = e^{i2\pi/N}\).