Numerical Methods on 四方喫茶舘

Numerical Methods: Periodic Steady State Simulation

Thu, 01 Oct 2026 21:24:34 -0700

We explained some commonly used numerical methods for solving IVP problems in transient simulations in the last post. However, there are sometimes situations where IVP problem doesn’t characterize the system’s behavior adequately.

One such situation is when we are interested in the system’s periodic steady-state behavior rather than its transient response. In these cases, we are more interested in what’s called the Boundary Value Problem (BVP), where the solution is sought over a period with specified boundary conditions rather than initial conditions.

Boundary Value Problem (BVP)

Let’s say you have a switched-capacitor filter, or simply our RC circuit from last time, driven by a 1 GHz clock, and you want to know its steady-state output ripple. With a transient simulation, you would have to start from some initial condition and wait for hundreds of cycles until all the start-up transients die out, and then throw most of the waveform away. What we really want is the waveform of one period after the circuit has settled, and the condition that tells us we’ve settled is that the state at the end of the period comes back to where it started. You have just created a boundary value problem (BVP): instead of specifying the state at one point in time and marching forward, we specify conditions that the solution must satisfy at both ends of an interval.

Mathematically, a two-point BVP can be expressed as:

$$ \frac{d\mathbf{x}(t)}{dt} = \mathbf{f}(\mathbf{x}(t), t), \quad f(t_0) = A, \quad f(t_0 + T) = B $$

where $\mathbf{x}(t)$ and $\mathbf{f}(\mathbf{x}(t), t)$ are the same as in the IVP, $T$ is the length of the interval. In the special case of periodic steady-state analysis, the boundary conditions are simply:

$$ f(t_0) = f(t_0 + T) $$

meaning that the solution repeats itself after one period.

BVPs are generally more challenging to solve than IVPs because the solution must satisfy conditions at multiple points rather than just an initial condition. We will see how we are going to convert a BVP into an IVP, and use whatever we have learned about IVPs to tackle the problem. However, before that we must understand what are the scenarios that BVP might not have a solution.

Preventing the Butterfly Effect

You probably have heard of the “butterfly effect,” which refers to the sensitive dependence on initial conditions in chaotic systems. Sometimes this can create problems when we would like to find the exact solution.

The Lorenz Attractor Example

One famous butterfly effect example is the Lorenz attractor.

In 1963, Edward Lorenz simplified a model of atmospheric convection down to just three state variables, and arrived at the following system of ODEs:

$$ \begin{align*} \begin{cases} \dot{x} &= \sigma (y - x) \\ \dot{y} &= x (\rho - z) - y \\ \dot{z} &= xy - \beta z \end{cases} \end{align*} $$

where $\sigma$, $\rho$ and $\beta$ are positive constants. Lorenz used $\sigma = 10$, $\rho = 28$ and $\beta = 8/3$, which are still the standard values today. In our notation, the state vector is $\mathbf{x} = (x, y, z)^T$, and $\mathbf{f}(\mathbf{x})$ is the right-hand side above.

The system looks innocent: it’s only three equations, the only nonlinear terms are the products $xz$ and $xy$, and there is no randomness anywhere. Yet for these parameter values, every trajectory gets pulled onto a bounded, butterfly-shaped set, called the attractor, and keeps looping around its two “wings” forever, switching between them in a pattern that never repeats.

The butterfly effect shows up when we start two simulations from almost the same point. Let’s integrate the system with the RK4 method from the last post, with $h = 0.005$, from $\mathbf{x}_0 = (1, 1, 1)$ and from a second initial state that differs only by $10^{-8}$ in $x$:

For the first 25 time units, the two waveforms are indistinguishable. Then they suddenly go their separate ways, and after $t \approx 33$ they have nothing to do with each other. On the right, we see why: after an initial settling phase, the separation $\delta(t)$ between the two trajectories grows exponentially,

$$ \delta(t) \approx \delta_0 \, e^{\lambda t} $$

until it saturates at the size of the attractor itself. The growth rate $\lambda$ is called the largest Lyapunov exponent. Fitting the growth phase gives a slope of 0.95, and averaging over 1000 time units gives $\lambda \approx 0.90$, i.e. any initial error grows by about $e^{0.9} \approx 2.5\times$ per unit time. A $10^{-8}$ difference, far below anything we could ever measure, reaches order one at $t \approx 28$: about 13 time units of settling, then roughly 20 of exponential growth.

Bear in mind that this is not a numerical artifact: a smaller time step only makes RK4 follow the exact trajectory more closely, and the exact trajectories themselves diverge like this. This is a property of the system, not of the solver.

Continuous Dependence on Initial Conditions

So, is there anything that prevents the butterfly effect? To answer this, let’s go back to the mapping $\psi$ from the IVP, and be a bit more precise about where it lands. For a fixed final time $T$, $\psi$ maps an initial state to a whole trajectory on $[t_0, T]$:

$$ \psi: \mathbb{R}^n \to \mathcal{C}_n[t_0, T], \quad \psi(\mathbf{x}_0)(t) := \mathbf{x}(t, t_0, \mathbf{x}_0) $$

where $\mathcal{C}_n[t_0, T]$ is the space of continuous functions from $[t_0, T]$ to $\mathbb{R}^n$. We say $\psi$ is continuous at $\mathbf{x}_0$ if:

$$ \forall \varepsilon > 0, \ \exists \delta > 0 \ \text{ s.t. } \ \|\mathbf{x}_0 - \mathbf{z}_0\| < \delta \Rightarrow \|\psi(\mathbf{x}_0) - \psi(\mathbf{z}_0)\|_\infty < \varepsilon $$

On the left, $ \|\cdot\| $ can be any norm on $\mathbb{R}^n$. On the right, the norm is the infinity norm over the whole trajectory:

$$ \|\psi(\mathbf{x}_0) - \psi(\mathbf{z}_0)\|_\infty = \sup_{t_0 \le t \le T} \|\mathbf{x}(t) - \mathbf{z}(t)\| $$

Or, in terms of balls:

$$ \mathbf{z}_0 \in B_\delta(\mathbf{x}_0) \Rightarrow \psi(\mathbf{z}_0) \in B_\varepsilon(\psi(\mathbf{x}_0)) $$

In plain words: if we start close enough, the worst-case difference between the two trajectories over the entire interval stays bounded by $\varepsilon$. This is exactly what “preventing the butterfly effect” means.

Theorem (Continuous Dependence on Initial Conditions): Consider $\dot{\mathbf{x}} = \mathbf{f}(t, \mathbf{x})$, $\mathbf{x} \in \mathbb{R}^n$, where $\mathbf{f}(t, \mathbf{x})$ satisfies the conditions of the global existence and uniqueness theorem. Fix $T \in [t_0, +\infty)$, and suppose $\mathbf{x}(\cdot)$ and $\mathbf{z}(\cdot)$ are two solutions satisfying
$$ \begin{align*} \dot{\mathbf{x}} &= \mathbf{f}(t, \mathbf{x}), \quad \mathbf{x}(t_0) = \mathbf{x}_0 \\ \dot{\mathbf{z}} &= \mathbf{f}(t, \mathbf{z}), \quad \mathbf{z}(t_0) = \mathbf{z}_0 \end{align*} $$
Then $\forall \varepsilon > 0, \ \exists \delta > 0$ such that
$$ \|\mathbf{x}_0 - \mathbf{z}_0\| < \delta \Rightarrow \sup_{t_0 \le t \le T} \|\mathbf{x}(t) - \mathbf{z}(t)\| < \varepsilon $$

Wait, doesn’t the Lorenz attractor contradict this? Not really. The key word is fix $T$: the theorem promises a $\delta$ for every finite interval, but says nothing about how small that $\delta$ has to be. For the Lorenz system, the separation grows like $e^{\lambda t}$, so keeping the trajectories within $\varepsilon$ up to time $T$ requires roughly $ \delta \sim \varepsilon \, e^{-\lambda (T - t_0)} $. Continuity still holds, but $\delta$ shrinks exponentially as the interval gets longer.

Here is why this matters for BVPs. Recall that the periodic steady state is a root of $F(\mathbf{x}_0) = \psi(\mathbf{x}_0)(t_0 + T) - \mathbf{x}_0$. To solve it with Newton’s method, we need the sensitivity of the final state with respect to the initial state, $\partial \psi / \partial \mathbf{x}_0$, which grows like $e^{\lambda T}$ for a chaotic system. Over a long interval, a tiny change in $\mathbf{x}_0$ changes $F$ by an enormous amount, and the problem becomes hopelessly ill-conditioned.

Continuous Dependence on Initial Conditions

We defined function $\psi$ in the last post. As a reminder:

$$ \psi(\mathbf{x}_0)(t) = \mathbf{x}(t) $$

where $\mathbf{x}(t)$ is the solution of the initial value problem

$$ \begin{cases} \dot{\mathbf{x}} = \mathbf{f}(\mathbf{x}), \\ \mathbf{x}(t_0) = \mathbf{x}_0. \end{cases} $$

To prevent small changes in the initial state from causing large deviations in the solution, we require the system to have continuous dependence on initial conditions. Formally, this means that for any $\epsilon > 0$, there exists a $\delta > 0$ such that if $ \|\mathbf{x}_0 - \mathbf{y}_0\| < \delta $, then

$$ \|\psi(\mathbf{x}_0)(t) - \psi(\mathbf{y}_0)(t)\| < \epsilon \quad \text{for all } t \in [t_0, t_0 + T]. $$

Shooting Method

Now we are ready to solve a BVP. Let’s use the following problem as our running example:

$$ \ddot{x} + x = 0, \qquad x(0) = 1, \qquad x(\pi/2) = 2 $$

This one has a closed-form solution, so we can check our answers. Every solution of the ODE has the form $x(t) = A\cos t + B\sin t$. From $x(0) = 1$ we get $A = 1$, and from $x(\pi/2) = 2$ we get $B = 2$:

$$ x(t) = \cos t + 2\sin t, \qquad \dot{x}(0) = 2 $$

📝 Note

A second-order ODE needs two conditions, but not every choice works. Don’t use $x(\pi)$ as the second condition: every solution with $x(0) = 1$ has $x(\pi) = -1$, so that condition gives either no solution or infinitely many.

Idea

Remember that we promised to convert a BVP into an IVP. The shooting method does exactly that. If we knew the initial slope $s = \dot{x}(0)$, this would just be an IVP, and we know how to solve those. So let’s guess $s$, integrate forward, and see by how much we miss the far boundary:

$$ F(s) = x(\pi/2; s) - 2 $$

where $x(\pi/2; s)$ is the solution at $t = \pi/2$ when we start with slope $s$. The BVP is now a root-finding problem: find $s$ such that $F(s) = 0$. Just like aiming a cannon: fire, see where the ball lands, adjust the angle, fire again.

For periodic steady state, the idea is the same, except that the unknown is the whole initial state, and the miss is measured against the initial state itself:

$$ F(\mathbf{x}_0) = \psi(\mathbf{x}_0)(t_0 + T) - \mathbf{x}_0 = 0 $$

Steps

Rewrite the problem as a first-order system with $y_1 = x$ and $y_2 = \dot{x}$:
$$ \dot{y}_1 = y_2, \qquad \dot{y}_2 = -y_1, \qquad \mathbf{y}(0) = [1, s]^T $$
Guess $s_0 = 0$. Integrating gives $x(\pi/2) = 0$, so $F_0 = -2$.
Guess $s_1 = 1$. Integrating gives $x(\pi/2) = 1$, so $F_1 = -1$.
Secant update:
$$ s_2 = s_1 - F_1 \frac{s_1 - s_0}{F_1 - F_0} = 1 - (-1)\frac{1}{1} = 2 $$
$F(2) \approx 0$, so we have converged to $\dot{x}(0) = 2$.

Because the ODE is linear, $F(s)$ is linear in $s$, and the secant method converges in one step. A nonlinear ODE would take a few iterations.

Code

Let’s reuse the RK4 integrator from the last post:

 1T, X0, XT = np.pi / 2, 1.0, 2.0 # x(0) = 1, x(T) = 2
 2f = lambda y, t: np.array([y[1], -y[0]]) # y = [x, x']
 3
 4def F(s, h=T / 200):
 5 """Integrate with x(0) = 1, x'(0) = s; return the miss at the far boundary."""
 6 t, y = integrate(rk4_step, f, np.array([X0, s]), 0.0, T, h)
 7 return y[-1, 0] - XT
 8
 9def shoot(s0=0.0, s1=1.0, tol=1e-10):
10 F0, F1 = F(s0), F(s1)
11 history = [(s0, F0), (s1, F1)]
12 while abs(F1) > tol:
13 s0, F0, s1 = s1, F1, s1 - F1 * (s1 - s0) / (F1 - F0) # secant step
14 F1 = F(s1)
15 history.append((s1, F1))
16 return s1, history

Running it, the three guesses give:

Guess	$s$	$F(s)$
0	0	−2.000
1	1	−1.000
2	1.999999999951	$-1.8 \times 10^{-15}$

The remaining $5 \times 10^{-11}$ error in $s$ comes from RK4’s truncation error with $h = T/200$, not from the shooting itself.

Bear in mind that for a nonlinear system with many states, we’d use Newton’s method instead of the secant method, which requires the Jacobian $\partial \psi / \partial \mathbf{x}_0$, the sensitivity of the final state to the initial state. This is exactly where the butterfly effect bites: for a chaotic (or just very sensitive) system, this Jacobian grows exponentially with the interval length.

Harmonic Balance

Idea

The shooting method works in the time domain. Harmonic balance goes the other way: assume the solution is a truncated Fourier series, substitute it into the ODE, and require the coefficient of each harmonic in the residual to vanish. This turns the ODE into a set of algebraic equations for the Fourier coefficients, plus the frequency $\omega$ if it is unknown.

Steps

Ansatz with $N$ harmonics:
$$ x(t) = a_0 + \sum_{k=1}^{N} \left[ a_k \cos(k\omega t) + b_k \sin(k\omega t) \right] $$
Substitute. Since $\ddot{x} = -\sum_k (k\omega)^2 [a_k\cos(k\omega t) + b_k\sin(k\omega t)]$, we have
$$ \ddot{x} + x = a_0 + \sum_{k=1}^{N} (1 - k^2\omega^2)\left[a_k\cos(k\omega t) + b_k\sin(k\omega t)\right] $$
Balance each harmonic:
- DC: $a_0 = 0$
- $k$-th harmonic: $(1 - k^2\omega^2) a_k = 0$ and $(1 - k^2\omega^2) b_k = 0$
A nonzero solution needs $k\omega = 1$ for some $k$. Taking $k = 1$ gives $\omega = 1$, and every other harmonic vanishes:
$$ x(t) = A\cos t + B\sin t $$
Note that the balance fixes the frequency, but not the amplitudes, because the ODE is linear and unforced.
Apply the boundary conditions: $x(0) = A = 1$ and $x(\pi/2) = B = 2$, so
$$ x(t) = \cos t + 2\sin t $$

The result is exact because the true solution happens to be a single harmonic. In a nonlinear ODE, e.g. the Duffing equation $\ddot{x} + x + \varepsilon x^3 = 0$, the harmonics couple to each other ($\cos^3 t$ produces $\cos 3t$), so the balance equations become nonlinear, and $\omega$ depends on the amplitude.

Code

Numerically, we stack the unknowns $[\omega, a_0, \ldots, a_N, b_1, \ldots, b_N]$ into one vector, and solve the harmonic balance equations together with the two boundary conditions. There are more equations than unknowns, so we solve it in the least-squares sense with Gauss-Newton:

 1N = 3 # number of harmonics
 2
 3def unpack(p): # p = [w, a0..aN, b1..bN]
 4 return p[0], p[1:N + 2], np.r_[0.0, p[N + 2:]]
 5
 6def x_hb(t, a, b, w):
 7 k = np.arange(N + 1)[:, None]
 8 return (a[:, None] * np.cos(k * w * t) + b[:, None] * np.sin(k * w * t)).sum(0)
 9
10def residual(p):
11 w, a, b = unpack(p)
12 k = np.arange(N + 1)
13 hb = np.r_[(1 - (k * w) ** 2) * a, ((1 - (k * w) ** 2) * b)[1:]] # each harmonic of x'' + x
14 bc = [x_hb(np.array([0.0]), a, b, w)[0] - X0, # boundary conditions
15 x_hb(np.array([T]), a, b, w)[0] - XT]
16 return np.r_[hb, bc]
17
18def gauss_newton(r, p, tol=1e-14, max_iter=100):
19 """Least-squares Newton: p <- p - lstsq(J, r(p)), finite-difference Jacobian."""
20 for it in range(max_iter):
21 rp = r(p)
22 J = np.empty((rp.size, p.size))
23 for j in range(p.size):
24 dp = np.zeros_like(p); dp[j] = 1e-7
25 J[:, j] = (r(p + dp) - rp) / dp[j]
26 step = np.linalg.lstsq(J, rp, rcond=None)[0]
27 p = p - step
28 if np.max(np.abs(step)) < tol:
29 break
30 return p, it + 1
31
32p0 = np.r_[0.9, 0.5 * np.ones(N + 1), 0.5 * np.ones(N)] # rough initial guess
33p, iters = gauss_newton(residual, p0)

Starting from a rough guess of $\omega = 0.9$ and all coefficients at 0.5, it converges in 8 iterations to $\omega = 1$, $a = [0, 1, 0, 0]$ and $b = [0, 2, 0, 0]$. The 2nd and 3rd harmonics come out as zero, and the reconstructed waveform matches $\cos t + 2\sin t$ to machine precision.

From BVP to Periodic Steady State: Two Practical Scenarios

So far, our example came with two nice boundary conditions handed to us. Real PSS problems are not that kind. When we ask the simulator for the periodic steady state of a circuit, we usually don’t know the state at the start of the period, and for an oscillator we don’t even know how long the period is. Let’s look at how each of these situations is turned into something the shooting method and harmonic balance can solve.

Scenario 1: Driven Circuit, Unknown Initial State

This is the typical case: a circuit driven by a periodic source with a known period $T$, like a clock or an LO. We want the steady state, but we have no idea what the state looks like at the start of a period; that’s exactly what we’re trying to find.

The trick is that periodicity itself replaces the missing initial condition:

$$ \mathbf{x}(t_0 + T) = \mathbf{x}(t_0) $$

That’s $n$ equations for the $n$ unknown components of $\mathbf{x}_0 = \mathbf{x}(t_0)$, which is a well-posed problem.

Shooting: we solve $F(\mathbf{x}_0) = \psi(\mathbf{x}_0)(t_0 + T) - \mathbf{x}_0 = 0$ with Newton’s method. Each Newton iteration integrates one period of transient, and the Jacobian is
$$ \frac{\partial F}{\partial \mathbf{x}_0} = \frac{\partial \psi(\mathbf{x}_0)(t_0 + T)}{\partial \mathbf{x}_0} - I $$
The first term is the sensitivity of the end-of-period state to the start-of-period state, also known as the monodromy matrix. For a linear circuit, $F$ is linear, and Newton converges in one step.
Harmonic balance: here periodicity comes for free. Every term of a Fourier series with fundamental $\omega = 2\pi/T$ is already $T$-periodic, so there is no initial condition to find at all; we only solve for the coefficients. For a linear circuit, the harmonics don’t even talk to each other, and harmonic balance reduces to an AC analysis at each harmonic: $V_k = Z(jk\omega) I_k$.

Example. Let’s go back to the RC circuit from the last post ($ R = 10\,\text{k}\Omega $, $ C = 1\,\text{pF} $, $\tau = 10$ ns), and drive it with a square-wave current that switches between 0 and 100 µA at 1 GHz. This is the “RC driven by a clock” from the beginning of this post. The capacitor charges during the first half of the period and discharges during the second half, and in steady state the two exponential segments must connect. Solving for the end points gives the closed-form answer:

$$ v_{min} = IR\frac{e^{-a}}{1 + e^{-a}}, \qquad v_{max} = IR\frac{1}{1 + e^{-a}}, \qquad a = \frac{T}{2\tau} $$

i.e. $v_{min} = 487.503$ mV, $v_{max} = 512.497$ mV, a ripple of 24.99 mV around 500 mV.

The numbers:

Plain transient: starting from 0 V, it takes 131 periods before the waveform settles within 1 µV of the steady state, since the settling is governed by $\tau = 10T$.
Shooting: Newton converges after integrating only 6 periods in total (including the extra integrations for the finite-difference Jacobian), and lands on the exact $v_{min}$ to within $2 \times 10^{-15}$ V.
Harmonic balance: the square wave has sharp edges, so the waveform has a corner at every edge, and the Fourier coefficients decay slowly. The worst-case error over the period is 2.37 mV with 1 harmonic, 0.84 mV with 5, 0.195 mV with 25, and still 0.050 mV with 101 harmonics.

Note that the capacitor voltage itself is not a square wave: with $\tau = 10T$, the RC integrates the square-wave current into a triangle with 25 mV of ripple. The trouble for harmonic balance is the sharp corner at every source edge, which the zoom on the right shows: with 1 harmonic the corner is badly rounded off, with 5 it’s better, and only around 25 harmonics does it follow the exact solution closely.

⚠️ Warning

The source edge at $T/2$ is a discontinuity, and an RK4 step that straddles it applies the wrong current for part of the step. In my first attempt, this alone left an error of about 0.5 mV in the steady state. The fix is what every circuit simulator does with breakpoints: end a time step exactly at the edge, and restart the integration from there.

Scenario 2: Autonomous Circuit, Unknown Period

Now consider an oscillator: a ring oscillator, an LC VCO, or anything without a periodic input. Two new problems show up:

The period $T$ is unknown. It’s set by the circuit itself, so $T$ becomes an extra unknown, and we now have $n + 1$ unknowns but only $n$ periodicity equations.
The phase is arbitrary. Since nothing drives the circuit, any time-shifted copy of a periodic solution is also a periodic solution, so there are infinitely many of them.

Both problems are solved together by adding one phase condition, which pins down where the period starts. That gives $n + 1$ equations for $n + 1$ unknowns. On top of that, the DC operating point $\mathbf{x}(t) = \mathbf{x}_{DC}$ trivially satisfies periodicity for any $T$. This is why oscillator PSS needs a good initial guess, and why simulators typically run a short transient first to get the oscillation going.

Shooting: the unknowns are $(\mathbf{x}_0, T)$. A convenient phase condition is to start the period at a peak of one state variable, i.e. $\dot{x}_j(t_0) = 0$.
Harmonic balance: the unknowns are the Fourier coefficients plus $\omega$. A convenient phase condition is to set the sine coefficient of the fundamental of one variable to zero, $b_1 = 0$, which lines up its fundamental with a cosine.

Example. The classic model of an oscillator is the Van der Pol oscillator:

$$ \ddot{x} - \mu (1 - x^2) \dot{x} + x = 0 $$

which is (in normalized units) an LC tank in parallel with a nonlinear conductance $-g_1 v + g_3 v^3$, a crude model of a cross-coupled pair: negative resistance at small amplitude to start the oscillation, and compression at large amplitude to limit it. We take $\mu = 1$, where there is no closed-form solution.

For shooting, the phase condition $\dot{x}(0) = 0$ means $x(0) = A$ is the amplitude, and the unknowns are just $[A, T]$:

1def shoot_vdp(A0=1.5, T0=2 * np.pi):
2 """Unknowns [A, T]; phase condition x'(0) = 0, i.e. start the period at a peak x(0) = A."""
3 def F(p):
4 A, T = p
5 y_end = integrate(rk4_step, f_vdp, np.array([A, 0.0]), 0.0, T, T / N_STEPS)[1][-1]
6 return np.array([y_end[0] - A, y_end[1] - 0.0])
7 return newton(F, np.array([A0, T0]))

Starting from a guess of $A = 1.5$ and $T = 2\pi$, it converges to $A = 2.0086$ and $T = 6.6633$, which matches the known period for $\mu = 1$.

For harmonic balance, the nonlinear term $x^2 \dot{x}$ is awkward to expand by hand. Instead, we evaluate the residual at sample points in the time domain, and project it back onto the harmonics with a Fourier transform. This time-domain/frequency-domain round trip is also how harmonic balance simulators handle nonlinear devices. The results:

Harmonics $N$	Period $T$	Error vs. shooting
1	6.283185	$3.8 \times 10^{-1}$
3	6.665495	$2.2 \times 10^{-3}$
5	6.663382	$9.5 \times 10^{-5}$
7	6.663287	$1.8 \times 10^{-7}$
11	6.663287	$1.1 \times 10^{-9}$

With a single harmonic, harmonic balance gives exactly $T = 2\pi$ and an amplitude of 2. This is the describing function result that you may remember from analyzing oscillators by hand: it assumes the waveform is a pure sinusoid, and therefore misses the frequency shift caused by the harmonics. With more harmonics, the error drops very quickly, because the waveform is smooth.

Note that the two methods use different phase conditions, so I aligned the HB waveforms to the peak at $t = 0$ in the left plot.

Summary

	Driven circuit	Autonomous circuit
Known	Period $T$	Nothing
Unknowns	$\mathbf{x}_0$	$\mathbf{x}_0$ and $T$
Equations	Periodicity ($n$)	Periodicity ($n$) + phase condition ($1$)
Shooting	Newton on $\psi(\mathbf{x}_0)(T) - \mathbf{x}_0$	Same, with $T$ as an unknown and e.g. $\dot{x}_j(0) = 0$
Harmonic balance	Coefficients at known $\omega$	Coefficients and $\omega$, with e.g. $b_1 = 0$

The two examples also show the trade-off between the two methods nicely. The switching waveform of the driven RC is exact with shooting, but with harmonic balance the error only falls like $1/N$: going from 25 to 101 harmonics buys just a 4× improvement. The smooth waveform of the Van der Pol oscillator is captured to $10^{-9}$ with 11 harmonics.

Numerical Methods: Transient Simulation

Thu, 01 Oct 2026 20:58:32 -0700

Although I’m an IC designer, it’s sometimes necessary to understand what my EDA tool is doing under the hood, especially when the simulation takes so long for extracted simulation. You might wonder why the extracted simulation takes so much longer than the schematic simulation by just adding a bunch of parasitic elements. The reason lies in the numerical methods used to solve the underlying differential equations, which become more complex and computationally intensive as the circuit model becomes more detailed.

I will discuss some of the commonly used numerical methods used in transient simulation of circuits in this post.

Initial Value Problem (IVP)

Let’s say you want to measure an RC circuit’s settling time after it’s subjected to a step current input, and you ask the simulator to simulate for 50ns. You have just created an initial value problem (IVP), where the initial condition is the circuit’s state at the beginning of the simulation, and the goal is to find the circuit’s state at subsequent time points.

Mathematically, an IVP can be expressed as:

$$ \frac{d\mathbf{x}(t)}{dt} = \mathbf{f}(\mathbf{x}(t), t), \quad \mathbf{x}(t_0) = \mathbf{x}_0 $$

where $\mathbf{x}(t)$ represents the state vector of the circuit at time $t$, $\mathbf{f}(\mathbf{x}(t), t)$ represents the system of differential equations governing the circuit, $t_0$ is the initial time, and $\mathbf{x}_0$ is the initial state of the circuit.

The IVP problem can also be interpreted as we performing a mapping from the initial state $\mathbf{x}_0$ at time $t_0$ to the solution vector $\mathbf{x}(t)$ at subsequent time points. In this case, people usually define it in following form:

Given the system
$$ \dot{x} = f(x, t), \quad x(t_0) = x_0 \in \mathbb{R} $$
Define mapping $ \psi: \mathbb{R}^n \to \mathbb{C}^n $ such that
$$ \psi(x_0)(t) = x(t, t_0, x_0) \quad \text{for} \quad t \ge t_0 $$

A system must be Lipschitz continuous with respect to the state vector $\mathbf{x}$ to guarantee the existence and uniqueness of the solution to the IVP. This can be either local or global, and whether a solution exists globally or only locally also depends on the Lipschitz condition. This is also known as the Picard-Lindelöf theorem, also known as the uniqueness and existence theorem for ordinary differential equations.

Forward Euler Method

The Forward Euler Method is one of the simplest and most widely used numerical methods for solving IVPs. It is an explicit method, meaning that the state at the next time step is computed directly from the state at the current time step. The method can be expressed as:

$$ x_{k+1} = x_k + h f(x_k, t_k) $$

where $x_k$ is the state at time $t_k$, $h$ is the time step size, and $f(x_k, t_k)$ is the derivative of the state with respect to time evaluated at the current state and time.

The Forward Euler Method is conditionally stable, meaning that the choice of time step $h$ can affect the accuracy and stability of the solution. For stiff problems, where the system exhibits rapid changes in some components, the Forward Euler Method may require very small time steps to maintain stability, making it computationally expensive.

Here is a python code snippet demonstrating the Forward Euler Method:

 1import numpy as np
 2
 3def forward_euler(f, x0, t0, t_end, h):
 4 """Solve dx/dt = f(x, t), x(t0) = x0 with x_{k+1} = x_k + h * f(x_k, t_k)."""
 5 n_steps = int(round((t_end - t0) / h))
 6 t = t0 + h * np.arange(n_steps + 1)
 7 x = np.zeros((n_steps + 1,) + np.shape(x0))
 8 x[0] = x0
 9 for k in range(n_steps):
10 x[k + 1] = x[k] + h * f(x[k], t[k])
11 return t, x
12
13# RC circuit driven by a step current: C dv/dt = I - v/R
14R, C, I = 10e3, 1e-12, 100e-6 # 10 kOhm, 1 pF, 100 uA -> tau = RC = 10 ns
15tau = R * C
16f = lambda v, t: (I - v / R) / C
17
18t, v = forward_euler(f, x0=0.0, t0=0.0, t_end=50e-9, h=0.1 * tau)
19
20v_exact = I * R * (1 - np.exp(-t / tau))
21print(f"final value: {v[-1]:.4f} V (exact {v_exact[-1]:.4f} V), "
22 f"max error: {np.max(np.abs(v - v_exact)) * 1e3:.2f} mV")

With $h = 0.1\tau$, the final value is 0.9948 V against the exact 0.9933 V, and the worst-case error along the waveform is about 19 mV. Not bad for three lines of math.

Stability: the Test Equation

What does “conditionally stable” actually mean? The standard way to answer this is to feed the method the simplest possible linear system, also known as the test equation:

$$ \dot{x} = \lambda x, \quad \lambda \in \mathbb{C} $$

Most linear circuits can be diagonalized into a bunch of these, with $\lambda$ being the poles of the circuit. Applying Forward Euler:

$$ x_{k+1} = x_k + h \lambda x_k = (1 + h\lambda) x_k $$

so after $k$ steps we have $x_k = (1 + h\lambda)^k x_0$. Let’s define $z = h\lambda$, and call $R(z) = 1 + z$ the stability function of the method. The numerical solution only decays (like a stable circuit should) if $|R(z)| < 1$, i.e. $z$ has to sit inside a unit circle centered at $-1$.

For our RC circuit, $\lambda = -1/\tau$, therefore:

$$ |1 - h/\tau| < 1 \Rightarrow h < 2\tau $$

If we pick a time step larger than $2\tau$, the simulation blows up, even though the circuit itself is perfectly stable. Bear in mind that this limit is set by the fastest pole in the system, not the one we care about.

Backward Euler Method

What if we evaluate the derivative at the end of the step instead of the beginning?

$$ x_{k+1} = x_k + h f(x_{k+1}, t_{k+1}) $$

We realize that $x_{k+1}$ now shows up on both sides of the equation, which makes Backward Euler an implicit method: we can’t just compute the next state, we have to solve for it. For a nonlinear $f$, this is done using Newton-Raphson iteration at every time step, which is exactly what your circuit simulator is doing.

Applying the test equation:

$$ x_{k+1} = x_k + h\lambda x_{k+1} \Rightarrow R(z) = \frac{1}{1 - z} $$

$|R(z)| < 1$ whenever $\Re(z) < 0$, meaning that Backward Euler is stable for any step size, as long as the circuit itself is stable. This property is called A-stability. On top of that, $R(z) \to 0$ as $z \to -\infty$, so very fast poles are damped out immediately instead of lingering around; this stronger property is called L-stability.

Since the time-marching loop is the same for every method, from now on let’s split it out, and only write down how one step is taken:

 1def integrate(step, f, x0, t0, t_end, h):
 2 """March x_{k+1} = step(f, x_k, t_k, h) from t0 to t_end."""
 3 n_steps = int(round((t_end - t0) / h))
 4 t = t0 + h * np.arange(n_steps + 1)
 5 x = np.zeros((n_steps + 1,) + np.shape(x0))
 6 x[0] = x0
 7 for k in range(n_steps):
 8 x[k + 1] = step(f, x[k], t[k], h)
 9 return t, x
10
11def forward_euler_step(f, x, t, h):
12 return x + h * f(x, t)

For the implicit methods, we need a small Newton-Raphson solver. Here I use a finite-difference Jacobian to keep it generic:

 1def newton(g, x, tol=1e-12, max_iter=50):
 2 """Solve g(x) = 0 with Newton-Raphson and a finite-difference Jacobian."""
 3 x = np.atleast_1d(np.asarray(x, dtype=float)).copy()
 4 for _ in range(max_iter):
 5 gx = np.atleast_1d(g(x))
 6 J = np.empty((x.size, x.size))
 7 for j in range(x.size):
 8 dx = np.zeros_like(x)
 9 dx[j] = 1e-8 * max(1.0, abs(x[j]))
10 J[:, j] = (np.atleast_1d(g(x + dx)) - gx) / dx[j]
11 delta = np.linalg.solve(J, -gx)
12 x += delta
13 if np.max(np.abs(delta)) < tol * max(1.0, np.max(np.abs(x))):
14 break
15 return x
16
17def backward_euler_step(f, x, t, h):
18 g = lambda y: y - x - h * f(y, t + h) # x_{k+1} appears on both sides
19 return newton(g, x).reshape(np.shape(x))

The price we pay: every time step now requires building a Jacobian and solving a linear system, instead of a single function evaluation. And Backward Euler is still only first-order accurate, just like Forward Euler.

Trapezoidal Method

Forward Euler uses the slope at the beginning of the step, Backward Euler uses the slope at the end. A natural idea is to take the average of the two:

$$ x_{k+1} = x_k + \frac{h}{2} \left[ f(x_k, t_k) + f(x_{k+1}, t_{k+1}) \right] $$

This is the trapezoidal rule of integration applied to the IVP, and it’s still implicit. The stability function is:

$$ R(z) = \frac{1 + z/2}{1 - z/2} $$

Here is the interesting part: for $\Re(z) < 0$, the numerator is always smaller than the denominator in magnitude, so the trapezoidal method is also A-stable. On the imaginary axis, $|R(j\omega h)| = 1$ exactly, which means a lossless oscillator stays lossless. And it’s second-order accurate, which is a big upgrade over both Euler methods at almost the same cost.

1def trapezoidal_step(f, x, t, h):
2 fk = f(x, t)
3 g = lambda y: y - x - h / 2 * (fk + f(y, t + h))
4 return newton(g, x).reshape(np.shape(x))

⚠️ Warning

The trapezoidal method is A-stable but not L-stable: as $z \to -\infty$, $R(z) \to -1$. A very fast pole (for example, a tiny parasitic RC) is therefore not damped out; instead its component flips sign every time step. This is the infamous “trapezoidal ringing” you may have seen in SPICE waveforms, typically right after a sharp edge. This is also why simulators commonly mix in Backward Euler or Gear (BDF2) steps around discontinuities.

Heun’s Method (Improved Euler)

What if we want the averaging idea of the trapezoidal method, but don’t want to solve an implicit equation? We can simply predict $x_{k+1}$ using Forward Euler first, then use the prediction to evaluate the slope at the end of the step:

$$ \begin{align*} \tilde{x}_{k+1} &= x_k + h f(x_k, t_k) \\ x_{k+1} &= x_k + \frac{h}{2} \left[ f(x_k, t_k) + f(\tilde{x}_{k+1}, t_{k+1}) \right] \end{align*} $$

This is called Heun’s method, or the improved Euler method. It’s a predictor-corrector scheme: explicit, two function evaluations per step, and second-order accurate.

1def heun_step(f, x, t, h):
2 k1 = f(x, t)
3 x_pred = x + h * k1 # predictor: one Forward Euler step
4 k2 = f(x_pred, t + h)
5 return x + h / 2 * (k1 + k2) # corrector: average the two slopes

However, being explicit, there is no free lunch on stability. The stability function is:

$$ R(z) = 1 + z + \frac{z^2}{2} $$

which, just like Forward Euler, requires $h < 2\tau$ on the negative real axis. Worse, on the imaginary axis $|R(j\omega h)|^2 = 1 + (\omega h)^4 / 4 > 1$, so an LC tank slowly gains energy.

Runge-Kutta Methods

Heun’s method is in fact the simplest member of a larger family: the Runge-Kutta methods. The idea is to sample the slope at several intermediate points within one step, and combine them with carefully chosen weights so that the error terms cancel up to a certain order. The most famous one is the classical fourth-order Runge-Kutta, usually just called RK4:

$$ \begin{align*} k_1 &= f(x_k, t_k) \\ k_2 &= f(x_k + \tfrac{h}{2} k_1, t_k + \tfrac{h}{2}) \\ k_3 &= f(x_k + \tfrac{h}{2} k_2, t_k + \tfrac{h}{2}) \\ k_4 &= f(x_k + h k_3, t_k + h) \\ x_{k+1} &= x_k + \frac{h}{6} (k_1 + 2k_2 + 2k_3 + k_4) \end{align*} $$

1def rk4_step(f, x, t, h):
2 k1 = f(x, t)
3 k2 = f(x + h / 2 * k1, t + h / 2)
4 k3 = f(x + h / 2 * k2, t + h / 2)
5 k4 = f(x + h * k3, t + h)
6 return x + h / 6 * (k1 + 2 * k2 + 2 * k3 + k4)

Its stability function is the Taylor series of $e^z$ truncated at the fourth order:

$$ R(z) = 1 + z + \frac{z^2}{2} + \frac{z^3}{6} + \frac{z^4}{24} $$

The stability region is noticeably larger than Forward Euler’s (about $h < 2.79\tau$ on the negative real axis), and the accuracy is fourth order. But it’s still bounded: RK4 is explicit, and no explicit method can be A-stable.

Comparison

Putting everything together:

Method	Type	Order	Stability function $R(z)$	A-stable	Cost per step
Forward Euler	Explicit	1	$1 + z$	No	1 evaluation of $f$
Backward Euler	Implicit	1	$\frac{1}{1 - z}$	Yes (also L-stable)	Newton solve
Trapezoidal	Implicit	2	$\frac{1 + z/2}{1 - z/2}$	Yes (not L-stable)	Newton solve
Heun	Explicit	2	$1 + z + \frac{z^2}{2}$	No	2 evaluations of $f$
RK4	Explicit	4	$1 + z + \ldots + \frac{z^4}{24}$	No	4 evaluations of $f$

And the pros and cons:

Forward Euler: dead simple and cheap per step. However it’s only first order, and the step size is limited by the fastest pole. It also adds energy to oscillators.
Backward Euler: unconditionally stable and kills fast parasitic modes immediately, which makes it extremely robust. The downside is that it’s only first order, and it adds artificial damping: an oscillator rings down even if it’s lossless.
Trapezoidal: second order, A-stable, and preserves the amplitude of lossless oscillations. But it is not L-stable, which leads to the numerical ringing on stiff circuits, and it still needs a Newton solve every step.
Heun: second order without any implicit solve, so it’s a cheap accuracy upgrade over Forward Euler. Stability is just as limited, though, and it slowly pumps energy into oscillators.
RK4: very accurate per step for smooth problems, and still explicit. But four function evaluations per step, and the stability region is bounded, so a stiff circuit will force tiny steps anyway.

An Example with a Closed-Form Solution

Talk is cheap. Let’s put all five methods side by side on circuits where we know the exact answer.

LC Tank

Consider an ideal LC tank with $ L = 1\,\text{nH} $, $ C = 1\,\text{pF} $, initially charged to 1 V:

$$ \begin{align*} C \frac{dv}{dt} &= -i \\ L \frac{di}{dt} &= v \end{align*} $$

with the closed-form solution $v(t) = \cos(\omega_0 t)$, $\omega_0 = 1/\sqrt{LC}$. The poles sit right on the imaginary axis, so this is the perfect test for whether a method adds or removes energy. We simulate five periods with $h = T_0/16$:

After five periods ($v(5T_0) = 1$ V exactly):

Method	$v(5T_0)$	Energy / $E_0$
Forward Euler	+28.05 V	96131
Backward Euler	+0.0003 V	≈ 0
Trapezoidal	+0.923 V	1.0000
Heun	+0.912 V	1.61
RK4	+0.998 V	0.996

This matches what the stability functions told us. Forward Euler explodes, because $|1 + j\omega_0 h| > 1$ and the energy grows by a fixed factor every step. Backward Euler damps the tank to nothing within five periods. The trapezoidal method keeps the energy exactly, but the waveform slowly drifts in phase, which is why $v(5T_0)$ is off. Heun slowly gains energy, and RK4 is the closest to the exact solution with a tiny bit of damping.

RC Step Response with a Large Step

Now back to our RC circuit from the beginning ($\tau = 10$ ns), but this time we deliberately use a step size $h = 2.5\tau$, which is larger than what Forward Euler can handle:

On the left:

Forward Euler and Heun both diverge, since $h > 2\tau$. After 250 ns, they are at −56.7 V and −127.4 V respectively.
RK4 is still stable ($2.5 < 2.79$), but it’s inaccurate: after one step it’s at 0.352 V while the exact answer is 0.918 V. Stable doesn’t mean correct.
Backward Euler settles without any overshoot, just a bit slower than the real circuit.
Trapezoidal overshoots to 1.111 V on the first step and then rings around the final value: exactly the “trapezoidal ringing” from earlier, since $R(-2.5) = -0.11$ is negative.

On the right, we sweep the step size on the LC tank and plot the maximum error over one period. The slopes on the log-log plot give the order of each method: 1.0 for both Euler methods, 2.0 for trapezoidal and Heun, and 4.0 for RK4, exactly as advertised. Notice also that trapezoidal and Heun share the same slope, but trapezoidal has the smaller error constant.

Back to the Question: Why Is My Extracted Simulation So Slow?

Now we are able to answer the question from the beginning. Circuits are stiff: the time constants of a post-layout netlist span many orders of magnitude, from the slow settling we care about down to the femtosecond-level RC of a tiny parasitic. With an explicit method, the step size would be dictated by that fastest parasitic pole, so explicit methods like RK4 are rarely used in circuit simulators.

Instead, SPICE-family simulators use implicit methods, typically trapezoidal and Gear (BDF2), together with Backward Euler around discontinuities. In Spectre, you can choose among them using the method option of the transient analysis. Implicit methods remove the stability limit, but every time step now requires several Newton-Raphson iterations, and every iteration requires factorizing a matrix whose size grows with the number of nodes. Adding thousands of parasitic nodes therefore makes each time step much more expensive, and the fast parasitic dynamics still force the simulator to take smaller steps to keep the accuracy within tolerance.