Control¶
Steady-state LQR action selection, the action-side dual of the Kalman filter. Under a fixed sensor and fixed dynamics noise it is the action expected free energy chooses 1. The Agent builds one when you give it a StateGoal. Build one yourself to hand the Agent an LQRSelector beside a state Preference.
LQRController ¶
LQRController(
model: LinearGaussianModel,
*,
precision: ArrayLike,
effort: ArrayLike,
tol: float = 1e-12,
max_iter: int = 1000,
)
Steady-state LQR action selection — the action-side dual of the filter.
It front-loads action: solve the control Riccati once for L∞, then
action = -L∞·(mean − target) on every step. The estimation Riccati is its dual.
A steady-state KalmanBackend solves that one once for
K∞, and the pair is steady-state LQG. The Agent does not
freeze its filter. It recomputes the Kalman gain every step, so its loop is this
controller on a per-step filter.
The load-bearing claim (ADR-003) is that LQR is active inference here, not a substitute for it. For a fixed linear-Gaussian sensor and fixed dynamics noise the covariance recursion is control-independent, so Expected Free Energy's epistemic term is identical for every action and drops out of the argmin; EFE-minimising selection reduces to its pragmatic term, and the pragmatic term under a Gaussian preference is a quadratic cost whose optimum is exactly LQR. The epistemic term re-enters once the sensor noise or the dynamics noise depends on the state (ADR-006).
The two cost matrices are named for the preference they encode, not by LQR's
traditional letters, which already mean the noise covariances on the model. That
is the collision ADR-003 warns about. The names are the ones a
StateGoal carries, precision and effort, and an
Agent hands a goal's two straight through.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
LinearGaussianModel
|
The linear-Gaussian model to act in. Must carry a |
required |
precision
|
ArrayLike
|
How sharply the agent prefers the target, an |
required |
effort
|
ArrayLike
|
How much action costs, a |
required |
tol
|
float
|
Absolute tolerance on successive cost-to-go iterates; convergence is declared when they stop moving by more than this. |
1e-12
|
max_iter
|
int
|
Iteration cap before the Riccati recursion is declared to have failed to converge. |
1000
|
Raises:
| Type | Description |
|---|---|
TypeError
|
If |
ValueError
|
If the model has no |
RuntimeError
|
If the control Riccati does not converge within |
Source code in src/cpomdp/control.py
model
property
¶
The model the gain was solved for, fixed at construction.
action ¶
The action that drives the estimated state toward target.
One matrix-vector product, -L∞·(mean − target). All the work was
front-loaded into L∞ at construction, so there is no optimisation in
the loop. The mean − target shift turns the regulator (which drives its
state to zero) into a controller that drives the state to target.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
mean
|
ArrayLike
|
The current belief mean — the best estimate of the state,
shape |
required |
target
|
ArrayLike
|
The state to steer toward, shape |
required |
Returns:
| Type | Description |
|---|---|
Float64[Array, p]
|
The action, shape |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Source code in src/cpomdp/control.py
__repr__ ¶
The steady-state gain, the one number the Riccati solve leaves behind.
The finite-horizon schedule¶
In plain terms
A controller steers a system toward a goal. LQRController builds one by solving a
puzzle that asks how hard to push right now, given where the state is and given
forever to reach the goal. It answers by repeating one calculation until the answer
stops changing. That final answer is the steady-state gain, and the controller keeps
only it.
The intermediate answers were never junk. After one repetition the answer is how hard
to push with one step left. After five, with five steps left. lqr_schedule runs
the same calculation a chosen number of times, H, and keeps every answer. The
result is a schedule: the right push with H steps left, then with H − 1, down to
the last one.
The expected-free-energy planner looks a fixed number of steps ahead, applies the
first action and plans again. A planner like that is using the "H steps left" rule
at every step. The "forever" rule is close to it when H is large and visibly
different when H is small. Held against the forever rule, the planner shows a gap
that fades as H grows and looks exactly like a bug. first_gain is the matching
rule, so no such gap is manufactured.
One convention to know. The cost is charged on the state each action arrives at, and nothing is charged after the last action. The planner's pragmatic term does the same accounting, so the two line up without adjustment.
lqr_schedule ¶
lqr_schedule(
model: LinearGaussianModel,
*,
horizon: int,
precision: ArrayLike,
effort: ArrayLike,
) -> LQRSchedule
Run the control Riccati recursion backward over horizon steps.
The same Bellman step LQRController iterates to a fixed point, run a declared
number of times from a zero terminal cost and with every gain kept. With j
steps remaining and W = precision + P_{j−1} the cost of what remains::
L_j = (effort + Bᵀ W B)⁻¹ (Bᵀ W A)
P_j = Aᵀ W A − (Aᵀ W B) L_j
starting from P_0 = 0. At horizon = 1, W is precision and the
gain is the one-step regulator. The Control page of the API reference opens the
same account in plain terms.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
LinearGaussianModel
|
The linear-Gaussian model to act in. Must carry a |
required |
horizon
|
int
|
|
required |
precision
|
ArrayLike
|
The stage cost on the state, an |
required |
effort
|
ArrayLike
|
The stage cost on the action, a |
required |
Returns:
| Type | Description |
|---|---|
LQRSchedule
|
The schedule, with |
LQRSchedule
|
|
Raises:
| Type | Description |
|---|---|
TypeError
|
If |
ValueError
|
If |
Source code in src/cpomdp/control.py
LQRSchedule
dataclass
¶
LQRSchedule(
model: LinearGaussianModel,
precision: Float64[Array, "n n"],
effort: Float64[Array, "p p"],
gains: Float64[Array, "H p n"],
costs_to_go: Float64[Array, "H+1 n n"],
)
The gain schedule and cost-to-go of an H-step regulator.
The stage cost is charged on the state each action arrives at, and nothing is
charged after the last one: the terminal cost is zero. That is the sum a
receding-horizon planner scores over its lookahead, so first_gain is the gain
such a planner applies at every step, and it is what a comparison against one
has to use. LQRController.gain is its limit as H grows and differs from it
at every finite H, by an amount that shrinks with H and reads as an error
when the horizons are not matched.
The schedule carries the model and the two costs it was built from, so anything priced against it reads the same costs and noise the recursion did.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
LinearGaussianModel
|
The model the schedule regulates. |
required |
precision
|
Float64[Array, 'n n']
|
The stage cost on the state it was built with. (LQR's |
required |
effort
|
Float64[Array, 'p p']
|
The stage cost on the action it was built with. (LQR's |
required |
gains
|
Float64[Array, 'H p n']
|
|
required |
costs_to_go
|
Float64[Array, 'H+1 n n']
|
|
required |
kalman_schedule ¶
kalman_schedule(
model: LinearGaussianModel,
*,
horizon: int,
prior: Belief | None = None,
) -> KalmanSchedule
The exact filter's gain and covariance at each of horizon readings.
The covariance recursion the per-step KalmanBackend runs, run ahead of time
and kept, since under fixed noise it never sees a reading.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
LinearGaussianModel
|
The model the filter runs under, with fixed noise. |
required |
horizon
|
int
|
How many readings, at least one. |
required |
prior
|
Belief | None
|
The belief before the first reading. Defaults to the model's prior. |
None
|
Returns:
| Type | Description |
|---|---|
KalmanSchedule
|
The gains and covariances, indexed as the plan indexes its steps. |
Raises:
| Type | Description |
|---|---|
TypeError
|
If |
ValueError
|
If |
Source code in src/cpomdp/control.py
KalmanSchedule
dataclass
¶
The exact filter's gains and error covariances over H readings.
Both are data-independent, so they can be written down before a reading exists.
The indexing follows the plan's: action k is chosen from the belief after
k readings, and reading k + 1 arrives after it.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
gains
|
Float64[Array, 'H n m']
|
|
required |
covariances
|
Float64[Array, 'H+1 n n']
|
|
required |
-
Magnus T. Koudahl, Wouter M. Kouw, and Bert de Vries. On epistemics in expected free energy for linear Gaussian state space models. Entropy, 23(12):1565, 2021. URL: https://doi.org/10.3390/e23121565, doi:10.3390/e23121565. ↩