Agent¶
The stateful façade you drive: infer_states to perceive, sample_action to act.
Variations of Agent
Agent(model, goal) filters with the per-step Kalman filter.
Agent.from_backend(backend, goal) takes any other engine, a graph backend
included. A flat backend plans under backend.model, and a graph backend plans on
the graph, so a StateGoal on a coupled graph is refused. Both lead into the same
choices below. Either one
takes a Preference and selector= in place of the goal.
A curious agent is one whose information term changes with the action. The Agent
builds one for an ObservationGoal wherever a noise depends on the state. That is
the sensor's R(x), or the dynamics noise Q(x) of a
CallableDynamicsNoise, which is claimed at horizon 1
only, so a Q(x)-only goal that needs a longer horizon is refused. A graph carries
no Q(x), so on a graph only R(x) makes a goal curious. With both noises fixed
the information term is the same for every action, and the goal is refused.
Green marks a curious agent, blue LQR, grey a tracker, a red outline a refusal at
construction, and a dashed red outline a combination that builds and then raises or
does not do what its name suggests.
flowchart LR
none["no goal<br/>Agent(model)"]
state["StateGoal(target)"]
obs["ObservationGoal(target, …)"]
none -- "model without control" --> track["Tracker: perceives, never acts"]:::track
none -- "model with control" --> stuck["Tracker whose infer_states raises:<br/>it has no action to predict with"]:::gap
state -- "fixed sensor,<br/>target held still" --> lqr["LQR: no information term"]:::lqr
state -- "fixed sensor,<br/>target drifts" --> r1["Refused: LQR would settle short"]:::refused
state -- "R(x) sensor" --> r2["Refused: a state goal on varying sensing"]:::refused
state -- "coupled graph" --> uncoupled["Refused: LQR would leave out the couplings"]:::refused
obs -- "R(x) or Q(x)" --> c1["Curious EFE: horizon read off the model"]:::curious
obs -- "Q(x) only,<br/>horizon past 1" --> rq2["Refused: Q(x) is claimed at horizon 1 only"]:::refused
obs -- "R(x) or Q(x),<br/>horizon set too short" --> r3["Refused: the target does not move the scores"]:::refused
obs -- "both noises fixed,<br/>flat model or graph" --> r4["Refused: output regulation, deferred"]:::refused
obs -- "graph model,<br/>R(x) node sensors" --> c2["Curious EFE: graph selector, horizon 1"]:::curious
classDef curious fill:#26a69a33,stroke:#26a69a,stroke-width:2px
classDef lqr fill:#5c8fd633,stroke:#5c8fd6,stroke-width:2px
classDef track fill:#90a4ae26,stroke:#90a4ae,stroke-width:2px
classDef refused fill:#e5737314,stroke:#e57373,stroke-width:2px
classDef gap fill:#e5737326,stroke:#e57373,stroke-width:2px,stroke-dasharray:5 4
A goal configures the selector the Agent builds. To choose with a selector of your
own, pass a Preference in its place. The selector then carries the whole search,
and the Preference carries the target and precision it chooses toward, in the space
the selector reads.
| You pass | Both noises fixed | R(x) or Q(x) |
|---|---|---|
observation Preference + selector=EFESelector(…) or FFGEFESelector(…) |
EFE with a flat information term, the ablation of a curious agent | Curious EFE at the horizon given, which is not checked |
observation Preference + selector=RecedingHorizonSelector(…) or OpenLoopSelector(…) |
Search with a flat information term | Curious search over varying action sequences |
state Preference + selector=LQRSelector(…) |
LQR with your controller | LQR with your controller, blind to the information term |
a Preference of the wrong length for a built-in selector |
Refused | Refused |
a goal + selector=, or one of Preference and selector= without the other |
Refused | Refused |
An ObservationGoal with no horizon set gets the shortest horizon at which the
target moves the scores. agent.selector.horizon reads it back,
agent.selector.cost_per_cycle gives what each sample_action costs, and printing
the agent shows both. A longer horizon looks further ahead for more work per step,
and can change what the agent does. To compare, build one agent per horizon and run
each. The sensor below is sharpest at the goal, position 1, and the push reaches the
reading a step after it changes the velocity, so the shortest horizon is 2.
import jax.numpy as jnp
from cpomdp import Agent, Belief, CallableSensor, LinearGaussianModel, ObservationGoal
def sharper_near_goal(x, params):
return jnp.array([[0.01 + (x[0] - 1.0) ** 2]])
model = LinearGaussianModel(
[[1.0, 0.1], [0.0, 1.0]],
control_matrix=[[0.0], [0.1]],
observation_matrix=[[1.0, 0.0]],
dynamics_noise=jnp.eye(2) * 1e-6,
observation_model=CallableSensor([[1.0, 0.0]], sharper_near_goal, ()),
prior=Belief(mean=[0.0, 0.0], cov=jnp.eye(2)),
)
for horizon in (None, 4, 8):
goal = ObservationGoal([1.0], action_bounds=(-2.0, 2.0), horizon=horizon)
agent = Agent(model, goal)
state = jnp.zeros(2)
for _ in range(40):
agent.infer_states(model.observation_matrix @ state)
action = agent.sample_action()
state = model.dynamics_matrix @ state + model.control_matrix @ action
print(agent.selector.horizon, agent.selector.cost_per_cycle, jnp.round(state, 2))
# 2 42 [1.08 0.4 ]
# 4 84 [1. 0.]
# 8 168 [0.99 0. ]
At the shortest horizon the agent overshoots and is still moving after 40 steps. At 4 it settles on the goal, for twice the work per step.
The Agent is eager. Its methods update the belief it holds, so jax.jit, vmap and
grad do not apply to them. A jitted infer_states filters every observation against
the belief it was traced with, so the belief never advances, and the next eager call
raises UnexpectedTracerError. To compile a loop, drive a backend directly under
jax.lax.scan.
A backend driven under jax.lax.scan
import jax
import jax.numpy as jnp
from cpomdp import Belief, KalmanBackend, LinearGaussianModel
model = LinearGaussianModel(
[[1.0, 0.1], [0.0, 1.0]],
control_matrix=[[0.0], [0.1]],
observation_matrix=[[1.0, 0.0]],
dynamics_noise=jnp.eye(2) * 1e-6,
observation_noise=[[1e-2]],
prior=Belief(mean=[0.0, 0.0], cov=jnp.eye(2)),
)
backend = KalmanBackend(model)
def step(belief, reading):
observation, action = reading
belief = backend.infer_states(observation, belief, action)
return belief, belief.mean
observations = jnp.ones((50, 1))
actions = jnp.zeros((50, 1))
final, means = jax.jit(lambda xs: jax.lax.scan(step, model.prior, xs))(
(observations, actions)
)
A selector's select(belief, preference) is pure as well. Call it inside step to
choose each action within the compiled loop.
Agent ¶
Agent(
model: LinearGaussianModel,
objective: StateGoal
| ObservationGoal
| Preference
| None = None,
*,
selector: ActionSelector | None = None,
)
A continuous active-inference agent — the continuous sibling of pymdp's Agent.
The backend and action selector underneath are pure: belief in, belief out;
belief + preference in, action out. The Agent is the one stateful piece
— it owns the
current belief (the continuous analog of pymdp's qs) and carries it
forward across calls, so you drive it in the same perceive → act loop pymdp
users already know::
agent = Agent(model, StateGoal(target))
belief = agent.infer_states(observation) # perceive
action = agent.sample_action() # act
The agent remembers the action it last sampled and feeds it to its own
predict step on the next infer_states — so you never thread actions back
in by hand, and a perceive-only model (no control matrix) needs no action at
all. This mirrors the LQG loop exactly: the filter predicts with the action
that was actually applied to the plant between observations.
The vocabulary maps onto pymdp's like this:
| pymdp (discrete) | cpomdp (continuous) | role |
|---|---|---|
Agent |
Agent |
the stateful façade |
qs |
belief |
posterior over the state |
infer_states |
infer_states |
fold an observation in |
sample_action |
sample_action |
choose an action |
A |
observation_matrix, observation_noise |
the likelihood |
B |
dynamics_matrix, control_matrix, dynamics_noise |
transition |
C |
objective |
a StateGoal, an ObservationGoal, or a Preference |
D |
model.prior |
belief before any observation |
infer_policies |
folded into sample_action |
score the actions |
The model has no letter aliases. Control theory's A is pymdp's B, and its
C is pymdp's A, so each field goes by its role name only.
One honest difference from pymdp: sample_action is deterministic here,
not a draw from a policy posterior. With a fixed linear-Gaussian sensor and fixed
dynamics noise the EFE-minimising action is the LQR optimum (ADR-003), so it
returns the single best action. The name keeps the pymdp muscle-memory; the
behaviour is exact.
An agent built without an objective is a pure tracker: infer_states
works, but sample_action raises — there is nothing to act toward.
The Agent is eager. It keeps the belief and the last action on itself, so
jax.jit, vmap and grad do not apply to its methods. A jitted
infer_states filters every observation against the belief it was traced with,
so the belief never advances, and the next eager call raises
UnexpectedTracerError. To compile a loop, call a backend's infer_states
and a selector's select inside jax.lax.scan, as the API page shows.
Build an agent over model, optionally one that can act.
The objective says what the agent pursues, and one place says how it searches.
A goal carries its own search and the Agent builds the selector from it. A
StateGoal steers in state space via LQR (it needs a fixed sensor). An
ObservationGoal seeks a preferred observation via EFE, over the goal's
horizon or, left unset, the shortest at which the target moves the scores. It
needs a control matrix, and a sensor noise R(x) or dynamics noise Q(x)
that depends on the state, on a flat model and a graph alike. To choose with a
selector of your own, pass a Preference and
selector=. The selector then carries the search. Omit the objective for a
perceive-only tracker.
The agent filters with a per-step KalmanBackend
built from model. For any other engine, use from_backend.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
LinearGaussianModel
|
The linear-Gaussian generative model the agent perceives and
(with an objective) acts under. Its |
required |
objective
|
StateGoal | ObservationGoal | Preference | None
|
What the agent pursues. A |
None
|
selector
|
ActionSelector | None
|
An |
None
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If the objective and model are incompatible. That covers a
|
TypeError
|
If |
Source code in src/cpomdp/agent.py
model
property
¶
The model the agent plans and filters under, read-only.
It is the backend's model. The selector and any controller were built from it
at construction, so acting under a different model needs a new Agent.
belief
property
writable
¶
The current posterior over the state, replaced by each infer_states.
Assigning a Belief over the model's state restarts
filtering from it. Anything else is refused, since the next step would fail
on it or filter from the wrong state.
selector
property
¶
The selector that chooses the agent's actions, read-only.
None for a tracker. For an ObservationGoal with no horizon set,
selector.horizon is the horizon read off the model, and
selector.cost_per_cycle is what each sample_action costs.
from_backend
classmethod
¶
from_backend(
backend: InferenceBackend,
objective: StateGoal
| ObservationGoal
| Preference
| None = None,
*,
selector: ActionSelector | None = None,
) -> Agent
Build an agent that filters with backend and plans on the same plant.
For an engine other than the default per-step Kalman filter: a steady-state
KalmanBackend, the RxInfer oracle, or a
CouplingGraphBackend, which has no model of
its own to pass. The agent's model is backend.model. A flat backend plans
under it. A graph backend plans on the graph, since a coupled graph's model
leaves out the couplings.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
backend
|
InferenceBackend
|
The inference engine, any
|
required |
objective
|
StateGoal | ObservationGoal | Preference | None
|
What the agent pursues, as for the constructor. |
None
|
selector
|
ActionSelector | None
|
A selector of your own, beside a |
None
|
Returns:
| Type | Description |
|---|---|
Agent
|
The agent. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the objective and the backend's model are incompatible, or
a built-in |
TypeError
|
On an objective and |
Source code in src/cpomdp/agent.py
__repr__ ¶
The objective, the selector and the backend, as built.
Source code in src/cpomdp/agent.py
infer_states ¶
Fold one observation into the belief and return the updated belief.
The agent's current belief goes in as the prior and the posterior
comes back out and is stored — that reassignment is the recursive
filter, advanced one step. The belief is never mutated in place; each call
replaces it with a fresh Belief.
No action is passed in: the agent supplies its own last sampled action to
the predict step (zero before the first sample_action), and a
perceive-only model carries no action at all. This is the action actually
applied to the plant since the previous observation — exactly what the
Kalman predict step needs.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
observation
|
ArrayLike
|
The latest sensor reading, shape |
required |
Returns:
| Type | Description |
|---|---|
Belief
|
The updated belief (also stored on |
Raises:
| Type | Description |
|---|---|
ValueError
|
On a shape mismatch in |
Source code in src/cpomdp/agent.py
sample_action ¶
The action the agent's selector chooses for the current belief.
Delegates to the agent's ActionSelector, handing it the current belief
and the objective's Preference. For a StateGoal under a fixed
sensor that selection is exactly the LQR optimum, -L∞·(mean − target)
with L∞ the steady-state gain solved at construction (ADR-003). That is
one matrix-vector product. For an ObservationGoal it is the
EFE-minimising action over the front-loaded candidate grid. Deterministic,
not a sample, and it takes no rng_key (see the class docstring). Policy
evaluation is folded in here, so there is no separate infer_policies
step. On the EFE path its per-cycle cost reads back as
agent.selector.cost_per_cycle. The chosen action is remembered so the next
infer_states predicts with it.
Returns:
| Type | Description |
|---|---|
Float64[Array, p]
|
The action, shape |
Raises:
| Type | Description |
|---|---|
ValueError
|
If this is a perceive-only agent, built without an
|