Skip to content

Agent

The stateful façade you drive: infer_states to perceive, sample_action to act.

Variations of Agent

Agent(model, goal) filters with the per-step Kalman filter. Agent.from_backend(backend, goal) takes any other engine, a graph backend included. A flat backend plans under backend.model, and a graph backend plans on the graph, so a StateGoal on a coupled graph is refused. Both lead into the same choices below. Either one takes a Preference and selector= in place of the goal.

A curious agent is one whose information term changes with the action. The Agent builds one for an ObservationGoal wherever a noise depends on the state. That is the sensor's R(x), or the dynamics noise Q(x) of a CallableDynamicsNoise, which is claimed at horizon 1 only, so a Q(x)-only goal that needs a longer horizon is refused. A graph carries no Q(x), so on a graph only R(x) makes a goal curious. With both noises fixed the information term is the same for every action, and the goal is refused. Green marks a curious agent, blue LQR, grey a tracker, a red outline a refusal at construction, and a dashed red outline a combination that builds and then raises or does not do what its name suggests.

flowchart LR
    none["no goal<br/>Agent(model)"]
    state["StateGoal(target)"]
    obs["ObservationGoal(target, …)"]

    none -- "model without control" --> track["Tracker: perceives, never acts"]:::track
    none -- "model with control" --> stuck["Tracker whose infer_states raises:<br/>it has no action to predict with"]:::gap
    state -- "fixed sensor,<br/>target held still" --> lqr["LQR: no information term"]:::lqr
    state -- "fixed sensor,<br/>target drifts" --> r1["Refused: LQR would settle short"]:::refused
    state -- "R(x) sensor" --> r2["Refused: a state goal on varying sensing"]:::refused
    state -- "coupled graph" --> uncoupled["Refused: LQR would leave out the couplings"]:::refused
    obs -- "R(x) or Q(x)" --> c1["Curious EFE: horizon read off the model"]:::curious
    obs -- "Q(x) only,<br/>horizon past 1" --> rq2["Refused: Q(x) is claimed at horizon 1 only"]:::refused
    obs -- "R(x) or Q(x),<br/>horizon set too short" --> r3["Refused: the target does not move the scores"]:::refused
    obs -- "both noises fixed,<br/>flat model or graph" --> r4["Refused: output regulation, deferred"]:::refused
    obs -- "graph model,<br/>R(x) node sensors" --> c2["Curious EFE: graph selector, horizon 1"]:::curious

    classDef curious fill:#26a69a33,stroke:#26a69a,stroke-width:2px
    classDef lqr fill:#5c8fd633,stroke:#5c8fd6,stroke-width:2px
    classDef track fill:#90a4ae26,stroke:#90a4ae,stroke-width:2px
    classDef refused fill:#e5737314,stroke:#e57373,stroke-width:2px
    classDef gap fill:#e5737326,stroke:#e57373,stroke-width:2px,stroke-dasharray:5 4

A goal configures the selector the Agent builds. To choose with a selector of your own, pass a Preference in its place. The selector then carries the whole search, and the Preference carries the target and precision it chooses toward, in the space the selector reads.

You pass Both noises fixed R(x) or Q(x)
observation Preference + selector=EFESelector(…) or FFGEFESelector(…) EFE with a flat information term, the ablation of a curious agent Curious EFE at the horizon given, which is not checked
observation Preference + selector=RecedingHorizonSelector(…) or OpenLoopSelector(…) Search with a flat information term Curious search over varying action sequences
state Preference + selector=LQRSelector(…) LQR with your controller LQR with your controller, blind to the information term
a Preference of the wrong length for a built-in selector Refused Refused
a goal + selector=, or one of Preference and selector= without the other Refused Refused

An ObservationGoal with no horizon set gets the shortest horizon at which the target moves the scores. agent.selector.horizon reads it back, agent.selector.cost_per_cycle gives what each sample_action costs, and printing the agent shows both. A longer horizon looks further ahead for more work per step, and can change what the agent does. To compare, build one agent per horizon and run each. The sensor below is sharpest at the goal, position 1, and the push reaches the reading a step after it changes the velocity, so the shortest horizon is 2.

import jax.numpy as jnp

from cpomdp import Agent, Belief, CallableSensor, LinearGaussianModel, ObservationGoal


def sharper_near_goal(x, params):
    return jnp.array([[0.01 + (x[0] - 1.0) ** 2]])


model = LinearGaussianModel(
    [[1.0, 0.1], [0.0, 1.0]],
    control_matrix=[[0.0], [0.1]],
    observation_matrix=[[1.0, 0.0]],
    dynamics_noise=jnp.eye(2) * 1e-6,
    observation_model=CallableSensor([[1.0, 0.0]], sharper_near_goal, ()),
    prior=Belief(mean=[0.0, 0.0], cov=jnp.eye(2)),
)

for horizon in (None, 4, 8):
    goal = ObservationGoal([1.0], action_bounds=(-2.0, 2.0), horizon=horizon)
    agent = Agent(model, goal)
    state = jnp.zeros(2)
    for _ in range(40):
        agent.infer_states(model.observation_matrix @ state)
        action = agent.sample_action()
        state = model.dynamics_matrix @ state + model.control_matrix @ action
    print(agent.selector.horizon, agent.selector.cost_per_cycle, jnp.round(state, 2))
# 2 42 [1.08 0.4 ]
# 4 84 [1. 0.]
# 8 168 [0.99 0.  ]

At the shortest horizon the agent overshoots and is still moving after 40 steps. At 4 it settles on the goal, for twice the work per step.

The Agent is eager. Its methods update the belief it holds, so jax.jit, vmap and grad do not apply to them. A jitted infer_states filters every observation against the belief it was traced with, so the belief never advances, and the next eager call raises UnexpectedTracerError. To compile a loop, drive a backend directly under jax.lax.scan.

A backend driven under jax.lax.scan
import jax
import jax.numpy as jnp

from cpomdp import Belief, KalmanBackend, LinearGaussianModel

model = LinearGaussianModel(
    [[1.0, 0.1], [0.0, 1.0]],
    control_matrix=[[0.0], [0.1]],
    observation_matrix=[[1.0, 0.0]],
    dynamics_noise=jnp.eye(2) * 1e-6,
    observation_noise=[[1e-2]],
    prior=Belief(mean=[0.0, 0.0], cov=jnp.eye(2)),
)
backend = KalmanBackend(model)


def step(belief, reading):
    observation, action = reading
    belief = backend.infer_states(observation, belief, action)
    return belief, belief.mean


observations = jnp.ones((50, 1))
actions = jnp.zeros((50, 1))
final, means = jax.jit(lambda xs: jax.lax.scan(step, model.prior, xs))(
    (observations, actions)
)

A selector's select(belief, preference) is pure as well. Call it inside step to choose each action within the compiled loop.

Agent

Agent(
    model: LinearGaussianModel,
    objective: StateGoal
    | ObservationGoal
    | Preference
    | None = None,
    *,
    selector: ActionSelector | None = None,
)

A continuous active-inference agent — the continuous sibling of pymdp's Agent.

The backend and action selector underneath are pure: belief in, belief out; belief + preference in, action out. The Agent is the one stateful piece — it owns the current belief (the continuous analog of pymdp's qs) and carries it forward across calls, so you drive it in the same perceive → act loop pymdp users already know::

agent = Agent(model, StateGoal(target))
belief = agent.infer_states(observation)   # perceive
action = agent.sample_action()             # act

The agent remembers the action it last sampled and feeds it to its own predict step on the next infer_states — so you never thread actions back in by hand, and a perceive-only model (no control matrix) needs no action at all. This mirrors the LQG loop exactly: the filter predicts with the action that was actually applied to the plant between observations.

The vocabulary maps onto pymdp's like this:

pymdp (discrete) cpomdp (continuous) role
Agent Agent the stateful façade
qs belief posterior over the state
infer_states infer_states fold an observation in
sample_action sample_action choose an action
A observation_matrix, observation_noise the likelihood
B dynamics_matrix, control_matrix, dynamics_noise transition
C objective a StateGoal, an ObservationGoal, or a Preference
D model.prior belief before any observation
infer_policies folded into sample_action score the actions

The model has no letter aliases. Control theory's A is pymdp's B, and its C is pymdp's A, so each field goes by its role name only.

One honest difference from pymdp: sample_action is deterministic here, not a draw from a policy posterior. With a fixed linear-Gaussian sensor and fixed dynamics noise the EFE-minimising action is the LQR optimum (ADR-003), so it returns the single best action. The name keeps the pymdp muscle-memory; the behaviour is exact.

An agent built without an objective is a pure tracker: infer_states works, but sample_action raises — there is nothing to act toward.

The Agent is eager. It keeps the belief and the last action on itself, so jax.jit, vmap and grad do not apply to its methods. A jitted infer_states filters every observation against the belief it was traced with, so the belief never advances, and the next eager call raises UnexpectedTracerError. To compile a loop, call a backend's infer_states and a selector's select inside jax.lax.scan, as the API page shows.

Build an agent over model, optionally one that can act.

The objective says what the agent pursues, and one place says how it searches. A goal carries its own search and the Agent builds the selector from it. A StateGoal steers in state space via LQR (it needs a fixed sensor). An ObservationGoal seeks a preferred observation via EFE, over the goal's horizon or, left unset, the shortest at which the target moves the scores. It needs a control matrix, and a sensor noise R(x) or dynamics noise Q(x) that depends on the state, on a flat model and a graph alike. To choose with a selector of your own, pass a Preference and selector=. The selector then carries the search. Omit the objective for a perceive-only tracker.

The agent filters with a per-step KalmanBackend built from model. For any other engine, use from_backend.

Parameters:

Name Type Description Default
model LinearGaussianModel

The linear-Gaussian generative model the agent perceives and (with an objective) acts under. Its prior becomes the starting belief.

required
objective StateGoal | ObservationGoal | Preference | None

What the agent pursues. A StateGoal (a state to reach, carrying the LQR weights) or an ObservationGoal (an observation to prefer, carrying the action search). A Preference only beside selector=. None builds a pure tracker, which perceives and never acts.

None
selector ActionSelector | None

An ActionSelector of your own, for a search no goal describes: EFE with both noises fixed, a varying-sequence search driver, an LQRSelector over a controller of your own, or a custom rule. It needs a Preference as the objective, in the space the selector reads. That is an observation for the EFE selectors and the search drivers, and a state for LQRSelector.

None

Raises:

Type Description
ValueError

If the objective and model are incompatible. That covers a StateGoal on a state-dependent sensor, an ObservationGoal on a control-free model, and an ObservationGoal on a model whose noises are both fixed (output regulation, deferred). Also raised for a target of the wrong length for the space it is read in, a target the dynamics do not hold at zero action on the LQR path, an ObservationGoal horizon too short for its target to move the scores, selector= with a control-free model, and a built-in selector= built for another model or graph.

TypeError

If objective is not a StateGoal, ObservationGoal or Preference, if a goal comes with selector=, or if a Preference or selector= comes without the other. Also raised for a coupled CouplingGraphBackend's model, which leaves out the couplings.

Source code in src/cpomdp/agent.py
def __init__(
    self,
    model: LinearGaussianModel,
    objective: StateGoal | ObservationGoal | Preference | None = None,
    *,
    selector: ActionSelector | None = None,
) -> None:
    """Build an agent over ``model``, optionally one that can act.

    The objective says what the agent pursues, and one place says how it searches.
    A goal carries its own search and the Agent builds the selector from it. A
    ``StateGoal`` steers in state space via LQR (it needs a fixed sensor). An
    ``ObservationGoal`` seeks a preferred observation via EFE, over the goal's
    horizon or, left unset, the shortest at which the target moves the scores. It
    needs a control matrix, and a sensor noise ``R(x)`` or dynamics noise ``Q(x)``
    that depends on the state, on a flat model and a graph alike. To choose with a
    selector of your own, pass a [`Preference`][cpomdp.Preference] and
    ``selector=``. The selector then carries the search. Omit the objective for a
    perceive-only tracker.

    The agent filters with a per-step [`KalmanBackend`][cpomdp.KalmanBackend]
    built from ``model``. For any other engine, use ``from_backend``.

    Args:
        model: The linear-Gaussian generative model the agent perceives and
            (with an objective) acts under. Its ``prior`` becomes the starting
            belief.
        objective: What the agent pursues. A ``StateGoal`` (a state to reach,
            carrying the LQR weights) or an ``ObservationGoal`` (an observation
            to prefer, carrying the action search). A ``Preference`` only beside
            ``selector=``. ``None`` builds a pure tracker, which perceives and
            never acts.
        selector: An [`ActionSelector`][cpomdp.ActionSelector] of your own, for a
            search no goal describes: EFE with both noises fixed, a
            varying-sequence search driver, an ``LQRSelector`` over a controller
            of your own, or a custom rule. It needs a ``Preference`` as the
            objective, in the space the selector reads. That is an observation
            for the EFE selectors and the search drivers, and a state for
            ``LQRSelector``.

    Raises:
        ValueError: If the objective and model are incompatible. That covers a
            ``StateGoal`` on a state-dependent sensor, an ``ObservationGoal`` on a
            control-free model, and an ``ObservationGoal`` on a model whose noises
            are both fixed (output regulation, deferred). Also raised for a target
            of the wrong length for the space it is read in, a target the
            dynamics do not hold at zero action on the LQR path, an
            ``ObservationGoal`` horizon too short for its target to move the
            scores, ``selector=`` with a control-free model, and a built-in
            ``selector=`` built for another model or graph.
        TypeError: If ``objective`` is not a ``StateGoal``, ``ObservationGoal``
            or ``Preference``, if a goal comes with ``selector=``, or if a
            ``Preference`` or ``selector=`` comes without the other. Also raised
            for a coupled ``CouplingGraphBackend``'s model, which leaves out the
            couplings.
    """
    _refuse_graph_interface(model, "Agent")
    self._setup(KalmanBackend(model), objective, selector)

model property

model: LinearGaussianModel

The model the agent plans and filters under, read-only.

It is the backend's model. The selector and any controller were built from it at construction, so acting under a different model needs a new Agent.

belief property writable

belief: Belief

The current posterior over the state, replaced by each infer_states.

Assigning a Belief over the model's state restarts filtering from it. Anything else is refused, since the next step would fail on it or filter from the wrong state.

selector property

selector: ActionSelector | None

The selector that chooses the agent's actions, read-only.

None for a tracker. For an ObservationGoal with no horizon set, selector.horizon is the horizon read off the model, and selector.cost_per_cycle is what each sample_action costs.

from_backend classmethod

from_backend(
    backend: InferenceBackend,
    objective: StateGoal
    | ObservationGoal
    | Preference
    | None = None,
    *,
    selector: ActionSelector | None = None,
) -> Agent

Build an agent that filters with backend and plans on the same plant.

For an engine other than the default per-step Kalman filter: a steady-state KalmanBackend, the RxInfer oracle, or a CouplingGraphBackend, which has no model of its own to pass. The agent's model is backend.model. A flat backend plans under it. A graph backend plans on the graph, since a coupled graph's model leaves out the couplings.

Parameters:

Name Type Description Default
backend InferenceBackend

The inference engine, any InferenceBackend.

required
objective StateGoal | ObservationGoal | Preference | None

What the agent pursues, as for the constructor.

None
selector ActionSelector | None

A selector of your own, beside a Preference, as for the constructor.

None

Returns:

Type Description
Agent

The agent.

Raises:

Type Description
ValueError

If the objective and the backend's model are incompatible, or a built-in selector= was built for another model or graph, as for the constructor.

TypeError

On an objective and selector= that do not pair, as for the constructor, or on a StateGoal with a coupled graph backend, whose model leaves out the couplings LQR would need.

Source code in src/cpomdp/agent.py
@classmethod
def from_backend(
    cls,
    backend: InferenceBackend,
    objective: StateGoal | ObservationGoal | Preference | None = None,
    *,
    selector: ActionSelector | None = None,
) -> "Agent":
    """Build an agent that filters with ``backend`` and plans on the same plant.

    For an engine other than the default per-step Kalman filter: a steady-state
    [`KalmanBackend`][cpomdp.KalmanBackend], the RxInfer oracle, or a
    [`CouplingGraphBackend`][cpomdp.CouplingGraphBackend], which has no model of
    its own to pass. The agent's model is ``backend.model``. A flat backend plans
    under it. A graph backend plans on the graph, since a coupled graph's model
    leaves out the couplings.

    Args:
        backend: The inference engine, any
            [`InferenceBackend`][cpomdp.InferenceBackend].
        objective: What the agent pursues, as for the constructor.
        selector: A selector of your own, beside a ``Preference``, as for the
            constructor.

    Returns:
        The agent.

    Raises:
        ValueError: If the objective and the backend's model are incompatible, or
            a built-in ``selector=`` was built for another model or graph, as for
            the constructor.
        TypeError: On an objective and ``selector=`` that do not pair, as for the
            constructor, or on a ``StateGoal`` with a coupled graph backend, whose
            model leaves out the couplings LQR would need.
    """
    agent = cls.__new__(cls)
    agent._setup(backend, objective, selector)
    return agent

__repr__

__repr__() -> str

The objective, the selector and the backend, as built.

Source code in src/cpomdp/agent.py
def __repr__(self) -> str:
    """The objective, the selector and the backend, as built."""
    objective = None if self._objective is None else type(self._objective).__name__
    return (
        f"Agent(objective={objective}, selector={_short_repr(self._selector)}, "
        f"backend={type(self._backend).__name__})"
    )

infer_states

infer_states(observation: ArrayLike) -> Belief

Fold one observation into the belief and return the updated belief.

The agent's current belief goes in as the prior and the posterior comes back out and is stored — that reassignment is the recursive filter, advanced one step. The belief is never mutated in place; each call replaces it with a fresh Belief.

No action is passed in: the agent supplies its own last sampled action to the predict step (zero before the first sample_action), and a perceive-only model carries no action at all. This is the action actually applied to the plant since the previous observation — exactly what the Kalman predict step needs.

Parameters:

Name Type Description Default
observation ArrayLike

The latest sensor reading, shape (m,).

required

Returns:

Type Description
Belief

The updated belief (also stored on self.belief).

Raises:

Type Description
ValueError

On a shape mismatch in observation, which the backend enforces.

Source code in src/cpomdp/agent.py
def infer_states(self, observation: ArrayLike) -> Belief:
    """Fold one observation into the belief and return the updated belief.

    The agent's current ``belief`` goes in as the prior and the posterior
    comes back out and is stored — that reassignment *is* the recursive
    filter, advanced one step. The belief is never mutated in place; each call
    replaces it with a fresh ``Belief``.

    No action is passed in: the agent supplies its own last sampled action to
    the predict step (zero before the first ``sample_action``), and a
    perceive-only model carries no action at all. This is the action actually
    applied to the plant since the previous observation — exactly what the
    Kalman predict step needs.

    Args:
        observation: The latest sensor reading, shape ``(m,)``.

    Returns:
        The updated belief (also stored on ``self.belief``).

    Raises:
        ValueError: On a shape mismatch in ``observation``, which the backend
            enforces.
    """
    observation = jnp.asarray(observation, dtype=float)
    self._belief = self._backend.infer_states(
        observation, self._belief, self._last_action
    )
    return self._belief

sample_action

sample_action() -> Float64[Array, p]

The action the agent's selector chooses for the current belief.

Delegates to the agent's ActionSelector, handing it the current belief and the objective's Preference. For a StateGoal under a fixed sensor that selection is exactly the LQR optimum, -L∞·(mean − target) with L∞ the steady-state gain solved at construction (ADR-003). That is one matrix-vector product. For an ObservationGoal it is the EFE-minimising action over the front-loaded candidate grid. Deterministic, not a sample, and it takes no rng_key (see the class docstring). Policy evaluation is folded in here, so there is no separate infer_policies step. On the EFE path its per-cycle cost reads back as agent.selector.cost_per_cycle. The chosen action is remembered so the next infer_states predicts with it.

Returns:

Type Description
Float64[Array, p]

The action, shape (p,).

Raises:

Type Description
ValueError

If this is a perceive-only agent, built without an objective, with nothing to act toward. Also raised when the selector returns NaN, which the grid selectors do when no candidate scores a finite G. The action is then not remembered. The check is one host read of the p action entries per cycle.

Source code in src/cpomdp/agent.py
def sample_action(self) -> Float64[Array, "p"]:
    """The action the agent's selector chooses for the current belief.

    Delegates to the agent's ``ActionSelector``, handing it the current belief
    and the objective's ``Preference``. For a ``StateGoal`` under a fixed
    sensor that selection is exactly the LQR optimum, ``-L∞·(mean − target)``
    with ``L∞`` the steady-state gain solved at construction (ADR-003). That is
    one matrix-vector product. For an ``ObservationGoal`` it is the
    EFE-minimising action over the front-loaded candidate grid. Deterministic,
    not a sample, and it takes no ``rng_key`` (see the class docstring). Policy
    evaluation is folded in here, so there is no separate ``infer_policies``
    step. On the EFE path its per-cycle cost reads back as
    ``agent.selector.cost_per_cycle``. The chosen action is remembered so the next
    ``infer_states`` predicts with it.

    Returns:
        The action, shape ``(p,)``.

    Raises:
        ValueError: If this is a perceive-only agent, built without an
            ``objective``, with nothing to act toward. Also raised when the
            selector returns NaN, which the grid selectors do when no candidate
            scores a finite ``G``. The action is then not remembered. The check
            is one host read of the ``p`` action entries per cycle.
    """
    if self._selector is None:
        raise ValueError(
            "this Agent has no objective, so it can only perceive. Pass a "
            "StateGoal or an ObservationGoal to Agent(...), or a Preference with "
            "selector=, to enable sample_action()."
        )
    assert self._preference is not None  # set with _selector; narrows the type
    action = self._selector.select(self.belief, self._preference)
    if bool(jnp.isnan(action).any()):
        raise ValueError(
            "the selector returned a non-finite action at this belief. For an "
            "EFE selector, no candidate scored a finite expected free energy, "
            "which a state-dependent R(x) or Q(x) that is not a covariance at "
            "the predicted states does."
        )
    self._last_action = action
    return action