cpomdp — Continuous Active Inference for Python

Continuous active inference for Python. The continuous-state sibling of pymdp.

cpomdp is a JAX-native active inference toolbox for continuous state spaces. You give it a linear-Gaussian generative model and you get an agent that tracks hidden states and selects actions by minimising expected free energy.

Continuous means continuous: positions, velocities, concentrations, temperatures. State-dependent observation noise keeps the epistemic term alive, so information-seeking behaviour is available in a regime where it is provably absent. Inference runs as message passing over a Forney factor graph, JIT-compilable and vmappable end to end.

The loop is infer_states / sample_action, which pymdp users will recognise. cpomdp is its continuous-state sibling. The machinery underneath is its own.

Full documentation, including API reference and guides, lives at cpomdp.inferogenesis.com.

What cpomdp does

Exact continuous perception. Linear mean dynamics and Gaussian noise make the filter an exact Kalman filter, closed form, nothing sampled. Under fixed noise there is no variational gap. The same code path runs whether you are tracking or acting.

Action that reaches the posterior covariance. cpomdp lets the observation noise depend on the state, R(x) (Corva 2026), or the process noise, Q(x). The mean stays linear. Because the covariance now responds to what the agent does, the epistemic term of expected free energy differs across policies and information-seeking behaviour becomes available. Under a fixed linear-Gaussian sensor that term is identical for every policy (Koudahl, Kouw & de Vries 2021); state-dependent noise is the smallest deviation from that model which escapes the result.

One knob between exploiting and exploring. StateGoal and ObservationGoal carry a precision Λ. The pragmatic term scales with Λ and the epistemic term does not, so Λ alone decides whether an agent beelines for the goal or detours to sharpen its belief first. The four bacilli above differ in that number and nothing else.

Multi-step EFE, searched and certified. cpomdp.efe.policy_efe scores an H-step horizon. cpomdp.enumeration.EnumeratedEfeSearch sweeps the full action space A^H and returns a CompletenessCertificate, so "this is the best plan" is decided over the declared action set rather than sampled from it, and the object that says so is checkable. Receding-horizon and open-loop selectors wrap the search, and both report the true per-cycle cost |A|^H · H. Drop the epistemic term and steady-state LQR is what remains.

Inference as message passing. Beliefs propagate over a Forney factor graph through the CouplingGraph backend, with a temporal-edge partition controlling which off-diagonal precision blocks cross the time boundary. Where a model cannot be flattened, cpomdp raises IncompatibleLinearizationError rather than returning a quietly wrong number: state-dependent R(x) combined with a mean-shifting coupling makes flattening structurally impossible, and the two-pass linearisation is the supported route.

What is not here. The mean stays linear. Genuinely nonlinear sensors, a curved g(x) needing a second-order moment match, are the next step rather than a current feature.

Horizon length decides whether the epistemic machinery pays off. With R(x) alive a short-horizon agent still walks past the information; stretch the horizon and it detours to collect it. Curiosity needs the term and the lookahead together.

Example

Four bacilli seeking food in the same world — the continuous-state answer to pymdp's mouse-seeking-cheese, now with an epistemic term. The twist: the food's position is hidden, and a beacon marks where the agent can see it. Visiting the beacon doesn't sharpen where the agent thinks it is — it sharpens where it thinks the food is, which it can't act on directly. That makes the information genuinely instrumental: resolving it changes where the agent then heads. Each body sits at its true hidden state; the blue + is where it believes it is, the diamond is where it believes the food is (both with their uncertainty ellipses), and the star is the food's true, hidden location. The four differ in one number only — the goal precision Λ each is built with. They all minimise the same Expected Free Energy G = pragmatic − epistemic; because the pragmatic (goal) term scales with Λ while the epistemic (information) term doesn't, Λ alone tips the balance: classic LQR and a sharp Λ beeline to the agent's current food guess and never detour; a balanced Λ detours to the beacon, learns where the food really is, then heads there with confidence; a weak Λ is so over-curious it parks at the beacon and never eats. One real knob — the precision you'd actually pass — four behaviours.

Reproduce it with examples/bacillus_uncertain_food.py (pip install "cpomdp[examples]").

How far ahead before information is worth a detour?

An agent on an open plane wants a goal it cannot locate. Its prior points the wrong way. A beacon well off that line is the only thing that can say where the goal really is. Walking there costs ground.

Run the same world once per planning horizon. At H = 2 the agent walks straight to the spot it already believed in and settles there. It never checks. At H = 14 it walks away from that spot, reads the beacon, finds out it was wrong, and then goes to the real goal. Only H changed.

The margin between the two plans, ΔG(H) = G(detour) − G(direct) in nats, crosses zero exactly once. The epistemic pull is flat, because sensing once is worth what sensing once is worth. The pragmatic gradient decays under it. Freeze R at a constant and the sweep never crosses at any horizon, so the behaviour belongs to the state-dependent sensor and the horizon together.

Reproduce it with examples/crossover_horizon_figure.py.

Why the horizon is the question. State-dependent observation noise reintroduces epistemic value in linear-Gaussian active inference (Corva 2026) establishes that R(x) makes the epistemic term non-constant, so a linear-Gaussian agent can be curious at all. Non-constant is not the same as decision-changing. The horizon at which curiosity starts changing which plan an agent picks is a separate question, and it has a measured answer.

The answer is one-dimensional, and it is proven. On the corridor cue task, H* = 7: the first horizon whose argmin is a two-phase sense-then-commit walk rather than a direct reach. That comes from enumerating every sequence in the declared action set {0, ±1, ±2} to depth H, so it decides the flip rather than sampling for it, and the search returns a CompletenessCertificate saying how many policies it was obliged to visit and how many it did. Zero the epistemic term and the crossing moves out to H ≈ 10, which is what makes the pull load-bearing rather than incidental. The numbers are registered in warrant_numbers.md, the model is examples/ffg/crossover.py, and tests/test_example_checks.py::test_crossover_check asserts it on every merge and release. The 7 is an upper bound, because the declared set clips the reach at −2 while reaching the goal from the start takes −3. On the wider {−3…2} the flip is at 6. That wider number is computed rather than certified. No completeness certificate reaches a report and no test asserts it, so it is stated here as a qualifier on the 7 rather than as a result of its own. Certifying it, and measuring whether wider sets still move it, is issue #65.

The plane above is the readable version, not the proof. It contrasts two named plans instead of searching, so its crossing is an exact statement about those two plans and says nothing about the argmin over all plans. Its H = 7 and the corridor's H* = 7 are different quantities on different models that happen to coincide.

More in the examples gallery, including the FFG examples, where a branch-coupled state-dependent sensor resolves a hidden context and can't be flattened to a Kalman filter.

Install

pip install cpomdp

Or the latest from source:

pip install git+https://github.com/inferogenesis/cpomdp

That's all you need for normal use. There's also an optional RxInfer (Julia) backend that the test suite leans on as a correctness oracle. You almost certainly don't need it, but if you want it:

pip install "cpomdp[rxinfer]"

It pulls in a Julia bridge and bootstraps itself the first time you use it.

Quickstart

Here's an agent steering a point mass to a target. It can push the mass and it can see where the mass is, but it never sees the velocity. The filter has to work that out from how the position moves.

import jax.numpy as jnp
from cpomdp import Agent, Belief, LinearGaussianModel, StateGoal

# State is [position, velocity]. A push changes velocity, velocity carries
# position along, and we only ever observe position (through a noisy sensor).
dt = 0.1
model = LinearGaussianModel(
    dynamics_matrix=[[1, dt], [0, 1]],   # velocity carries position along
    control_matrix=[[0], [dt]],          # a push nudges velocity
    observation_matrix=[[1, 0]],               # we observe position only
    dynamics_noise=jnp.eye(2) * 1e-6,
    observation_noise=[[1e-2]],
    prior=Belief(mean=[0, 0], cov=jnp.eye(2)),
)

# Tell it where to go: sit still at position 1.
agent = Agent(model, StateGoal([1.0, 0.0]))

true_state = jnp.array([0.0, 0.0])
for _ in range(100):
    obs = model.observation_matrix @ true_state            # what the agent gets to see
    agent.infer_states(obs)                           # perceive
    action = agent.sample_action()                    # act
    true_state = model.dynamics_matrix @ true_state + model.control_matrix @ action

print(jnp.round(agent.belief.mean, 3))   # ≈ [1, 0]

Run that and the belief lands on [1, 0]. The agent worked out it was at position 1 and sitting still, which is exactly where we asked it to go, and it did it without ever seeing the velocity it had to control.

The pymdp parallel

If you've used pymdp, the loop is the same and most of the names are too. Four carry over verbatim:

Agent · qs · infer_states · sample_action

(qs is a read-only alias for belief, cpomdp's canonical name — so agent.qs and agent.belief are the same posterior. Use whichever your fingers reach for.)

Only two things are spelled differently:

pymdp	cpomdp	what it is
C	StateGoal / ObservationGoal	the goal you pursue, and how sharply
D	model.prior	belief before you've seen anything

One honest difference in behaviour. sample_action here is deterministic, not a sample from a policy posterior. For a linear-Gaussian sensor the action that minimises expected free energy turns out to be exactly the LQR optimum, so there's a single best action and that's what comes back. Same loop, exact answer. The reasoning is in DECISIONS.md (ADR-003) if you want it.

Just want to track, not act?

A model with no control_matrix is a pure tracker. Drop the goal and infer_states still folds in observations and sharpens the belief, while sample_action stops you — there's nothing to steer toward, and nothing to steer with.

tracker = LinearGaussianModel(        # no control matrix -> pure tracking
    dynamics_matrix=[[1, dt], [0, 1]],
    observation_matrix=[[1, 0]],
    dynamics_noise=jnp.eye(2) * 1e-6,
    observation_noise=[[1e-2]],
    prior=Belief(mean=[0, 0], cov=jnp.eye(2)),
)
agent = Agent(tracker)                # no objective
agent.infer_states([0.5])             # perceiving is fine
agent.sample_action()                 # ValueError: this Agent has no objective ...

What's in the box

you want to	reach for
perceive, exactly	Agent.infer_states, KalmanBackend
reach a target state	StateGoal with LQRSelector
act on expected free energy, one step	ObservationGoal with EFESelector, expected_free_energy
...over an H-step horizon	cpomdp.efe.policy_efe, policy_efe_trace
...searched exhaustively, with a certificate	cpomdp.enumeration, EnumeratedEfeSearch, RecedingHorizonSelector, OpenLoopSelector
sense more sharply in some places than others	CallableSensor, state-dependent R(x)
diffuse more in some states than others	CallableProcessNoise, state-dependent Q(x)
declare a branching model	CouplingGraph, Coupling, CouplingGraphBackend
ask whether a state-dependent sensor earns its keep	probe_model → SensorReport
check a rollout stayed well conditioned	cpomdp.diagnostics.rollout_conditioning
perceive but never act	a model with no control_matrix, Agent(model) alone
re-plan every step, closed loop	cpomdp.enumeration.RecedingHorizonSelector (replan_interval = 1)
commit to one plan and execute it	cpomdp.enumeration.OpenLoopSelector (replan_interval = H)
swap the inference engine	the InferenceBackend protocol, or cpomdp.backends.rxinfer.RxInferBackend

The state-dependence is in the noise. The mean stays linear. Genuinely nonlinear sensors, a curved g(x) needing a second-order moment match, are the next step and are not here yet.

How provable is your result? (experimental)

Most toolboxes let you build an agent and stop there. cpomdp also labels how well each of your results is established, so you do not have to invent a warrant scheme of your own before you can report one honestly.

Every check the suite runs carries three labels. A warrant says what established the claim: PROVED for a theorem, a symbolic identity, or a finite domain exhausted under a completeness certificate; CERTIFIED for validated numerics over a compact domain; CORROBORATED for a sample of a continuum, which settles existence and refutes a universal by counterexample and decides no universal at any sample count. A tier says how well the number itself is known: EXACT, BOUNDED, or COMPUTED. An outcome says what the falsifier did, and a falsifier does not pass — it fires or it does not. PROVED with nothing behind it does not construct.

research/warrant_ledger.md is the canonical table, and every other document points at it. warrant_numbers.md records the declared numbers those claims are measured against. The vocabulary itself is packages/warrantlib, published separately and re-exported as cpomdp.warrant, documented at cpomdp.inferogenesis.com/api/warrant.

Treat this as experimental. It may move out of cpomdp into a standalone inferogenesis tool before the 1.0 release, which is undecided.

Swappable backends

You can swap the inference engine if you want to. KalmanBackend is the default and does the real work; RxInferBackend re-derives the same answers through Julia and exists mainly so the fast path has something independent to check itself against. Both sit behind the InferenceBackend protocol, so you can write your own.

Status

Still pre-1.0. v0.4 secures the public API around the factor-graph backend and the enumeration layer. If you have a request or a suggestion that would make the front-facing API more usable, open a GitHub issue, I am happy to listen. Until 1.0, a minor version is where breaking changes can land.

Development

I designed and built cpomdp — the architecture, the conditionally-linear-Gaussian formulation, the API, and every decision in DECISIONS.md are mine. The design draws on my day-to-day work as a full-time software engineer and on hands-on expertise integrating and developing large machine-learning models at scale using event-driven microservice architecture.

I used AI coding assistants (Claude Opus 4.8, and Opus 5 from v0.4) as tools under close review: to draft docstrings, probe for edge cases and candidate bugs, and expand the test suite, including adversarial ones. Everything they produced I read, checked, and approved before it landed. None of it is taken on trust. The numbers are validated independently against the RxInfer (Julia) and analytic NumPy oracles described above. Correctness rests on those checks, not on the tools that helped write the code.

Contributions

If you would like to contribute either your dev time or help steer the direction of the toolbox, please add a GitHub issue or discussion thread. I am monitoring this repository closely and would love to collaborate. CONTRIBUTING.md covers the setup, the hooks, and the bar a PR has to clear.

If you notice a better method in something I've already done or are just curious and want to chat I am more than happy to talk through my decision processes and improve on my work. I intend to blog my construction of cpomdp provided it doesn't interfere with developing it.

Acknowledgements

Thanks to Kevin Backhouse (Postgraduate Researcher in Cognitive Neuroscience, Durham University) for guidance on the active-inference formulation, collaboration on related discrete generative-model projects, and for being a consistent sounding board throughout the design of this work.

Where next

• Agent — the façade you drive
• Model & belief — LinearGaussianModel and Belief
• Backends — Kalman (default) and the RxInfer oracle
• Control — the steady-state LQR controller