postjevsql.git / crates / postjevsql-sidecar

Chapter 10: postjevsql-sidecar, for databases that won't let you in

Most people's Postgres is somebody else's: RDS, Aurora, Cloud SQL, Azure, Supabase. None of them will load a custom compiled extension, so CREATE EXTENSION postjevsql is not an option there. The sidecar is the way round. You run a small Postgres of your own beside the real one, install the extension there, and let it see the real database's tables through postgres_fdw, Postgres's own way of reading another server's tables as if they were local. Your app connects to the sidecar with any ordinary client.

flowchart LR
  app["your app, psql, a GUI"] -->|"the ordinary protocol"| side
  subgraph side["the sidecar (a Postgres you run)"]
    ext["postjevsql + jev_cache"]
    ft["foreign tables, one per target table"]
  end
  ft -->|"postgres_fdw over verify-full TLS:<br/>only the rows that pass the other filters"| tgt[("the target (managed, untouched)")]
  ext --> J["Jev"]

The split falls out of a rule postgres_fdw already has: it ships a function to the remote only if it is immutable, and every jev* function is volatile. So the ordinary WHERE clauses run on the target, the rows that pass come over, and the judging happens on the sidecar. The cache lives on the sidecar too, and the target is never written to for bookkeeping.

Setting all of that up (the foreign server, the user mappings, one foreign table per target table, kept in step as the target's schema changes) is the job of this crate. It is one engine, called by every launcher: the NixOS module (nix/sidecar.nix), the container image (nix/sidecar-image.nix, whose entrypoint is this CLI's serve), and anyone pointing it at a Postgres they already run.

Aside: what it costs. Nothing here is free, and the contract says so. Every row that passes the target's filters crosses the network. A query whose WHERE has a jev call cannot push its joins, aggregates or LIMIT down to the target, so a count(*) … WHERE jev(…) reads every candidate row. And postgres_fdw does not do two-phase commit: the target commits first, so a failure between the two loses only the sidecar's cache rows.

Try it. cargo test -p postjevsql-sidecar runs the config, secret and sync-plan tests, no database needed. The CLI's own tests, against a real throwaway target and sidecar, are nix develop -c buck2 test //tests:sidecar_cli.

For the people who maintain it

PathWhat
src/The parts with no I/O: the config, the secrets, the sync plan. Chapter 11.
cli/postjevsql-sidecar, the binary that talks to Postgres: converge, sync, serve. Chapter 12.
BUCKThe library, :cli, and :test, the library's unit tests.
Cargo.tomlGenerated by tools/cargo-gen (chapter 20).

The design and its invariants are the root CLAUDE.md's Deployment section.

← Previous: Chapter 9, postjevsql-core/src/ · Up: crates · Next: Chapter 11, postjevsql-sidecar/src/ →