Jōkamachi Systems

Why Kubernetes?

Mon, 01 Jan 0001 00:00:00 +0000

Jōkamachi Systems has roots in functional programming and classic Linux systems administration. The natural gravitational pull of that combination is towards prescriptive tools — Terraform, NixOS, terranix. You declare how the world ought to be, run the apply, and the tool drives reality towards your specification. Apply once. Done.

This works beautifully when reality cooperates by holding its shape.

The polite fiction of a snapshot

In What functional programmers get wrong about systems (2026), Ian Duncan observes that

Overview

Mon, 01 Jan 0001 00:00:00 +0000

This booklet covers how a Jōkamachi cluster is laid out: control plane, worker pools, multi-tenancy, isolation between customers, and the deliberately boring networking choices that hold the whole thing together.

The booklet you read before you trust us with your workloads.

Overview

Mon, 01 Jan 0001 00:00:00 +0000

This booklet introduces the technologies that make a Jōkamachi cluster run: Kubernetes, the container runtime, the network plane, storage, ingress, and the supply-chain pieces around them.

Written for someone who has heard of Kubernetes but has never had to operate one.

Overview

Mon, 01 Jan 0001 00:00:00 +0000

The booklet covers, with numbers:

Hetzner Load Balancer vs. custom NAT gateway — when each is the right choice for your traffic mix, with throughput and latency measurements under sustained load.
Shared, dedicated, and performance CPU cores — what each costs, what each delivers, and where the boundaries actually live under contention from a noisy neighbour.
Storage and network throughput between Hetzner cloud VMs, dedicated servers, and the connection between them in the same datacentre.
End-to-end SLO numbers for a reference workload running on each configuration we sell.

Overview

Mon, 01 Jan 0001 00:00:00 +0000

This booklet walks you from a fresh Hetzner account to a working Jōkamachi Kubernetes cluster you operate yourself. No prior Kubernetes experience assumed.

If you’re evaluating whether Kubernetes is the right shape for your team at all, start with the introduction booklet first — it covers the why before this booklet covers the how.

Why Kubenix?

Mon, 01 Jan 0001 00:00:00 +0000

Kubernetes runs whatever spec you hand it; the question this chapter answers is what kind of artefact the spec should be. The short answer is: not a pile of Helm charts.

Helm charts are the Kubernetes ecosystem’s standard way to ship a service. Each chart, in isolation, works fine. But a real cluster is not one service; it’s many services, often inter-dependent, often sharing config — DNS names, certificates, secrets, network policies, RBAC bindings — that no individual chart owns.

You don't need KAOS!

Mon, 01 Jan 0001 00:00:00 +0000

The motivation behind KAOS is to derive and automate collective operating experience. But “collective experience” is built one incident at a time, and every individual contribution starts the same way: someone — a human, or a robot — runs kubectl against the cluster, or receives an alert notification, notices something is off, and fixes it.

Real experience is earned through trial. You can run those kubectl commands yourself, or you can hand them to an agentic CLI with access to the cluster — either path produces experience. Each path leaves a different gap, though. Humans produce organisational knowledge, but bring it with them when they leave the organisation. Standalone agentic CLIs don’t retain knowledge across sessions without extensive prompting. And human operators of agentic CLIs tend to overlook the solution entirely — because the agent did the fixing, and you only learn through pain.

How far one prompt takes you

Mon, 01 Jan 0001 00:00:00 +0000

If you’ve never tried this: open a terminal against a cluster you have admin on, start whichever agentic CLI your team already trusts (Claude Code, Aider with a shell tool, an in-house equivalent), and type the following prompt — exactly the following prompt, with no system message, no playbook, no pre-defined toolbelt:

❯ Are there any incidents in the Kubernetes cluster? Use kubectl, it's already authenticated.

A typical frontier model, in a typical agentic CLI, will respond with something close to this:

How an agentic skill is formed

Mon, 01 Jan 0001 00:00:00 +0000

Frontier models, given access to a cluster, will read the event log, walk every namespace, and recognise an error pattern when they see one. They do this without much prompting — the ability is built into the model. What they don’t have, out of the box, is incident- management context. They don’t track timelines, they don’t know whether an issue is ongoing or resolved, and they don’t remember whether the same pattern was seen and fixed last Tuesday.

Property testing Kubenix

Mon, 01 Jan 0001 00:00:00 +0000

The chapter covers the property-testing techniques we apply to Kubenix-defined clusters: invariants that span services, the test runner setup, examples of properties we ship by default, and how to add tenant-specific properties.

A property test for a cluster looks much like a property test for any other piece of typed code: state an invariant in the type system or in code, run a generator that produces representative cluster states, and check that the invariant holds for every generated state. The difference is the substrate — the “code” being tested is the desired state of a cluster, not a function — and the value of the technique scales with how much your cluster’s correctness depends on relationships you can’t see by reading any one chart.

Guardrails and incident maturity

Mon, 01 Jan 0001 00:00:00 +0000

KAOS is never granted blanket autonomy. Its authority is scoped per class of incident, and each class is promoted through three states as the playbook matures:

Propose — KAOS describes what it would do; a human reviews and takes the action manually.
Approve — KAOS proposes an action with the exact change prepared; a human signs off and KAOS applies it.
Act — KAOS handles the incident end-to-end, logging the decision; a human reviews after the fact.

Promotion happens incident-class by incident-class, only after the class has been resolved enough times the same way that “same way” is a fact, not a guess.

What KAOS sees, and what it doesn't

Mon, 01 Jan 0001 00:00:00 +0000

Training on incidents

KAOS gets better the more incidents it has seen. We train on incidents we resolve across all customers to improve the substrate for every customer; the alternative is that each customer’s KAOS starts from scratch, which would be a worse product. We don’t pretend otherwise.

About

Mon, 01 Jan 0001 00:00:00 +0000

What we do

Jōkamachi Systems runs production Kubernetes for teams that need high-availability clusters at predictable cost, in availability zones they choose. We provision, operate, upgrade, and on-call your clusters on Hetzner cloud VMs and bare-metal servers, across all availability zones we support.

You ship workloads. We keep the lights on.

How we run

Substrate. Hetzner cloud VMs for elastic capacity; Hetzner dedicated servers for steady-state workloads where bare-metal pricing beats every hyperscaler at every spec.
Availability zones. Pick the zones that fit your data-sovereignty and latency requirements; we deploy there. Hetzner today; the platform itself isn’t pinned to a continent.
Stack. A fully open-source Kubernetes platform that we maintain and you can read end-to-end. No proprietary control plane, no licence server.
Pricing. Predictable. A fixed management fee on top of the lowest-cost hardware you can find. Free if you’d rather self-host the platform — see plans for the full breakdown.

Why we exist

We think production Kubernetes should be affordable, sovereign, and on your terms — and that all three are only achievable off the public cloud. The longer version of that argument lives on the Philosophy page.

Philosophy

Mon, 01 Jan 0001 00:00:00 +0000

The historical jōkamachi

A jōkamachi (城下町, “town below the castle”) is a class of fortified settlement that emerged in Japan during the Sengoku period and was systematically deployed across the country throughout the Edo period. Rather than growing organically, a jōkamachi was a deliberately engineered system: a complete civic, military, and economic environment organized in concentric layers around a central stronghold.

At the core sat the castle itself — the seat of the daimyō and the administrative kernel of the entire settlement. Every other component of the town existed to serve, supply, or defend this center.

Pricing

Mon, 01 Jan 0001 00:00:00 +0000

Same platform. Pick your operator.

Every Jōkamachi cluster runs the same open-source software. What changes between plans is who watches the cluster — your operators, our operators, KAOS, or any combination of the three.

Sign up

Mon, 01 Jan 0001 00:00:00 +0000

Pick a plan, pick a response SLA, and tell us about the workload. We read every submission ourselves; expect a reply within one business day.