Neon database provisioning

Retired. Production no longer runs on Neon. The hosted managed service and the private tailnet deployment both use the shared in-cluster PostgreSQL cluster described in Managed Service Production Configuration. Neon was retired on cost grounds.

This guide is kept because the create-only provisioning operator is still present in appsoftware-infra under scripts/neon/, so returning to Neon would not mean rebuilding it. Do not run these commands against the live deployment: they would rewrite the two database URLs in each EtherPK Secret back to Neon endpoints.

This guide describes the operator workflow that created the Neon databases used by the hosted managed service and the private etherpkclient.raptor-hen.ts.net tailnet deployment. General self-hosters can use any suitable PostgreSQL provider and never needed this AppSoftware infrastructure workflow.

What the command creates

The appsoftware-infra repository defines three independent Neon projects:

Workload Project Database Database owner
Corporate etherpk-corporate-production etherpk_corporate etherpk_corporate_owner
Managed Sync etherpk-server-production etherpk_server etherpk_server_owner
Private standalone Sync appsoftware-etherpk-server-production appsoftware_etherpk_server appsoftware_etherpk_server_owner

The appsoftware- prefix records that AppSoftware owns and operates the private standalone Server. It does not make that Server part of the public managed EtherPK service.

All three use these reviewed options:

Option Configured value Why
Neon region aws-eu-west-2 Keeps all production databases in AWS London
PostgreSQL Version 17 Pins a reviewed major version instead of taking a changing default
Branch production Gives the production data branch an explicit name
History 21600 seconds Uses the Free plan's maximum six-hour point-in-time history
Minimum compute 0.25 CU Keeps the idle production footprint small
Maximum compute 1 CU Allows initial production workloads to autoscale within a bounded cost
Scale to zero Neon plan default Uses the plan's supported idle timeout
Stored passwords Enabled Allows the operator to obtain the application connection strings
Branch protection Disabled Protected branches are not available on the Free plan

The Free restore window is also limited to 1 GB of data changes. A write-heavy database can therefore have less than six hours available. The version-controlled source of truth is scripts/neon/projects.json in appsoftware-infra. Review that file before the first provisioning run. Its private standalone target is apps/appsoftware-server/appsoftware-etherpk-server/secrets.yaml; the public managed Server Secret remains at apps/appsoftware-server/etherpk-server/secrets.yaml.

protectBranch is deliberately false. If the organisation is moved to a Neon plan which supports protected branches, it can be changed to true and applied by another provisioning run. Do not enable it whilst the organisation remains on Free.

One organisation API key

Create one organisation-scoped API key in the Neon Console. A useful name is appsoftware-infra-local-provisioning. One organisation key manages all three projects.

Do not use a personal or project-scoped key. The command checks Neon's authentication response and refuses either type before inspecting or creating resources.

Keep the key in your password manager. Do not put it in Git, an application SOPS Secret, Kubernetes, CI or Flux. The provisioning command never prints it, but an organisation key has full Neon administrative authority if it is used by another program.

Preview the changes

From the appsoftware-infra repository, read the key without displaying it or adding it to shell history:

read -rsp 'Neon organisation API key: ' NEON_API_KEY
export NEON_API_KEY
echo

pnpm neon:plan

The plan is read-only. It reports each project as:

An invalid project is never silently changed or replaced.

Create missing resources

Run:

pnpm neon:provision

The command creates only resources which are missing. It can create a project and its initial branch / role / database, or a missing role and database inside an otherwise valid project. It waits for Neon to finish each operation and verifies the result. With the current Free-plan configuration it does not request branch protection.

For each database it obtains:

Neon-generated database passwords

Neon generates the password for each PostgreSQL role. For example:

postgresql://etherpk_corporate_owner:npg_...@host/database

In this URL:

The pooled application URI and direct migration URI for one project intentionally share the same role and password. The host distinguishes the pooled and direct connections. Each of the three projects has separate credentials.

Do not generate these passwords yourself, copy one between projects or put one in a ConfigMap. The operator retrieves the complete URLs from Neon and writes them directly into the correct SOPS-encrypted Secret.

neon:provision writes encrypted output itself. If you decrypt a Secret to inspect its connection strings, re-encrypt it before testing or committing:

pnpm encrypt
pnpm status

The command verifies the database, owner, password and required TLS, then replaces only those two fields in the correct application SOPS Secret. Connection strings are not printed. Plaintext is passed to SOPS through stdin and is never written to the repository.

Finish with:

pnpm encrypt
pnpm test
pnpm status
unset NEON_API_KEY

All Secret files must report as encrypted. Review the complete uncommitted diff before committing it. A repeated provisioning run reuses matching resources and leaves already-current Secrets unchanged.

Deletion is always manual

There is no neon:delete or neon:recreate package command. The infrastructure code contains no Neon API call for deleting a database, role, branch, endpoint, snapshot or project. Flux and application startup cannot delete or recreate Neon infrastructure.

If a future reset is authorised:

  1. stop the affected application;
  2. confirm the backup and restore evidence;
  3. record who authorised the data loss;
  4. delete the exact database or project manually in the Neon Console;
  5. run pnpm neon:plan;
  6. run pnpm neon:provision to create what is missing and write the new SOPS connection strings.

If the replacement has a different database or owner name, create it before deleting the old one. The command leaves differently named resources untouched. Verify the new encrypted URLs and a successful application startup, then remove the superseded database and role manually in Neon.

The Free plan does not prevent a branch or project being deleted through the Neon Console or by a different API client. This is why the organisation key must remain outside deployed systems. The repository operator still exposes no destructive operation.

What happens after creation

Commit the reviewed encrypted Secrets and allow Flux to reconcile them. Corporate and both Server images own their application schemas:

  1. Corporate takes its PostgreSQL advisory lock, applies the Corporate baseline and materialises the three configured OAuth clients. Its logs should contain three Materialised OAuth ... messages before the web process starts.
  2. Managed Sync takes its separate advisory lock and applies the Server baseline.
  3. The managed Client can be released after both managed services pass their probes.
  4. Private standalone Sync applies the Server baseline independently in its own Neon project.

Production does not run pnpm oauth:seed:corporate. The initial rollout uses STRIPE_ENABLED=false, so it does not need a Stripe seed. Catalogue creation remains a separate deliberate operator action when paid billing is enabled, as described in Managed Service Production Configuration.

If Corporate reports Dynamic require of "fs" is not supported from build/oauth-seed.js, the database has not caused the failure. That pod is running the superseded image which embedded a local .env loader in its production ESM bundle. Deploy the current image; do not recreate Neon or rotate credentials for this error.