Databases and Storage

State is the part of the system that cannot be redeployed from git.

2guides Feed

Everything else in a stack can be rebuilt from a git repository. The database cannot, which is why this part of the system deserves more care than the rest of it usually gets.

The recurring decisions are the same regardless of engine. Where does it listen, and can anything outside the host reach it. Which credentials does the application use, and can they alter the schema. How is it backed up, and has that backup been restored. Answering those three well matters more than the choice of engine for most self-hosted systems.

On engine choice: reach for PostgreSQL for anything relational or transactional, including a lot of what people use document stores for. Reach for a dedicated search engine when relevance, typo tolerance and faceted filtering are a product feature rather than a query filter. Reach for a cache when you have measured the thing you are caching.

The failure that costs most is backups nobody has restored. Scheduled dumps that run green for a year and cannot be loaded are common, and the cause is usually mundane — a misconfigured storage endpoint failing quietly, a dump file that has been zero bytes since a flag changed, a database tarred while it was running.

The second is access control. Development starts with a superuser connection because it is convenient, production inherits it, and a SQL injection that would have been a data leak becomes a host compromise instead. Separating the schema owner from the application login takes ten minutes and closes that gap permanently.

Start here

How to Back Up Docker Volumes Automatically

A nightly backup for a self-hosted Docker host: database dumps taken with the right tools, file volumes snapshotted with restic, a systemd timer to run it, and a restore test that proves it works.

Intermediate · 4 min read

Beginner to advanced

A reading order that builds Databases knowledge in the sequence it is actually needed.

  1. Start How to Back Up Docker Volumes Automatically 4 min
  2. Build How to Run Meilisearch in Production with Docker 4 min

Common problems

Failure modes that come up repeatedly with Databases, and where each one is solved.

Backups run green for a year and cannot be restored
Nobody tried. Restore into a scratch container on a schedule and query the result — it is the only check that means anything.
The application connects with an over-privileged account
Convenient during development, unbounded during an incident. Separate the schema owner from the login role and grant only what the app uses.
A data service is exposed on a public interface
Bind to loopback or a private interface, and remember that Docker's published ports do not respect the host firewall by default.

Latest Databases guides

Browse the archive
  • advanced

    How to Run PostgreSQL Securely on a VPS

    Keep the listener off the public internet, get pg_hba.conf right, require TLS for remote connections, and give the application a role that cannot drop your tables — on PostgreSQL 17 and Ubuntu 24.04.

    4 min
  • intermediate

    How to Run Meilisearch in Production with Docker

    A Meilisearch deployment that survives a real workload — master key handling, scoped API keys, the index settings you must set before loading data, snapshots, and what its memory usage actually means.

    4 min AI-assisted

Frequently asked questions

Can PostgreSQL handle full-text search instead of a search engine?

For filtering and exact-ish matching within an application, yes, and it saves you a service. For typo tolerance, relevance ranking and sub-50-millisecond faceted search that users notice, a dedicated engine is a different class of result.

How often should backups be tested?

Monthly, restoring into a scratch instance and running a real query. The failure this catches — backups that have been green for a year and cannot be restored — is common and entirely preventable.

Where should backups be stored?

Somewhere with a different failure domain from the server: another provider, or at least another account. Use append-only credentials where the provider supports them so a compromised host cannot delete its history.

Sources

  1. PostgreSQL documentation PostgreSQL Global Development Group Primary source
  2. Meilisearch documentation Meilisearch Primary source
  3. Redis documentation Redis Primary source