Previously, all container volumes initialized via the `container-volume`
service would have local rotating backups performed by pushing `tar`
archives to the `/var/lib/backups/coreos-home-server` directory.
This proved to be a simple and effective mechanism for storing historic
volume state locally; however, the use-case for historic backup is
usually data loss, either partial (e.g. by deleting files inadvertently)
or complete (e.g. by loss of disk), which is likely better mitigated by
more concrete mechanisms of retention.
In addition, this need to store historic volume state locally, in its
totality, proved to be a barrier for performing partial backups, which
is an issue especially for larger volumes.
This commit deprecates this simple, generic volume backup/restore
mechanism, and instead has us rely directly on Rclone reading from the
volume in question.
This allows us to move to a more configurable base, driven by service
configuration files (which can be extended by private configuration)
rather than systemd-driven configuration, which wasn't ever as robust as
it should've been.
This allows for serving any Podman volume over WebDAV, with the ability
to serve a dedicated subdirectory per user. Authentication is provided
by LDAP, and users are expected to be in the `rclone_user` group.
This is, apparently, the easiest solution to set up, while still being
as featureful as required for simple use (readable and PDF archive of
bookmarks).
We would previously use the `letsencrypt` volume used as state by Lego
itself, which contains a number of private files not intended to be
accessed widely; the `letsencrypt-certificates` volume used now contains
only certificate chains and private keys, under dedicated folders.
Container builds using the `container-build@` systemd service will
generally tag any container image built with the `latest` tag, which is
then referred to pervasively in container executions.
However, this tag is overwritten when building new images, and, combined
with how `podman auto-update` will prune old image digests, may cause us
to lack the ability to roll back, automatically or otherwise.
This commit sets a `previous` tag on container re-builds, which should
only generally happen when source files change (due to the `ExecCondition`)
present on the service, which in turn should ensure that images are not
spuriously tagged as such.
This commit adds an `ExecCondition` directive on the `container-build@`
service, used as a pre-requisite for all other Podman-based services,
skipping `podman build` invocations unless local `Containerfile` or any
files in the `container` sub-directories have changed.
Container builds are responsible for the majority of time taken during
boot, even with cache in place; this will help alleviate pressure and
hopefully speed up boot considerably.
Container auto-updates are scheduled for 30 minutes past every hour, or
approximately 30 minutes after `coreos-home-server-update` runs, in
order to give enough time for container builds to complete; only
containers with auto-updates enabled are eligible, however.
This also enables health-checks for Redis, and updates the version to
7.2.
Podman-generated systemd unit files aren't usually generated into
`/etc/systemd/system`, and this directory is preferred for any
pre-existing unit files.
Services will now have an additional set of security-oriented response
headers attached, and cache times re-jigged.
In addition, the `nginx-serve-static@` service has been removed in
favour of `nginx-serve-volume@`, which is simpler to set up and use.
Slidge replaces Spectrum with immense improvement to bridging
capabilities, albeit with only experimental MUC support. Nevertheless,
the current state is sufficiently stable for a complete replacement.
This commit moves the `nginx-proxy-http` service back to separate
`UPSTREAM_HOST`, `UPSTREAM_PORT`, and `UPSTREAM_PATH` variables, which
allows for more granular configuration, e.g. `proxy_redirect` patterns.
This commit switches Hugo to a webhook-based building process, with
support for Github, Gitlab, and Gitea hooks (including local versions of
Gitea) initially. In addition, Hugo-based sites are now intended to be
served under a single volume, with ingress configuration pointing to
sub-paths into the volume.
Documentation for webhook setup and NGINX proxy configuration is still
underway, and will be filled in later.
This commit unifies the `UPSTREAM_HOST` and `UPSTREAM_PORT` environment
variables to a new `UPSTREAM_ENDPOINT` variable, making additional
customizations (such as a `proxy_pass` to a sub-path) possible.
This commit extends our `coreos-home-server-update` script with support
for updating host directories with configuration collected across
multiple remote directories of the same name. This will, essentially,
allow for extending systemd services with custom configuration, as
sometimes required of base systemd service files.
This sets the stage for moving common authentication from IMAP/Dovecot
to LDAP, which allows for more control over user information, as well as
a basic form of RBAC.
No services are currently set up to support LDAP -- support will follow
soon after this commit.
This commit removes host configuration that is only useful for my own
personal use-cases, and leaves base and virtual hosts, which are more
commonly useful. It is intended that additional hosts are tracked as
either submodules or subtrees, as required by individual use-cases.
This commit adds a new service for Gitea, exposing HTTP and SSH ports by
default (SSH over 7920), and accepting authentication via the local SMTP
server. No users are otherwise created by default, and administration is
expected to happen either via CLI, or via a custom admin user.
Container volume backup logic has been moved to a (largely equivalent)
external script, allowing for future expansion of functionality. In
addition, a `rclone-pull@` service has been added and set up as a default
dependency for the `container-volume-restore@` service, allowing for
automatic set up of servers based on latest remote backups.
This commit adds two services, `grafana` and `prometheus`, and sets up
some existing services (`dovecot` and `prosody`) to expose metrics into
Grafana. In addition, systemd services have been added to facilitate
registering metrics for services into Prometheus, as well as
automatically provisioning Grafana dashboards based on static JSON
representations.
This work will continue to evolve as more services gain proper Grafana
dashboards, and Loki is also integrated for access to the systemd
journal.
By default, a `[email protected]` systemd service is included,
for building Hugo-based sites as stored in a local Git repository (as
served by the `git` service), and into a volume (which is intended to be
served by the `nginx-serve-volume` service).
This brings a number of changes and improvements, and moves from MariaDB
to SQLite for storage, which requires manual migration for pre-existing
deployments using `prosody-migrator`.
Pointers to the latest backup are useful for restore operations, but
should be ignored in all other cases, and thus are more appropriately
handled as symbolic links.
This commit adds a new systemd service, `[email protected]`,
templated against the absolute path of a directory to keep in sync with
a (presumably) remote store.
Support for Backblaze B2 endpoints has been set up by default, but the
specific remote type can be configured via the `RCLONE_REMOTE_TYPE` host
variable. In addition, a default-passthrough remote that encrypts data
against a static password and salt has been defined under the `crypt`
name, and can also be used as the `RCLONE_DEST` of choice.
We no longer copy service directories into `/etc/coreos-home-server` if
these have not had their respective `spec.bu` files included; these
directories are not needed in these cases, and would be erroneously
considered as eligible in subsequent `coreos-home-server-update`
invocations.
Container volume backups will now be skipped if no changes have been
made in source files against the latest backup. In addition the default
timer has been changed for performing backups once a day, at 02:00,
likely a time where there's less traffic on the server.
This commit contains a fairly large diff for a fairly small change:
moving the `config/common` directory to `host/base` to better reflect
its intended use, and promoting `config/service` to the root directory.
These changes unlock some improvements in `coreos-home-server-update`
processes, which will (assuming `/etc/coreos-home-server/base` exists)
keep host-wide systemd services in sync in addition to service-specific
ones.
Changes have been make to the `Makefile` and a few other places where
`config/common` was referenced, but most of this work is renames that
are not intended to break compatibility with new or running servers.
Most importantly, this helps make WebRTC calls in XMPP more reliable
when either (or both) endpoints are behind NAT (as is the case with most
mobile devices), and avoids depending on a third-party service.
Default configuration has been applied in the virtual environment file;
this allows for setting up most host-dependent configuration easily.
This service allows for easy serving of static content in a volume,
typically HTML files in directory structures mapping to the navigation
structure for the content served.
Alex Palaistras