haumea/zrepl: reduce snapshot count

[vcunat]

Reduce snapshot count. We repeatedly run out of space on Haumea.

[vcunat]

Oops, I expect we need have the regex = part here as well.

[vcunat]

Or maybe not. Their examples set doesn't have it:
https://zrepl.github.io/configuration/prune.html#pruning-policies

[vcunat]

Maybe we should drop this line anyway. In case the remote end is down, we probably want to keep pruning the sender to reduce the risk of running out of space. Also it might reduce the time to sync up when the receiver becomes reachable again.

[vcunat]

Let me dump a bit about why.

The problematic situations that we see have lots of data unique to (some) in-between snapshots, i.e. dropping some of those snapshots (manually) could release lots of space. Consequently:

• making/transferring snapshots less often should decrease the total transfer amount and reduce this pressure on the remote
• larger spacing between snapshots kept on Haumea should decrease the total space needed there, perhaps even if we didn't significantly decrease the total time span covered by snapshots

EDIT:

• interesting note is that those problematic snapshots seem to have also larger total size (i.e. if we didn't have any snapshotting, the disk usage would be larger at those points)

[vcunat]

🤔 as for the backup location(s), it feels wasteful to keep every week uniformly for a year. Can you see any reason for it? I'd intuitively again go for some exponentially increasing spacing. I assume we can afford more space than on Haumea itself, so e.g. this slower one?

              "2x1h"
              "2x2h"
              "2x4h"
              "4x8h"
              # At this point the grid spans 2 days (-2h) by 10 snapshots.
              # (See note above about 8h -> 24h.)
              "2x1d"
              "2x2d"
              "2x4d"
              "2x8d"
              "2x16d"
              "2x32d"
              "2x64d"
              "2x128d"
              # At this point we keep 26 snapshots spanning 384--512 days (depends on moment),
              # with exponentially increasing spacing (almost).

Perhaps note the docs that the specified intervals do not overlap. All the intervals are stacked in the specified order and multiplicity, forming a fixed grid.

[vcunat]

Not sure if it's worth trying to explain the weirdness. I base it not on actual experience but on definition in their docs – and how such a model behaves then when running continuously.