An in-memory key-value store with gRPC API.
$ make build
built ./dist/bin/keyquarryBy default, the server listens on localhost:33969, without TLS:
$ ./dist/bin/keyquarry serve
time=2023-11-26T15:13:50.083-05:00 level=INFO msg="starting server" address=:33969 config.ssl-certfile="" config.ssl-keyfile="" config.backup="" config.MaxNumberOfKeys=0 config.DefaultMaxValueSize=1000000 config.History.RevisionLimit=5Alternatively, you can specify a unix socket like:
$ ./dist/bin/keyquarry serve --listen unix:///tmp/keyquarry.sockOr bind to all interfaces, accepting connections from anywhere:
$ ./dist/bin/keyquarry serve --listen :39969Set a key foo to value bar:
$ ./dist/bin/keyquarry client set foo bar
{"success":true,"isNew":true}Get the value of foo:
$ ./dist/bin/keyquarry client get foo
barUpdate the value of foo:
$ ./dist/bin/keyquarry client set foo baz
$ ./dist/bin/keyquarry client get foo
bazGet metadata for foo, indenting the output with two spaces:
$ ./dist/bin/keyquarry client --indent=2 info foo
{
"key":"foo",
"hash":16101355973854746,
"created":{"seconds":1704058237,"nanos":591124661},
"version":1,
"size":3,
"content_type":"text/plain; charset=utf-8"
}Note: version is incremented whenever the value of a key is updated, based
on the hash of the value.
Lock foo, fail to update its value, then unlock and delete foo:
$ ./dist/bin/keyquarry client lock foo 30s
{"success":true}
$ ./dist/bin/keyquarry client --client-id=bar set foo baz
time=2023-11-29T14:57:53.813-05:00 level=ERROR source=/home/edward/sdk/keyquarry/cmd/root.go:498 msg="rpc error: code = PermissionDenied desc = key is locked"
exit status 1
$ ./dist/bin/keyquarry client unlock foo
{"success":true}
$ ./dist/bin/keyquarry client delete foo
{"deleted":true}By default, --client-id is set as {user}@{hostname}. You can override this.
Only the client ID that locked a key can unlock it, extend the lock timeout,
or get/set the value.
You can persist the current state of the server to a SQLite or Postgres
database. First, set KEYQUARRY_SNAPSHOT_ENABLED=true.
Set KEYQUARRY_SNAPSHOT_DATABASE environment variable to your connection
string (ex: sqlite:///path/to/file.db or
postgres://user:password@host:port/dbname).
By default, the server will load this state on start (if any snapshots exist),
and save its state on shutdown. To disable loading on start, set
KEYQUARRY_START_FRESH=true. To create new snapshots periodically,
set KEYQUARRY_SNAPSHOT_INTERVAL with a duration (ex: 5m). If no interval
is specified, a snapshot will only be created on shutdown.
On startup, it will look for the most recent snapshot record based on KEYQUARRY_NAME (by
default, set as {user}@{host}), attempting to restore from that data.
Adds a new client ID to the server. This is used to identify the client
in other methods for the purpose of locking/unlocking keys. Subsequent
calls of other methods will fail if the client ID is not registered,
or included in the request metadata as client_id.
Creates a new key with the provided value. If the key already exists,
it will be updated.
If lock_duration is provided, the key will be locked for that duration,
and automatically unlocked after it has passed.
If lifespan is provided, the key will be deleted after that duration. The
lifespan will be extended if the key is updated before it expires. Subsequent
calls to Set with a lifespan will reset the lifespan to the given duration.
This takes precedence over lock_duration, in that the key may be deleted
prior to being automatically unlocked, if the lifespan is shorter than the
lock duration.
If a key is locked, only the client_id that set the lock can update the key.
Locks a key for the given duration. If the key is already locked, the lock will be extended by the given duration. If the key is not already locked, a new lock is created. After the lock duration has passed, the key will be automatically unlocked.
Locked keys can only be updated by the same client_id that created the lock.
If the lock duration is shorter than the lifespan of the key, the key may be deleted prior to the lock expiring.
Creating/updating a lock will not reset the lifespan of the key.
Unlocks a key. If the key is not locked, nothing will happen. Only the
same client_id that created the lock can unlock the key.
Unlocking a key will not reset the lifespan of the key.
Gets the value of a key. If the key does not exist, or the key is
currently locked by a different client_id, this will return an error.
Inspect returns metadata about a key. If the key does not exist, this will return an error.
If include_value is true, the value of the key will be included in the response,
unless the key is currently locked by another client_id, in which case an error
will be returned.
If include_metrics is true, additional access/set/lock data will be included in
the response.
Deletes a key. If the key does not exist, or is currently locked
by a different client_id, this will return an error.
Indicates whether a key exists.
Gets the value of a key, and deletes the key. If the key does not exist, or is
currently locked by a different client_id, this will return an error.
Returns a list of keys. Results can be filtered with pattern, which,
if provided, must be a valid regex for regexp.MatchString. Limit the
number of results with limit (0 for no limit).
Returns a stream of events. Watch specific keys with keys (if empty, all
keys will be watched). Filter for specific events with events (if empty,
all events will be watched).
Returns a stream which will emit the value of a key whenever it is updated.
The current value of the key will be emitted immediately. Additional metadata
about the key is also provided. Values will only be sent when a key is
created, updated, deleted, expired or expunged. Use KeyEvent to determine
whether the value was actually updated, and when the key no longer exists.
If the key is locked by another client_id, values will not be sent until
the key is unlocked. For an Unlock event, the current value will be
sent immediately.
For Deleted, Expired or Expunged events, the value of the key at
the time of the event will be sent, whether or not it has changed.
Gets the value of a key for a specific revision. If the key does not exist, or the revision does not exist, this will return an error.
Returns metrics related to a key. If the key does not exist, this will return an error.
Example YAML configuration:
listen_address: localhost:33969
name: foo@bar
graceful_shutdown_timeout: 30s
log_level: NOTICE
log_json: false
log_events: false
event_stream_buffer_size: 1000
event_stream_send_timeout: 1s
event_stream_subscriber_limit: 0
snapshot:
database: postgres://keyquarry:keyquarry@localhost:5432/keyquarry
enabled: true
interval: 5m
monitor_address: localhost:33970
metrics: false
expvar: false
pprof: false
trace: false
service_name: keyquarry
revision_limit: 5
max_key_length: 1024
max_value_size: 1000000
min_lock_duration: 5s
max_lock_duration: 5m
min_lifespan: 5s
max_keys: 10000
prune_interval: 1h
prune_at: 10000
prune_to: 9000
eager_prune_at: 20000
eager_prune_to: 15000Each of these options can be set as an environment variable, or in a YAML
configuration file. The environment variable name is the same as the option,
prefixed with KEYQUARRY_, uppercased, with any period replaced by an underscore.
For example, listen_address can be set as KEYQUARRY_LISTEN_ADDRESS in the
environment, while snapshot.database would be KEYQUARRY_SNAPSHOT_DATABASE
in the environment.
nameservice_namelisten_addressgraceful_shutdown_timeoutlog_levellog_jsonlog_eventsevent_stream_buffer_sizeevent_stream_send_timeoutevent_stream_subscriber_limitmonitor_addressprometheusexpvarpproftracemax_key_lengthmax_keysmax_value_sizemin_lock_duration
$ docker build -t keyquarry:dev .(Or docker compose build server)
$ docker compose up -dThe compose file enables telemetry by default (KEYQUARRY_TRACE=true), and traces
can be viewed in jaeger at http://localhost:16686
You can monitor events with the client with docker compose logs -f client.