Skip to content

Latest commit

 

History

History
173 lines (116 loc) · 5.13 KB

influxdb-post-processing.md

File metadata and controls

173 lines (116 loc) · 5.13 KB
Raw

Tutorial: InfluxDB 1.8 post-processing

User credentials and access rights do not survive a round trip through backup and restore. This is a known issue and the lack of activity suggests it is unlikely to be fixed.

Technically, the same statement applies to continuous queries. However, IOTstackBackup provides a solution for CQs.

background

What continuous queries, user credentials and access rights have in common is that they are stored in:

~/IOTstack/volumes/influxdb/data/meta/meta.db

The meta.db file is included in the portable backup generated by influxd and it is faithfully archived and restored by IOTstackBackup. But, for some unknown reason, influxd does not reconstitute those properties from the portable backup. This means it is reasonably likely that all properties which are stored in meta.db will suffer a similar fate.

workarounds

IOTstackBackup provides two distinct workarounds for this problem:

  1. Automatic extraction and restoration of continuous queries; and
  2. A manual workaround in the form of an optional epilog file where you can define users and grants.

The workarounds are supported in both:

  • iotstack_restore_influxdb; and
  • iotstack_reload_influxdb.

To take advantage of the optional epilog you need to create a text file at the following path:

~/IOTstack/services/influxdb/iotstack_restore_influxdb.epilog

The epilog file is expected to contain InfluxQL commands. There is no mechanism for automating its construction. You have to set it up yourself, by hand, and take responsibility for its validity.

If the epilog file exists when iotstack_restore_influxdb runs, its contents will be passed to the container for execution. This occurs after the databases have been restored.

You can test your epilog file like this:

$ cd ~/IOTstack/services/influxdb
$ docker cp iotstack_restore_influxdb.epilog influxdb:.
$ docker exec influxdb influx -import -path "iotstack_restore_influxdb.epilog"

The epilog file is kept in the services directory so it is included in the general backup and restore. Because the general restore runs before the database restores, the epilog file will already be in place when the InfluxDB restore runs.

practical example

Make the following assumptions:

  1. You are working in the influx CLI:

    $ influx
>

  2. You define the following users:

    > CREATE USER "dba" WITH PASSWORD 'supremo' WITH ALL PRIVILEGES
> CREATE USER "nodered" WITH PASSWORD 'nodereds_big_secret'
> CREATE USER "grafana" WITH PASSWORD 'grafanas_little_secret'

  3. You have a power database to which you grant the following access rights:

    > GRANT WRITE ON "power" TO "nodered"
> GRANT READ ON "power" TO "grafana"

The content of iotstack_restore_influxdb.epilog would simply be those same commands:

CREATE USER "dba" WITH PASSWORD 'supremo' WITH ALL PRIVILEGES
CREATE USER "nodered" WITH PASSWORD 'nodereds_big_secret'
CREATE USER "grafana" WITH PASSWORD 'grafanas_little_secret'
GRANT WRITE ON "power" TO "nodered"
GRANT READ ON "power" TO "grafana"

Note:

  • InfluxQL follows SQL conventions on comments. The following are valid comments:

     -- this is a valid comment
 /* this is also a valid comment but it must be on one line */

    These comments are invalid:

     # this will cause influx to complain
 // this will also cause influx to complain
 /* influx does
    not like C-style
    multiline comments
 */

command reconstruction

If you want to take advantage of the epilog workaround but you do not have a record of the necessary commands, you can usually reconstruct them by hand.

users

Usernames can be retrieved via:

> SHOW USERS
user    admin
----    -----
dba     true
nodered false
grafana false

Passwords are stored as hashes so there is no way to recover and display the plain-text versions.

Templates:

  • admin user:

     CREATE USER "«user»" WITH PASSWORD '«password»' WITH ALL PRIVILEGES

  • non-admin user:

     CREATE USER "«user»" WITH PASSWORD '«password»'

Note:

  • The use of both single- and double-quotes is intentional. Usernames must be wrapped in double-quotes and passwords must be wrapped in single quotes.

grants

Grants can be retrieved by iterating the non-admin users:

> SHOW GRANTS FOR nodered
database privilege
-------- ---------
power    WRITE

> SHOW GRANTS FOR grafana
database privilege
-------- ---------
power    READ

Template:

GRANT «privilege» ON "«database»" TO "«user»"

continuous queries

The iotstack_backup_influxdb and iotstack_reload_influxdb scripts extract your continuous queries and save them into the file continuous-queries.influxql, along with the other files InfluxDB creates when asked to produce a portable backup.

That file is then included in the .tar produced by iotstack_backup_influxdb, from which it is extracted when iotstack_restore_influxdb runs.

Note:

  • If you have been using the epilog to reload your queries, InfluxDB will not object as the (implicit) duplicate is created when the epilog is run.