-
Notifications
You must be signed in to change notification settings - Fork 256
NODE
Tune node into desired state and monitor it.
Node is an abstraction of hardware resource, which can be bare metal, virtual machine, or even k8s pods.
There are different types of node in pigsty:
- common node, nodes that managed by pigsty
- admin node, the node where pigsty is installed on and issue admin commands
- infra node, the node where INFRA module installed, admin node is usually the first one of all infra nodes.
Common Node
You can manage nodes with Pigsty, and install modules on them. The node.yml
playbook will adjust the node to desired state.
Some service will be added to all nodes by default:
Component | Port | Description |
---|---|---|
Node Exporter | 9100 | Node Monitoring Metrics Exporter |
HAProxy Admin | 9101 | HAProxy admin page |
Docker Daemon | 9323 | Docker daemon (disable by default) |
Promtail | 9080 | Log collecting agent |
Admin Node
There is one and only one admin node in a pigsty deployment, which is specified by admin_ip
, it is set to local primary IP during configure.
The node will have ssh / sudo access to all other nodes, which is critical, make sure it's fully secured.
INFRA Node
A pigsty deployment may have one or more infra nodes, usually 2 ~ 3 in large production environment.
Infra nodes are specified by the infra
group in the inventory. and infra nodes will have INFRA module installed (dns,nginx,prometheus,grafana,etc...),
The admin node is also used as the only one infra node by default, and infra nodes can be used as 'backup' admin nodes.
PGSQL Node
Node with PGSQL module installed is called PGSQL node, the node and pg instance are 1:1 deployed. And node instance can be borrowed from corresponding pg instances with node_id_from_pg
.
Component | Port | Description |
---|---|---|
Postgres | 5432 | Pigsty CMDB |
Pgbouncer | 6432 | Pgbouncer Connection Pooling Service |
Patroni | 8008 | Patroni HA Component |
Haproxy Primary | 5433 | Primary connection pool: Read/Write Service |
Haproxy Replica | 5434 | Replica connection pool: Read-only Service |
Haproxy Default | 5436 | Primary Direct Connect Service |
Haproxy Offline | 5438 | Offline Direct Connect: Offline Read Service |
Haproxy service
|
543x | Customized Services |
Haproxy Admin | 9101 | Monitoring metrics and traffic management |
PG Exporter | 9630 | PG Monitoring Metrics Exporter |
PGBouncer Exporter | 9631 | PGBouncer Monitoring Metrics Exporter |
Node Exporter | 9100 | Node Monitoring Metrics Exporter |
Promtail | 9080 | Collection of Postgres, Pgbouncer, Patroni logs (Optional) |
vip-manager | - | Bind VIP to the primary |
Add Node
To add a node into pigsty, you need to have nopass ssh/sudo access to the node
bin/node-add [ip...] # add node to pigsty: ./node.yml -l <cls|ip|group>
Remove Node
To remove node from pigsty, you can use:
bin/node-rm [ip...] # remove node from pigsty: ./node-rm.yml -l <cls|ip|group>
Create Admin
If current user does not have nopass ssh/sudo access to the node, you can use another admin user to bootstrap the node:
node.yml -t node_admin -k -K -e ansible_user=<another admin> # input ssh/sudo password for another admin
-
node.yml
: Init node for pigsty -
node-rm.yml
: Remove node from pigsty
There are 10 sections, 58 parameters about NODE
module.
-
NODE_ID
: Node identity parameters -
NODE_DNS
: Node Domain Name Resolution -
NODE_PACKAGE
: Upstream Repo & Install Packages -
NODE_TUNE
: Node Tuning & Features -
NODE_ADMIN
: Admin User & SSH Keys -
NODE_TIME
: Timezone, NTP, Crontab -
HAPROXY
: Expose services with HAProxy -
DOCKER
: Docker daemon on node -
NODE_EXPORTER
: Node monitoring agent -
PROMTAIL
: Promtail logging agent
Parameters
Parameter | Section | Type | Level | Comment |
---|---|---|---|---|
nodename |
NODE_ID |
string | I | node instance identity, use hostname if missing, optional |
node_cluster |
NODE_ID |
string | C | node cluster identity, use 'nodes' if missing, optional |
nodename_overwrite |
NODE_ID |
bool | C | overwrite node's hostname with nodename? |
nodename_exchange |
NODE_ID |
bool | C | exchange nodename among play hosts? |
node_id_from_pg |
NODE_ID |
bool | C | use postgres identity as node identity if applicable? |
node_default_etc_hosts |
NODE_DNS |
string[] | G | static dns records in /etc/hosts
|
node_etc_hosts |
NODE_DNS |
string[] | C | extra static dns records in /etc/hosts
|
node_dns_method |
NODE_DNS |
enum | C | how to handle dns servers: add,none,overwrite |
node_dns_servers |
NODE_DNS |
string[] | C | dynamic nameserver in /etc/resolv.conf
|
node_dns_options |
NODE_DNS |
string[] | C | dns resolv options in /etc/resolv.conf
|
node_repo_method |
NODE_PACKAGE |
enum | C | how to setup node repo: none,local,public |
node_repo_remove |
NODE_PACKAGE |
bool | C | remove existing repo on node? |
node_repo_local_urls |
NODE_PACKAGE |
string[] | C | local repo url, if node_repo_method = local |
node_packages |
NODE_PACKAGE |
string[] | C | packages to be installed current nodes |
node_default_packages |
NODE_PACKAGE |
string[] | G | default packages to be installed on all nodes |
node_disable_firewall |
NODE_TUNE |
bool | C | disable node firewall? true by default |
node_disable_selinux |
NODE_TUNE |
bool | C | disable node selinux? true by default |
node_disable_numa |
NODE_TUNE |
bool | C | disable node numa, reboot required |
node_disable_swap |
NODE_TUNE |
bool | C | disable node swap, use with caution |
node_static_network |
NODE_TUNE |
bool | C | preserve dns resolver settings after reboot |
node_disk_prefetch |
NODE_TUNE |
bool | C | setup disk prefetch on HDD to increase performance |
node_kernel_modules |
NODE_TUNE |
string[] | C | kernel modules to be enabled on this node |
node_hugepage_ratio |
NODE_TUNE |
float | C | node mem hugepage ratio, 0 disable it by default |
node_tune |
NODE_TUNE |
enum | C | node tuned profile: none,oltp,olap,crit,tiny |
node_sysctl_params |
NODE_TUNE |
dict | C | sysctl parameters in k:v format in addition to tuned |
node_data |
NODE_ADMIN |
path | C | node main data directory, /data by default |
node_admin_enabled |
NODE_ADMIN |
bool | C | create a admin user on target node? |
node_admin_uid |
NODE_ADMIN |
int | C | uid and gid for node admin user |
node_admin_username |
NODE_ADMIN |
username | C | name of node admin user, dba by default |
node_admin_ssh_exchange |
NODE_ADMIN |
bool | C | exchange admin ssh key among node cluster |
node_admin_pk_current |
NODE_ADMIN |
bool | C | add current user's ssh pk to admin authorized_keys |
node_admin_pk_list |
NODE_ADMIN |
string[] | C | ssh public keys to be added to admin user |
node_timezone |
NODE_TIME |
string | C | setup node timezone, empty string to skip |
node_ntp_enabled |
NODE_TIME |
bool | C | enable chronyd time sync service? |
node_ntp_servers |
NODE_TIME |
string[] | C | ntp servers in /etc/chrony.conf
|
node_crontab_overwrite |
NODE_TIME |
bool | C | overwrite or append to /etc/crontab ? |
node_crontab |
NODE_TIME |
string[] | C | crontab entries in /etc/crontab
|
haproxy_enabled |
HAPROXY |
bool | C | enable haproxy on this node? |
haproxy_clean |
HAPROXY |
bool | G/C/A | cleanup all existing haproxy config? |
haproxy_reload |
HAPROXY |
bool | A | reload haproxy after config? |
haproxy_auth_enabled |
HAPROXY |
bool | G | enable authentication for haproxy admin page |
haproxy_admin_username |
HAPROXY |
username | G | haproxy admin username, admin by default |
haproxy_admin_password |
HAPROXY |
password | G | haproxy admin password, pigsty by default |
haproxy_exporter_port |
HAPROXY |
port | C | haproxy admin/exporter port, 9101 by default |
haproxy_client_timeout |
HAPROXY |
interval | C | client side connection timeout, 24h by default |
haproxy_server_timeout |
HAPROXY |
interval | C | server side connection timeout, 24h by default |
haproxy_services |
HAPROXY |
service[] | C | list of haproxy service to be exposed on node |
docker_enabled |
DOCKER |
bool | C | enable docker on this node? |
docker_cgroups_driver |
DOCKER |
enum | C | docker cgroup fs driver: cgroupfs,systemd |
docker_registry_mirrors |
DOCKER |
string[] | C | docker registry mirror list |
docker_image_cache |
DOCKER |
path | C | docker image cache dir, /tmp/docker by default |
node_exporter_enabled |
NODE_EXPORTER |
bool | C | setup node_exporter on this node? |
node_exporter_port |
NODE_EXPORTER |
port | C | node exporter listen port, 9100 by default |
node_exporter_options |
NODE_EXPORTER |
arg | C | extra server options for node_exporter |
promtail_enabled |
PROMTAIL |
bool | C | enable promtail logging collector? |
promtail_clean |
PROMTAIL |
bool | G/A | purge existing promtail status file during init? |
promtail_port |
PROMTAIL |
port | C | promtail listen port, 9080 by default |
promtail_positions |
PROMTAIL |
path | C | promtail position status file path |