Skip to content

NODE

Jump to bottom
Feng Ruohang edited this page Feb 11, 2023 · 3 revisions

NODE

Tune node into desired state and monitor it.

Concept

Node is an abstraction of hardware resource, which can be bare metal, virtual machine, or even k8s pods.

There are different types of node in pigsty:

  • common node, nodes that managed by pigsty
  • admin node, the node where pigsty is installed on and issue admin commands
  • infra node, the node where INFRA module installed, admin node is usually the first one of all infra nodes.

Common Node

You can manage nodes with Pigsty, and install modules on them. The node.yml playbook will adjust the node to desired state.

Some service will be added to all nodes by default:

Component Port Description
Node Exporter 9100 Node Monitoring Metrics Exporter
HAProxy Admin 9101 HAProxy admin page
Docker Daemon 9323 Docker daemon (disable by default)
Promtail 9080 Log collecting agent

Admin Node

There is one and only one admin node in a pigsty deployment, which is specified by admin_ip, it is set to local primary IP during configure.

The node will have ssh / sudo access to all other nodes, which is critical, make sure it's fully secured.

INFRA Node

A pigsty deployment may have one or more infra nodes, usually 2 ~ 3 in large production environment.

Infra nodes are specified by the infra group in the inventory. and infra nodes will have INFRA module installed (dns,nginx,prometheus,grafana,etc...),

The admin node is also used as the only one infra node by default, and infra nodes can be used as 'backup' admin nodes.

PGSQL Node

Node with PGSQL module installed is called PGSQL node, the node and pg instance are 1:1 deployed. And node instance can be borrowed from corresponding pg instances with node_id_from_pg.

Component Port Description
Postgres 5432 Pigsty CMDB
Pgbouncer 6432 Pgbouncer Connection Pooling Service
Patroni 8008 Patroni HA Component
Haproxy Primary 5433 Primary connection pool: Read/Write Service
Haproxy Replica 5434 Replica connection pool: Read-only Service
Haproxy Default 5436 Primary Direct Connect Service
Haproxy Offline 5438 Offline Direct Connect: Offline Read Service
Haproxy service 543x Customized Services
Haproxy Admin 9101 Monitoring metrics and traffic management
PG Exporter 9630 PG Monitoring Metrics Exporter
PGBouncer Exporter 9631 PGBouncer Monitoring Metrics Exporter
Node Exporter 9100 Node Monitoring Metrics Exporter
Promtail 9080 Collection of Postgres, Pgbouncer, Patroni logs (Optional)
vip-manager - Bind VIP to the primary

Administration

Add Node

To add a node into pigsty, you need to have nopass ssh/sudo access to the node

bin/node-add [ip...]      # add node to pigsty:  ./node.yml -l <cls|ip|group>

Remove Node

To remove node from pigsty, you can use:

bin/node-rm [ip...]       # remove node from pigsty: ./node-rm.yml -l <cls|ip|group>

Create Admin

If current user does not have nopass ssh/sudo access to the node, you can use another admin user to bootstrap the node:

node.yml -t node_admin -k -K -e ansible_user=<another admin>   # input ssh/sudo password for another admin

Playbooks

Parameters

There are 10 sections, 58 parameters about NODE module.

Parameters
Parameter Section Type Level Comment
nodename NODE_ID string I node instance identity, use hostname if missing, optional
node_cluster NODE_ID string C node cluster identity, use 'nodes' if missing, optional
nodename_overwrite NODE_ID bool C overwrite node's hostname with nodename?
nodename_exchange NODE_ID bool C exchange nodename among play hosts?
node_id_from_pg NODE_ID bool C use postgres identity as node identity if applicable?
node_default_etc_hosts NODE_DNS string[] G static dns records in /etc/hosts
node_etc_hosts NODE_DNS string[] C extra static dns records in /etc/hosts
node_dns_method NODE_DNS enum C how to handle dns servers: add,none,overwrite
node_dns_servers NODE_DNS string[] C dynamic nameserver in /etc/resolv.conf
node_dns_options NODE_DNS string[] C dns resolv options in /etc/resolv.conf
node_repo_method NODE_PACKAGE enum C how to setup node repo: none,local,public
node_repo_remove NODE_PACKAGE bool C remove existing repo on node?
node_repo_local_urls NODE_PACKAGE string[] C local repo url, if node_repo_method = local
node_packages NODE_PACKAGE string[] C packages to be installed current nodes
node_default_packages NODE_PACKAGE string[] G default packages to be installed on all nodes
node_disable_firewall NODE_TUNE bool C disable node firewall? true by default
node_disable_selinux NODE_TUNE bool C disable node selinux? true by default
node_disable_numa NODE_TUNE bool C disable node numa, reboot required
node_disable_swap NODE_TUNE bool C disable node swap, use with caution
node_static_network NODE_TUNE bool C preserve dns resolver settings after reboot
node_disk_prefetch NODE_TUNE bool C setup disk prefetch on HDD to increase performance
node_kernel_modules NODE_TUNE string[] C kernel modules to be enabled on this node
node_hugepage_ratio NODE_TUNE float C node mem hugepage ratio, 0 disable it by default
node_tune NODE_TUNE enum C node tuned profile: none,oltp,olap,crit,tiny
node_sysctl_params NODE_TUNE dict C sysctl parameters in k:v format in addition to tuned
node_data NODE_ADMIN path C node main data directory, /data by default
node_admin_enabled NODE_ADMIN bool C create a admin user on target node?
node_admin_uid NODE_ADMIN int C uid and gid for node admin user
node_admin_username NODE_ADMIN username C name of node admin user, dba by default
node_admin_ssh_exchange NODE_ADMIN bool C exchange admin ssh key among node cluster
node_admin_pk_current NODE_ADMIN bool C add current user's ssh pk to admin authorized_keys
node_admin_pk_list NODE_ADMIN string[] C ssh public keys to be added to admin user
node_timezone NODE_TIME string C setup node timezone, empty string to skip
node_ntp_enabled NODE_TIME bool C enable chronyd time sync service?
node_ntp_servers NODE_TIME string[] C ntp servers in /etc/chrony.conf
node_crontab_overwrite NODE_TIME bool C overwrite or append to /etc/crontab?
node_crontab NODE_TIME string[] C crontab entries in /etc/crontab
haproxy_enabled HAPROXY bool C enable haproxy on this node?
haproxy_clean HAPROXY bool G/C/A cleanup all existing haproxy config?
haproxy_reload HAPROXY bool A reload haproxy after config?
haproxy_auth_enabled HAPROXY bool G enable authentication for haproxy admin page
haproxy_admin_username HAPROXY username G haproxy admin username, admin by default
haproxy_admin_password HAPROXY password G haproxy admin password, pigsty by default
haproxy_exporter_port HAPROXY port C haproxy admin/exporter port, 9101 by default
haproxy_client_timeout HAPROXY interval C client side connection timeout, 24h by default
haproxy_server_timeout HAPROXY interval C server side connection timeout, 24h by default
haproxy_services HAPROXY service[] C list of haproxy service to be exposed on node
docker_enabled DOCKER bool C enable docker on this node?
docker_cgroups_driver DOCKER enum C docker cgroup fs driver: cgroupfs,systemd
docker_registry_mirrors DOCKER string[] C docker registry mirror list
docker_image_cache DOCKER path C docker image cache dir, /tmp/docker by default
node_exporter_enabled NODE_EXPORTER bool C setup node_exporter on this node?
node_exporter_port NODE_EXPORTER port C node exporter listen port, 9100 by default
node_exporter_options NODE_EXPORTER arg C extra server options for node_exporter
promtail_enabled PROMTAIL bool C enable promtail logging collector?
promtail_clean PROMTAIL bool G/A purge existing promtail status file during init?
promtail_port PROMTAIL port C promtail listen port, 9080 by default
promtail_positions PROMTAIL path C promtail position status file path
Clone this wiki locally