Skip to content

Commit

Permalink
add docker to default packages
Browse files Browse the repository at this point in the history
  • Loading branch information
Vonng committed Apr 8, 2022
1 parent 4ef1dca commit 68f8c8d
Show file tree
Hide file tree
Showing 9 changed files with 351 additions and 226 deletions.
167 changes: 167 additions & 0 deletions docs/zh-cn/t-docker.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
# 容器指南

Pigsty v1.5 带有Docker与Kubernetes部署支持,其中,Docker Daemon将默认在管理节点上启用,以供安装更多SaaS服务

您可以使用Docker,快速部署启动软件应用,在容器中,您可以直接使用连接串访问部署于宿主机上的PostgreSQL/Redis数据库。


## 样例:模式迁移工具:ByteBase

```bash
docker run \
--init --name bytebase \
--restart always \
--detach \
--add-host host.docker.internal:host-gateway \
--publish 8080:8080 --volume ~/.bytebase/data:/var/opt/bytebase \
bytebase/bytebase:1.0.2 --data /var/opt/bytebase --host http://localhost --port 8080
```

例如,上述命令将在8080端口启动一个ByteBase,可用于数据库模式迁移。

```bash
# 添加新的Nginx服务,然后更新配置
nginx_upstreams:
- { name: bb, domain: bb.pigsty, endpoint: "10.10.10.10:8080" }

./infra.yml -t nginx_config
ssh meta 'sudo nginx -s reload'

# 在您本地添加解析 /etc/hosts
10.10.10.10 bb.pigsty
```
## 样例:开源代码仓库:Gitlab
请参考[Gitlab Docker部署文档](https://docs.gitlab.com/ee/install/docker.html) 完成Docker部署。
```bash
export GITLAB_HOME=/data/gitlab

sudo docker run --detach \
--hostname gitlab.example.com \
--publish 443:443 --publish 80:80 --publish 23:22 \
--name gitlab \
--restart always \
--volume $GITLAB_HOME/config:/etc/gitlab \
--volume $GITLAB_HOME/logs:/var/log/gitlab \
--volume $GITLAB_HOME/data:/var/opt/gitlab \
--shm-size 256m \
gitlab/gitlab-ee:latest

sudo docker exec -it gitlab grep 'Password:' /etc/gitlab/initial_root_password
```
## 样例:开源技术论坛:Discourse
搭建开源论坛Discourse,需要调整配置 `app.yml` ,重点是SMTP部分的配置
<details><summary>Discourse配置样例</summary>
```yaml
templates:
- "templates/web.china.template.yml"
- "templates/postgres.template.yml"
- "templates/redis.template.yml"
- "templates/web.template.yml"
- "templates/web.ratelimited.template.yml"
## Uncomment these two lines if you wish to add Lets Encrypt (https)
# - "templates/web.ssl.template.yml"
# - "templates/web.letsencrypt.ssl.template.yml"
expose:
- "80:80" # http
- "443:443" # https
params:
db_default_text_search_config: "pg_catalog.english"
db_shared_buffers: "768MB"
env:
LC_ALL: en_US.UTF-8
LANG: en_US.UTF-8
LANGUAGE: en_US.UTF-8
EMBER_CLI_PROD_ASSETS: 1
UNICORN_WORKERS: 4
DISCOURSE_HOSTNAME: forum.pigsty
DISCOURSE_DEVELOPER_EMAILS: '[email protected],[email protected]'
DISCOURSE_SMTP_ENABLE_START_TLS: false
DISCOURSE_SMTP_AUTHENTICATION: login
DISCOURSE_SMTP_OPENSSL_VERIFY_MODE: none
DISCOURSE_SMTP_ADDRESS: smtpdm.server.address
DISCOURSE_SMTP_PORT: 80
DISCOURSE_SMTP_USER_NAME: [email protected]
DISCOURSE_SMTP_PASSWORD: "<password>"
DISCOURSE_SMTP_DOMAIN: mail.pigsty.cc
volumes:
- volume:
host: /var/discourse/shared/standalone
guest: /shared
- volume:
host: /var/discourse/shared/standalone/log/var-log
guest: /var/log

hooks:
after_code:
- exec:
cd: $home/plugins
cmd:
- git clone https://github.com/discourse/docker_manager.git
run:
- exec: echo "Beginning of custom commands"
# - exec: rails r "SiteSetting.notification_email='[email protected]'"
- exec: echo "End of custom commands"
```
</details>
然后,执行以下命令,拉起Discourse即可。
```sql
./launcher rebuild app
```
## 样例:开源社交网站:Mastodon
## 样例:自动的后端API:PostgREST
[PostgREST](https://postgrest.org/en/stable/index.html)是一个自动根据 PostgreSQL 数据库模式生成 REST API的二进制组件。
例如,以下命令将使用docker拉起 postgrest (本地8123端口,使用默认管理员用户,暴露Pigsty CMDB模式)
```bash
docker run --rm --net=host -p 8123:8123 \
-e PGRST_DB_URI="postgres://dbuser_dba:[email protected]/meta" \
-e PGRST_DB_SCHEMA="pigsty" \
-e PGRST_DB_ANON_ROLE="dbuser_dba" \
-e PGRST_SERVER_PORT=8123 \
-e PGRST_JWT_SECRET=haha \
postgrest/postgrest
```
访问 http://10.10.10.10:8123 会展示所有自动生成API的定义,在 [Swagger Editor](https://editor.swagger.io) 中可以自动生成API文档。
`curl http://10.10.10.10:8123/cluster` 会匿名访问数据表`pigsty.cluster`
如果您想要进行增删改查,设计更精细的权限控制,请参考 [Tutorial 1 - The Golden Key](https://postgrest.org/en/stable/tutorials/tut1.html),生成一个签名JWT。
## 样例:数据库模式报表SchemaSPY
使用以下`docker`生成数据库模式报表,以CMDB为例:
```bash
docker run -v /www/schema/pg-meta/meta/pigsty:/output \
andrewjones/schemaspy-postgres:latest \
-host 10.10.10.10 -port 5432 -u dbuser_dba -p DBUser.DBA -db meta -s pigsty
```
然后访问 http://pigsty/schema/pg-meta/meta/pigsty 即可访问Schema报表
31 changes: 20 additions & 11 deletions files/conf/pigsty-auto.yml
Original file line number Diff line number Diff line change
Expand Up @@ -182,7 +182,15 @@ all:
skip_if_unavailable: true
gpgcheck: no
baseurl: https://download.copr.fedorainfracloud.org/results/harbottle/main/epel-$releasever-$basearch/
repo_packages: # which packages to be included # what to download #
- name: docker-ce # for latest docker
description: Docker CE Stable - $basearch
skip_if_unavailable: true
gpgcheck: no
baseurl:
- https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/$basearch/stable
- https://download.docker.com/linux/centos/$releasever/$basearch/stable

repo_packages: # which packages to be included # what to download #
- epel-release nginx wget yum-utils yum createrepo sshpass zip unzip # ---- boot ---- #
- ntp chrony uuid lz4 bzip2 nc pv jq vim-enhanced make patch bash lsof wget git tuned perf ftp lrzsz rsync # ---- node ---- #
- numactl grubby sysstat dstat iotop bind-utils net-tools tcpdump socat ipvsadm telnet ca-certificates keepalived # ----- utils ----- #
Expand All @@ -207,6 +215,8 @@ all:
- pgquarrel-0.7.0-1 pgsql_tweaks_14 pgtap_14 pgtt_14 postgresql-unit_14 postgresql_anonymizer_14 postgresql_faker_14
- safeupdate_14 semver_14 set_user_14 sslutils_14 table_version_14 # pgrouting_14 osm2pgrouting_14
- clang coreutils diffutils rpm-build rpm-devel rpmlint rpmdevtools bison flex # gcc gcc-c++ # - build utils - #
- docker-ce docker-compose kubelet kubectl kubeadm kubernetes-cni helm # - cloud native- #

repo_url_packages: # extra packages from url
- https://github.com/cybertec-postgresql/vip-manager/releases/download/v1.0.1/vip-manager_1.0.1-1_amd64.rpm
- https://github.com/Vonng/pg_exporter/releases/download/v0.4.1/pg_exporter-0.4.1-1.el7.x86_64.rpm
Expand Down Expand Up @@ -240,14 +250,13 @@ all:
- { name: consul, domain: c.pigsty, endpoint: "127.0.0.1:8500" }
- { name: pgweb, domain: cli.pigsty, endpoint: "127.0.0.1:8081" }
- { name: jupyter, domain: lab.pigsty, endpoint: "127.0.0.1:8888" }
app_list: # application nav links on home page
- { name: Pev2 , url : '/pev2' , comment: 'postgres explain visualizer 2' }
- { name: Logs , url : '/logs' , comment: 'realtime pgbadger log sample' }
- { name: Report , url : '/report' , comment: 'daily log summary report ' }
- { name: Pkgs , url : '/pigsty' , comment: 'local yum repo packages' }
- { name: Repo , url : '/pigsty.repo' , comment: 'local yum repo file' }
- { name: ISD , url : '${grafana}/d/isd-overview' , comment: 'noaa isd data visualization' }
- { name: Covid , url : '${grafana}/d/covid-overview' , comment: 'covid data visualization' }
app_list: # application nav links on home page
- { name: Explain , url : '/pev2' , comment: 'postgres explain visualizer' }
- { name: Package , url : '/pigsty' , comment: 'local yum repo packages' }
- { name: YumRepo , url : '/pigsty.repo' , comment: 'local yum repo file' }
- { name: PGLogs , url : '/logs' , comment: 'postgres raw csv logs' }
- { name: Schemas , url : '/schema' , comment: 'schemaspy summary report' }
- { name: Reports , url : '/report' , comment: 'pgbadger summary report' }

docs_enabled: true # setup local document under nginx?
pev2_enabled: true # setup pev2 query visualizer under nginx?
Expand Down Expand Up @@ -372,12 +381,12 @@ all:
#-----------------------------------------------------------------
node_packages: # common packages for all nodes
- wget,sshpass,ntp,chrony,tuned,uuid,lz4,make,patch,bash,lsof,wget,unzip,git,ftp,vim-minimal
- numactl,grubby,sysstat,dstat,iotop,bind-utils,net-tools,tcpdump,socat,ipvsadm,telnet,tuned,pv,jq,perf,ca-certificates
- numactl,grubby,sysstat,dstat,iotop,bind-utils,net-tools,tcpdump,socat,ipvsadm,telnet,tuned,nc,pv,jq,perf,ca-certificates
- readline,zlib,openssl,openssl-libs,openssh-clients,python3,python36-requests,node_exporter,redis_exporter,consul,etcd,promtail
node_extra_packages: [ ] # extra packages for all nodes
node_meta_packages: # extra packages for meta nodes
- grafana,prometheus2,alertmanager,loki,nginx_exporter,blackbox_exporter,pushgateway,redis,postgresql14
- nginx,ansible,pgbadger,python-psycopg2,dnsmasq,coreutils,diffutils,polysh
- nginx,ansible,pgbadger,python-psycopg2,dnsmasq,coreutils,diffutils,polysh,docker-ce,docker-compose
node_meta_pip_install: jupyterlab # extra pip packages to be installed on meta nodes

#-----------------------------------------------------------------
Expand Down
32 changes: 20 additions & 12 deletions files/conf/pigsty-citus.yml
Original file line number Diff line number Diff line change
Expand Up @@ -261,7 +261,15 @@ all:
skip_if_unavailable: true
gpgcheck: no
baseurl: https://download.copr.fedorainfracloud.org/results/harbottle/main/epel-$releasever-$basearch/
repo_packages: # which packages to be included # what to download #
- name: docker-ce # for latest docker
description: Docker CE Stable - $basearch
skip_if_unavailable: true
gpgcheck: no
baseurl:
- https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/$basearch/stable
- https://download.docker.com/linux/centos/$releasever/$basearch/stable

repo_packages: # which packages to be included # what to download #
- epel-release nginx wget yum-utils yum createrepo sshpass zip unzip # ---- boot ---- #
- ntp chrony uuid lz4 bzip2 nc pv jq vim-enhanced make patch bash lsof wget git tuned perf ftp lrzsz rsync # ---- node ---- #
- numactl grubby sysstat dstat iotop bind-utils net-tools tcpdump socat ipvsadm telnet ca-certificates keepalived # ----- utils ----- #
Expand All @@ -286,6 +294,8 @@ all:
- pgquarrel-0.7.0-1 pgsql_tweaks_14 pgtap_14 pgtt_14 postgresql-unit_14 postgresql_anonymizer_14 postgresql_faker_14
- safeupdate_14 semver_14 set_user_14 sslutils_14 table_version_14 # pgrouting_14 osm2pgrouting_14
- clang coreutils diffutils rpm-build rpm-devel rpmlint rpmdevtools bison flex # gcc gcc-c++ # - build utils - #
- docker-ce docker-compose kubelet kubectl kubeadm kubernetes-cni helm # - cloud native- #

repo_url_packages: # extra packages from url
- https://github.com/cybertec-postgresql/vip-manager/releases/download/v1.0.1/vip-manager_1.0.1-1_amd64.rpm
- https://github.com/Vonng/pg_exporter/releases/download/v0.4.1/pg_exporter-0.4.1-1.el7.x86_64.rpm
Expand Down Expand Up @@ -319,14 +329,13 @@ all:
- { name: consul, domain: c.pigsty, endpoint: "127.0.0.1:8500" }
- { name: pgweb, domain: cli.pigsty, endpoint: "127.0.0.1:8081" }
- { name: jupyter, domain: lab.pigsty, endpoint: "127.0.0.1:8888" }
app_list: # application nav links on home page
- { name: Pev2 , url : '/pev2' , comment: 'postgres explain visualizer 2' }
- { name: Logs , url : '/logs' , comment: 'realtime pgbadger log sample' }
- { name: Report , url : '/report' , comment: 'daily log summary report ' }
- { name: Pkgs , url : '/pigsty' , comment: 'local yum repo packages' }
- { name: Repo , url : '/pigsty.repo' , comment: 'local yum repo file' }
- { name: ISD , url : '${grafana}/d/isd-overview' , comment: 'noaa isd data visualization' }
- { name: Covid , url : '${grafana}/d/covid-overview' , comment: 'covid data visualization' }
app_list: # application nav links on home page
- { name: Explain , url : '/pev2' , comment: 'postgres explain visualizer' }
- { name: Package , url : '/pigsty' , comment: 'local yum repo packages' }
- { name: YumRepo , url : '/pigsty.repo' , comment: 'local yum repo file' }
- { name: PGLogs , url : '/logs' , comment: 'postgres raw csv logs' }
- { name: Schemas , url : '/schema' , comment: 'schemaspy summary report' }
- { name: Reports , url : '/report' , comment: 'pgbadger summary report' }

docs_enabled: true # setup local document under nginx?
pev2_enabled: true # setup pev2 query visualizer under nginx?
Expand Down Expand Up @@ -461,12 +470,12 @@ all:
#-----------------------------------------------------------------
node_packages: # common packages for all nodes
- wget,sshpass,ntp,chrony,tuned,uuid,lz4,make,patch,bash,lsof,wget,unzip,git,ftp,vim-minimal
- numactl,grubby,sysstat,dstat,iotop,bind-utils,net-tools,tcpdump,socat,ipvsadm,telnet,tuned,pv,jq,perf,ca-certificates
- numactl,grubby,sysstat,dstat,iotop,bind-utils,net-tools,tcpdump,socat,ipvsadm,telnet,tuned,nc,pv,jq,perf,ca-certificates
- readline,zlib,openssl,openssl-libs,openssh-clients,python3,python36-requests,node_exporter,redis_exporter,consul,etcd,promtail
node_extra_packages: [ ] # extra packages for all nodes
node_meta_packages: # extra packages for meta nodes
- grafana,prometheus2,alertmanager,loki,nginx_exporter,blackbox_exporter,pushgateway,redis,postgresql14
- nginx,ansible,pgbadger,python-psycopg2,dnsmasq,coreutils,diffutils,polysh
- nginx,ansible,pgbadger,python-psycopg2,dnsmasq,coreutils,diffutils,polysh,docker-ce,docker-compose
node_meta_pip_install: jupyterlab # extra pip packages to be installed on meta nodes

#-----------------------------------------------------------------
Expand Down Expand Up @@ -719,7 +728,6 @@ all:
- host all +dbrole_offline 172.16.0.0/12 md5
- host all +dbrole_offline 192.168.0.0/16 md5


pgbouncer_hba_rules: # pgbouncer host-based authentication rules
- title: local password access
role: common
Expand Down
Loading

0 comments on commit 68f8c8d

Please sign in to comment.