Skip to content

Commit 0e55dd0

Browse files
author
Yannic Nevado
committed
initial commit
0 parents  commit 0e55dd0

15 files changed

+784
-0
lines changed

‎Readme.md‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
# Custom Postgres Operator
2+
3+
This repository contains a custom setup for an [HA](https://www.postgresql.org/docs/current/high-availability.html) [Postgres](https://www.postgresql.org/) cluster, based on [Postgres Operator](https://github.com/zalando/postgres-operator) from [Patroni](https://github.com/zalando/spilo).
4+
5+
The Cluster runs on an [K8s](https://kubernetes.io/) Cluster which is managed by [Rancher](https://rancher.com/) and hosted at the infrastructure of Proventa AG.
6+
7+
8+
## Setup on K8s cluster(Rancher)
9+
10+
* *Projectname:* zalando-postgres
11+
* *Namespace:* zalando-postgres
12+
* 3 Nodes
13+
* *LoadBalancer:* [Metallb](https://metallb.universe.tf)
14+
* access actual Postgres master over external-IP and specified port
15+
* required VPN connection!!
16+
17+
## Installation
18+
19+
The installation will be executed by applying manifests in a specific order, based on the operator installation from patroni. The setup is deployed to an specific namespace named *zalando-postgres* which must be ceated on the K8s cluster. You also can setup the operator(+Postgres cluster) with Helm chart or Operator Lifecycle Manager.
20+
(fyi: you can also set the current-context for kubectl, here ns=zalando-postgres for example)
21+
22+
### Create the operator
23+
24+
```shell
25+
# operator configuration
26+
$ kubectl create -f manifests/configmap.yaml --namespace zalando-postgres
27+
# identity and permissions
28+
$ kubectl create -f manifests/operator-service-account-rbac.yaml --namespace zalando-postgres
29+
# deploy operator to K8s
30+
$ kubectl create -f manifests/postgres-operator.yaml --namespace zalando-postgres
31+
```
32+
33+
### Check operator status
34+
35+
* get all pods in the `namespace zalando-postgres` with the operator label `name=postgres-operator`
36+
37+
```shell
38+
$ kubectl get pod -l name=postgres-operator --namespace zalando-postgres
39+
```
40+
41+
### Create Postgres cluster
42+
43+
* create a Postgres cluster from [postgres manifest](manifest/minimal-postgres-manifest.yml) with the specified values, like databases, users and *postgresql.conf* parameters
44+
45+
```shell
46+
$ kubectl create -f manifests/minimal-postgres-manifest.yaml --namespace zalando-postgres
47+
```
48+
49+
### Check Postgres cluster status
50+
51+
```shell
52+
# check the deployed cluster
53+
$ kubectl get postgresql
54+
# check created database pods with spilo-role (master, replica)
55+
$ kubectl get pods -l application=spilo -L spilo-role --namespace zalando-postgres
56+
# check created service resources and get ext. loadbalancer ip and port
57+
$ kubectl get svc -l application=spilo -L spilo-role --namespace zalando-postgres
58+
```
59+
60+
### Update Postgres cluster
61+
62+
You can update the Postgres cluster with the *manifest file* or directly with *psql-client*.
63+
64+
* After editting Postgres cluster file apply the modified configuration
65+
* `kubectl get svc -l application=spilo -L spilo-role --namespace zalando-postgres`
66+
* Restart(Reload?) Patroni
67+
* `kubectl exec -i <Pod-Name> supervisorctl restart patroni --namespace zalando-postgres`
68+
69+
### Connect to Postgres master
70+
71+
* required `pg_hba.conf` entry for the external LoadBalancer IP
72+
73+
```shell
74+
$ psql -h <LoadBalancer-IP> -p <LoadBalancer-Port> -d <databasename> -U <databaseuser>
75+
```
76+
77+
* connect directly from shell in Rancher
78+
79+
```shell
80+
$ psql -d <databasename> -U <databaseuser>
81+
```
82+
83+
## Referenced
84+
85+
* [Patroni Postgres Operator](https://github.com/zalando/postgres-operator)
86+
* [Documentation](https://postgres-operator.readthedocs.io/en/latest/)

‎manifests/Readme.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
# Manifest Files

‎manifests/complete-postgres-manifest.yaml‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
apiVersion: "acid.zalan.do/v1"
2+
kind: postgresql
3+
metadata:
4+
name: acid-test-cluster
5+
# labels:
6+
# environment: demo
7+
spec:
8+
dockerImage: registry.opensource.zalan.do/acid/spilo-11:1.6-p1
9+
initContainers:
10+
- name: date
11+
image: busybox
12+
command: [ "/bin/date" ]
13+
teamId: "ACID"
14+
volume:
15+
size: 1Gi
16+
# storageClass: my-sc
17+
numberOfInstances: 2
18+
users: # Application/Robot users
19+
zalando:
20+
- superuser
21+
- createdb
22+
enableMasterLoadBalancer: true
23+
enableReplicaLoadBalancer: true
24+
allowedSourceRanges: # load balancers' source ranges for both master and replica services
25+
- 127.0.0.1/32
26+
databases:
27+
foo: zalando
28+
29+
# Expert section
30+
31+
enableShmVolume: true
32+
# spiloFSGroup: 103
33+
postgresql:
34+
version: "11"
35+
parameters:
36+
shared_buffers: "32MB"
37+
max_connections: "10"
38+
log_statement: "all"
39+
resources:
40+
requests:
41+
cpu: 10m
42+
memory: 100Mi
43+
limits:
44+
cpu: 300m
45+
memory: 300Mi
46+
patroni:
47+
initdb:
48+
encoding: "UTF8"
49+
locale: "en_US.UTF-8"
50+
data-checksums: "true"
51+
pg_hba:
52+
- hostssl all all 0.0.0.0/0 md5
53+
- host all all 0.0.0.0/0 md5
54+
# slots:
55+
# - permanent_physical_1:
56+
# type: physical
57+
# - permanent_logical_1:
58+
# type: logical
59+
# database: foo
60+
# plugin: pgoutput
61+
ttl: 30
62+
loop_wait: &loop_wait 10
63+
retry_timeout: 10
64+
maximum_lag_on_failover: 33554432
65+
# restore a Postgres DB with point-in-time-recovery
66+
# with a non-empty timestamp, clone from an S3 bucket using the latest backup before the timestamp
67+
# with an empty/absent timestamp, clone from an existing alive cluster using pg_basebackup
68+
# clone:
69+
# uid: "efd12e58-5786-11e8-b5a7-06148230260c"
70+
# cluster: "acid-batman"
71+
# timestamp: "2017-12-19T12:40:33+01:00" # timezone required (offset relative to UTC, see RFC 3339 section 5.6)
72+
# s3_wal_path: "s3://custom/path/to/bucket"
73+
74+
# run periodic backups with k8s cron jobs
75+
enableLogicalBackup: true
76+
logicalBackupSchedule: "30 00 * * *"
77+
s3_access_key_id: "production_volume_backup"
78+
s3_secret_access_key: "qXQzlfeXop8MhCd9oP5H"
79+
maintenanceWindows:
80+
- 01:00-06:00 #UTC
81+
- Sat:00:00-04:00
82+
# sidecars:
83+
# - name: "telegraf-sidecar"
84+
# image: "telegraf:latest"
85+
# resources:
86+
# limits:
87+
# cpu: 500m
88+
# memory: 500Mi
89+
# requests:
90+
# cpu: 100m
91+
# memory: 100Mi
92+
# env:
93+
# - name: "USEFUL_VAR"
94+
# value: "perhaps-true"

‎manifests/configmap.yaml‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
apiVersion: v1
2+
kind: ConfigMap
3+
metadata:
4+
name: postgres-operator
5+
data:
6+
# additional_secret_mount: "some-secret-name"
7+
# additional_secret_mount_path: "/some/dir"
8+
api_port: "8080"
9+
aws_region: eu-central-1
10+
cluster_domain: cluster.local
11+
cluster_history_entries: "1000"
12+
cluster_labels: application:spilo
13+
cluster_name_label: version
14+
# custom_service_annotations:
15+
# "keyx:valuez,keya:valuea"
16+
db_hosted_zone: db.example.com
17+
debug_logging: "true"
18+
# default_cpu_limit: "3"
19+
# default_cpu_request: 100m
20+
# default_memory_limit: 1Gi
21+
# default_memory_request: 100Mi
22+
docker_image: registry.opensource.zalan.do/acid/spilo-11:1.6-p1
23+
# enable_admin_role_for_users: "true"
24+
# enable_database_access: "true"
25+
enable_master_load_balancer: "true"
26+
# enable_pod_antiaffinity: "false"
27+
# enable_pod_disruption_budget: "true"
28+
enable_replica_load_balancer: "false"
29+
# enable_shm_volume: "true"
30+
# enable_team_superuser: "false"
31+
enable_teams_api: "false"
32+
# etcd_host: ""
33+
# infrastructure_roles_secret_name: postgresql-infrastructure-roles
34+
# inherited_labels: application,environment
35+
# kube_iam_role: ""
36+
# log_s3_bucket: ""
37+
# logical_backup_docker_image: "registry.opensource.zalan.do/acid/logical-backup"
38+
# logical_backup_s3_bucket: "my-bucket-url"
39+
# logical_backup_schedule: "30 00 * * *"
40+
master_dns_name_format: '{cluster}.{team}.staging.{hostedzone}'
41+
# master_pod_move_timeout: 10m
42+
# max_instances: "-1"
43+
# min_instances: "-1"
44+
# node_readiness_label: ""
45+
# oauth_token_secret_name: postgresql-operator
46+
# pam_configuration: |
47+
# https://info.example.com/oauth2/tokeninfo?access_token= uid realm=/employees
48+
# pam_role_name: zalandos
49+
pdb_name_format: "postgres-{cluster}-pdb"
50+
# pod_antiaffinity_topology_key: "kubernetes.io/hostname"
51+
pod_deletion_wait_timeout: 10m
52+
# pod_environment_configmap: ""
53+
pod_label_wait_timeout: 10m
54+
pod_management_policy: "ordered_ready"
55+
pod_role_label: spilo-role
56+
pod_service_account_name: "zalando-postgres-operator"
57+
pod_terminate_grace_period: 5m
58+
# postgres_superuser_teams: "postgres_superusers"
59+
# protected_role_names: "admin"
60+
ready_wait_interval: 3s
61+
ready_wait_timeout: 30s
62+
repair_period: 5m
63+
replica_dns_name_format: '{cluster}-repl.{team}.staging.{hostedzone}'
64+
replication_username: standby
65+
resource_check_interval: 3s
66+
resource_check_timeout: 10m
67+
resync_period: 5m
68+
ring_log_lines: "100"
69+
secret_name_template: '{username}.{cluster}.credentials'
70+
# sidecar_docker_images: ""
71+
# set_memory_request_to_limit: "false"
72+
spilo_privileged: "false"
73+
super_username: postgres
74+
# team_admin_role: "admin"
75+
# team_api_role_configuration: "log_statement:all"
76+
# teams_api_url: http://fake-teams-api.default.svc.cluster.local
77+
# toleration: ""
78+
# wal_s3_bucket: ""
79+
watched_namespace: "*" # listen to all namespaces
80+
workers: "4"

‎manifests/fake-teams-api.yaml‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
apiVersion: extensions/v1beta1
2+
kind: Deployment
3+
metadata:
4+
name: fake-teams-api
5+
spec:
6+
replicas: 1
7+
template:
8+
metadata:
9+
labels:
10+
name: fake-teams-api
11+
spec:
12+
containers:
13+
- name: fake-teams-api
14+
image: ikitiki/fake-teams-api:latest
15+
16+
---
17+
18+
apiVersion: v1
19+
kind: Service
20+
metadata:
21+
name: fake-teams-api
22+
spec:
23+
selector:
24+
name: fake-teams-api
25+
ports:
26+
- name: server
27+
port: 80
28+
protocol: TCP
29+
targetPort: 80
30+
type: NodePort
31+
32+
---
33+
34+
apiVersion: v1
35+
kind: Secret
36+
metadata:
37+
name: postgresql-operator
38+
namespace: default
39+
type: Opaque
40+
data:
41+
apiVersion: v1
42+
data:
43+
read-only-token-secret: dGVzdHRva2Vu
44+
read-only-token-type: QmVhcmVy

‎manifests/infrastructure-roles-configmap.yaml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
apiVersion: v1
2+
kind: ConfigMap
3+
metadata:
4+
name: postgresql-infrastructure-roles
5+
data:
6+
batman: |
7+
inrole: [admin] # following roles will be assigned to the new user
8+
user_flags:
9+
- createdb
10+
db_parameters: # db parameters, applyed for this particular user
11+
log_statement: all

‎manifests/infrastructure-roles.yaml‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
apiVersion: v1
2+
data:
3+
# required format (w/o quotes): 'propertyNumber: value'
4+
# allowed properties: 'user', 'password', 'inrole'
5+
# numbers >= 1 are mandatory
6+
# alternatively, supply the user: password pairs and
7+
# provide other options in the configmap.
8+
# robot_zmon_acid_monitoring
9+
user1: cm9ib3Rfem1vbl9hY2lkX21vbml0b3Jpbmc=
10+
# robot_zmon
11+
inrole1: cm9ib3Rfem1vbg==
12+
# testuser
13+
user2: dGVzdHVzZXI=
14+
# foobar
15+
password2: Zm9vYmFy
16+
# user batman with the password justice
17+
# look for other fields in the infrastructure roles configmap
18+
batman: anVzdGljZQ==
19+
kind: Secret
20+
metadata:
21+
name: postgresql-infrastructure-roles
22+
namespace: default
23+
type: Opaque

‎manifests/kustomization.yaml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
apiVersion: kustomize.config.k8s.io/v1beta1
2+
kind: Kustomization
3+
resources:
4+
- configmap.yaml
5+
- operator-service-account-rbac.yaml
6+
- postgres-operator.yaml

‎manifests/minimal-postgres-manifest.yaml‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
apiVersion: "acid.zalan.do/v1"
2+
kind: postgresql
3+
metadata:
4+
name: acid-minimal-cluster
5+
namespace: zalando-postgres
6+
spec:
7+
teamId: "ACID"
8+
volume:
9+
size: 1Gi
10+
numberOfInstances: 3
11+
users:
12+
zalando: # database owner
13+
- superuser
14+
- createdb
15+
foo_user: [] # role for application foo
16+
databases:
17+
foo: zalando # dbname: owner
18+
test: zalando
19+
test2: zalando
20+
postgresql:
21+
version: "11"
22+
parameters:
23+
shared_buffers: "128MB"
24+
max_connections: "10"
25+
log_statement: "all"
26+
log_destination: "stderr"
27+
log_directory: "/var/log/postgresql"
28+
patroni:
29+
initdb:
30+
encoding: "UTF8"
31+
locale: "de_DE.UTF-8"
32+
data-checksums: "true"
33+
pg_hba:
34+
- hostssl all all 0.0.0.0/0 md5
35+
- host all all 0.0.0.0/0 md5
36+
- host all zalando 192.168.232.109/24 md5

0 commit comments

Comments
 (0)