1166 lines
31 KiB
Markdown
1166 lines
31 KiB
Markdown
# PostgreSQL 实战指南
|
||
|
||
## 目录
|
||
|
||
1. [PostgreSQL 介绍](#1-postgresql-介绍)
|
||
2. [PostgreSQL 安装](#2-postgresql-安装)
|
||
3. [PostgreSQL 管理](#3-postgresql-管理)
|
||
4. [PostgreSQL 体系架构](#4-postgresql-体系架构)
|
||
5. [PostgreSQL 备份还原](#5-postgresql-备份还原)
|
||
6. [PostgreSQL 高可用](#6-postgresql-高可用)
|
||
|
||
---
|
||
|
||
## 1. PostgreSQL 介绍
|
||
|
||
### 1.1 什么是 PostgreSQL
|
||
|
||
PostgreSQL 是一个功能强大的开源对象关系型数据库系统（ORDBMS），支持 SQL 标准并具有丰富的企业级特性。作为最先进的开源数据库，PostgreSQL 以其可靠性、数据完整性和扩展性著称。
|
||
|
||
### 1.2 PostgreSQL 特点
|
||
|
||
| 特性 | 说明 |
|
||
|------|------|
|
||
| 开源免费 | 完全开源，BSD/MIT 许可证 |
|
||
| ACID 完整 | 完全支持 ACID 事务特性 |
|
||
| MVCC | 多版本并发控制，支持高并发 |
|
||
| 丰富数据类型 | 支持数组、JSON、UUID、几何类型等 |
|
||
| 索引类型 | 支持 B-tree、Hash、GIN、GiST、BRIN 等 |
|
||
| 扩展性 | 支持自定义函数、操作符、数据类型 |
|
||
| 外键约束 | 完整的外键约束支持 |
|
||
| 视图 | 支持物化视图、普通视图 |
|
||
| 触发器 | 支持行级和语句级触发器 |
|
||
| 存储过程 | 支持存储过程（多种语言） |
|
||
| 分区表 | 支持范围分区、列表分区、哈希分区 |
|
||
| 复制 | 支持流复制、逻辑复制 |
|
||
| 集群 | 支持 Citus 分布式集群 |
|
||
|
||
### 1.3 PostgreSQL 应用场景
|
||
|
||
| 场景 | 说明 |
|
||
|------|------|
|
||
| 企业应用 | ERP、CRM 等企业级应用数据库 |
|
||
| GIS 应用 | PostGIS 扩展支持地理信息系统 |
|
||
| 数据仓库 | 分析型数据库，大数据处理 |
|
||
| Web 应用 | LAMP/LEMP 架构数据存储 |
|
||
| 金融系统 | 对事务要求严格的系统 |
|
||
| 科研数据 | 科研数据存储和分析 |
|
||
|
||
### 1.4 PostgreSQL 与 MySQL 对比
|
||
|
||
| 特性 | PostgreSQL | MySQL |
|
||
|------|------------|-------|
|
||
| 事务支持 | 完整 ACID | ACID（InnoDB） |
|
||
| 并发控制 | MVCC | MVCC（InnoDB） |
|
||
| 数据类型 | 丰富 | 基础 |
|
||
| 索引类型 | 多样 | 较少 |
|
||
| 扩展性 | 强大 | 一般 |
|
||
| 复制 | 流复制/逻辑复制 | 主从复制/GTID |
|
||
| 性能 | 复杂查询快 | 简单查询快 |
|
||
| 许可证 | BSD | GPL |
|
||
|
||
---
|
||
|
||
## 2. PostgreSQL 安装
|
||
|
||
### 2.1 环境要求
|
||
|
||
- 操作系统：Linux（CentOS/Ubuntu/Debian）、macOS、Windows
|
||
- 内存：至少 1GB（生产环境建议 4GB+）
|
||
- 磁盘：至少 100GB
|
||
|
||
### 2.2 YUM 安装（CentOS/RHEL）
|
||
|
||
```bash
|
||
# 安装 PostgreSQL Yum 源
|
||
yum install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-8-x86_64/pgdg-redhat-repo-latest.noarch.rpm
|
||
|
||
# 安装 PostgreSQL 服务器和客户端
|
||
yum install -y postgresql15-server postgresql15
|
||
|
||
# 初始化数据库
|
||
/usr/pgsql-15/bin/postgresql-15-setup initdb
|
||
|
||
# 启动服务
|
||
systemctl start postgresql-15
|
||
systemctl enable postgresql-15
|
||
|
||
# 检查服务状态
|
||
systemctl status postgresql-15
|
||
```
|
||
|
||
### 2.3 APT 安装（Debian/Ubuntu）
|
||
|
||
```bash
|
||
# 安装 PostgreSQL APT 源
|
||
sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
|
||
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
|
||
|
||
# 更新并安装
|
||
apt update
|
||
apt install -y postgresql-15 postgresql-client-15
|
||
|
||
# 启动服务
|
||
systemctl start postgresql
|
||
systemctl enable postgresql
|
||
```
|
||
|
||
### 2.4 源码编译安装
|
||
|
||
```bash
|
||
# 安装依赖
|
||
yum install -y gcc make libxml2 libuuid openssl readline zlib
|
||
|
||
# 下载源码
|
||
wget https://ftp.postgresql.org/pub/source/v15.4/postgresql-15.4.tar.gz
|
||
tar -zxf postgresql-15.4.tar.gz
|
||
cd postgresql-15.4
|
||
|
||
# 配置
|
||
./configure --prefix=/usr/local/pgsql-15 \
|
||
--with-openssl \
|
||
--with-readline \
|
||
--with-uuid=libuuid \
|
||
--with-libxml \
|
||
--with-zlib
|
||
|
||
# 编译安装
|
||
make -j4
|
||
make install
|
||
|
||
# 创建用户
|
||
useradd -m -s /bin/bash postgres
|
||
mkdir -p /usr/local/pgsql-15/data
|
||
chown -R postgres:postgres /usr/local/pgsql-15
|
||
|
||
# 切换用户初始化
|
||
su - postgres
|
||
/usr/local/pgsql-15/bin/initdb -D /usr/local/pgsql-15/data
|
||
|
||
# 启动数据库
|
||
/usr/local/pgsql-15/bin/pg_ctl -D /usr/local/pgsql-15/data -l /usr/local/pgsql-15/logfile start
|
||
```
|
||
|
||
### 2.5 Docker 安装
|
||
|
||
```bash
|
||
# 拉取镜像
|
||
docker pull postgres:15
|
||
|
||
# 运行容器
|
||
docker run -d \
|
||
--name postgres \
|
||
-e POSTGRES_PASSWORD=your_password \
|
||
-e POSTGRES_USER=admin \
|
||
-e POSTGRES_DB=mydb \
|
||
-p 5432:5432 \
|
||
-v /data/postgres/data:/var/lib/postgresql/data \
|
||
postgres:15
|
||
|
||
# 使用配置文件启动
|
||
docker run -d \
|
||
--name postgres \
|
||
-e POSTGRES_PASSWORD=your_password \
|
||
-v /data/postgres/postgresql.conf:/etc/postgresql/postgresql.conf \
|
||
-v /data/postgres/data:/var/lib/postgresql/data \
|
||
postgres:15 -c config-file=/etc/postgresql/postgresql.conf
|
||
```
|
||
|
||
### 2.6 基础配置
|
||
|
||
```bash
|
||
# postgresql.conf 主要配置
|
||
|
||
# 连接配置
|
||
listen_addresses = '*'
|
||
port = 5432
|
||
max_connections = 200
|
||
|
||
# 内存配置
|
||
shared_buffers = 4GB
|
||
effective_cache_size = 12GB
|
||
work_mem = 256MB
|
||
maintenance_work_mem = 1GB
|
||
|
||
# 持久化配置
|
||
wal_level = replica
|
||
max_wal_size = 1GB
|
||
min_wal_size = 80MB
|
||
|
||
# 日志配置
|
||
log_destination = 'stderr'
|
||
logging_collector = on
|
||
log_directory = 'log'
|
||
log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log'
|
||
log_statement = 'ddl'
|
||
log_min_duration_statement = 1000
|
||
|
||
# 复制配置
|
||
wal_sender_timeout = 60s
|
||
wal_receiver_timeout = 60s
|
||
|
||
# 客户端连接配置
|
||
authentication_timeout = 1min
|
||
password_encryption = scram-sha-256
|
||
```
|
||
|
||
### 2.7 pg_hba.conf 配置
|
||
|
||
```bash
|
||
# pg_hba.conf - 客户端认证
|
||
|
||
# 本地连接
|
||
local all all trust
|
||
|
||
# IPv4 本地回环
|
||
host all all 127.0.0.1/32 scram-sha-256
|
||
|
||
# IPv6 本地回环
|
||
host all all ::1/128 scram-sha-256
|
||
|
||
# 允许特定网段
|
||
host all all 192.168.1.0/24 scram-sha-256
|
||
|
||
# 允许所有网段（生产环境谨慎）
|
||
# host all all 0.0.0.0/0 scram-sha-256
|
||
|
||
# 复制连接
|
||
local replication all trust
|
||
host replication all 127.0.0.1/32 scram-sha-256
|
||
host replication all ::1/128 scram-sha-256
|
||
host replication all 192.168.1.0/24 scram-sha-256
|
||
```
|
||
|
||
### 2.8 启动和停止
|
||
|
||
```bash
|
||
# 服务方式（使用 RPM/APT 安装）
|
||
systemctl start postgresql-15
|
||
systemctl stop postgresql-15
|
||
systemctl restart postgresql-15
|
||
systemctl status postgresql-15
|
||
|
||
# 手动方式
|
||
pg_ctl -D /var/lib/pgsql/15/data start
|
||
pg_ctl -D /var/lib/pgsql/15/data stop
|
||
|
||
# 连接数据库
|
||
psql -U postgres
|
||
psql -h localhost -U postgres -d mydb
|
||
```
|
||
|
||
---
|
||
|
||
## 3. PostgreSQL 管理
|
||
|
||
### 3.1 连接和基本操作
|
||
|
||
```bash
|
||
# 连接数据库
|
||
psql -U username -d database_name
|
||
psql -h hostname -p 5432 -U username -d database_name
|
||
|
||
# 常用命令
|
||
\l # 列出所有数据库
|
||
\d # 列出当前数据库的所有表
|
||
\d table_name # 查看表结构
|
||
\du # 列出所有用户
|
||
\di # 列出所有索引
|
||
\dv # 列出所有视图
|
||
\dt # 列出所有表
|
||
\db # 列出所有表空间
|
||
\x # 展开显示
|
||
\? # 帮助
|
||
\q # 退出
|
||
```
|
||
|
||
### 3.2 数据库操作
|
||
|
||
```bash
|
||
# 创建数据库
|
||
CREATE DATABASE mydb;
|
||
|
||
# 创建数据库（指定所有者、编码）
|
||
CREATE DATABASE mydb
|
||
OWNER postgres
|
||
ENCODING 'UTF8'
|
||
TEMPLATE template0;
|
||
|
||
# 删除数据库
|
||
DROP DATABASE mydb;
|
||
|
||
# 修改数据库
|
||
ALTER DATABASE mydb RENAME TO newdb;
|
||
ALTER DATABASE mydb SET work_mem = '256MB';
|
||
|
||
# 切换数据库
|
||
\c mydb
|
||
\connect mydb
|
||
```
|
||
|
||
### 3.3 表操作
|
||
|
||
```bash
|
||
# 创建表
|
||
CREATE TABLE users (
|
||
id SERIAL PRIMARY KEY,
|
||
username VARCHAR(50) UNIQUE NOT NULL,
|
||
email VARCHAR(100) NOT NULL,
|
||
password VARCHAR(255) NOT NULL,
|
||
age INTEGER CHECK (age >= 0),
|
||
status SMALLINT DEFAULT 1,
|
||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||
);
|
||
|
||
# 创建表（带约束）
|
||
CREATE TABLE orders (
|
||
id SERIAL PRIMARY KEY,
|
||
user_id INTEGER REFERENCES users(id) ON DELETE CASCADE,
|
||
total_amount DECIMAL(10, 2) NOT NULL,
|
||
status VARCHAR(20) DEFAULT 'pending',
|
||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||
CONSTRAINT valid_amount CHECK (total_amount >= 0)
|
||
);
|
||
|
||
# 查看表结构
|
||
\d users
|
||
|
||
# 修改表
|
||
ALTER TABLE users ADD COLUMN phone VARCHAR(20);
|
||
ALTER TABLE users DROP COLUMN phone;
|
||
ALTER TABLE users RENAME COLUMN email TO user_email;
|
||
ALTER TABLE users ALTER COLUMN age SET DEFAULT 18;
|
||
ALTER TABLE users ALTER COLUMN age DROP DEFAULT;
|
||
ALTER TABLE users ADD CONSTRAINT unique_email UNIQUE (user_email);
|
||
|
||
# 删除表
|
||
DROP TABLE users;
|
||
DROP TABLE IF EXISTS users;
|
||
```
|
||
|
||
### 3.4 索引操作
|
||
|
||
```bash
|
||
# 创建索引
|
||
CREATE INDEX idx_users_email ON users(email);
|
||
CREATE INDEX idx_users_status ON users(status);
|
||
CREATE INDEX idx_users_created ON users(created_at DESC);
|
||
|
||
# 创建唯一索引
|
||
CREATE UNIQUE INDEX idx_users_username ON users(username);
|
||
|
||
# 创建复合索引
|
||
CREATE INDEX idx_orders_user_status ON orders(user_id, status);
|
||
|
||
# 创建部分索引
|
||
CREATE INDEX idx_orders_active ON orders(user_id) WHERE status = 'active';
|
||
|
||
# 创建表达式索引
|
||
CREATE INDEX idx_users_lower_email ON users(LOWER(email));
|
||
|
||
# 查看索引
|
||
\d users
|
||
SELECT * FROM pg_indexes WHERE tablename = 'users';
|
||
|
||
# 删除索引
|
||
DROP INDEX idx_users_email;
|
||
```
|
||
|
||
### 3.5 用户和权限管理
|
||
|
||
```bash
|
||
# 创建用户
|
||
CREATE USER username WITH PASSWORD 'password';
|
||
|
||
# 创建超级用户
|
||
CREATE USER admin WITH SUPERUSER PASSWORD 'password';
|
||
|
||
# 修改用户
|
||
ALTER USER username WITH PASSWORD 'newpassword';
|
||
ALTER USER username VALID UNTIL '2025-01-01';
|
||
ALTER USER username CREATEDB;
|
||
ALTER USER username CREATEROLE;
|
||
|
||
# 删除用户
|
||
DROP USER username;
|
||
|
||
# 创建角色
|
||
CREATE ROLE readonly;
|
||
CREATE ROLE readwrite;
|
||
|
||
# 角色授权
|
||
GRANT SELECT ON ALL TABLES IN SCHEMA public TO readonly;
|
||
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO readwrite;
|
||
GRANT USAGE ON ALL SEQUENCES IN SCHEMA public TO readwrite;
|
||
|
||
# 默认权限
|
||
ALTER DEFAULT PRIVILEGES IN SCHEMA public
|
||
GRANT SELECT ON TABLES TO readonly;
|
||
|
||
# 撤销权限
|
||
REVOKE ALL PRIVILEGES ON TABLE users FROM username;
|
||
REVOKE INSERT ON ALL TABLES IN SCHEMA public FROM readwrite;
|
||
|
||
# 查看权限
|
||
\du username
|
||
```
|
||
|
||
### 3.6 视图操作
|
||
|
||
```bash
|
||
# 创建视图
|
||
CREATE VIEW user_orders AS
|
||
SELECT u.username, o.id, o.total_amount, o.status, o.created_at
|
||
FROM users u
|
||
JOIN orders o ON u.id = o.user_id;
|
||
|
||
# 创建物化视图
|
||
CREATE MATERIALIZED VIEW monthly_sales AS
|
||
SELECT DATE_TRUNC('month', created_at) AS month, SUM(total_amount) AS total
|
||
FROM orders
|
||
GROUP BY DATE_TRUNC('month', created_at);
|
||
|
||
# 刷新物化视图
|
||
REFRESH MATERIALIZED VIEW monthly_sales;
|
||
|
||
# 删除视图
|
||
DROP VIEW user_orders;
|
||
DROP MATERIALIZED VIEW monthly_sales;
|
||
```
|
||
|
||
### 3.7 函数和存储过程
|
||
|
||
```bash
|
||
# 创建函数
|
||
CREATE OR REPLACE FUNCTION get_user_count()
|
||
RETURNS INTEGER AS $$
|
||
BEGIN
|
||
RETURN (SELECT COUNT(*) FROM users);
|
||
END;
|
||
$$ LANGUAGE plpgsql;
|
||
|
||
# 创建存储过程（ PostgreSQL 11+）
|
||
CREATE OR REPLACE PROCEDURE update_user_status(user_id INT, new_status INT)
|
||
LANGUAGE plpgsql AS $$
|
||
BEGIN
|
||
UPDATE users SET status = new_status, updated_at = CURRENT_TIMESTAMP
|
||
WHERE id = user_id;
|
||
END;
|
||
$$;
|
||
|
||
# 调用存储过程
|
||
CALL update_user_status(1, 2);
|
||
```
|
||
|
||
### 3.8 触发器
|
||
|
||
```bash
|
||
# 创建触发器函数
|
||
CREATE OR REPLACE FUNCTION update_timestamp()
|
||
RETURNS TRIGGER AS $$
|
||
BEGIN
|
||
NEW.updated_at = CURRENT_TIMESTAMP;
|
||
RETURN NEW;
|
||
END;
|
||
$$ LANGUAGE plpgsql;
|
||
|
||
# 创建触发器
|
||
CREATE TRIGGER update_users_timestamp
|
||
BEFORE UPDATE ON users
|
||
FOR EACH ROW
|
||
EXECUTE FUNCTION update_timestamp();
|
||
|
||
# 查看触发器
|
||
\dT+ users
|
||
|
||
# 删除触发器
|
||
DROP TRIGGER update_users_timestamp ON users;
|
||
```
|
||
|
||
### 3.9 分区表
|
||
|
||
```bash
|
||
# 创建范围分区表
|
||
CREATE TABLE orders (
|
||
id BIGSERIAL,
|
||
user_id INTEGER NOT NULL,
|
||
total_amount DECIMAL(10, 2) NOT NULL,
|
||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||
) PARTITION BY RANGE (created_at);
|
||
|
||
# 创建分区
|
||
CREATE TABLE orders_2024 PARTITION OF orders
|
||
FOR VALUES FROM ('2024-01-01') TO ('2025-01-01');
|
||
|
||
CREATE TABLE orders_2025 PARTITION OF orders
|
||
FOR VALUES FROM ('2025-01-01') TO ('2026-01-01');
|
||
|
||
# 创建列表分区
|
||
CREATE TABLE products (
|
||
id SERIAL,
|
||
name VARCHAR(100),
|
||
category VARCHAR(50)
|
||
) PARTITION BY LIST (category);
|
||
|
||
CREATE TABLE products_electronics PARTITION OF products
|
||
FOR VALUES IN ('electronics', 'computers');
|
||
|
||
CREATE TABLE products_clothing PARTITION OF products
|
||
FOR VALUES IN ('clothing', 'fashion');
|
||
```
|
||
|
||
### 3.10 性能分析
|
||
|
||
```bash
|
||
# 查看慢查询
|
||
SELECT query, calls, mean_time, total_time
|
||
FROM pg_stat_statements
|
||
ORDER BY total_time DESC
|
||
LIMIT 10;
|
||
|
||
# 查看表大小
|
||
SELECT relname, pg_size_pretty(pg_total_relation_size(relid))
|
||
FROM pg_catalog.pg_statio_user_tables
|
||
ORDER BY pg_total_relation_size(relid) DESC;
|
||
|
||
# 查看索引使用情况
|
||
SELECT indexrelname, idx_scan, idx_tup_read, idx_tup_fetch
|
||
FROM pg_stat_user_indexes
|
||
ORDER BY idx_scan DESC;
|
||
|
||
# 分析表
|
||
ANALYZE users;
|
||
|
||
# 清理缓存
|
||
DISCARD ALL;
|
||
```
|
||
|
||
---
|
||
|
||
## 4. PostgreSQL 体系架构
|
||
|
||
### 4.1 架构概述
|
||
|
||
PostgreSQL 采用**进程/线程混合架构**，主要包括：
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────┐
|
||
│ PostgreSQL 进程结构 │
|
||
├─────────────────────────────────────────────────────┤
|
||
│ │
|
||
│ ┌──────────────┐ │
|
||
│ │ Postmaster │ ← 主进程（父进程） │
|
||
│ │ (PID 1234) │ │
|
||
│ └──────┬───────┘ │
|
||
│ │ │
|
||
│ ┌─────┼─────┬─────────────┐ │
|
||
│ │ │ │ │ │
|
||
│ ▼ ▼ ▼ ▼ │
|
||
│ ┌────┐ ┌────┐ ┌──────┐ ┌──────┐ │
|
||
│ │Writer│ │Wal │ │Check-│ │Auto-│ │
|
||
│ │Writer│ │Writer│ │pointer│ │vacuum│ │
|
||
│ └────┘ └────┘ └──────┘ └──────┘ │
|
||
│ │
|
||
│ ┌──────────────────────────────────────────────┐ │
|
||
│ │ Client Connections │ │
|
||
│ │ ┌──────┐ ┌──────┐ ┌──────┐ │ │
|
||
│ │ │postgres│ │postgres│ │postgres│ ... │ │
|
||
│ │ │(PID 1) │ │(PID 2) │ │(PID 3) │ │ │
|
||
│ │ └──────┘ └──────┘ └──────┘ │ │
|
||
│ └──────────────────────────────────────────────┘ │
|
||
│ │
|
||
└─────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
### 4.2 进程说明
|
||
|
||
| 进程 | 说明 |
|
||
|------|------|
|
||
| **Postmaster** | 主进程，监听连接请求，创建子进程 |
|
||
| **Writer** | 负责将数据写入磁盘 |
|
||
| **Wal Writer** | 负责将 WAL 写入磁盘 |
|
||
| **Checkpointer** | 检查点进程，触发检查点 |
|
||
| **Autovacuum** | 自动 vacuum 进程 |
|
||
| **Archiver** | WAL 归档进程 |
|
||
| **Stats Collector** | 统计信息收集进程 |
|
||
| **Bgwriter** | 后台写进程 |
|
||
| **Client Process** | 每个客户端连接对应的服务进程 |
|
||
|
||
### 4.3 内存结构
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────┐
|
||
│ PostgreSQL 内存结构 │
|
||
├─────────────────────────────────────────────────────┤
|
||
│ │
|
||
│ 共享内存（Shared Memory） │
|
||
│ ┌─────────────────────────────────────────────┐ │
|
||
│ │ shared_buffers ← 数据缓存 │ │
|
||
│ │ wal_buffers ← WAL 缓冲区 │ │
|
||
│ │ commit_timestamp buffers │ │
|
||
│ │ locks ← 锁管理 │ │
|
||
│ │ predicate_locks ← 谓词锁 │ │
|
||
│ │ transaction logs ← 事务日志 │ │
|
||
│ └─────────────────────────────────────────────┘ │
|
||
│ │
|
||
│ 进程内存（Process Memory） │
|
||
│ ┌─────────────────────────────────────────────┐ │
|
||
│ │ work_mem ← 排序/哈希操作内存 │ │
|
||
│ │ maintenance_work_mem ← 维护操作内存 │ │
|
||
│ │ temp_buffers ← 临时表缓存 │ │
|
||
│ │ parallel_worker ← 并行工作进程内存 │ │
|
||
│ └─────────────────────────────────────────────┘ │
|
||
│ │
|
||
└─────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
### 4.4 核心概念
|
||
|
||
#### 4.4.1 共享内存
|
||
|
||
| 参数 | 说明 |
|
||
|------|------|
|
||
| `shared_buffers` | 数据页缓存，建议设置为系统内存的 25% |
|
||
| `wal_buffers` | WAL 缓冲区，默认 16MB |
|
||
| `effective_cache_size` | 操作系统缓存估计，影响查询规划 |
|
||
|
||
#### 4.4.2 进程内存
|
||
|
||
| 参数 | 说明 |
|
||
|------|------|
|
||
| `work_mem` | 排序和哈希操作内存，每个操作独立分配 |
|
||
| `maintenance_work_mem` | 维护操作（VACUUM、CREATE INDEX）内存 |
|
||
| `temp_buffers` | 临时表缓存 |
|
||
|
||
### 4.5 存储结构
|
||
|
||
#### 4.5.1 物理存储
|
||
|
||
```
|
||
PostgreSQL 数据目录：$PGDATA
|
||
|
||
$PGDATA/
|
||
├── base/ # 表数据文件
|
||
│ ├── 12345/ # 数据库 OID
|
||
│ │ ├── 12345 # 表文件
|
||
│ │ ├── 12345.1 # 表文件（超过 1GB 自动分卷）
|
||
│ │ └── 12345_vm # 可见性映射
|
||
│ └── ...
|
||
├── global/ # 系统表
|
||
├── pg_wal/ # WAL 日志
|
||
├── pg_xact/ # 事务状态
|
||
├── pg_notify/ # LISTEN/NOTIFY
|
||
├── pg_serial/ # 可序列化事务
|
||
├── pg_subtrans/ # 子事务
|
||
├── pg_tblspc/ # 表空间链接
|
||
├── pg_dynshmem/ # 动态共享内存
|
||
├── pg_replslot/ # 复制槽
|
||
├── pg_stat/ # 统计信息
|
||
├── pg_stat_tmp/ # 临时统计信息
|
||
├── pg_snapshots/ # 导出快照
|
||
├── pg_logical/ # 逻辑解码
|
||
├── pg_walwriter/ # WAL 写入器
|
||
├── pg_conf/ # 配置文件
|
||
├── postgresql.conf # 主配置文件
|
||
├── pg_hba.conf # 客户端认证
|
||
└── pg_ident.conf # 用户映射
|
||
```
|
||
|
||
#### 4.5.2 逻辑存储
|
||
|
||
- **表（Table）**：由行组成的数据集合
|
||
- **行（Row）**：一条记录
|
||
- **页（Page）**：默认 8KB，存储基本单元
|
||
- **元组（Tuple）**：行的物理表示
|
||
|
||
### 4.6 事务管理
|
||
|
||
#### 4.6.1 MVCC 机制
|
||
|
||
PostgreSQL 使用 MVCC（多版本并发控制）实现事务隔离：
|
||
|
||
- 每次更新创建新版本，旧版本保留
|
||
- 事务通过查看版本号判断可见性
|
||
- 读写不冲突，提高并发性能
|
||
|
||
#### 4.6.2 事务隔离级别
|
||
|
||
```sql
|
||
-- 查看当前隔离级别
|
||
SHOW transaction_isolation;
|
||
|
||
-- 设置隔离级别
|
||
SET TRANSACTION ISOLATION LEVEL READ COMMITTED;
|
||
SET SESSION CHARACTERISTICS AS TRANSACTION ISOLATION LEVEL SERIALIZABLE;
|
||
```
|
||
|
||
| 级别 | 说明 | 脏读 | 不可重复读 | 幻读 |
|
||
|------|------|------|------------|------|
|
||
| READ UNCOMMITTED | 读未提交 | 可能 | 可能 | 可能 |
|
||
| READ COMMITTED | 读已提交 | 不可能 | 可能 | 可能 |
|
||
| REPEATABLE READ | 可重复读 | 不可能 | 不可能 | 可能 |
|
||
| SERIALIZABLE | 串行化 | 不可能 | 不可能 | 不可能 |
|
||
|
||
### 4.7 WAL 日志
|
||
|
||
#### 4.7.1 WAL 机制
|
||
|
||
WAL（Write Ahead Logging）预写日志，确保数据持久性：
|
||
|
||
1. 事务提交时，先写 WAL
|
||
2. WAL 写入磁盘后，返回成功
|
||
3. 后台将数据写入磁盘
|
||
|
||
#### 4.7.2 检查点
|
||
|
||
```sql
|
||
-- 手动执行检查点
|
||
CHECKPOINT;
|
||
|
||
-- 查看检查点统计
|
||
SELECT * FROM pg_stat_checkpoints;
|
||
```
|
||
|
||
### 4.8 存储引擎
|
||
|
||
PostgreSQL 使用单一存储引擎（heap），通过不同索引实现不同访问模式：
|
||
|
||
| 索引类型 | 适用场景 |
|
||
|----------|----------|
|
||
| B-tree | 范围查询、排序 |
|
||
| Hash | 精确匹配 |
|
||
| GiST | 几何、地理数据 |
|
||
| GIN | 全文搜索、数组 |
|
||
| BRIN | 顺序扫描、大表 |
|
||
| SP-GiST | 空间分割 |
|
||
|
||
---
|
||
|
||
## 5. PostgreSQL 备份还原
|
||
|
||
### 5.1 备份类型
|
||
|
||
| 类型 | 说明 | 工具 |
|
||
|------|------|------|
|
||
| SQL 导出 | 逻辑备份，可跨版本 | pg_dump |
|
||
| 物理备份 | 直接复制数据文件 | pg_basebackup |
|
||
| 归档备份 | 增量备份 | WAL 归档 |
|
||
|
||
### 5.2 逻辑备份（pg_dump）
|
||
|
||
```bash
|
||
# 备份单个数据库
|
||
pg_dump -U postgres -Fc mydb > mydb.dump
|
||
pg_dump -U postgres -Fp mydb > mydb.sql
|
||
|
||
# 备份所有数据库
|
||
pg_dumpall -U postgres > all_databases.sql
|
||
|
||
# 备份指定表
|
||
pg_dump -U postgres -t users -t orders mydb > tables.sql
|
||
|
||
# 备份表结构
|
||
pg_dump -U postgres -s mydb > schema.sql
|
||
|
||
# 备份数据
|
||
pg_dump -U postgres -a mydb > data.sql
|
||
|
||
# 压缩备份
|
||
pg_dump -U postgres -Fc mydb > mydb.dump.gz
|
||
|
||
# 并行备份（PostgreSQL 13+）
|
||
pg_dump -U postgres -Fd -j 4 -f backup_dir mydb
|
||
```
|
||
|
||
### 5.3 逻辑还原（pg_restore）
|
||
|
||
```bash
|
||
# 还原数据库
|
||
psql -U postgres -d mydb < mydb.sql
|
||
|
||
# 还原压缩备份
|
||
pg_restore -U postgres -d mydb mydb.dump
|
||
|
||
# 还原前删除原有对象
|
||
pg_restore -U postgres -d mydb --clean mydb.dump
|
||
|
||
# 还原到新数据库
|
||
createdb -U postgres newdb
|
||
pg_restore -U postgres -d newdb mydb.dump
|
||
|
||
# 还原并行备份
|
||
pg_restore -U postgres -d mydb -j 4 backup_dir/
|
||
```
|
||
|
||
### 5.4 物理备份（pg_basebackup）
|
||
|
||
```bash
|
||
# 基础物理备份
|
||
pg_basebackup -U postgres -D /backup/base -Ft -z -P
|
||
|
||
# 流复制备份
|
||
pg_basebackup -U postgres -D /backup/base -Xs -P
|
||
|
||
# 带 WAL 的备份
|
||
pg_basebackup -U postgres -D /backup/base -X stream -P
|
||
|
||
# 备份到远程
|
||
pg_basebackup -U postgres -h remote_host -D /backup/base -Xs -P
|
||
```
|
||
|
||
### 5.5 PITR（时间点恢复）
|
||
|
||
#### 5.5.1 配置 WAL 归档
|
||
|
||
```bash
|
||
# postgresql.conf
|
||
wal_level = replica
|
||
archive_mode = on
|
||
archive_command = 'test ! -f /backup/wal/%f && cp %p /backup/wal/%f'
|
||
archive_timeout = 300
|
||
```
|
||
|
||
#### 5.5.2 执行基础备份
|
||
|
||
```bash
|
||
pg_basebackup -U postgres -D /backup/base -Ft -z -P
|
||
```
|
||
|
||
#### 5.5.3 恢复配置
|
||
|
||
```bash
|
||
# postgresql.conf 恢复配置
|
||
restore_command = 'cp /backup/wal/%f %p'
|
||
recovery_target_time = '2024-01-15 10:00:00'
|
||
|
||
# 创建恢复信号文件
|
||
touch /var/lib/pgsql/15/data/recovery.signal
|
||
```
|
||
|
||
#### 5.5.4 启动恢复
|
||
|
||
```bash
|
||
# 停止 PostgreSQL
|
||
pg_ctl -D /var/lib/pgsql/15/data stop
|
||
|
||
# 修改配置后启动
|
||
pg_ctl -D /var/lib/pgsql/15/data -l /var/lib/pgsql/15/data/logfile start
|
||
```
|
||
|
||
### 5.6 定时备份脚本
|
||
|
||
```bash
|
||
#!/bin/bash
|
||
|
||
# 备份配置
|
||
BACKUP_DIR="/backup/postgresql"
|
||
DATE=$(date +%Y%m%d_%H%M%S)
|
||
DB_NAME="mydb"
|
||
DB_USER="postgres"
|
||
|
||
# 创建备份目录
|
||
mkdir -p $BACKUP_DIR
|
||
|
||
# 删除 7 天前的备份
|
||
find $BACKUP_DIR -type f -mtime +7 -delete
|
||
|
||
# 执行备份
|
||
pg_dump -U $DB_USER -Fc $DB_NAME > $BACKUP_DIR/${DB_NAME}_${DATE}.dump
|
||
|
||
# 记录日志
|
||
echo "[$(date)] Backup completed: ${DB_NAME}_${DATE}.dump" >> /var/log/pg_backup.log
|
||
```
|
||
|
||
### 5.7 自动备份（cron）
|
||
|
||
```bash
|
||
# 编辑 crontab
|
||
crontab -e
|
||
|
||
# 每天凌晨 2 点执行备份
|
||
0 2 * * * /usr/local/bin/backup.sh
|
||
|
||
# 每天凌晨 2 点和下午 2 点执行备份
|
||
0 2,14 * * * /usr/local/bin/backup.sh
|
||
```
|
||
|
||
---
|
||
|
||
## 6. PostgreSQL 高可用
|
||
|
||
### 6.1 高可用方案概述
|
||
|
||
| 方案 | 说明 | 复杂度 | 延迟 |
|
||
|------|------|--------|------|
|
||
| 流复制 | 主从同步复制 | 低 | 低 |
|
||
| 逻辑复制 | 表级别复制 | 中 | 低 |
|
||
| Patroni + etcd | 自动故障转移 | 高 | 低 |
|
||
| PgBouncer | 连接池 + 负载均衡 | 中 | - |
|
||
| Citus | 分布式集群 | 高 | 中 |
|
||
|
||
### 6.2 流复制配置
|
||
|
||
#### 6.2.1 主库配置
|
||
|
||
```bash
|
||
# postgresql.conf
|
||
wal_level = replica
|
||
max_wal_senders = 10
|
||
max_replication_slots = 10
|
||
wal_keep_size = 1GB
|
||
|
||
# 可选：同步复制
|
||
synchronous_commit = on
|
||
synchronous_standby_names = 'standby1'
|
||
```
|
||
|
||
#### 6.2.2 从库配置
|
||
|
||
```bash
|
||
# postgresql.conf
|
||
primary_conninfo = 'host=192.168.1.100 port=5432 user=replica password=replica_pass'
|
||
|
||
# 可选：只读模式
|
||
hot_standby = on
|
||
max_standby_streaming_delay = 30s
|
||
```
|
||
|
||
#### 6.2.3 创建复制用户
|
||
|
||
```sql
|
||
-- 主库执行
|
||
CREATE USER replica REPLICATION LOGIN PASSWORD 'replica_pass';
|
||
|
||
-- 配置 pg_hba.conf
|
||
host replication replica 192.168.1.0/24 scram-sha-256
|
||
```
|
||
|
||
#### 6.2.4 启动从库
|
||
|
||
```bash
|
||
# 从主库基础备份
|
||
pg_basebackup -U postgres -h 192.168.1.100 -D /var/lib/pgsql/15/data -Xs -P
|
||
|
||
# 启动从库
|
||
pg_ctl -D /var/lib/pgsql/15/data -l /var/lib/pgsql/15/data/logfile start
|
||
```
|
||
|
||
### 6.3 逻辑复制配置
|
||
|
||
#### 6.3.1 发布和订阅
|
||
|
||
```sql
|
||
-- 在发布库创建发布
|
||
CREATE PUBLICATION mypub FOR TABLE users, orders;
|
||
|
||
-- 在订阅库创建订阅
|
||
CREATE SUBSCRIPTION mysub
|
||
CONNECTION 'host=192.168.1.100 port=5432 dbname=mydb user=postgres'
|
||
PUBLICATION mypub;
|
||
```
|
||
|
||
#### 6.3.2 复制槽
|
||
|
||
```sql
|
||
-- 创建复制槽（逻辑复制必须）
|
||
SELECT * FROM pg_create_logical_replication_slot('slot_name', 'pgoutput');
|
||
|
||
-- 查看复制槽
|
||
SELECT * FROM pg_replication_slots;
|
||
```
|
||
|
||
### 6.4 Patroni 高可用
|
||
|
||
#### 6.4.1 Patroni 架构
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────┐
|
||
│ Patroni 高可用架构 │
|
||
├─────────────────────────────────────────────────────┤
|
||
│ │
|
||
│ ┌─────────────┐ ┌─────────────┐ │
|
||
│ │ etcd 1 │──────│ etcd 2 │ │
|
||
│ └─────────────┘ └─────────────┘ │
|
||
│ │ │ │
|
||
│ └──────────┬─────────┘ │
|
||
│ ▼ │
|
||
│ ┌─────────────────┐ │
|
||
│ │ Patroni Leader │ │
|
||
│ │ 192.168.1.100 │ │
|
||
│ └────────┬────────┘ │
|
||
│ │ │
|
||
│ ┌────────┴────────┐ │
|
||
│ ▼ ▼ │
|
||
│ ┌──────────┐ ┌──────────┐ │
|
||
│ │ Standby 1│ │ Standby 2│ │
|
||
│ │192.168.1│ │192.168.1│ │
|
||
│ └──────────┘ └──────────┘ │
|
||
│ │
|
||
└─────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
#### 6.4.2 Patroni 配置
|
||
|
||
```yaml
|
||
# patroni.yml
|
||
scope: postgres-cluster
|
||
name: postgres-1
|
||
|
||
etcd:
|
||
hosts: 192.168.1.101:2379,192.168.1.102:2379,192.168.1.103:2379
|
||
|
||
postgresql:
|
||
data_dir: /var/lib/pgsql/15/data
|
||
pgpass: /tmp/pgpass
|
||
parameters:
|
||
wal_level: replica
|
||
max_wal_senders: 10
|
||
max_replication_slots: 10
|
||
hot_standby: on
|
||
|
||
bootstrap:
|
||
dcs: postgresql
|
||
method: basebackup
|
||
basebackup:
|
||
host: 192.168.1.100
|
||
user: postgres
|
||
|
||
restapi:
|
||
listen: 192.168.1.100:8008
|
||
connect_address: 192.168.1.100:8008
|
||
```
|
||
|
||
#### 6.4.3 启动 Patroni
|
||
|
||
```bash
|
||
# 安装 Patroni
|
||
pip install patroni[etcd] etcd
|
||
|
||
# 启动 Patroni
|
||
patroni patroni.yml
|
||
```
|
||
|
||
### 6.5 PgBouncer 连接池
|
||
|
||
```bash
|
||
# 安装 PgBouncer
|
||
yum install pgbouncer -y
|
||
|
||
# 配置 pgbouncer.ini
|
||
[databases]
|
||
mydb = host=192.168.1.100 port=5432 dbname=mydb
|
||
|
||
[pgbouncer]
|
||
listen_addr = *
|
||
listen_port = 6432
|
||
auth_type = md5
|
||
auth_file = /etc/pgbouncer/userlist.txt
|
||
pool_mode = transaction
|
||
max_client_conn = 1000
|
||
default_pool_size = 20
|
||
|
||
# 配置 userlist.txt
|
||
"postgres" "md5d7f7a8..."
|
||
|
||
# 启动 PgBouncer
|
||
systemctl start pgbouncer
|
||
systemctl enable pgbouncer
|
||
```
|
||
|
||
### 6.6 读写分离
|
||
|
||
```bash
|
||
# 应用程序配置
|
||
|
||
# 主库（写）
|
||
jdbc:postgresql://192.168.1.100:5432/mydb
|
||
|
||
# 从库（读）
|
||
jdbc:postgresql://192.168.1.101:5432/mydb
|
||
jdbc:postgresql://192.168.1.102:5432/mydb
|
||
|
||
# 伪代码：读写分离
|
||
def execute_query(sql):
|
||
if sql.strip().upper().startswith('SELECT'):
|
||
return execute_on_slave(sql)
|
||
else:
|
||
return execute_on_master(sql)
|
||
```
|
||
|
||
### 6.7 故障转移
|
||
|
||
```sql
|
||
-- 手动故障转移（从库执行）
|
||
SELECT pg_promote();
|
||
|
||
-- 查看复制状态
|
||
SELECT * FROM pg_stat_replication;
|
||
|
||
-- 查看复制延迟
|
||
SELECT now() - pg_last_xact_replay_timestamp() AS replication_lag;
|
||
```
|
||
|
||
### 6.8 监控指标
|
||
|
||
```sql
|
||
-- 查看连接数
|
||
SELECT count(*) FROM pg_stat_activity;
|
||
|
||
-- 查看慢查询
|
||
SELECT query, state, wait_event, query_start
|
||
FROM pg_stat_activity
|
||
WHERE state != 'idle'
|
||
ORDER BY query_start DESC;
|
||
|
||
-- 查看复制延迟
|
||
SELECT client_addr, state, lag
|
||
FROM pg_stat_replication;
|
||
|
||
-- 查看锁等待
|
||
SELECT * FROM pg_locks WHERE NOT granted;
|
||
```
|
||
|
||
---
|
||
|
||
## 附录
|
||
|
||
### 常用命令速查
|
||
|
||
| 命令 | 说明 |
|
||
|------|------|
|
||
| `psql -U postgres` | 连接数据库 |
|
||
| `\l` | 列出数据库 |
|
||
| `\d` | 列出表 |
|
||
| `\du` | 列出用户 |
|
||
| `\x` | 展开显示 |
|
||
| `CREATE DATABASE db` | 创建数据库 |
|
||
| `DROP DATABASE db` | 删除数据库 |
|
||
| `CREATE USER user` | 创建用户 |
|
||
| `GRANT ALL ON db` | 授权 |
|
||
|
||
### 配置文件参数速查
|
||
|
||
| 参数 | 说明 |
|
||
|------|------|
|
||
| `listen_addresses` | 监听地址 |
|
||
| `port` | 端口 |
|
||
| `max_connections` | 最大连接数 |
|
||
| `shared_buffers` | 共享缓存 |
|
||
| `work_mem` | 工作内存 |
|
||
| `wal_level` | WAL 级别 |
|
||
| `max_wal_senders` | 最大 WAL 发送数 |
|
||
|
||
### 端口说明
|
||
|
||
| 端口 | 说明 |
|
||
|------|------|
|
||
| 5432 | PostgreSQL 主服务端口 |
|
||
| 5433 | PostgreSQL 主服务（多实例） |
|
||
| 6432 | PgBouncer 连接池端口 |
|
||
| 8008 | Patroni REST API 端口 |
|
||
|
||
---
|
||
|
||
*文档创建时间：2024年*
|