ETCD 部署并启用证书认证

原 作者 周攀 | 发表于 | 分类于 2014级

证书生成

证书生成的方式非常多，同类的文档也很容易找到，这里推荐此篇文章：https://coreos.com/os/docs/latest/generate-self-signed-certificates.html，但是由于etcd的特殊性，server端配置的证书，也会被用于去进行客户端认证，因此需要在 server 的 usages 里面加上：client auth选项，否则会出现此种问题：certificate specifies an incompatible key usage

下面是完整的ca-config.json内容：

{
    "signing": {
        "default": {
            "expiry": "43800h"
        },
        "profiles": {
            "server": {
                "expiry": "43800h",
                "usages": [
                    "signing",
                    "key encipherment",
                    "server auth",
                    "client auth" // 此处需要特别注意，由于etcd实现方式不同，server的证书也会用于进行客户端认证，此选项必须的
                ]
            },
            "client": {
                "expiry": "43800h",
                "usages": [
                    "signing",
                    "key encipherment",
                    "client auth"
                ]
            },
            "peer": {
                "expiry": "43800h",
                "usages": [
                    "signing",
                    "key encipherment",
                    "server auth",
                    "client auth"
                ]
            }
        }
    }
}

还有需要注意的点就是在生成 server证书的时候，hosts字段需要加上etcd全部节点的IP/主机名信息及127.0.0.1，peer证书可以统一，也可以分别生成，如果需要统一，则需要在hosts字段加上所有节点的IP/主机名信息，如果分开生成，则hosts字段只需要填写对应节点的IP/主机名信息即可

编写配置

准备使用三台机器，以下是主机及端口相关信息规划：

Name IP client port peer port
node-0 192.168.0.1 4001 2379
node-1 192.168.0.2 4001 2379
node-2 192.168.0.3 4001 2379

数据目录我们统一使用：/home/work/etcd/data

证书目录我们统一为：/home/work/etcd/conf/ssl/，其下有 ca.crt、server.crt、server.key、peer.crt、peer.key，其中 server.crt 与 server.key 用于服务端验证，peer.crt与peer.key用于etcd各节点之间的认证，都是由上面我们统一生成的

配置参数解析：

参数名称 参数含义 对应命令行参数
name 全集群唯一，对应命令行参数 –name
data-dir 服务运行数据存储目录 –data-dir
listen-client-urls 对外提供服务，供客户端连接的端口 –listen-client-urls
advertise-client-urls 对外公告的该节点客户端监听地址，这个值会告诉集群中其他节点 –advertise-client-urls
listen-peer-urls 和集群其他节点进行通信的地址 –listen-peer-urls
initial-advertise-peer-urls 该节点同伴监听地址，这个值会告诉集群中其他节点 –initial-advertise-peer-urls
initial-cluster 集群中所有节点的信息，采用name=ip:port 形式，多个值采用逗号分隔，如下：node-0=https://192.168.0.1:2379,node-1=https://192.168.0.2:2379…… –initial-cluster
initial-cluster-state 初始化集群的状态；对于新集群的时候，这个值默认为 new，对已经存在的集群，这个值为 existing –initial-cluster-state
initial-cluster-token 集群token，需要每个集群保持唯一 –initial-cluster-token
client-cert-auth 是否启用证书认证功能 –client-cert-auth / --peer-client-cert-auth
trusted-ca-file CA证书路径 –trusted-ca-file / --peer-trusted-ca-file
cert-file 证书路径 –cert-file / --peer-cert-file
key-file 私钥路径 –key-file / --peer-key-file

下面是各个节点的配置内容信息：

# node-0
name: node-0
data-dir: /home/work/etcd/data
listen-client-urls: https://0.0.0.0:4001
advertise-client-urls: https://192.168.0.1:4001
listen-peer-urls: https://0.0.0.0:2379
initial-advertise-peer-urls: https://192.168.0.1:2379
initial-cluster: node-0=https://192.168.0.1:2379,node-1=https://192.168.0.2:2379,node-2=https://192.168.0.3:2379
initial-cluster-token: "febc275dc726ae0ac16348d3d58fe0f3"
initial-cluster-state: new
client-transport-security:
  client-cert-auth: true
  trusted-ca-file: /home/work/etcd/conf/ssl/ca.crt
  cert-file: /home/work/etcd/conf/ssl/server.crt
  key-file: /home/work/etcd/conf/ssl/server.key
peer-transport-security:
  trusted-ca-file: /home/work/etcd/conf/ssl/ca.crt
  cert-file: /home/work/etcd/conf/ssl/peer.crt
  key-file: /home/work/etcd/conf/ssl/peer.key
  client-cert-auth: true

# node-1
name: node-1
data-dir: /home/work/etcd/data
listen-client-urls: https://0.0.0.0:4001
advertise-client-urls: https://192.168.0.2:4001
listen-peer-urls: https://0.0.0.0:2379
initial-advertise-peer-urls: https://192.168.0.2:2379
initial-cluster: node-0=https://192.168.0.1:2379,node-1=https://192.168.0.2:2379,node-2=https://192.168.0.3:2379
initial-cluster-token: "febc275dc726ae0ac16348d3d58fe0f3"
initial-cluster-state: new
client-transport-security:
  client-cert-auth: true
  trusted-ca-file: /home/work/etcd/conf/ssl/ca.crt
  cert-file: /home/work/etcd/conf/ssl/server.crt
  key-file: /home/work/etcd/conf/ssl/server.key
peer-transport-security:
  trusted-ca-file: /home/work/etcd/conf/ssl/ca.crt
  cert-file: /home/work/etcd/conf/ssl/peer.crt
  key-file: /home/work/etcd/conf/ssl/peer.key
  client-cert-auth: true


# node-2
name: node-2
data-dir: /home/work/etcd/data
listen-client-urls: https://0.0.0.0:4001
advertise-client-urls: https://192.168.0.3:4001
listen-peer-urls: https://0.0.0.0:2379
initial-advertise-peer-urls: https://192.168.0.3:2379
initial-cluster: node-0=https://192.168.0.1:2379,node-1=https://192.168.0.2:2379,node-2=https://192.168.0.3:2379
initial-cluster-token: "febc275dc726ae0ac16348d3d58fe0f3"
initial-cluster-state: new
client-transport-security:
  client-cert-auth: true
  trusted-ca-file: /home/work/etcd/conf/ssl/ca.crt
  cert-file: /home/work/etcd/conf/ssl/server.crt
  key-file: /home/work/etcd/conf/ssl/server.key
peer-transport-security:
  trusted-ca-file: /home/work/etcd/conf/ssl/ca.crt
  cert-file: /home/work/etcd/conf/ssl/peer.crt
  key-file: /home/work/etcd/conf/ssl/peer.key
  client-cert-auth: true

服务部署

下载二进制包并解压

从 https://github.com/etcd-io/etcd/releases 选择对应的版本即可，如果不知道该选哪个版本，直接选最新即可

我这里选择最新的v3.4.9版：

cd /home/work/etcd/
mkdir bin
wget https://github.com/etcd-io/etcd/releases/download/v3.4.9/etcd-v3.4.9-linux-amd64.tar.gz # 下载二进制
tar -zvxf etcd-v3.4.9-linux-amd64 # 解压
mv etcd-v3.4.9-linux-amd64/etcd  etcd-v3.4.9-linux-amd64/etcdctl bin/ # 将二进制文件移入指定目录
export PATH=$PATH:/home/work/etcd/bin # 添加环境变量
export ETCDCTL_API=2 # 设置etcdctl的API版本为v2版本，否则参数不识别

启动服务

假设配置文件我们已经写入 /home/work/etcd/conf 目录下的 etcd.yml 文件中，我们可通过如下命令启动：

etcd --config-file conf/etcd.yml

使用god守护ETCD

关于god参见：God 使用手册

以下是god的配置文件：

God.watch do |w|
  w.autostart = false
  w.grace = 1
  w.name = "etcd"
  w.uid = "work"
  w.gid = "work"
  w.start = "/home/work/etcd/bin/etcd --config-file /home/work/etcd/conf/etcd.yml"
  w.log = "/home/work/etcd/log/etcd.log"
  w.dir = "/home/work/etcd"
  w.keepalive(
  )
  w.behavior(:clean_pid_file)
  w.stop_timeout = 300.seconds
  w.lifecycle do |on|
    on.condition(:flapping) do |c|
      c.to_state = [:start, :restart]
      c.times = 6
      c.within = 1.minute
      c.transition = :unmonitored
      c.retry_in = 1.minutes
      c.retry_times = 200000
      c.retry_within = 1.hours
      c.notify = 'proc_down'
    end
  end
end