etcdctl的v3版本与v2版本使用命令有所不同，本文介绍etcdctl v3版本的命令工具的使用方式。

1. etcdctl的安装

etcdctl的二进制文件可以在 github.com/coreos/etcd/releases 选择对应的版本下载，例如可以执行以下install_etcdctl.sh的脚本，修改其中的版本信息。

#!/bin/bash
ETCD_VER=v3.3.4
ETCD_DIR=etcd-download
DOWNLOAD_URL=https://github.com/coreos/etcd/releases/download

# Download
mkdir ${ETCD_DIR}
cd ${ETCD_DIR}
wget ${DOWNLOAD_URL}/${ETCD_VER}/etcd-${ETCD_VER}-linux-amd64.tar.gz 
tar -xzvf etcd-${ETCD_VER}-linux-amd64.tar.gz

# install
cd etcd-${ETCD_VER}-linux-amd64
cp etcdctl /usr/local/bin/

2. etcdctl V3

使用etcdctlv3的版本时，需设置环境变量ETCDCTL_API=3。

export ETCDCTL_API=3

或者在`/etc/profile`文件中添加环境变量
vi /etc/profile
...
export ETCDCTL_API=3
...
source /etc/profile

查看当前etcdctl的版本信息etcdctl version。

[root@k8s-dbg-master-1 etcd]# etcdctl version
etcdctl version: 3.3.4
API version: 3.3

更多命令帮助可以查询etcdctl —help。

[root@k8s-dbg-master-1 etcd]# etcdctl --help
NAME:
	etcdctl - A simple command line client for etcd3.

USAGE:
	etcdctl

VERSION:
	3.3.4

API VERSION:
	3.3


COMMANDS:
	get			Gets the key or a range of keys
	put			Puts the given key into the store
	del			Removes the specified key or range of keys [key, range_end)
	txn			Txn processes all the requests in one transaction
	compaction		Compacts the event history in etcd
	alarm disarm		Disarms all alarms
	alarm list		Lists all alarms
	defrag			Defragments the storage of the etcd members with given endpoints
	endpoint health		Checks the healthiness of endpoints specified in `--endpoints` flag
	endpoint status		Prints out the status of endpoints specified in `--endpoints` flag
	endpoint hashkv		Prints the KV history hash for each endpoint in --endpoints
	move-leader		Transfers leadership to another etcd cluster member.
	watch			Watches events stream on keys or prefixes
	version			Prints the version of etcdctl
	lease grant		Creates leases
	lease revoke		Revokes leases
	lease timetolive	Get lease information
	lease list		List all active leases
	lease keep-alive	Keeps leases alive (renew)
	member add		Adds a member into the cluster
	member remove		Removes a member from the cluster
	member update		Updates a member in the cluster
	member list		Lists all members in the cluster
	snapshot save		Stores an etcd node backend snapshot to a given file
	snapshot restore	Restores an etcd member snapshot to an etcd directory
	snapshot status		Gets backend snapshot status of a given file
	make-mirror		Makes a mirror at the destination etcd cluster
	migrate			Migrates keys in a v2 store to a mvcc store
	lock			Acquires a named lock
	elect			Observes and participates in leader election
	auth enable		Enables authentication
	auth disable		Disables authentication
	user add		Adds a new user
	user delete		Deletes a user
	user get		Gets detailed information of a user
	user list		Lists all users
	user passwd		Changes password of user
	user grant-role		Grants a role to a user
	user revoke-role	Revokes a role from a user
	role add		Adds a new role
	role delete		Deletes a role
	role get		Gets detailed information of a role
	role list		Lists all roles
	role grant-permission	Grants a key to a role
	role revoke-permission	Revokes a key from a role
	check perf		Check the performance of the etcd cluster
	help			Help about any command

OPTIONS:
      --cacert=""				verify certificates of TLS-enabled secure servers using this CA bundle
      --cert=""					identify secure client using this TLS certificate file
      --command-timeout=5s			timeout for short running command (excluding dial timeout)
      --debug[=false]				enable client-side debug logging
      --dial-timeout=2s				dial timeout for client connections
  -d, --discovery-srv=""			domain name to query for SRV records describing cluster endpoints
      --endpoints=[127.0.0.1:2379]		gRPC endpoints
      --hex[=false]				print byte strings as hex encoded strings
      --insecure-discovery[=true]		accept insecure SRV records describing cluster endpoints
      --insecure-skip-tls-verify[=false]	skip server certificate verification
      --insecure-transport[=true]		disable transport security for client connections
      --keepalive-time=2s			keepalive time for client connections
      --keepalive-timeout=6s			keepalive timeout for client connections
      --key=""					identify secure client using this TLS key file
      --user=""					username[:password] for authentication (prompt if password is not supplied)
  -w, --write-out="simple"			set the output format (fields, json, protobuf, simple, table)

3. etcdctl 常用命令

3.1. 指定etcd集群

HOST_1=10.240.0.17
HOST_2=10.240.0.18
HOST_3=10.240.0.19
ENDPOINTS=$HOST_1:2379,$HOST_2:2379,$HOST_3:2379

etcdctl --endpoints=$ENDPOINTS member list

执行查询时前缀是固定的，如下所示，使用这个前缀再加上etcd的查找命令即可成功查询：

ETCDCTL_API=3 /usr/local/bin/etcdctl --endpoints=https://127.0.0.1:2379 \
--cacert=/etc/kubernetes/pki/etcd/ca.crt \
--cert=/etc/kubernetes/pki/etcd/healthcheck-client.crt \
--key=/etc/kubernetes/pki/etcd/healthcheck-client.key

命令行太长了，不容易记住，使用alias命令用来设置指令的别名。

alias etcdctl="ETCDCTL_API=3 /usr/local/bin/etcdctl --endpoints=https://127.0.0.1:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/healthcheck-client.crt --key=/etc/kubernetes/pki/etcd/healthcheck-client.key"

3.2. 增删改查

1、增

etcdctl --endpoints=$ENDPOINTS put foo "Hello World!"

2、查

etcdctl --endpoints=$ENDPOINTS get foo
etcdctl --endpoints=$ENDPOINTS --write-out="json" get foo
# 查看所有etcd的所有key，执行以下命令
etcdctl get / --prefix --keys-only


# 查看某个POD信息
# 输出信息中，有少量不可见字符，这是因为etcd中存储的并不是json的原文，而是protocol buffer序列化后的数据，不过还是有部分内容是可读的；
etcdctl get /registry/pods/default/nginx-deployment-6b474476c4-qsscn

基于相同前缀查找

etcdctl --endpoints=$ENDPOINTS put web1 value1
etcdctl --endpoints=$ENDPOINTS put web2 value2
etcdctl --endpoints=$ENDPOINTS put web3 value3

etcdctl --endpoints=$ENDPOINTS get web --prefix

列出所有的key

etcdctl --endpoints=$ENDPOINTS get / --prefix --keys-only

3、删

etcdctl --endpoints=$ENDPOINTS put key myvalue
etcdctl --endpoints=$ENDPOINTS del key

etcdctl --endpoints=$ENDPOINTS put k1 value1
etcdctl --endpoints=$ENDPOINTS put k2 value2
etcdctl --endpoints=$ENDPOINTS del k --prefix

3.3. 集群状态

集群状态主要是etcdctl endpoint status 和etcdctl endpoint health两条命令。

etcdctl --write-out=table --endpoints=$ENDPOINTS endpoint status

+------------------+------------------+---------+---------+-----------+-----------+------------+
|     ENDPOINT     |        ID        | VERSION | DB SIZE | IS LEADER | RAFT TERM | RAFT INDEX |
+------------------+------------------+---------+---------+-----------+-----------+------------+
| 10.240.0.17:2379 | 4917a7ab173fabe7 | 3.0.0   | 45 kB   | true      |         4 |      16726 |
| 10.240.0.18:2379 | 59796ba9cd1bcd72 | 3.0.0   | 45 kB   | false     |         4 |      16726 |
| 10.240.0.19:2379 | 94df724b66343e6c | 3.0.0   | 45 kB   | false     |         4 |      16726 |
+------------------+------------------+---------+---------+-----------+-----------+------------+

etcdctl --endpoints=$ENDPOINTS endpoint health

10.240.0.17:2379 is healthy: successfully committed proposal: took = 3.345431ms
10.240.0.19:2379 is healthy: successfully committed proposal: took = 3.767967ms
10.240.0.18:2379 is healthy: successfully committed proposal: took = 4.025451ms

3.4. 集群成员

跟集群成员相关的命令如下：

	member add		    Adds a member into the cluster
	member remove		Removes a member from the cluster
	member update		Updates a member in the cluster
	member list		    Lists all members in the cluster

例如 etcdctl member list列出集群成员的命令。

etcdctl --endpoints=http://172.16.5.4:12379 member list -w table

+-----------------+---------+-------+------------------------+-----------------------------------------------+
|       ID        | STATUS  | NAME  |       PEER ADDRS       |                 CLIENT ADDRS                  |
+-----------------+---------+-------+------------------------+-----------------------------------------------+
| c856d92a82ba66a | started | etcd0 | http://172.16.5.4:2380 | http://172.16.5.4:2379,http://172.16.5.4:4001 |
+-----------------+---------+-------+------------------------+-----------------------------------------------+

4. etcdctl get

使用etcdctl {command} --help可以查看具体命令的帮助信息。

# etcdctl get --help
NAME:
	get - Gets the key or a range of keys

USAGE:
	etcdctl get [options] <key> [range_end]

OPTIONS:
      --consistency="l"			Linearizable(l) or Serializable(s)
      --from-key[=false]		Get keys that are greater than or equal to the given key using byte compare
      --keys-only[=false]		Get only the keys
      --limit=0				Maximum number of results
      --order=""			Order of results; ASCEND or DESCEND (ASCEND by default)
      --prefix[=false]			Get keys with matching prefix
      --print-value-only[=false]	Only write values when using the "simple" output format
      --rev=0				Specify the kv revision
      --sort-by=""			Sort target; CREATE, KEY, MODIFY, VALUE, or VERSION

GLOBAL OPTIONS:
      --cacert=""				verify certificates of TLS-enabled secure servers using this CA bundle
      --cert=""					identify secure client using this TLS certificate file
      --command-timeout=5s			timeout for short running command (excluding dial timeout)
      --debug[=false]				enable client-side debug logging
      --dial-timeout=2s				dial timeout for client connections
      --endpoints=[127.0.0.1:2379]		gRPC endpoints
      --hex[=false]				print byte strings as hex encoded strings
      --insecure-skip-tls-verify[=false]	skip server certificate verification
      --insecure-transport[=true]		disable transport security for client connections
      --key=""					identify secure client using this TLS key file
      --user=""					username[:password] for authentication (prompt if password is not supplied)
  -w, --write-out="simple"			set the output format (fields, json, protobuf, simple, table)

数据结构说明

/registry/apiregistration.k8s.io/apiservices/{版本}.{api名称}

包含 Kubernetes 中 API 服务的定义，因此我们可以在其中找到 Kubernetes 使用的所有现有核心 API，例如 /registry/apiregistration.k8s.io/apiservices/v1.batch或 /registry/apiregistration.k8s.io/apiservices/v1beta1.rbac。 authorization.k8s.io或自定义 API 定义（请参阅 https://github.com/kubernetes-incubator/apiserver-builder/blob/master/docs/concepts/aggregation.md）。您可以通过在 Etcd 中读取该键的值（您将获得人类可读的 json）或更友好的方式使用 kubectl get apiservice v1beta1.authorization.k8s.io -o json（与直接值相同）来获取有关 API 的信息等访问）

/registry/clusterroles/{角色名称}

包含 Kubernetes 中所有集群范围角色的定义，因此我们可以在那里找到 /registry/clusterroles/cluster-admin 或 /registry/clusterroles/system:kube-scheduler 之类的内容。Etcd 中的数据是人类可读但难以理解的——我们可以看到一些操作，如获取、修补、更新 API 的某些部分

/registry/clusterrolebindings/{实体名称}

包含集群范围内的角色和用户/组/服务帐户之间的绑定，因此我们可以在那里找到 /registry/clusterrolebindings/cluster-admin或 /registry/clusterrolebindings/kubeadm:kubelet-bootstrap 之类的东西。Etcd 中的数据是人类可读但难以理解的。

/registry/roles/{namespace}/{role name} 和 /registry/rolebindings/{namespace}/{entity name}

与集群角色/绑定中的故事相同，但受命名空间限制，例如 /registry/roles/kube-system/system:controller:token-cleaner

/registry/serviceaccounts/{namespace}/{name}

所有服务帐户的定义

/registry/configmaps/{命名空间}/{地图名称}

所有配置映射存储为 yamls

/registry/controllerrevisions/{namespace}/{pod}

我发现 ControllerRevision 资源用于在 DaemonSet 和 StatefulSet ( https://kubernetes.io/docs/tasks/manage-daemon/rollback-daemon-set/ ) 中提供回滚可能性。在 Etcd 中，我们可以找到 pod 规范的快照。

/registry/daemonsets/{namespace}/{name} 和 /registry/deployments/{namespace}/{name} 等。

在这些键下，Kubernetes 存储有关不同部署的信息，例如 DaemonSet、Deployment、ReplicaSet、Job 等。在部署的情况下有趣的是，我们看到那里描述了 last-applied-configuration https://kubernetes.io/docs/concepts/overview/对象管理-kubectl/declarative-config/#merge-patch-calculation

/registry/minions/{节点名称}

Kubernetes 节点以前被称为“minions”，所以在 Etcd 中的名称仍然没有改变。我们看到有大量数据描述节点，例如：

CPU内核
内存大小
kubelet 的状态：例如 kubelet 有足够的可用磁盘空间或 kubelet 有足够的 PID 可用
IP地址
主机名
Docker 版本
Docker image /registry/ranges/servicenodeportss 在节点上可用

/registry/namespaces/{namespace}

只是定义命名空间。还有特定命名空间的状态，如 Active 或 Terminating。

/registry/pods/{namespace}/{pod 名称}

集群中运行的每个 pod 的状态。包含很多信息，如 pod IP、挂载的卷、docker 映像等。

/registry/ranges/serviceips

服务的 CIDR

/registry/ranges/servicenodeports

暴露服务的端口范围

/registry/secrets/{namespace}/{pod}

集群中的所有秘密都以默认模式存储为纯文本。有关加密，请参阅 https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/

/registry/services/endpoints/{namespace}/{name}

服务定义。Kubernetes 计算特定服务选择了哪些 Pod，并将该信息存储在服务值中，以便我们可以在那里看到 Pod 的 IP 地址和名称。

文章参考：

https://coreos.com/etcd/docs/latest/demo.html
https://jakubbujny.wordpress.com/2018/09/02/what-stores-kubernetes-in-etcd/