mirror of
https://github.com/etcd-io/etcd.git
synced 2024-09-27 06:25:44 +00:00
185 lines
7.8 KiB
Markdown
185 lines
7.8 KiB
Markdown
# Clustering Guide
|
||
|
||
This guide will walk you through configuring a three machine etcd cluster with
|
||
the following details:
|
||
|
||
|Name |Address |
|
||
|-------|---------------|
|
||
|infra0 |10.0.1.10 |
|
||
|infra1 |10.0.1.11 |
|
||
|infra2 |10.0.1.12 |
|
||
|
||
## Static
|
||
|
||
As we know the cluster members, their addresses and the size of the cluster
|
||
before starting we can use an offline bootstrap configuration. Each machine
|
||
will get either the following command line or environment variables:
|
||
|
||
```
|
||
ETCD_INITIAL_CLUSTER=”infra0=http://10.0.1.10:2379,infra1=http://10.0.1.11:2379,infra2=http://10.0.1.12:2379”
|
||
ETCD_INITIAL_CLUSTER_STATE=new
|
||
```
|
||
|
||
```
|
||
-initial-cluster infra0=http://10.0.1.10:2379,http://10.0.1.11:2379,infra2=http://10.0.1.12:2379 \
|
||
-initial-cluster-state new
|
||
```
|
||
|
||
On each machine you would start etcd with these flags:
|
||
|
||
```
|
||
$ etcd -name infra0 -initial-advertise-peer-urls https://10.0.1.10:2379 \
|
||
-initial-cluster infra0=http://10.0.1.10:2379,infra1=http://10.0.1.11:2379,infra2=http://10.0.1.12:2379 \
|
||
-initial-cluster-state new
|
||
$ etcd -name infra1 -initial-advertise-peer-urls https://10.0.1.11:2379 \
|
||
-initial-cluster infra0=http://10.0.1.10:2379,infra1=http://10.0.1.11:2379,infra2=http://10.0.1.12:2379 \
|
||
-initial-cluster-state new
|
||
$ etcd -name infra2 -initial-advertise-peer-urls https://10.0.1.12:2379 \
|
||
-initial-cluster infra0=http://10.0.1.10:2379,infra1=http://10.0.1.11:2379,infra2=http://10.0.1.12:2379 \
|
||
-initial-cluster-state new
|
||
```
|
||
|
||
The command line parameters starting with `-initial-cluster` will be ignored on
|
||
subsequent runs of etcd. You are free to remove the environment variables or
|
||
command line flags after the initial bootstrap process. If you need to make
|
||
changes to the configuration later see our guide on runtime configuration.
|
||
|
||
### Error Cases
|
||
|
||
In the following case we have not included our new host in the list of
|
||
enumerated nodes. If this is a new cluster, the node must be added to the list
|
||
of initial cluster members.
|
||
```
|
||
$ etcd -name infra1 -initial-advertise-peer-urls http://10.0.1.11:2379 \
|
||
-initial-cluster infra0=http://10.0.1.10:2379 \
|
||
-initial-cluster-state new
|
||
etcd: infra1 not listed in the initial cluster config
|
||
exit 1
|
||
```
|
||
|
||
In this case we are attempting to map a node (infra0) on a different address
|
||
(127.0.0.1:2379) than its enumerated address in the cluster list
|
||
(10.0.1.10:2379). If this node is to listen on multiple addresses, all
|
||
addresses must be reflected in the “initial-cluster” configuration directive.
|
||
|
||
```
|
||
$ etcd -name infra0 -initial-advertise-peer-urls http://127.0.0.1:2379 \
|
||
-initial-cluster infra0=http://10.0.1.10:2379,infra1=http://10.0.1.11:2379,infra2=http://10.0.1.12:2379 \
|
||
-initial-cluster-state=new
|
||
etcd: infra0 has different advertised URLs in the cluster and advertised peer URLs list
|
||
exit 1
|
||
```
|
||
|
||
If you configure a peer with a different set of configuration and attempt to
|
||
join this cluster you will get a cluster ID mismatch and etcd will exit.
|
||
|
||
```
|
||
$ etcd -name infra3 -initial-advertise-peer-urls http://10.0.1.13:2379 \
|
||
-initial-cluster infra0=http://10.0.1.10:2379,infra1=http://10.0.1.11:2379,infra3=http://10.0.1.13:2379 \
|
||
-initial-cluster-state=new
|
||
etcd: conflicting cluster ID to the target cluster (c6ab534d07e8fcc4 != bc25ea2a74fb18b0). Exiting.
|
||
exit 1
|
||
```
|
||
|
||
|
||
## Discovery
|
||
|
||
In a number of cases you might not know the IPs of your cluster peers ahead of
|
||
time. This is common when utilizing cloud providers or when your network uses
|
||
DHCP. In these cases you can use an existing etcd cluster to bootstrap a new
|
||
one. We call this process “discovery”.
|
||
|
||
Discovery uses an existing cluster to bootstrap itself. If you are using your
|
||
own etcd cluster you can create a URL like so:
|
||
|
||
```
|
||
$ curl -X PUT https://myetcd.local/v2/keys/discovery/6c007a14875d53d9bf0ef5a6fc0257c817f0fb83/_config/size -d value=5
|
||
```
|
||
|
||
The URL you will use in this case will be
|
||
`https://myetcd.local/v2/keys/discovery/6c007a14875d53d9bf0ef5a6fc0257c817f0fb83`
|
||
and the machines will use the
|
||
`https://myetcd.local/v2/keys/discovery/6c007a14875d53d9bf0ef5a6fc0257c817f0fb83`
|
||
directory for registration as they start.
|
||
|
||
If you do not have access to an existing cluster you can use the hosted
|
||
discovery.etcd.io service. You can create a private discovery URL using the
|
||
"new" endpoint like so:
|
||
|
||
```
|
||
$ curl https://discovery.etcd.io/new?size=3
|
||
https://discovery.etcd.io/3e86b59982e49066c5d813af1c2e2579cbf573de
|
||
```
|
||
|
||
This will create the cluster with an initial expected size of 3 members. If you
|
||
do not specify a size a default of 3 will be used.
|
||
|
||
```
|
||
ETCD_DISCOVERY=https://discovery.etcd.io/3e86b59982e49066c5d813af1c2e2579cbf573de
|
||
```
|
||
|
||
```
|
||
-discovery https://discovery.etcd.io/3e86b59982e49066c5d813af1c2e2579cbf573de
|
||
```
|
||
|
||
Now we start etcd with those relevant flags on each machine:
|
||
|
||
```
|
||
$ etcd -name infra0 -initial-advertise-peer-urls http://10.0.1.10:2379 -discovery https://discovery.etcd.io/3e86b59982e49066c5d813af1c2e2579cbf573de
|
||
$ etcd -name infra1 -initial-advertise-peer-urls http://10.0.1.11:2379 -discovery https://discovery.etcd.io/3e86b59982e49066c5d813af1c2e2579cbf573de
|
||
$ etcd -name infra2 -initial-advertise-peer-urls http://10.0.1.12:2379 -discovery https://discovery.etcd.io/3e86b59982e49066c5d813af1c2e2579cbf573de
|
||
```
|
||
|
||
This will cause each machine to register itself with the etcd service and begin
|
||
the cluster once all machines have been registered.
|
||
|
||
You can use the environment variable `ETCD_DISCOVERY_PROXY` to cause etcd to use an HTTP proxy to connect to the discovery service.
|
||
|
||
### Error and Warning Cases
|
||
|
||
#### Discovery Server Errors
|
||
|
||
```
|
||
$ etcd -name infra0 -initial-advertise-peer-urls http://10.0.1.10:2379 -discovery https://discovery.etcd.io/3e86b59982e49066c5d813af1c2e2579cbf573de
|
||
etcd: error: the cluster doesn’t have a size configuration value in https://discovery.etcd.io/3e86b59982e49066c5d813af1c2e2579cbf573de/_config
|
||
exit 1
|
||
```
|
||
|
||
#### User Errors
|
||
|
||
```
|
||
$ etcd -name infra0 -initial-advertise-peer-urls http://10.0.1.10:2379 -discovery https://discovery.etcd.io/3e86b59982e49066c5d813af1c2e2579cbf573de
|
||
etcd: error: the cluster using discovery https://discovery.etcd.io/3e86b59982e49066c5d813af1c2e2579cbf573de has already started with all 5 members
|
||
exit 1
|
||
```
|
||
|
||
#### Warnings
|
||
|
||
This is a harmless warning notifying you that the discovery URL will be
|
||
ignored on this machine.
|
||
|
||
```
|
||
$ etcd -name infra0 -initial-advertise-peer-urls http://10.0.1.10:2379 -discovery https://discovery.etcd.io/3e86b59982e49066c5d813af1c2e2579cbf573de
|
||
etcd: warn: ignoring discovery URL: etcd has already been initialized and has a valid log in /var/lib/etcd
|
||
```
|
||
|
||
# 0.4 to 0.5+ Migration Guide
|
||
|
||
In etcd 0.5 we introduced the ability to listen on more than one address and to
|
||
advertise multiple addresses. This makes using etcd easier when you have
|
||
complex networking, such as private and public networks on various cloud
|
||
providers.
|
||
|
||
To make understanding this feature easier, we changed the naming of some flags,
|
||
but we support the old flags to make the migration from the old to new version
|
||
easier.
|
||
|
||
|Old Flag |New Flag |Migration Behavior |
|
||
|-----------------------|-----------------------|---------------------------------------------------------------------------------------|
|
||
|-peer-addr |-initial-advertise-peer-urls |If specified, peer-addr will be used as the only peer URL. Error if both flags specified.|
|
||
|-addr |-advertise-client-urls |If specified, addr will be used as the only client URL. Error if both flags specified.|
|
||
|-peer-bind-addr |-listen-peer-urls |If specified, peer-bind-addr will be used as the only peer bind URL. Error if both flags specified.|
|
||
|-bind-addr |-listen-client-urls |If specified, bind-addr will be used as the only client bind URL. Error if both flags specified.|
|
||
|-peers |none |Deprecated. The -initial-cluster flag provides a similar concept with different semantics. Please read this guide on cluster startup.|
|
||
|-peers-file |none |Deprecated. The -initial-cluster flag provides a similar concept with different semantics. Please read this guide on cluster startup.|
|