15971 Commits

Author SHA1 Message Date
Tobias Schottdorf
e039629907 raft: use half-populated joint quorum
To ease a future transition into joint quorums, this commit removes the
previous "ad-hoc" majority-based quorum and vote computations with that
introduced in the `raft/quorum` package.

More specifically, the progressTracker now uses a quorum.JointConfig for
which the "second" majority quorum is always empty; in this case the
quorum behaves like the one quorum.MajorityConfig that is actually
present. Or, more briefly, this change is a no-op, but it will take the
busywork out of actually starting to make use of joint quorums in the
future.

On a side node, I suspect that this might've fixed a bug regarding the
read index though I haven't been able to explicitly come up with a
counter-example. The problem was that the acks collected for the read
index weren't taking into account membership changes, so they'd run the
danger of using acks from nodes since removed to claim that a quorum of
acks had been received. There's a chance that there isn't a
counter-example (the only guarantee extracted from the "quorum" is that
there isn't another leader, but even if there's another leader all that
matters is that that leader doesn't have a divergent history from the
stale leader in the hypothetical counter-example), but either way there
is morally a bug here that is now fixed because VoteCommitted doesn't
care about votes from members that are not voters known to the currently
active configuration.
2019-06-19 14:19:35 +02:00
Tobias Schottdorf
0384c587eb raft: rename makeP{RS,rogressTracker} 2019-06-19 14:19:35 +02:00
Tobias Schottdorf
3def2364e4 raft: use membership sets in progress tracking
Instead of having disjoint mappings of ID to *Progress for voters and
learners, use a map[id]struct{} for each and share a map of *Progress
among them.

This is easier to handle when joint quorums are introduced, at which
point a node may be a voting member of two quorums.
2019-06-19 14:19:35 +02:00
Tobias Schottdorf
76c8ca5a55 quorum: introduce library for majority and joint quorums
The quorum package contains logic to reason about committed indexes as
well as vote outcomes for both majority and joint quorums. The package
is oblivious to the existence of learner replicas.

The plan is to hook this up to etcd/raft in subsequent commits.
2019-06-19 14:19:35 +02:00
Jingyi Hu
9ff7628577
Merge pull request #10801 from roe85/master
DOC: Fix referencing the wrong version number.
2019-06-18 13:24:28 -07:00
Gyuho Lee
53891cbf97
Merge pull request #10822 from tbg/airplane/learner-no-campaign
raft: prevent learners from becoming leader
2019-06-17 10:58:24 -07:00
Gyuho Lee
e5876c6ce2
Merge pull request #10826 from yznima/pr-race
Raft HTTP: fix pause/resume race condition
2019-06-17 09:58:18 -07:00
Nima Yahyazadeh
b1812a410f Raft HTTP: fix pause/resume race condition 2019-06-17 11:45:25 -04:00
Tobias Schottdorf
c844526002 raft: prevent learners from becoming leader
We were already taking some precautions against learners campaigning,
but there was no safeguard against an explicit call to `Campaign()`.
The newly added test also verifies that leadership transfers to
learners are ignored.
2019-06-17 09:20:45 +02:00
Richard
c27e1108f4 Doc: Fix referencing the wrong version number. 2019-06-14 14:35:34 +02:00
Xiang Li
2c5162af5c
Merge pull request #10523 from jingyih/fully_concurrent_reads
mvcc: fully concurrent read
2019-06-14 12:25:17 +08:00
Jingyi Hu
55066ebdc0 mvcc: address comments 2019-06-13 18:05:50 -07:00
Xiang Li
0de9b8abf5
Merge pull request #10815 from jingyih/fix_RLock
mvcc/backend: use RLock in test
2019-06-11 23:10:12 -07:00
Jingyi Hu
2a9320e944 mvcc: add TestConcurrentReadTxAndWrite
Add TestConcurrentReadTxAndWrite which creates random reads and writes,
and ensures reads always see latest writes.
2019-06-11 17:05:41 -07:00
Jingyi Hu
b873fbd127 mvcc/backend: correct RLock in test
Should use RLock instead of Lock.
2019-06-11 16:24:44 -07:00
Jingyi Hu
693afd8e5e mvcc/backend: add unit test for ConcurrentReadTx
Add TestConcurrentReadTx to ensure concurrentReadTx can see all the
prior writes which are stored on the read buffer.
2019-06-10 18:30:21 -07:00
Jingyi Hu
ad80752715 mvcc: add metrics dbOpenReadTxn
Expose the number of currently open read transactions in backend to
metrics endpoint.
2019-06-10 17:20:04 -07:00
Jingyi Hu
d6280f9ea5
Merge pull request #10809 from jingyih/update_changelog
CHANGELOG-3.4: update from #10808
2019-06-09 20:59:43 -07:00
Jingyi Hu
6906d07d1f CHANGELOG: update from PR10808 2019-06-09 20:16:13 -07:00
Jingyi Hu
833620b864
Merge pull request #10808 from WIZARD-CXY/updateboltdb2
vendor: update boltdb
2019-06-09 20:08:56 -07:00
宇慕
a2a077790b vendor: update boltdb 2019-06-10 10:25:59 +08:00
Jingyi Hu
48d144a3de
Merge pull request #10731 from WIZARD-CXY/learner_metric
etcdserver: add learner metrics
2019-06-08 22:43:03 -07:00
Xiang Li
ea70731f53
Merge pull request #10762 from FrozenAndrey/fix#10747
etcdmain: fix ignoring of ETCD_CONFIG_FILE env variable
2019-06-07 21:32:35 -07:00
Xiang Li
f6a9ebe579
Merge pull request #10710 from j2gg0s/refactor-lease-bench-test
lease: refactor lease's benchmark.
2019-06-07 21:32:04 -07:00
Xiang Li
372086cca1
Merge pull request #10803 from tbg/rna
raft: make relationship between node and RawNode explicit
2019-06-07 15:43:28 -07:00
Joe Betz
b5c6904cea
Merge pull request #10802 from jpbetz/jingyih-maintainer
Add @jingyih to MAINTAINERS
2019-06-07 14:59:39 -07:00
Joe Betz
54dcb9cf34
Add @jingyih to MAINTAINERS 2019-06-07 14:58:50 -07:00
Xiang Li
ea0be95387
Merge pull request #10792 from spzala/triagereflink
README: provide ref for issue and PR managenet doc
2019-06-07 14:09:47 -07:00
Tobias Schottdorf
cbb7730c26 raft: make relationship between node and RawNode explicit
This will keep them from diverging to much. In fact we should remove
some of the obvious differences that have crept in over time so that
what remains is structural. This isn't done in this commit since
it amounts to a change in the public API; we should lump this in
when we break the public API the next time.
2019-06-07 23:07:42 +02:00
Xiang Li
805b918715
Merge pull request #10800 from mitake/curl-test-nopassword
tests/e2e: initialize UserAddOptions{} field in testV3CurlAuth()
2019-06-06 17:39:50 -07:00
Hitoshi Mitake
7bbc536e1c tests/e2e: initialize UserAddOptions{} field in testV3CurlAuth() 2019-06-06 23:07:41 +09:00
Gyuho Lee
30ca4ae1e2
Merge pull request #10798 from WIZARD-CXY/learnerCLOG
CHANGELOG: add learner metrics
2019-06-06 01:49:49 -07:00
Xiang Li
9a73013004
Merge pull request #10797 from jingyih/lease_checkpoint_enabled_by_experimental_flag
*: enable lease checkpoint via experimental flag
2019-06-05 22:56:54 -07:00
宇慕
489675644a CHANGELOG: add learner metrics 2019-06-06 10:29:57 +08:00
Jingyi Hu
5af3723e28 CHANGELOG: update changelog-3.4 2019-06-05 15:45:58 -07:00
Jingyi Hu
e67b9829b6 *: enable lease checkpoint via experimental flag
Primary lessor persist lease remainingTTL only if experimental flag
"--experimental-enable-lease-checkpoint" is set.
2019-06-05 15:30:03 -07:00
Gyuho Lee
aa016eebf8
Merge pull request #10631 from spzala/goruntimeupdate112
*: test update for Go 1.12.5 and related changes
2019-06-05 15:28:37 -07:00
Gyuho Lee
1caaa9ed4a test: test update for Go 1.12.5 and related changes
Update to Go 1.12.5 testing. Remove deprecated unused and gosimple
pacakges, and mask staticcheck 1006. Also, fix unconvert errors related
to unnecessary type conversions and following staticcheck errors:
- remove redundant return statements
- use for range instead of for select
- use time.Since instead of time.Now().Sub
- omit comparison to bool constant
- replace T.Fatal and T.Fatalf in tests with T.Error and T.Fatalf respectively because the goroutine calls T.Fatal must be called in the same goroutine as the test
- fix error strings that should not be capitalized
- use sort.Strings(...) instead of sort.Sort(sort.StringSlice(...))
- use he status code of Canceled instead of grpc.ErrClientConnClosing which is deprecated
- use use status.Errorf instead of grpc.Errorf which is deprecated

Related #10528 #10438
2019-06-05 17:02:05 -04:00
Gyuho Lee
25412f9690
Merge pull request #10789 from jingyih/update_changelog_from_9540
CHANGELOG: update from cherry picks of #9540
2019-06-05 08:18:29 -07:00
Sahdev P. Zala
336c01b8d4 README: provide ref for issue and PR managenet doc
related: https://github.com/etcd-io/etcd/pull/10668

Provide a ref to the issue and PR management doc from the README,
similar to other references we have provided in the README.
2019-06-05 10:44:04 -04:00
Gyuho Lee
ea45cd61d0
Merge pull request #10788 from jingyih/add_missing_newline_EndpointHealth
ctlv3: add newline in EndpointHealth output
2019-06-05 02:19:54 -07:00
宇慕
0b8727b3f3 etcdserver: add learner metrics 2019-06-05 10:51:21 +08:00
yanjie.wyj
6a7ee7063c lease: refactor benchmark. 2019-06-05 10:01:01 +08:00
Jingyi Hu
ceb963e008 CHANGELOG: update from cherry pick of #9540 2019-06-04 16:47:19 -07:00
Jingyi Hu
17e10fe13f ctlv3: add missing newline in EndpointHealth
To make the output consistent with the output before #9540.
2019-06-04 15:52:29 -07:00
Gyuho Lee
5042c2751b
Merge pull request #10786 from rohitsardesai83/update_changelog3.3
CHANGELOG-3.3: add sigs.k8s.io yaml dependency
2019-06-04 09:52:53 -07:00
Rohit Sardesai
ee2b976254 CHANGELOG-3.3: add sigs.k8s.io yaml dependency 2019-06-04 14:13:57 +05:30
Gyuho Lee
35a67024f6
Merge pull request #10781 from gyuho/vv
module: require 1.12, remove "v3" import paths
2019-06-03 11:18:07 -07:00
Gyuho Lee
b40597ce46 module: require 1.12, remove "v3" import paths
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-06-03 11:15:19 -07:00
Xiang Li
ea0f919cdc
Merge pull request #10775 from jingyih/integration_member_promote_failed_cases
clientv3/integration: add member promote failure test cases
2019-06-02 20:39:00 -07:00