caojiamingalan
04cfb4c660
etcdserver: add cluster id check for hashKVHandler
...
Signed-off-by: caojiamingalan <alan.c.19971111@gmail.com >
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com >
2023-10-17 13:27:47 +02:00
James Blair
f62a894ae7
Fix goword failure in rafthttp/transport.go.
...
Signed-off-by: James Blair <mail@jamesblair.net >
2023-08-11 21:47:30 +12:00
Benjamin Wang
cd019255ba
etcdserver: Guarantee order of requested progress notifications
...
Progress notifications requested using ProgressRequest were sent
directly using the ctrlStream, which means that they could race
against watch responses in the watchStream.
This would especially happen when the stream was not synced - e.g. if
you requested a progress notification on a freshly created unsynced
watcher, the notification would typically arrive indicating a revision
for which not all watch responses had been sent.
This changes the behaviour so that v3rpc always goes through the watch
stream, using a new RequestProgressAll function that closely matches
the behaviour of the v3rpc code - i.e.
1. Generate a message with WatchId -1, indicating the revision for
*all* watchers in the stream
2. Guarantee that a response is (eventually) sent
The latter might require us to defer the response until all watchers
are synced, which is likely as it should be. Note that we do *not*
guarantee that the number of progress notifications matches the number
of requests, only that eventually at least one gets sent.
Signed-off-by: Benjamin Wang <wachao@vmware.com >
2023-04-11 09:51:48 +08:00
James Blair
1ea808b5ba
Backport go_srcs_in_module changes and fix goword failures.
...
Signed-off-by: James Blair <mail@jamesblair.net >
2023-02-24 22:01:41 +13:00
James Blair
183af509f6
Formatted source code for go 1.19.6.
...
Signed-off-by: James Blair <mail@jamesblair.net >
2023-02-20 21:33:59 +13:00
Kafuu Chino
dd983c662b
*: avoid closing a watch with ID 0 incorrectly
...
Signed-off-by: Kafuu Chino <KafuuChinoQ@gmail.com >
add test
1
1
1
1
1
1
2022-10-08 20:06:19 +08:00
Hitoshi Mitake
7b568f23ab
*: handle auth invalid token and old revision errors in watch
...
Signed-off-by: Hitoshi Mitake <h.mitake@gmail.com >
2022-10-03 23:00:13 +09:00
Marek Siarkowicz
7358362c99
server: Extract hasher to separate interface
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com >
2022-09-07 15:11:55 +02:00
Marek Siarkowicz
631107285a
server: Remove duplicated compaction revision
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com >
2022-09-07 15:11:55 +02:00
Marek Siarkowicz
a3f609d742
server: Return revision range that hash was calcualted for
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com >
2022-09-07 15:11:55 +02:00
Benjamin Wang
437f3778d0
Add flag --max-concurrent-streams to set the max concurrent stream each client can open at a time
...
Also refer to https://github.com/etcd-io/etcd/pull/14169#discussion_r917154243
Signed-off-by: Benjamin Wang <wachao@vmware.com >
2022-07-13 14:18:15 +08:00
ahrtr
bc5307de95
support linearizable renew lease
...
When etcdserver receives a LeaseRenew request, it may be still in
progress of processing the LeaseGrantRequest on exact the same
leaseID. Accordingly it may return a TTL=0 to client due to the
leaseID not found error. So the leader should wait for the appliedID
to be available before processing client requests.
2022-04-12 14:12:45 +08:00
ahrtr
66c7aab4d3
fix the data inconsistency issue by adding a txPostLockHook into the backend
...
Previously the SetConsistentIndex() is called during the apply workflow,
but it's outside the db transaction. If a commit happens between SetConsistentIndex
and the following apply workflow, and etcd crashes for whatever reason right
after the commit, then etcd commits an incomplete transaction to db.
Eventually etcd runs into the data inconsistency issue.
In this commit, we move the SetConsistentIndex into a txPostLockHook, so
it will be executed inside the transaction lock.
2022-04-08 20:37:34 +08:00
ahrtr
7db1051774
enhance health check endpoint to support serializable request
2022-02-17 15:03:22 +08:00
Hitoshi Mitake
dec6f72d68
*: implement a retry logic for auth old revision in the client
2021-11-15 00:09:16 +09:00
Chao Chen
7d44a7cd6e
server/etcdserver/api/etcdhttp: exclude the same alarm type activated by multiple peers
2021-11-12 14:21:14 -08:00
Marek Siarkowicz
58d2b12a50
client: Add grpc authority header integration tests
2021-09-30 12:15:32 +02:00
Marek Siarkowicz
e68c7ab4bc
server: Ensure that adding and removing members handle storev2 and backend out of sync
2021-09-15 14:36:41 +02:00
tangcong
dfd2fea4c5
fix health endpoint not usable when authentication is enabled
2021-07-30 07:53:40 +08:00
J. David Lowe
e27effa250
etcdserver: don't attempt to grant nil permission to a role
...
Prevent etcd from crashing when given a bad grant payload, e.g.:
$ curl -d '{"name": "foo"}' http://localhost:2379/v3/auth/role/add
{"header":{"cluster_id":"14841639068965178418", ...
$ curl -d '{"name": "foo"}' http://localhost:2379/v3/auth/role/grant
curl: (52) Empty reply from server
Signed-off-by: Gyuho Lee <leegyuho@amazon.com >
2021-06-04 15:34:22 -07:00
Piotr Tabor
3f13d3a2d5
integration.BeforeTest can be run without leak-detection.
2021-05-28 10:01:36 +02:00
Piotr Tabor
e6baf6d751
Represent bucket as object instead of []byte name.
...
Thanks to this change:
- all the maps bucket -> buffer are indexed by int's instead of
string. No need to do: byte[] -> string -> hash conversion on each
access.
- buckets are strongly typed in backend/mvcc API.
2021-05-25 09:22:25 +02:00
Piotr Tabor
e44fb40be5
Merge pull request #12962 from ptabor/20210513-write-conf-state
...
Save raftpb.ConfState in the backend.
2021-05-13 19:22:28 +02:00
Gyuho Lee
e2d67f2e3b
Merge pull request #12956 from gyuho/rename-to-main
...
*: rename "master" branch references to "main" in source code
2021-05-13 08:26:33 -07:00
Piotr Tabor
865df75714
Save raftpb.ConfState in the backend.
...
This makes (bbolt) backend a full feature snapshot in term of WAL/raft,
i.e. carries:
- commit : (applied_index)
- confState
Benefits:
- Backend will be a sufficient point in time definition sufficient to
start replaying WAL. We have applied_index & confState in consistent
state.
- In case of emergency a backend state can be used for recovery
2021-05-13 14:29:36 +02:00
Piotr Tabor
3cb1ba4b2b
Merge pull request #12954 from serathius/logger-new-ctx-client
...
client: Add logger argument to NewCtxClient
2021-05-13 09:03:38 +02:00
Gyuho Lee
77c8033739
server: rename "master" branch references
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com >
2021-05-12 10:37:35 -07:00
Piotr Tabor
f3b4a3e578
Detecting whether v2store is "empty" (metadata only).
2021-05-12 18:09:34 +02:00
Marek Siarkowicz
1189ee3f3d
client: Add logger argument to NewCtxClient
2021-05-12 16:40:55 +02:00
Lili Cosic
1a718a958e
Add initial Tracing with OpenTelemetry
2021-05-10 10:44:40 +02:00
Piotr Tabor
aeb9b5fc73
Merge pull request #12855 from ptabor/20210409-backend-hooks
...
(no)StoreV2 (Part 4): Backend hooks: precommit updates consistency_index
2021-05-08 09:34:31 +02:00
tangcong
b32bc914ff
learner support snapshot RPC ( #12890 )
...
* learner support snapshot RPC
* CHANGELOG: update for 12890
2021-05-04 13:26:33 -07:00
Piotr Tabor
2dbecea5b2
Simplify KVStore interaction with cindex thanks to hooks.
2021-05-04 18:21:23 +02:00
Piotr Tabor
a3fc535a5a
Merge pull request #12914 from ptabor/20210430-read-membership
...
No-storeV2: Read membership information from the backend (Part5)
2021-05-04 10:15:01 +02:00
Piotr Tabor
ffea1537d4
ClientV3 tests use integration.NewClient that configures proper logger.
2021-04-29 18:18:34 +02:00
Piotr Tabor
205a1a442a
Read membership information from the backend.
2021-04-29 13:45:45 +02:00
Piotr Tabor
067521981e
v2 etcdctl backup: producing consistent state of membership
2021-04-27 19:34:34 +02:00
Piotr Tabor
a70386a1a4
Simplify membership interface: Does not pass the 'unused' token.
2021-04-27 17:17:31 +02:00
Piotr Tabor
7ae3d25f91
Membership: Add additional methods to trim/manage membership data in backend.
2021-04-27 17:17:31 +02:00
Piotr Tabor
768da490ed
sever: v2store deprecation: Fix etcdctl snapshot restore to restore
...
correct 'backend' (bbolt) context in aspect of membership.
Prior to this change the 'restored' backend used to still contain:
- old memberid (mvcc deletion used, why the membership is in bolt
bucket, but not mvcc part):
```
mvs := mvcc.NewStore(s.lg, be, lessor, ci, mvcc.StoreConfig{CompactionBatchLimit: math.MaxInt32})
defer mvs.Close()
txn := mvs.Write(traceutil.TODO())
btx := be.BatchTx()
del := func(k, v []byte) error {
txn.DeleteRange(k, nil)
return nil
}
// delete stored members from old cluster since using new members
btx.UnsafeForEach([]byte("members"), del)
```
- didn't get new members added.
2021-04-27 17:17:30 +02:00
Piotr Tabor
9a4b2bdccc
Errors: context cancelled or context deadline exceeded are exposed as codes.Canceled, codes.DeadlineExceeded instead of 'codes.Unknown'
2021-04-22 14:35:24 +02:00
Piotr Tabor
ea287dd9f8
Merge pull request #12854 from ptabor/20210410-shouldApplyV3
...
(no)StoreV2 (Part 3): Applying consistency fix: ClusterVersionSet (and co) might get not applied on v2store
2021-04-21 09:31:38 +02:00
Sam Batschelet
0f2c940f64
Merge pull request #12880 from chaochn47/exclude_alarms_from_health_check
...
etcdhttp/metrics.go: exclude alarms from health check conditionally with `?exclude=NOSPACE`
2021-04-20 21:18:15 -04:00
Chao Chen
140ea4fa29
etcdhttp/metrics.go: exclude alarms from health check conditionally with ?exclude=NOSPACE
2021-04-20 13:17:09 -07:00
Piotr Tabor
17b982382e
Fix TestSnapshotV3RestoreMultiMemberAdd flakes (leaks)
...
- most important: unix's socket transport should not keep idle
connections. For top-level Transport we close them using:
f3c518025e/server/etcdserver/api/rafthttp/transport.go (L226)
2021-04-16 20:17:28 +02:00
Piotr Tabor
d69e46ea47
Make ShouldApplyV3 an enum - not bool
2021-04-13 23:01:03 +02:00
Piotr Tabor
b1c04ce043
Applying consistency fix: ClusterVersionSet (and co) might get no applied on v2store
...
ClusterVersionSet, ClusterMemberAttrSet, DowngradeInfoSet functions are
writing both to V2store and backend. Prior this CL there were
in a branch not executed if shouldApplyV3 was false,
e.g. during restore when Backend is up-to-date (has high
consistency-index) while v2store requires replay from WAL log.
The most serious consequence of this bug was that v2store after restore
could have different index (revision) than the same exact store before restore,
so potentially different content between replicas.
Also this change is supressing double-applying of Membership
(ClusterConfig) changes on Backend (store v3) - that lackilly are not
part of MVCC/KeyValue store, so they didn't caused Revisions to be
bumped.
Inspired by jingyih@ comment:
https://github.com/etcd-io/etcd/pull/12820#issuecomment-815299406
2021-04-12 09:43:48 +02:00
Piotr Tabor
3bb7acc8cf
Migrate dependencies pkg/foo -> client/pkg/foo
2021-04-07 00:38:47 +02:00
Piotr Tabor
55ccbe62a2
membership/cluster_test: Use zaptest logger.
2021-03-26 13:54:59 +01:00
Piotr Tabor
7d7a9c6f23
codec.go: should use google runtime
...
golang-proto runtime can deal with both: gogo & golang generated
protobufs. It does not work vice-versa with protobuf-1.5.1.
2021-03-24 22:06:47 +01:00
