Mirroristas/etcd
2
0
Fork 0
mirror of https://github.com/etcd-io/etcd.git synced 2024-09-27 06:25:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
17,162 Commits 15 Branches 491 Tags 148 MiB
Commit Graph

114 Commits

Author SHA1 Message Date
Marek Siarkowicz
e2740b4afa server,etcdutl: Preserve etcd version in backend allowing etcdutl to read it from snapshot 2021-06-25 14:06:56 +02:00
Piotr Tabor
8f9829cd2d
Merge pull request #13114 from lilic/fix-api-version 
Bump etcd version to 3.5.0 and 3.6.0-pre
 2021-06-24 14:26:17 +02:00
Piotr Tabor
72cb652332
Merge pull request #13132 from serathius/refactor-monitor 
etcdserver: Move version monitor logic to separate module
 2021-06-24 10:40:01 +02:00
Marek Siarkowicz
823f85dfc9 etcdserver: Move version monitor logic to separate module 2021-06-23 19:27:29 +02:00
Lili Cosic
b9d837183a server/etcdserver/api: Add 3.6 to supported version 2021-06-22 12:25:39 +02:00
Marek Siarkowicz
e1b1d93548 *: Snapshot returns local etcd version 
Co-authored-by: Lili Cosic <cosiclili@gmail.com>
 2021-06-14 16:36:50 +02:00
Gyuho Lee
ed790d9639
Merge pull request #13086 from dlowe/crash-on-missing-permission 
etcdserver: don't attempt to grant nil permission to a role
 2021-06-04 15:13:20 -07:00
J. David Lowe
115c694af6 etcdserver: don't attempt to grant nil permission to a role 
Prevent etcd from crashing when given a bad grant payload, e.g.:

$ curl -d '{"name": "foo"}' http://localhost:2379/v3/auth/role/add
{"header":{"cluster_id":"14841639068965178418", ...
$ curl -d '{"name": "foo"}' http://localhost:2379/v3/auth/role/grant
curl: (52) Empty reply from server
 2021-06-04 14:20:02 -07:00
J. David Lowe
8552d8ec20 etcdserver: don't activate alarm w/missing AlarmType 
Narrowly prevent etcd from crashing when given a bad ACTIVATE payload, e.g.:

$ curl -d "{\"action\":\"ACTIVATE\"}" ${ETCD}/v3/maintenance/alarm
curl: (52) Empty reply from server
 2021-06-04 12:18:56 -07:00
Gyuho Lee
57034e1b51
Merge pull request #13075 from wilsonwang371/no-unnecessary-proto_size 
server: skip unnecessary sprintf which executes proto.Size()
 2021-06-03 13:10:11 -07:00
Wilson Wang
11edc76b15 server: skip unnecessary sprintf which executes proto.Size() 2021-06-03 12:58:42 -07:00
Piotr Tabor
f15e0b8237 integration.BeforeTest can be run without leak-detection. 2021-05-27 20:55:11 +02:00
Sahdev Zala
05674c87fb
Merge pull request #13018 from ipixiu/feature/bolt-compaction-sleep-interval 
add bolt compaction sleep interval
 2021-05-25 10:06:47 -04:00
AlexStocks
184b0e5d49 add sleep interval 2021-05-24 16:22:00 +08:00
Pavan BG
d9c5e1f0a2 *: Replace internal testutil AssertEqual function in favour of github.com/stretchr/testify/assert.Equals 
To maintain uniformity in testing methodology, assert.Equals has been used everywhere while replacing testutil.AssertEqual

Fixes #13015
 2021-05-20 19:55:45 +05:30
Piotr Tabor
788bc5359d
Merge pull request #13000 from ptabor/20210519-shouldApplyV3 
applyV2 should apply on backend only once
 2021-05-19 02:09:36 +02:00
Piotr Tabor
0a972a3f05 applyV2 should reapply on backend only once 
During review of:  https://github.com/etcd-io/etcd/pull/12988 spotted
that PUT is actially writing to v3-backend.
If we are replaying WAL log, it might happened that backend's
applied_index is > than the WAL's log entry. In such situation we should
skip applying on backend V3.
I think both the methods (setVersion, setMembersAttributes) are in
practice idempotent so its not that 'serious' problem, but for
formal correctness adding the proper checks.
 2021-05-18 20:55:05 +02:00
Piotr Tabor
66752fef2f Represent bucket as object instead of []byte name. 
Thanks to this change:
  - all the maps bucket -> buffer are indexed by int's instead of
string. No need to do: byte[] -> string -> hash conversion on each
access.
  - buckets are strongly typed in backend/mvcc API.
 2021-05-18 18:58:53 +02:00
Chao Chen
783c5ad2d2 use v2 api to update cluster version 2021-05-17 11:45:54 -07:00
Piotr Tabor
85341e08f2
Merge pull request #12968 from serathius/logger-simplify 
server: Simplify passing logger setup by passing only logger
 2021-05-15 15:58:00 +02:00
Marek Siarkowicz
41ed74824e server: Simplify passing logger setup by passing only logger 2021-05-14 13:14:48 +02:00
Piotr Tabor
ab586cd463 Persists Term in the (bbolt) Backend. 
Additional layer of protection, that allows to validate whether we
start replaying log not only from the proper 'index', but also of the
right 'term'.
 2021-05-13 21:29:01 +02:00
Piotr Tabor
e44fb40be5
Merge pull request #12962 from ptabor/20210513-write-conf-state 
Save raftpb.ConfState in the backend.
 2021-05-13 19:22:28 +02:00
Gyuho Lee
e2d67f2e3b
Merge pull request #12956 from gyuho/rename-to-main 
*: rename "master" branch references to "main" in source code
 2021-05-13 08:26:33 -07:00
Piotr Tabor
865df75714 Save raftpb.ConfState in the backend. 
This makes (bbolt) backend a full feature snapshot in term of WAL/raft,
i.e. carries:
  - commit : (applied_index)
  - confState

Benefits:
  - Backend will be a sufficient point in time definition sufficient to
start replaying WAL. We have applied_index & confState in consistent
state.
  - In case of emergency a backend state can be used for recovery
 2021-05-13 14:29:36 +02:00
Piotr Tabor
3cb1ba4b2b
Merge pull request #12954 from serathius/logger-new-ctx-client 
client: Add logger argument to NewCtxClient
 2021-05-13 09:03:38 +02:00
Gyuho Lee
77c8033739 server: rename "master" branch references 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2021-05-12 10:37:35 -07:00
Piotr Tabor
ead81df948 Disallow -v2-deprecation>'not-yet' combined with --enable-v2 2021-05-12 18:09:34 +02:00
Piotr Tabor
f3b4a3e578 Detecting whether v2store is "empty" (metadata only). 2021-05-12 18:09:34 +02:00
Marek Siarkowicz
1189ee3f3d client: Add logger argument to NewCtxClient 2021-05-12 16:40:55 +02:00
Marek Siarkowicz
efc8505739 etcdserver: Implement running defrag if freeable space will exceed privided threshold 2021-05-11 14:00:29 +02:00
Lili Cosic
1a718a958e Add initial Tracing with OpenTelemetry 2021-05-10 10:44:40 +02:00
Piotr Tabor
aeb9b5fc73
Merge pull request #12855 from ptabor/20210409-backend-hooks 
(no)StoreV2 (Part 4): Backend hooks:  precommit updates consistency_index
 2021-05-08 09:34:31 +02:00
Gyuho Lee
344c9f3930
Merge pull request #12896 from wilsonwang371/profiling-txn2 
server: make applier use ReadTx() in Txn() instead of ConcurrentReadTx()
 2021-05-06 01:59:14 -07:00
tangcong
b32bc914ff
learner support snapshot RPC (#12890) 
* learner support snapshot RPC

* CHANGELOG: update for 12890
 2021-05-04 13:26:33 -07:00
Wilson Wang
98083ea914 server: add experimental flag for using shared buffer in transacton write 2021-05-04 11:59:08 -07:00
Piotr Tabor
2fb6f0a74b Simplify lease management after cindex update is moved to 'hooks'. 2021-05-04 18:21:23 +02:00
Piotr Tabor
2dbecea5b2 Simplify KVStore interaction with cindex thanks to hooks. 2021-05-04 18:21:23 +02:00
Piotr Tabor
fe3254aee3 Remove explicit authStore->ConsistencyIndex updates, as they are taken care by hook. 2021-05-04 15:38:23 +02:00
Piotr Tabor
50051675f9 Integrate backend::hooks with consistent_index. 
Every transaction committed to backend is writing most recent consistent_index.
Makes sure that even automatically trigger commits of batch-transactions
stays "really" consistent a.d. the most recent WAL log index applied.
 2021-05-04 15:38:23 +02:00
Piotr Tabor
a3fc535a5a
Merge pull request #12914 from ptabor/20210430-read-membership 
No-storeV2: Read membership information from the backend (Part5)
 2021-05-04 10:15:01 +02:00
Piotr Tabor
451f65d661
Merge pull request #12908 from ptabor/20210429-client-retry-logging 
Clientv3 (retry interceptor) logs should use the configured logger
 2021-04-29 19:25:04 +02:00
Piotr Tabor
cedbea6c81
Merge pull request #12904 from wpedrak/limit_mlocked_memory 
server: replace mlockall with `Mlock` in `--experimental-memory-mlock`
 2021-04-29 18:21:24 +02:00
Piotr Tabor
ffea1537d4 ClientV3 tests use integration.NewClient that configures proper logger. 2021-04-29 18:18:34 +02:00
Piotr Tabor
205a1a442a Read membership information from the backend. 2021-04-29 13:45:45 +02:00
wpedrak
1145c57601 server: replace mlockall with Mlock in --experimental-memory-mlock 
Implementation of `--experimental-memory-mlock` backed by `mlockall` syscall is replaced by `Mlock` flag (backed by mlock syscall) of bboltDB.
 2021-04-29 12:08:52 +02:00
wpedrak
927b3a3152 server: replace mlockall with Mlock in --experimental-memory-mlock 
Implementation of `--experimental-memory-mlock` backed by `mlockall` syscall is replaced by `Mlock` flag (backed by mlock syscall) of bboltDB.
 2021-04-29 12:08:20 +02:00
Piotr Tabor
e90504fe62 Unify shared code (and constants) with cindex package. 2021-04-29 11:51:25 +02:00
Piotr Tabor
f53b70facb Embed: In case KVStoreHash verification fails, close the backend. 
In case of failed verification, the server used to keep opened backend
(so the file was locked on OS level).
 2021-04-29 11:51:25 +02:00
Piotr Tabor
911204cd76 Fix ETCDCTL_API=2 etcdctl backup --with-v3 consistent index consistency 
Prior to this CL, `ETCDCTL_API=2 etcdctl backup --with-v3` was readacting WAL log
(by removal of some entries), but was NOT updating consistent_index in the backend.
Also the WAL editing logic was buggy, as it didn't took in consideration the fact
that when TERM changes, there can be entries with duplicated indexes in
the log. So its NOT sufficient to subtract number of removed entries to
get accurate log indexes.

The PR replaces removing and shifting of WAL entries with replacing them with an no-op entries.
Thanks to this consistent-index references are staying up to date.

The PR also:
  - updates 'verification' logic to check whether consistent_index does not lag befor last snapshot
  - env-gated execution of verification framework in `etcdctl backup`.

Tested with:
```
(./build.sh && cd tests && EXPECT_DEBUG=TRUE 'env' 'go' 'test' '-timeout=300m' 'go.etcd.io/etcd/tests/v3/e2e' -run=TestCtlV2Backup --count=1000 2>&1 | tee TestCtlV2BackupV3.log)
```
 2021-04-29 11:51:24 +02:00