Mirroristas/etcd
2
0
Fork 0
mirror of https://github.com/etcd-io/etcd.git synced 2024-09-27 06:25:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
17,246 Commits 15 Branches 491 Tags 148 MiB
Commit Graph

62 Commits

Author SHA1 Message Date
Kafuu Chino
dd983c662b *: avoid closing a watch with ID 0 incorrectly 
Signed-off-by: Kafuu Chino <KafuuChinoQ@gmail.com>

add test

1

1

1

1

1

1
 2022-10-08 20:06:19 +08:00
Benjamin Wang
6c26693ebe
Merge pull request #14178 from lavacat/release-3.5-txn-panic 
[3.5] server: don't panic in readonly serializable txn
 2022-09-13 14:44:38 +08:00
Marek Siarkowicz
21fb173f76 server: Implement compaction hash checking 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-09-07 15:11:56 +02:00
Marek Siarkowicz
037a898ba0 tests: Unify TestCompactionHash and extend it to also Delete keys and Defrag 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-09-07 15:11:55 +02:00
Marek Siarkowicz
1200b1006d server: Cache compaction hash for HashByRev API 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-09-07 15:11:55 +02:00
Marek Siarkowicz
7358362c99 server: Extract hasher to separate interface 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-09-07 15:11:55 +02:00
Marek Siarkowicz
631107285a server: Remove duplicated compaction revision 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-09-07 15:11:55 +02:00
Marek Siarkowicz
a3f609d742 server: Return revision range that hash was calcualted for 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-09-07 15:11:55 +02:00
Marek Siarkowicz
1ff59923d6 server: Store real rv range in hasher 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-09-07 15:11:55 +02:00
Marek Siarkowicz
991b429336 server: Move adjusting revision to hasher 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-09-07 15:11:55 +02:00
Marek Siarkowicz
2b8dd0de4e server: Pass revision as int 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-09-07 15:11:55 +02:00
Marek Siarkowicz
21e5d5d2b6 server: Calculate hash during compaction 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-09-07 15:11:55 +02:00
Marek Siarkowicz
f1a759a2c8 server: Fix range in mock not returning same number of keys and values 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-09-07 15:11:55 +02:00
Marek Siarkowicz
ea684db535 server: Move reading KV index inside scheduleCompaction function 
Makes it easier to test hash match between scheduleCompaction and
HashByRev.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-09-07 15:11:55 +02:00
Marek Siarkowicz
22d3e4ebd7 server: Return error from scheduleCompaction 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-09-07 15:11:55 +02:00
Marek Siarkowicz
679e327d5e server: Refactor hasher 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-09-07 15:11:55 +02:00
Marek Siarkowicz
f5ed371885 server: Extract kvHash struct 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-09-07 15:11:55 +02:00
Marek Siarkowicz
3f26995f99 server: Move unsafeHashByRev to new hash.go file 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-09-07 15:11:55 +02:00
Marek Siarkowicz
bc592c7b01 server: Extract unsafeHashByRev function 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-09-07 15:11:55 +02:00
Marek Siarkowicz
336fef4ce2 server: Test HashByRev values to make sure they don't change 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-09-07 15:11:55 +02:00
Marek Siarkowicz
35cbdf3961 server: Extract corruption detection to dedicated struct 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-09-07 15:11:55 +02:00
Bogdan Kanivets
204d883904 [backport 3.5] server: don't panic in readonly serializable txn 
Problem: We pass grpc context down to applier in readonly serializable txn.
This context can be cancelled for example due to timeout.
This will trigger panic inside applyTxn

Solution: Only panic for transactions with write operations

fixes https://github.com/etcd-io/etcd/issues/14110
main PR https://github.com/etcd-io/etcd/pull/14149

Signed-off-by: Bogdan Kanivets <bkanivets@apple.com>
 2022-09-01 01:01:50 -07:00
ahrtr
66c7aab4d3 fix the data inconsistency issue by adding a txPostLockHook into the backend 
Previously the SetConsistentIndex() is called during the apply workflow,
but it's outside the db transaction. If a commit happens between SetConsistentIndex
and the following apply workflow, and etcd crashes for whatever reason right
after the commit, then etcd commits an incomplete transaction to db.
Eventually etcd runs into the data inconsistency issue.

In this commit, we move the SetConsistentIndex into a txPostLockHook, so
it will be executed inside the transaction lock.
 2022-04-08 20:37:34 +08:00
Marek Siarkowicz
83538f342d server: Add verification of whether lock was called within out outside of apply 2022-04-06 11:22:51 +02:00
Bogdan Kanivets
631fa6fd65 server/storage/backend: restore original bolt db options after defrag 
Problem: Defrag was implemented before custom bolt options were added.
Currently defrag doesn't restore backend options.
For example BackendFreelistType will be unset after defrag.

Solution: save bolt db options and use them in defrag.
 2022-02-15 10:56:07 -08:00
leoyang.yl
55c16df997 fix runlock bug 2021-12-16 15:58:41 +00:00
Geeta Gharpure
446f7d6b6e storage/backend: Add a gauge to indicate if defrag is active (backport from 3.6) 2021-10-06 11:01:31 -07:00
Marek Siarkowicz
9a18742bd7 etcdserver: Fix invalid count returned on Range with Limit 2021-06-01 16:12:53 +02:00
Piotr Tabor
e6baf6d751 Represent bucket as object instead of []byte name. 
Thanks to this change:
  - all the maps bucket -> buffer are indexed by int's instead of
string. No need to do: byte[] -> string -> hash conversion on each
access.
  - buckets are strongly typed in backend/mvcc API.
 2021-05-25 09:22:25 +02:00
Piotr Tabor
8bddbdc1d6 Rename seq to bucket2seq. 2021-05-25 09:21:07 +02:00
Piotr Tabor
d3b3228c1f Minor formatting fix on top of https://github.com/etcd-io/etcd/pull/12568 2021-05-25 09:21:07 +02:00
mlmhl
261f8b1daf backend: set seq flag for each bucket buffer 2021-05-25 09:21:06 +02:00
Wilson Wang
7a7d6f94a7 server: set multiple concurrentReadTx instances share one txReadBuffer. 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2021-05-24 16:59:42 -07:00
Piotr Tabor
d99d0df5a5 Adding etcdutl test coverage. 2021-05-17 11:54:03 +02:00
Piotr Tabor
ab586cd463 Persists Term in the (bbolt) Backend. 
Additional layer of protection, that allows to validate whether we
start replaying log not only from the proper 'index', but also of the
right 'term'.
 2021-05-13 21:29:01 +02:00
Piotr Tabor
865df75714 Save raftpb.ConfState in the backend. 
This makes (bbolt) backend a full feature snapshot in term of WAL/raft,
i.e. carries:
  - commit : (applied_index)
  - confState

Benefits:
  - Backend will be a sufficient point in time definition sufficient to
start replaying WAL. We have applied_index & confState in consistent
state.
  - In case of emergency a backend state can be used for recovery
 2021-05-13 14:29:36 +02:00
Piotr Tabor
aeb9b5fc73
Merge pull request #12855 from ptabor/20210409-backend-hooks 
(no)StoreV2 (Part 4): Backend hooks:  precommit updates consistency_index
 2021-05-08 09:34:31 +02:00
Gyuho Lee
344c9f3930
Merge pull request #12896 from wilsonwang371/profiling-txn2 
server: make applier use ReadTx() in Txn() instead of ConcurrentReadTx()
 2021-05-06 01:59:14 -07:00
Piotr Tabor
a78d072b9a Simplify KVstore dependency on cindex. 2021-05-04 18:21:24 +02:00
Piotr Tabor
2dbecea5b2 Simplify KVStore interaction with cindex thanks to hooks. 2021-05-04 18:21:23 +02:00
Piotr Tabor
d53d2db1e2 Tests: Backend hooks support. 2021-05-04 15:38:22 +02:00
Piotr Tabor
9f11b16b2d backend: Hooks interface & implementation. 2021-05-04 15:38:22 +02:00
Wilson Wang
56154216b7 update variable declaration location 2021-05-03 10:30:15 -07:00
wpedrak
6623c008ee server: reapply Mlock flag after defrag 2021-05-03 11:01:02 +02:00
Piotr Tabor
cedbea6c81
Merge pull request #12904 from wpedrak/limit_mlocked_memory 
server: replace mlockall with `Mlock` in `--experimental-memory-mlock`
 2021-04-29 18:21:24 +02:00
wpedrak
927b3a3152 server: replace mlockall with Mlock in --experimental-memory-mlock 
Implementation of `--experimental-memory-mlock` backed by `mlockall` syscall is replaced by `Mlock` flag (backed by mlock syscall) of bboltDB.
 2021-04-29 12:08:20 +02:00
Piotr Tabor
e90504fe62 Unify shared code (and constants) with cindex package. 2021-04-29 11:51:25 +02:00
Wilson Wang
8d8d0377a2 server: applier uses ReadTx instead of ConcurrentTx 2021-04-28 11:06:24 -07:00
Piotr Tabor
7ae3d25f91 Membership: Add additional methods to trim/manage membership data in backend. 2021-04-27 17:17:31 +02:00
Makdon
ddcb463822
etcdserver/mvcc: update trace.Step condition 2021-04-25 23:07:45 +08:00