305 Commits

Author SHA1 Message Date
Marek Siarkowicz
2b090e86a6 server: Extract hasher to separate interface
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:20:18 +02:00
Marek Siarkowicz
80828b593a server: Remove duplicated compaction revision
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:20:18 +02:00
Marek Siarkowicz
34a02ba621 server: Return revision range that hash was calcualted for
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:20:17 +02:00
Marek Siarkowicz
01e85be159 server: Cover corruptionMonitor with tests
Get 100% coverage on InitialCheck and PeriodicCheck functions to avoid
any mistakes.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:25 +02:00
Marek Siarkowicz
7c35dadc25 server: Extract corruption detection to dedicated struct
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:24 +02:00
Marek Siarkowicz
70fbc0b3e7 server: Extract triggerCorruptAlarm to function
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:04:24 +02:00
Marek Siarkowicz
72a84ddbcf server: Fix broken code during merge 2022-06-09 13:34:33 +02:00
Sahdev Zala
d9e5460ca8
Merge pull request #13829 from qsyqian/feature/skip_compact_when_rev_not_change
skip compact when rev not change at period compact mode
2022-06-08 22:54:39 -04:00
ahrtr
25deb436af fix the race condition between goroutine and channel on the same leases to be revoked 2022-05-25 16:44:41 +08:00
Piotr Tabor
41ff2370e9 Remove unused code and apply code-quality suggestions. 2022-05-20 14:42:39 +02:00
Piotr Tabor
85b18c9b3e Rename WrapApply to Apply. 2022-05-20 14:32:04 +02:00
Piotr Tabor
87b80f16ca Fixing missing comment on the dispatch() function. 2022-05-20 14:32:04 +02:00
Piotr Tabor
42c6e08f22 Rename the txn, so as not to be the same as the package name. 2022-05-20 14:32:04 +02:00
Piotr Tabor
0da0cf4795 expose UberApplier as interface (not as implementation struct). 2022-05-20 14:32:04 +02:00
Piotr Tabor
5097b33ab9 Rename etcdserver/etcderrors package to etcdserver/errors. 2022-05-20 14:32:04 +02:00
Piotr Tabor
63b2f63cc1 Rename package alising "apply2" -> apply. 2022-05-20 14:32:04 +02:00
Piotr Tabor
c62f01e5fe Move CheckTxnAuth to txn. 2022-05-20 14:32:04 +02:00
Piotr Tabor
c78bf655a7 Simplify imports and improve comments. 2022-05-20 14:32:04 +02:00
Piotr Tabor
4e04770bac Apply encapsulation: Cleanup metrics reporting.
Side effect: applySec(0.4s) used to be reported as 0s, now it's correctly 0.4s.
2022-05-20 14:32:04 +02:00
Piotr Tabor
47a771871b Move apply to its own package (no dependency on etcdserver). 2022-05-20 14:32:04 +02:00
Piotr Tabor
fc6a6c3c27 Move etcdserver/errors.go to sepatate package to avoid cyclic dependencies. 2022-05-20 14:32:04 +02:00
Piotr Tabor
e2ae9b1d13 Move server/etcdserver/txn.go to new package: server/etcdserver/txn 2022-05-20 14:32:04 +02:00
Piotr Tabor
b073129d03 Applier does not depend on EtcdServer any longer.
All the depencies are explicily passed to the UberApplier factory method.
2022-05-20 14:32:04 +02:00
Piotr Tabor
651de5a057 Rename EtcdServer.Id with EtcdServer.MemberId.
It was misleading and error prone vs. ClusterId.
2022-05-20 14:32:04 +02:00
Piotr Tabor
f348134edd Marge applierV3Internal into applierV3 interface 2022-05-20 14:32:04 +02:00
Piotr Tabor
b7ad746bfe Encapsulating applier logic: UberApplier coordinates all appliers for server
This PR:
 - moves wrapping of appliers (due to Alarms) out of server.go into uber_applier.go
 - clearly devides the application logic into: chain of:
     a) 'WrapApply' (generic logic across all the methods)
     b) dispatcher (translation of Apply into specific method like 'Put')
     c) chain of 'wrappers' of the specific methods (like Put).
 - when we do recovery (restore from snapshot) we create new instance of appliers.

The purpose is to make sure we control all the depencies of the apply process, i.e.
we can supply e.g. special instance of 'backend' to the application logic.
2022-05-20 14:32:04 +02:00
Piotr Tabor
cdf9869d70 Encapsulation of applier logic: Move Txn related code out of applier.go.
The PR removes calls to applierV3base logic from server.go that is NOT part of 'application'.
The original idea was that read-only transaction and Range call shared logic with Apply,
so they can call appliers directly (but bypassing all 'corrupt', 'quota' and 'auth' wrappers).

This PR moves all the logic to a separate file (that later can become package on its own).
2022-05-20 14:32:04 +02:00
ahrtr
b32ac1f67b fix a typo: print the correct error info 2022-05-19 07:19:44 +08:00
ahrtr
e7f8bf7c44 enhance the /version endpoint to add storageVersion 2022-05-06 20:29:42 +08:00
Marek Siarkowicz
d58f971895 tests: Use zaptest.NewLogger in places we missed 2022-05-06 12:23:20 +02:00
Marek Siarkowicz
f09da32f9d
Merge pull request #13655 from serathius/health
Cleanup healthcheck code after V2 removal
2022-05-06 12:08:36 +02:00
Marek Siarkowicz
26f42e7a9e server: Apply review comments and split basic handler 2022-05-05 09:52:14 +02:00
Marek Siarkowicz
fb361e43f0 server: Split code for debug and version endpoints 2022-05-05 09:52:14 +02:00
Marek Siarkowicz
722ec487df server: Split metrics and health code 2022-05-05 09:52:14 +02:00
Marek Siarkowicz
600ee13ac0 server: Cover V3 health with tests 2022-05-05 09:52:14 +02:00
Marek Siarkowicz
e9dec74ded server: Refactor health checks 2022-05-05 09:52:14 +02:00
Marek Siarkowicz
191aed645e server: Run health check tests in subtests 2022-05-05 09:52:14 +02:00
Marek Siarkowicz
e4e391792a server: Rename test case expect fields 2022-05-05 09:52:13 +02:00
Marek Siarkowicz
0fb194d6f2 server: Use named struct initialization in healthcheck test 2022-05-05 09:52:13 +02:00
Marek Siarkowicz
0096d2ecdb server: Remove unused NewClientHandler 2022-05-05 09:52:13 +02:00
ahrtr
fb2eeb9027 verify consistent_index in snapshot must be equal to the snapshot index
Usually the consistent_index should be greater than the index of the
latest snapshot with suffix .snap. But for the snapshot coming from the
leader, the consistent_index should be equal to the snapshot index.
2022-05-03 20:02:47 +08:00
Piotr Tabor
887f95d0d3
Merge pull request #13963 from ptabor/20220412-verify-assert
Add verification consistent index is (nearly) never decreasing
2022-04-25 10:13:44 +02:00
Piotr Tabor
d69e07dd3a Verification framework and check whether cindex is not decreasing. 2022-04-22 12:32:05 +02:00
ahrtr
6eef7ede40 Update conssitent_index when applying fails
When clients have no permission to perform whatever operation, then
the applying may fail. We should also move consistent_index forward
in this case, otherwise the consitent_index may smaller than the
snapshot index.
2022-04-20 21:44:48 +08:00
ahrtr
484d2f01f3 set backend to cindex before recovering the lessor in applySnapshot 2022-04-12 10:36:29 +08:00
ahrtr
1b3d6cb0c8 set an separate applyTimeout for the waitAppliedIndex 2022-04-10 14:44:55 +08:00
ahrtr
fe3a57976e support linearizable renew lease
When etcdserver receives a LeaseRenew request, it may be still in
progress of processing the LeaseGrantRequest on exact the same
leaseID. Accordingly it may return a TTL=0 to client due to the
leaseID not found error. So the leader should wait for the appliedID
to be available before processing client requests.
2022-04-10 14:44:55 +08:00
Donal Hunt
d659403955
Update server/etcdserver/api/v3rpc/maintenance.go
Co-authored-by: Marek Siarkowicz <marek.siarkowicz@protonmail.com>
2022-04-08 18:45:13 +01:00
Donal Hunt
4c8ef011e0 *: drop use of humanize.Time() in favour of time.Duration.String()
humanize.Time() drops precision resulting in some events reporting they took
"now" time to complete. Using time.Duration.String() results in accurate
duration being reported.

Fixes #13905
2022-04-07 23:24:35 +01:00
Marek Siarkowicz
1ea53d527e server: Save consistency index and term to backend even when they decrease
Reason to store CI and term in backend was to make db fully independent
snapshot, it was never meant to interfere with apply logic. Skip of CI
was introduced for v2->v3 migration where we wanted to prevent it from
decreasing when replaying wal in
https://github.com/etcd-io/etcd/pull/5391. By mistake it was added to
apply flow during refactor in
https://github.com/etcd-io/etcd/pull/12855#commitcomment-70713670.

Consistency index and term should only be negotiated and used by raft to make
decisions. Their values should only driven by raft state machine and
backend should only be responsible for storing them.
2022-04-07 19:00:03 +02:00