162 Commits

Author SHA1 Message Date
AdamKorcz
0fcb2a628c server/storage/wal: fix oss-fuzz issue 44433 2022-02-14 11:50:27 +00:00
Chao Chen
fd79af9ee7 load all leases from backend 2022-02-13 20:30:20 -08:00
Bogdan Kanivets
01347a8f53 server/storage/backend: restore original bolt db options after defrag
Problem: Defrag was implemented before custom bolt options were added.
Currently defrag doesn't restore backend options.
For example BackendFreelistType will be unset after defrag.

Solution: save bolt db options and use them in defrag.
2022-02-11 11:01:27 -08:00
AdamKorcz
029c354316 server/storage/mvcc: fix oss-fuzz issue 44449 2022-02-11 10:57:02 +00:00
Marek Siarkowicz
f4187b4460 tools: Improve proto annotation documentation 2022-01-28 11:40:55 +01:00
Marek Siarkowicz
4bfcd84259 server: Handle V2Request when analysis WAL entries 2022-01-26 15:50:14 +01:00
Marek Siarkowicz
d865bb96f1 server: Refactor wal version to use visitor pattern 2022-01-26 15:50:14 +01:00
Marek Siarkowicz
6d808e5d7d *: Add static validation to etcd_version proto annotation 2022-01-26 15:50:14 +01:00
Eng Zer Jun
e9edc5432c
*: use T.TempDir to create temporary test directory
The directory created by `T.TempDir()` and is automatically removed when
the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-01-25 00:00:46 +08:00
Piotr Tabor
17568a19f4
Merge pull request #13568 from horizonzy/avoid-always-put-metaterm
Avoid always update meta term.
2022-01-22 16:02:50 +01:00
horizonzy
282e4e6e88 make code more readable. 2022-01-19 22:24:49 +08:00
Piotr Tabor
14a661e643
Merge pull request #13601 from lixd/op-put-lease
mvcc: add a fast return at put method
2022-01-19 14:47:26 +01:00
horizonzy
5b09de33a5 code clean: make rangeKeys close to storeTxnRead. 2022-01-17 11:12:25 +08:00
lixd
1d706179be mvcc: add test-case
a test-case for new code-path.
2022-01-16 10:43:15 +08:00
Piotr Tabor
0285f74aea
Merge pull request #13558 from gfuzz-asplos/main
fixing goroutine leaks
2022-01-15 16:58:19 +01:00
lixd
401cc1a575 mvcc: add a fast return at put method
return when two leaseID is equal, avoid invalid operations.

Fixes #13600
2022-01-15 13:16:44 +08:00
Piotr Tabor
68fa5dcf99
Merge pull request #13549 from songlh-psu/main
fixing the goroutine leaks in TestHashKVWhenCompacting
2022-01-14 13:58:16 +01:00
songlh
a9652b4b4e fixing the leaks in TestStressWatchCancelClose 2022-01-04 17:57:19 -05:00
Liang Zheng
0cc789d81d update dump db tool
Signed-off-by: Liang Zheng <zhengliang0901@gmail.com>
2022-01-01 00:13:33 +08:00
horizonzy
97d1f935f6 avoid always update meta term. 2021-12-29 11:23:38 +08:00
Linhai
98b0d901e8 fixing goroutine leaks 2021-12-24 15:57:38 -05:00
Linhai
a45c73d9b1 resolve the conflict 2021-12-21 17:49:47 -05:00
Piotr Tabor
69279532f4
Merge pull request #13540 from songlh-psu/fixing-3
fixing one panic and two goroutine leaks
2021-12-21 11:03:59 +01:00
Linhai
246e7eba09 fixing the goroutine in two unit tests 2021-12-21 04:46:39 -05:00
Linhai Song
5e8f50bb09 remove the extra stop 2021-12-17 20:03:19 -05:00
Linhai Song
0098dbf350 fixing two goroutine leaks and one panic 2021-12-15 22:38:25 -05:00
Linhai
3ebd0a7d00 fixing the goroutine leak in TestBackendClose 2021-12-15 01:54:51 -05:00
leoyang.yl
7e6c29c198 fix runlock bug 2021-11-26 11:05:36 +08:00
Piotr Tabor
6c2f5dc78a
Merge pull request #13405 from serathius/downgrade-b
Implement single node downgrades
2021-10-29 23:22:10 +02:00
Marek Siarkowicz
431adc5878 server: Implement storage downgrades
By validating if WAL doesn't include any incompatible entries we can
implement storage downgrades.
2021-10-29 12:47:19 +02:00
Marek Siarkowicz
f5d71fa389 server: Detect when WAL includes unapplied cluster version set to higher version
This is because etcd v3.5 will panic when it encounters
ClusterVersionSet entry with version >3.5.0. For downgrades to v3.5 to
work we need to make sure this entry is snapshotted.
2021-10-29 12:47:19 +02:00
Eng Zer Jun
2a151c8982
*: move from io/ioutil to io and os packages
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-10-28 00:05:28 +08:00
Chao Chen
7b6554fd30 non mutating requests pass through quotaKVServer when NOSPACE 2021-10-22 16:36:37 -07:00
Chao Chen
9038e65359 scripts/genproto.sh: fix module path of wal proto files 2021-10-15 22:00:16 -07:00
Piotr Tabor
5b226e0abf
Merge pull request #13391 from serathius/downgrade-refactor
Refactor code to make place for downgrade logic
2021-10-08 12:38:25 +02:00
Marek Siarkowicz
d039f016c5 server: Move Storage interface to storage package 2021-10-08 12:01:53 +02:00
Marek Siarkowicz
703df1c491 server: Move wal versioning functions to wal package 2021-10-08 12:01:53 +02:00
Marek Siarkowicz
1e5e57f268 server: Move downgrade detection code to version package 2021-10-08 10:41:37 +02:00
Aleksandr Razumov
3644c9d67b
client/pkg/fileutil: add missing logger to {Create,Touch}DirAll
Also populate it to every invocation.
2021-10-07 17:53:59 +03:00
Geeta Gharpure
817d2f40d1 storage/backend: Add a gauge to indicate if defrag is active 2021-09-27 17:02:13 -07:00
Piotr Tabor
183cc52bc3
Merge pull request #13230 from serathius/storage
Move storage bootstrap to its package
2021-09-25 17:34:30 +02:00
Sunny Liu
e7a09db019 mvcc: simplify watchable_store addVictim code 2021-09-24 17:49:03 +08:00
Marek Siarkowicz
a206ad2c96 server: Rename GetIDs to GetEffectiveNodeIDsFromWalEntries 2021-09-20 12:21:36 +02:00
Marek Siarkowicz
79f6faa5c7 Apply suggestions from code review
Co-authored-by: Lili Cosic <cosiclili@gmail.com>
2021-09-10 10:16:48 +02:00
Marek Siarkowicz
ff3729c4d5 server: Implement storage schema migration to follow cluster version change and panic if unknown storage version is found
Storage version should follow cluster version. During upgrades this
should be immidiate as storage version can be always upgraded as storage
is backward compatible. During downgrades it will be delayed and will
require time for incompatible changes to be snapshotted.

As storage version change can happen long after cluster is running, we
need to add a step during bootstrap to validate if loaded data can be
understood by migrator.
2021-09-10 10:16:48 +02:00
Marek Siarkowicz
0d15ff57e6 server: Implement schema migrations 2021-09-10 10:16:48 +02:00
Sahdev Zala
91a5089d17
Merge pull request #13288 from ben1009/etcd-wal-doc
doc, log: fix typo in wal doc, log
2021-09-06 19:47:44 -04:00
ben1009
af10e8791f fix typo in wal doc, log:
- fix logs in wal Repair

- unify broken file name to val
2021-08-31 11:59:05 +08:00
Manuel Rüger
c9b4c866ce Cleanup remaining TODOs for/past 3.5 release 2021-08-06 13:29:25 +02:00
Piotr Tabor
873f3691f1
Merge pull request #13216 from serathius/wal
Annotate proto messages with version and detect etcd version generated the wal
2021-08-06 10:58:57 +02:00