Mirroristas/etcd
2
0
Fork 0
mirror of https://github.com/etcd-io/etcd.git synced 2024-09-27 06:25:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
18,336 Commits 15 Branches 491 Tags 148 MiB
Commit Graph

18336 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
wayblink
428d21f5eb mvcc:add ut for Revisions/CountRevisions and remove RangeSince as it is not used 
Signed-off-by: wayblink <wayasxxx@gmail.com>
 2022-06-21 11:41:44 +08:00
Mikel Olasagasti Uranga
89637a4223 Tools/benchmark: migrate cheggaaa/pb.v1 to cheggaaa/pb/v3 
etcdctl/ctlv3: migrate cheggaaa/pb.v1 to cheggaaa/pb/v3

This commit also changes the format of the progress bar, from using a
custom progress bar to the default provided by the library.

Old behaviour:

./benchmarkv1  put
 0 / 10000 B                                                   !   0.00%
 3987 / 10000 Boooooooooooooom                                 !  39.87%
 10000 / 10000 Boooooooooooooooooooooooooooooooooooooooooooo! 100.00% 1s

New behaviour:

./benchmark put
 6536 / 10000 [----------------------->________________] 65.36% 7053 p/s
10000 / 10000 [---------------------------------------] 100.00% 7581 p/s

Signed-off-by: Mikel Olasagasti Uranga <mikel@olasagasti.info>
 2022-06-20 15:47:23 +02:00
Marek Siarkowicz
0bac609632
Merge pull request #14133 from ahrtr/changelog_auth_bundle_20220618 
update changelog for both 3.5 and 3.6 to cover the authBundle overwrite issue
 2022-06-20 10:47:09 +02:00
Benjamin Wang
449ff7b8cb update changelog for both 3.5 and 3.6 to cover the authBundle overwrite issue 
Signed-off-by: Benjamin Wang <wachao@vmware.com>
 2022-06-18 07:11:50 +08:00
Benjamin Wang
43434afd48
Merge pull request #14111 from patrocinio/check_perf_doc 
Doc: provide more clarify to the usage of check perf command
 2022-06-18 05:07:26 +08:00
Piotr Tabor
72b2a06fdb
Merge pull request #14126 from ptabor/20220615-update-grpc3 
Update dependencies - including zap (1.21)
 2022-06-17 20:25:43 +02:00
Eduardo Patrocinio
e1e3ed11e3 Doc: provide more clarify to the usage of check perf command 
This PR describes the fact that different workloads in the `check perf` command
are different, and the results might vary.

Signed-off-by: Eduardo Patrocinio <epatro@gmail.com>
 2022-06-17 13:13:58 -03:00
Piotr Tabor
88cd0fe695 Refresh minor dependencies. 
Signed-off-by: Piotr Tabor <ptab@google.com>
 2022-06-17 10:23:03 +02:00
Piotr Tabor
e7a84b69c8 Update zap to 1.21. 
Signed-off-by: Piotr Tabor <ptab@google.com>
 2022-06-17 10:05:25 +02:00
Piotr Tabor
8ac23dad88
Merge pull request #14125 from ptabor/20220615-update-grpc2 
Update to grpc-1.47 (and fix the connection-string format)
 2022-06-17 10:04:11 +02:00
Benjamin Wang
3e61412ae2
Merge pull request #14129 from ahrtr/changelog_wal_entry_limit 
Update 3.5 and 3.6 changelog to cover the fix of restricting the max WAL entry
 2022-06-17 15:03:52 +08:00
Benjamin Wang
6d1041dc62
Merge pull request #14128 from ahrtr/remove_ioutil 
replace ioutil with os package
 2022-06-17 15:03:07 +08:00
Piotr Tabor
cbf54a0421 Use proper grpc schema (unix:file) for unix sockets. 
Signed-off-by: Piotr Tabor <ptab@google.com>
 2022-06-17 08:57:23 +02:00
Piotr Tabor
17c1dcd614 Update grpc to 1.47 and go-cmp to 0.5.8. 
Signed-off-by: Piotr Tabor <ptab@google.com>
 2022-06-17 08:57:22 +02:00
Benjamin Wang
fa68664a26 update 3.5 and 3.6 changelog to cover the fix of restricting the max size of each WAL entry to remaining size of the WAL file 
Signed-off-by: Benjamin Wang <wachao@vmware.com>
 2022-06-17 10:17:02 +08:00
Benjamin Wang
8038e876d3 replace ioutil with os package 
Signed-off-by: Benjamin Wang <wachao@vmware.com>
 2022-06-17 10:11:30 +08:00
Benjamin Wang
4c95878f00
Merge pull request #14122 from ahrtr/threshold_20220615 
Restrict the max size of each WAL entry to the remaining size of the WAL file
 2022-06-17 08:19:42 +08:00
Benjamin Wang
ccf477d12b restrict the max size of each WAL entry to the remaining size of the file 
Currently the max size of each WAL entry is hard coded as 10MB. If users
set a value > 10MB for the flag --max-request-bytes, then etcd may run
into a situation that it successfully processes a big request, but fails
to decode it when replaying the WAL file on startup.

On the other hand, we can't just remove the limitation, because if a
WAL entry is somehow corrupted, and its recByte is a huge value, then
etcd may run out of memory. So the solution is to restrict the max size
of each WAL entry as a dynamic value, which is the remaining size of
the WAL file.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
 2022-06-17 05:41:49 +08:00
Piotr Tabor
427372b6ee
Merge pull request #14123 from tsonglew/patch-1 
fix: typo gouroutine
 2022-06-16 13:28:45 +02:00
Tsonglew
e5a80f5049
fix: typo gouroutine 
fix: typo gouroutine
 2022-06-16 16:35:06 +08:00
Benjamin Wang
d23917c576 Add FileReader and FileBufReader utilities 
The FileReader interface is the wrapper of io.Reader. It provides
the fs.FileInfo as well. The FileBufReader struct is the wrapper of
bufio.Reader, it also provides fs.FileInfo.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
 2022-06-16 15:32:16 +08:00
Piotr Tabor
a1fe0b8ea3
Merge pull request #14116 from ptabor/20220613-embed-errors 
Embed server should log errors (and not get stuck)
 2022-06-15 22:37:40 +02:00
Piotr Tabor
e61d431792
Merge pull request #14109 from SimFG/fifo_panic 
schedule: Provide logs when the fifo job panic happens
 2022-06-15 22:24:42 +02:00
Piotr Tabor
509a01861c
Merge pull request #14117 from ptabor/20220614-gitignore 
Extend .gitignore to ignore intelij config/generated files.
 2022-06-15 22:10:38 +02:00
Sahdev Zala
c4e7224946
Merge pull request #14121 from L2ncE/main 
Fix a syntax error in a code comment
 2022-06-15 15:41:30 -04:00
L2ncE
637afd359b Fix a syntax error in a code comment 
Signed-off-by: L2ncE <llance_24@foxmail.com>
 2022-06-15 23:19:12 +08:00
SimFG
d83925e357 schedule: Provide logs when the fifo job panic happens 
To make the fifo scheduler better debuggability.

Signed-off-by: SimFG <1142838399@qq.com>
 2022-06-15 20:58:17 +08:00
Piotr Tabor
1c401581e3 Close channel of started servers even if embed server fails to start. 
Before this change:

```
gwmux, err = sctx.registerGateway([]grpc.DialOption{grpc.WithInsecure()})
if err != nil { return err; }
```

caused the Etcd.Serve() method finish without sctx.serversC channel being closed.
The code closing Etcd got stuck waiting for all the servers to be running.

Signed-off-by: Piotr Tabor <ptab@google.com>
 2022-06-15 13:50:17 +02:00
Piotr Tabor
fcc8fce4d2 Expand logging in case of embed server not being able to successfully start. 
So far the errors were directed to Etcd.Errc (channel) that is not being consumed in practice.

Signed-off-by: Piotr Tabor <ptab@google.com>
 2022-06-15 13:50:17 +02:00
Piotr Tabor
f9e9f50a13 Extend .gitignore to ignore intelij config/generated files. 
Signed-off-by: Piotr Tabor <ptab@google.com>
 2022-06-15 13:14:08 +02:00
Marek Siarkowicz
71bba3c761
Merge pull request #14106 from SimFG/remove_case 
wal: remove the repeated test case
 2022-06-14 12:13:00 +02:00
SimFG
402188a98c wal: remove the repeated test case 
The TestFilePipelineFailLockFile case maybe test the line 77 in the alloc function. But the LockFile will not throw an error when the file lock is hold. And TestFilePipelineFailPreallocate and TestFilePipelineFailLockFile are exactly the same except for the comments.

Signed-off-by: SimFG <1142838399@qq.com>
 2022-06-14 17:26:20 +08:00
Marek Siarkowicz
2c12954158
Merge pull request #14049 from serathius/compact-hash 
Calculate hash during compaction
 2022-06-14 11:10:19 +02:00
Marek Siarkowicz
9612fc1194 tests: Unify TestCompactionHash and extend it to also Delete keys and Defrag 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:20:19 +02:00
Marek Siarkowicz
f5eadf5452 tests: Add tests for HashByRev HTTP API 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:20:19 +02:00
Marek Siarkowicz
1581eef888 tests: Add integration tests for compact hash 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:20:19 +02:00
Marek Siarkowicz
0e739da9a4 server: Cache compaction hash for HashByRev API 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:20:18 +02:00
Marek Siarkowicz
2b090e86a6 server: Extract hasher to separate interface 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:20:18 +02:00
Marek Siarkowicz
80828b593a server: Remove duplicated compaction revision 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:20:18 +02:00
Marek Siarkowicz
34a02ba621 server: Return revision range that hash was calcualted for 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:20:17 +02:00
Marek Siarkowicz
76d3c527ae server: Store real rv range in hasher 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:19:26 +02:00
Marek Siarkowicz
8f7b053480 server: Move adjusting revision to hasher 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:19:26 +02:00
Marek Siarkowicz
19941654fe server: Pass revision as int 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:19:26 +02:00
Marek Siarkowicz
887e53e61a server: Calculate hash during compaction 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:19:26 +02:00
Marek Siarkowicz
7381c81a7e server: Fix range in mock not returning same number of keys and values 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:19:26 +02:00
Marek Siarkowicz
08a43178fa server: Move reading KV index inside scheduleCompaction function 
Makes it easier to test hash match between scheduleCompaction and
HashByRev.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:19:26 +02:00
Marek Siarkowicz
638ca1006a server: Return error from scheduleCompaction 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:19:26 +02:00
Marek Siarkowicz
0f90359c4b server: Refactor hasher 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:19:25 +02:00
Marek Siarkowicz
e62c358793 server: Extract kvHash struct 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:19:25 +02:00
Marek Siarkowicz
fcaf76dbca server: Move unsafeHashByRev to new hash.go file 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:19:25 +02:00