385 Commits

Author SHA1 Message Date
Aleksandr Razumov
3644c9d67b
client/pkg/fileutil: add missing logger to {Create,Touch}DirAll
Also populate it to every invocation.
2021-10-07 17:53:59 +03:00
Marek Siarkowicz
90932324b1 client: Add grpc authority header integration tests 2021-09-29 12:42:16 +02:00
Piotr Tabor
752a206da4
Merge pull request #13238 from sakateka/etcdserver-logs
etcdserver: remove code duplication from the peer.send func
2021-09-27 09:01:34 +02:00
Piotr Tabor
ca5bad0e83
Merge pull request #13261 from njhill/sorting
etcdserver,clientv3: server-side ignore sort-ascend-key for range reqs
2021-09-25 17:47:10 +02:00
Marek Siarkowicz
35db0a5817 server: Refactor databaseFileMissing function 2021-09-20 12:21:36 +02:00
Marek Siarkowicz
39f92a32ca server: Move member dir creation up and introduce Close method to bootstrap structs 2021-09-20 12:21:36 +02:00
Marek Siarkowicz
a206ad2c96 server: Rename GetIDs to GetEffectiveNodeIDsFromWalEntries 2021-09-20 12:21:36 +02:00
Marek Siarkowicz
a450dc7f91 server: Rename function to NewConfigChangeEntries indicating we are not reading it from wal 2021-09-20 12:21:36 +02:00
Marek Siarkowicz
4884e7d8cf server: Move wal bootstrap from cluster to storage 2021-09-20 12:21:35 +02:00
Marek Siarkowicz
d3abf774ea server: Move cluster backend setting up the call hierarchy 2021-09-20 12:20:21 +02:00
Marek Siarkowicz
049e2d6ec0 server: Move raft up the bootstrap hierarchy 2021-09-20 12:20:19 +02:00
Marek Siarkowicz
138afa5be9 server: Split wal, cluster and raft bootstraping 2021-09-20 12:19:10 +02:00
Marek Siarkowicz
5d044563a8 server: Move raft and wal up the bootstrap hierarchy 2021-09-20 12:19:10 +02:00
Marek Siarkowicz
8b0d8ea2af server: Move cluster up the bootstrap hierarchy 2021-09-20 12:19:09 +02:00
Marek Siarkowicz
7c8f7166e7 server: Move bootstraping backend from snapshot to bootstrapBackend 2021-09-20 12:17:33 +02:00
Marek Siarkowicz
0211f5a2e8 server: Move snapshot recovery to separate function 2021-09-20 12:07:42 +02:00
Marek Siarkowicz
c97ab8f5e0 server: Move cluster up the bootstrap hierarchy 2021-09-20 12:07:41 +02:00
Marek Siarkowicz
648bac833f server: Move bootstrappedRaft up in file 2021-09-20 12:06:19 +02:00
Marek Siarkowicz
6a4ea70aef server: Move clusterID and nodeID up the bootstrap hierarchy 2021-09-20 12:06:18 +02:00
Marek Siarkowicz
db06a4ab28 server: Move wal bootstrap up the hierarchy 2021-09-20 12:04:44 +02:00
Marek Siarkowicz
aa0c050003 etcdserver: Add more hierarchy bootstap introducing a separate storage bootstrap step 2021-09-20 12:01:45 +02:00
Marek Siarkowicz
6c8a4fdcc5 server: Rename bootstrappedWal variables to bwal to separate it from wal package 2021-09-20 11:01:26 +02:00
Marek Siarkowicz
66d05e5496 Try updating storage version immidietly after cluster version is set 2021-09-10 10:16:48 +02:00
Marek Siarkowicz
ff3729c4d5 server: Implement storage schema migration to follow cluster version change and panic if unknown storage version is found
Storage version should follow cluster version. During upgrades this
should be immidiate as storage version can be always upgraded as storage
is backward compatible. During downgrades it will be delayed and will
require time for incompatible changes to be snapshotted.

As storage version change can happen long after cluster is running, we
need to add a step during bootstrap to validate if loaded data can be
understood by migrator.
2021-09-10 10:16:48 +02:00
Marek Siarkowicz
9d81dde082 server: Extract notifier struct 2021-09-10 10:16:48 +02:00
Piotr Tabor
af22382777
Merge pull request #13321 from patrocinio/main
Fix a few typos
2021-09-08 09:48:33 +02:00
Hitoshi Mitake
2a750a8dba *: implement a retry logic for auth old revision in the client 2021-09-05 01:13:52 +09:00
Eduardo Patrocinio
87f1dc7e40 Fix a few typos 2021-09-03 16:09:09 -04:00
Sahdev Zala
6a32bbad75
Merge pull request #13239 from sakateka/zap_raft
etcdserver: add zap.AddCallerSkip in NewRaftLoggerZap
2021-08-21 14:11:05 -04:00
Sahdev Zala
4739d3e9d3
Merge pull request #13252 from yuzhiquan/fix-always-true-or-false
etcdserver: remove always true or false in if statement
2021-08-21 14:09:43 -04:00
Marek Siarkowicz
83a325ac46 server: Move all functions needed for storage bootstrap to storage package
This is prerequestite to move storage bootstrap, splitted to separate PR
to make it easier to review.
2021-08-03 13:09:15 +02:00
Marek Siarkowicz
23b742cfd3 server: Remove Quota direct dependency on EtcdServer 2021-08-03 12:48:41 +02:00
Marek Siarkowicz
44b8ae145b etcdserver: Move datadir and wal to storage package 2021-08-03 12:47:37 +02:00
Sahdev Zala
2526463e44
Merge pull request #13236 from roytman/expensiveRequest
etcdserver: configure "expensive" requests duration
2021-08-02 09:33:43 -04:00
nickhill
99182f5404 etcdserver,clientv3: server-side ignore sort-ascend-key for range requests
A client-side optimization was made in #6100 to filter ascending key sorts to avoid an unnecessary re-sort since this is the order already returned by the back-end logic.

It seems to me that this really belongs on the server side since it's tied to the server implementation and should apply for any caller of the kv api (for example non-go clients).

Related, the client/v3 syncer depends on this default sorting which isn't explicit in the kv api contract. So I'm proposing the required sort parameters be included explicitly; it will take the fast path either way.
2021-07-30 15:02:15 -07:00
yuzhiquan
90773edb10 remove always true in if statement 2021-07-29 14:33:10 +08:00
Alexey Roytman
2a26f7ae4c
etcdserver: configure "expensive" requests duration
When a unary request takes more than predefined duration, this request
is defined as "expensive" and a warning is printed. The expensive request
duration is hard-coded to 300 ms. It can be not enough for example
for transactions with a lot of operations. The warnings just blow up
the log files and reduce throughput.

This fix allows user to configure the "expensive" request duration.

Signed-off-by: Alexey Roytman <roytman@il.ibm.com>
2021-07-27 08:33:44 +03:00
Sergey Kacheev
baf594b24a etcdserver: add zap.AddCallerSkip in NewRaftLoggerZap
The file `zap_raft.go` adds the raft.Logger proxy logger on top of `*zap.Logger`.
Adding a proxy requires adding the option `zap.AddCallerSkip(1)`,
so that the logging message specifies the correct caller,
two of the three constructors in the `zap_raft.go` adds this option.
This commit fixes the third constructor so that it also adds `zap.AddCallerSkip`.

Before fix:
`{"level":"info","ts":"2021-07-22T17:46:01.435Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"bd07d29169ff0c5a [logterm: 2, index: 8, vote: 38447ba545569bbe] ignored MsgPreVote from c7baeaad79d6d5ed [logterm: 2, index: 8] at term 2: lease is not expired (remaining ticks: 10)"}`

After fix:
`{"level":"info","ts":"2021-07-22T17:46:51.227Z","logger":"raft","caller":"raft/raft.go:859","msg":"bd07d29169ff0c5a [logterm: 2, index: 8, vote: c7baeaad79d6d5ed] ignored MsgPreVote from 38447ba545569bbe [logterm: 2, index: 8] at term 2: lease is not expired (remaining ticks: 9)"}`
2021-07-23 00:36:15 +07:00
Sergey Kacheev
daf7e0350a etcdserver: remove code duplication from the peer.send func
During the refactoring process, duplicate logging
of the send buffer overflow event was added.
Each of these log lines logs exactly the same information, the logging
context is sufficient to distinguish the cause.
Additionally, the unnecessary context (in parentheses) in the log
message was removed, which was necessary without the zap context (with
the old logger), but now only confuses.
2021-07-22 23:15:20 +07:00
Marek Siarkowicz
a0554a6bd3 etcdserver: Create AuthBackend interface 2021-07-20 18:09:53 +02:00
Marek Siarkowicz
6cd3633543 etcdserver: Rename membershipStore to membershipBackend 2021-07-20 17:56:52 +02:00
Marek Siarkowicz
2f31cc3fbc etcdserver: Create AlarmBackend interface 2021-07-20 17:53:44 +02:00
Marek Siarkowicz
f6534f1e92 server: Move setting storage version to schema 2021-07-12 15:37:21 +02:00
Marek Siarkowicz
5b6f4579fb server: Rename buckets to schema 2021-07-12 15:37:21 +02:00
Marek Siarkowicz
5e40a8b00c server: Create storage package and move mvcc files to it 2021-07-12 15:37:21 +02:00
Marek Siarkowicz
ef6c276fae etcdserver: Move server bootstraping to one file 2021-07-12 15:37:21 +02:00
Sahdev Zala
0cdd558361
Merge pull request #13188 from ahrtr/fix_excluded_alarm_issue
Skip empty query value(alarm) from the query parameter
2021-07-08 10:11:14 -04:00
Marek Siarkowicz
9824cc96ed etcdserver: Fix typos in bootstrap 2021-07-08 13:37:19 +02:00
Marek Siarkowicz
e1fa356fac etcdserver: Refactor standalone boostrap 2021-07-08 13:34:31 +02:00
Marek Siarkowicz
244e5c2cce etcdserver: Unify memory storage boostrap 2021-07-07 23:36:37 +02:00