This commit introduces the max inflight bytes setting at the Config level, and
tests that raft flow control honours it.
Signed-off-by: Pavel Kalinnikov <pavel@cockroachlabs.com>
This commit plumbs the max total byte size of the Inflights type higher up the
stack to the ProgressTracker.
Signed-off-by: Pavel Kalinnikov <pavel@cockroachlabs.com>
The Inflights type has limits on the message size and the number of inflight
messages. However, a single large entry that exceeds the size limit can still
be sent. In combination with the max messages count limit, many large messages
can be sent in a row and overflow the receiver. In effect, the "max" values act
as "target" rather than hard limits.
This commit adds an additional soft limit on the total size of inflight
messages, which catches such situations and prevents the receiver overflow.
Signed-off-by: Pavel Kalinnikov <pavel@cockroachlabs.com>
Tobias Grieger is actively working on etcd/raft, and has already
been added back as a raft maintainer, so no reason to keep him
in the Emeritus Maintainers list.
Signed-off-by: Benjamin Wang <wachao@vmware.com>
This commit removes the ability to apply log entries at the same time as
applying a snapshot. Doing so it possible, but it leads to complex code and
raises questions about what should be applied first. It also raises additional
complexity when we start allowing concurrent, asynchronous log appends and log
application. It's easiest to just disallow this.
Signed-off-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>
Also rename hasNextEnts to hasNextCommittedEnts.
Also rename maxNextEntsSize to maxNextCommittedEntsSize.
Pure refactor.
Signed-off-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>
When e2e test cases specify the DataDirPath and there are more than
one member in the cluster, we need to create a subdirectory for each
member. Otherwise all members share the same directory and accordingly
lead to conflict.
Signed-off-by: Benjamin Wang <wachao@vmware.com>
Deprecated: use WithTransportCredentials and insecure.NewCredentials() instead. Will be supported throughout 1.x.
Signed-off-by: Ramil Mirhasanov <ramil600@yahoo.com>
`unsafeCommit` is called by both `(*batchTxBuffered) commit` and
`(*backend) defrag`. When users perform the defragmentation
operation, etcd doesn't update the consistent index. If etcd
crashes(e.g. panicking) in the process for whatever reason, then
etcd replays the WAL entries starting from the latest snapshot,
accordingly it may re-apply entries which might have already been
applied, eventually the revision isn't consistent with other members.
Refer to discussion in https://github.com/etcd-io/etcd/pull/14685
Signed-off-by: Benjamin Wang <wachao@vmware.com>
Use array indexing to clean up the code and make it constant time.
Also, add a test for IsResponseMsg.
Signed-off-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>
This commit makes the rarely used `raftpb.Message.Snapshot` field nullable.
In doing so, it reduces the memory size of a `raftpb.Message` message from
264 bytes to 128 bytes — a 52% reduction in size.
While this commit does not change the protobuf encoding, it does change
how that encoding is used. `(gogoproto.nullable) = false` instruct the
generated proto marshaling logic to always encode a value for the field,
even if that value is empty. `(gogoproto.nullable) = true` instructs the
generated proto marshaling logic to omit an encoded value for the field
if the field is nil.
This raises compatibility concerns in both directions. Messages encoded
by new binary versions without a `Snapshot` field will be decoded as an
empty field by old binary versions. In other words, old binary versions
can't tell the difference. However, messages encoded by old binary versions
with an empty Snapshot field will be decoded as a non-nil, empty field by
new binary versions. As a result, new binary versions need to be prepared
to handle such messages.
While Message.Snapshot is not intentionally part of the external interface
of this library, it was possible for users of the library to access it and
manipulate it. As such, this change may be considered a breaking change.
Signed-off-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>
This fix avoids the assumption of knowing the current version of the
binary. We can query the binary with the version flag to get the actual
version of the given binary we upgrade and downgrade to. The
respectively reported versions should match what is returned by the
version endpoint.
Signed-off-by: Thomas Jungblut <tjungblu@redhat.com>
This commit removes the verbose comparisons in tests, in favor of using
assert/require helpers from the testify packages.
Signed-off-by: Pavel Kalinnikov <pavel@cockroachlabs.com>
Make the field name and comment clearer on the fact that it's used both in
StateProbe and StateReplicate. The old name ProbeSent was slightly confusing,
and also triggered thinking that it's used only in StateProbe.
Signed-off-by: Pavel Kalinnikov <pavel@cockroachlabs.com>
