Prior to this CL, `ETCDCTL_API=2 etcdctl backup --with-v3` was readacting WAL log
(by removal of some entries), but was NOT updating consistent_index in the backend.
Also the WAL editing logic was buggy, as it didn't took in consideration the fact
that when TERM changes, there can be entries with duplicated indexes in
the log. So its NOT sufficient to subtract number of removed entries to
get accurate log indexes.
The PR replaces removing and shifting of WAL entries with replacing them with an no-op entries.
Thanks to this consistent-index references are staying up to date.
The PR also:
- updates 'verification' logic to check whether consistent_index does not lag befor last snapshot
- env-gated execution of verification framework in `etcdctl backup`.
Tested with:
```
(./build.sh && cd tests && EXPECT_DEBUG=TRUE 'env' 'go' 'test' '-timeout=300m' 'go.etcd.io/etcd/tests/v3/e2e' -run=TestCtlV2Backup --count=1000 2>&1 | tee TestCtlV2BackupV3.log)
```
correct 'backend' (bbolt) context in aspect of membership.
Prior to this change the 'restored' backend used to still contain:
- old memberid (mvcc deletion used, why the membership is in bolt
bucket, but not mvcc part):
```
mvs := mvcc.NewStore(s.lg, be, lessor, ci, mvcc.StoreConfig{CompactionBatchLimit: math.MaxInt32})
defer mvs.Close()
txn := mvs.Write(traceutil.TODO())
btx := be.BatchTx()
del := func(k, v []byte) error {
txn.DeleteRange(k, nil)
return nil
}
// delete stored members from old cluster since using new members
btx.UnsafeForEach([]byte("members"), del)
```
- didn't get new members added.
We make v2 client code a module go.etcd.io/etcd/client/v2.
Pretty mechanical change that can be summarized as:
mkdir client/v2
cd client/v2 && git mod init go.etcd.io/etcd/client/v2
git mv client/*.go client/v2/
find -name '*.go' | xargs sed -i --follow-symlinks 's|/v3/client["]|/client/v2\"|g'
+ fixing changelog, bom, go.mod etc.
This CL fixes:
COVERDIR=./coverage PASSES="build_cov" && go test --tags cov -v ./tests/e2e
and is part of the effort to make:
COVERDIR=coverage PASSES="build_cov cov" ./test
fully pass.
The args passed to ./bin/etcd_test and ./bin/etcdctl_test binaries were
mismatched. The protocol of passing the arguments using
environment variables has been replaces with proper passing of flags.
How the measurement of coverage by e2e tests works:
1. COVERDIR=./coverage PASSES="build_cov" are generating
./bin/etcd_test and ./bin/etcdctl_test binaries.
2. These binaries are tests (as coverage can be computed only for
tests) [see ./main_test.go ./etcdctl/main_test.go], but this tests are
running the main logic of the server and uppon termination (or SIGTERM
signal) are writting proper .coverprofile files in the $COVERDIR folder.
The binaries used to take arguments using env variables, but its not
needed any longer. The binaries can consume any command line arguments
that either test (so --test.fooo) or the original binary can consume.
3. The tests/e2e (when compiled with the --tags cov) are starting the
_test binaries instead of the original binaries, such that the coverage
is being collected.
This change makes the etcd package compatible with the existing Go
ecosystem for module versioning.
Used this tool to update package imports:
https://github.com/KSubedi/gomove
Since syscall.Errno implements net.Error and all cases
are matched sequentially, it's a mistake to put syscall.Errno
case after net.Error since it will never be executed.
This change swaps syscall.Errno case with net.Error
to give that clause chance to execute.
etcdctl was checking if the user exists before applying mutable calls;
if etcdctl contacts a minority member, the member may not know the user
exists on the cluster yet, causing command failure when it should succeed.
If the user does not exist, it will be picked up once the command goes
through raft.
Fixes#6932
