From 01dbb202480f713af1f2b396bcc3d0f2b28fecc6 Mon Sep 17 00:00:00 2001 From: innoprenuer Date: Mon, 6 Aug 2018 14:03:36 +0200 Subject: [PATCH 01/15] moved java driver to officially supported category --- docs/server/source/drivers-clients/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/server/source/drivers-clients/index.rst b/docs/server/source/drivers-clients/index.rst index b124a598..3cf4d0fa 100644 --- a/docs/server/source/drivers-clients/index.rst +++ b/docs/server/source/drivers-clients/index.rst @@ -6,6 +6,7 @@ Libraries and Tools Maintained by the BigchainDB Team * `Python Driver `_ * `JavaScript / Node.js Driver `_ +* `Java driver `_ Community-Driven Libraries and Tools ------------------------------------ @@ -17,6 +18,5 @@ Community-Driven Libraries and Tools * `Haskell transaction builder `_ * `Go driver `_ -* `Java driver `_ * `Ruby driver `_ * `Ruby library for preparing/signing transactions and submitting them or querying a BigchainDB node (MIT licensed) `_ From 32b64ccc2a208f38162566f3e088ad49baced79f Mon Sep 17 00:00:00 2001 From: Troy McConaghy Date: Mon, 6 Aug 2018 15:20:31 +0200 Subject: [PATCH 02/15] Problem: k8s docs not clear about who they're for (#2439) Solution: Add a note atop k8s docs pages to clarify who they're for --- .../architecture.rst | 22 ++++++++++++++----- .../bigchaindb-network-on-kubernetes.rst | 11 ++++++++++ .../production-deployment-template/index.rst | 17 ++++++++++---- .../node-config-map-and-secrets.rst | 11 ++++++++++ .../node-on-kubernetes.rst | 13 ++++++++++- .../tectonic-azure.rst | 11 ++++++++++ .../template-kubernetes-azure.rst | 11 ++++++++++ .../upgrade-on-kubernetes.rst | 11 ++++++++++ .../workflow.rst | 14 ++++++++++-- 9 files changed, 109 insertions(+), 12 deletions(-) diff --git a/docs/server/source/production-deployment-template/architecture.rst b/docs/server/source/production-deployment-template/architecture.rst index 4d029f5c..2bfc9e4c 100644 --- a/docs/server/source/production-deployment-template/architecture.rst +++ b/docs/server/source/production-deployment-template/architecture.rst @@ -1,13 +1,25 @@ -Architecture of a BigchainDB Node -================================== +Architecture of a BigchainDB Node Running in a Kubernetes Cluster +================================================================= -A BigchainDB Production deployment is hosted on a Kubernetes cluster and includes: +.. note:: + + A highly-available Kubernetes cluster requires at least five virtual machines + (three for the master and two for your app's containers). + Therefore we don't recommend using Kubernetes to run a BigchainDB node + if that's the only thing the Kubernetes cluster will be running. + Instead, see **How to Set Up a BigchainDB Network**. + If your organization already *has* a big Kubernetes cluster running many containers, + and your organization has people who know Kubernetes, + then this Kubernetes deployment template might be helpful. + +If you deploy a BigchainDB node into a Kubernetes cluster +as described in these docs, it will include: * NGINX, OpenResty, BigchainDB, MongoDB and Tendermint `Kubernetes Services `_. -* NGINX, OpenResty, BigchainDB and MongoDB Monitoring Agent. +* NGINX, OpenResty, BigchainDB and MongoDB Monitoring Agent `Kubernetes Deployments `_. -* MongoDB and Tendermint `Kubernetes StatefulSet `_. +* MongoDB and Tendermint `Kubernetes StatefulSets `_. * Third party services like `3scale `_, `MongoDB Cloud Manager `_ and the `Azure Operations Management Suite diff --git a/docs/server/source/production-deployment-template/bigchaindb-network-on-kubernetes.rst b/docs/server/source/production-deployment-template/bigchaindb-network-on-kubernetes.rst index 100dafb5..d3caf6a3 100644 --- a/docs/server/source/production-deployment-template/bigchaindb-network-on-kubernetes.rst +++ b/docs/server/source/production-deployment-template/bigchaindb-network-on-kubernetes.rst @@ -3,6 +3,17 @@ Kubernetes Template: Deploying a BigchainDB network =================================================== +.. note:: + + A highly-available Kubernetes cluster requires at least five virtual machines + (three for the master and two for your app's containers). + Therefore we don't recommend using Kubernetes to run a BigchainDB node + if that's the only thing the Kubernetes cluster will be running. + Instead, see **How to Set Up a BigchainDB Network**. + If your organization already *has* a big Kubernetes cluster running many containers, + and your organization has people who know Kubernetes, + then this Kubernetes deployment template might be helpful. + This page describes how to deploy a static BigchainDB + Tendermint network. If you want to deploy a stand-alone BigchainDB node in a BigchainDB cluster, diff --git a/docs/server/source/production-deployment-template/index.rst b/docs/server/source/production-deployment-template/index.rst index 64a834db..fa9d64c1 100644 --- a/docs/server/source/production-deployment-template/index.rst +++ b/docs/server/source/production-deployment-template/index.rst @@ -1,10 +1,19 @@ Production Deployment Template ============================== -This section outlines how *we* deploy production BigchainDB, -integrated with Tendermint(backend for BFT consensus), -clusters on Microsoft Azure using -Kubernetes. We improve it constantly. +.. note:: + + A highly-available Kubernetes cluster requires at least five virtual machines + (three for the master and two for your app's containers). + Therefore we don't recommend using Kubernetes to run a BigchainDB node + if that's the only thing the Kubernetes cluster will be running. + Instead, see **How to Set Up a BigchainDB Network**. + If your organization already *has* a big Kubernetes cluster running many containers, + and your organization has people who know Kubernetes, + then this Kubernetes deployment template might be helpful. + +This section outlines a way to deploy a production BigchainDB node (or BigchainDB cluster) +on Microsoft Azure using Kubernetes. You may choose to use it as a template or reference for your own deployment, but *we make no claim that it is suitable for your purposes*. Feel free change things to suit your needs or preferences. diff --git a/docs/server/source/production-deployment-template/node-config-map-and-secrets.rst b/docs/server/source/production-deployment-template/node-config-map-and-secrets.rst index 7bf8d0de..d308bc11 100644 --- a/docs/server/source/production-deployment-template/node-config-map-and-secrets.rst +++ b/docs/server/source/production-deployment-template/node-config-map-and-secrets.rst @@ -3,6 +3,17 @@ How to Configure a BigchainDB Node ================================== +.. note:: + + A highly-available Kubernetes cluster requires at least five virtual machines + (three for the master and two for your app's containers). + Therefore we don't recommend using Kubernetes to run a BigchainDB node + if that's the only thing the Kubernetes cluster will be running. + Instead, see **How to Set Up a BigchainDB Network**. + If your organization already *has* a big Kubernetes cluster running many containers, + and your organization has people who know Kubernetes, + then this Kubernetes deployment template might be helpful. + This page outlines the steps to set a bunch of configuration settings in your BigchainDB node. They are pushed to the Kubernetes cluster in two files, diff --git a/docs/server/source/production-deployment-template/node-on-kubernetes.rst b/docs/server/source/production-deployment-template/node-on-kubernetes.rst index b6fbeedd..96f4098a 100644 --- a/docs/server/source/production-deployment-template/node-on-kubernetes.rst +++ b/docs/server/source/production-deployment-template/node-on-kubernetes.rst @@ -3,7 +3,18 @@ Kubernetes Template: Deploy a Single BigchainDB Node ==================================================== -This page describes how to deploy a BigchainDB + Tendermint node +.. note:: + + A highly-available Kubernetes cluster requires at least five virtual machines + (three for the master and two for your app's containers). + Therefore we don't recommend using Kubernetes to run a BigchainDB node + if that's the only thing the Kubernetes cluster will be running. + Instead, see **How to Set Up a BigchainDB Network**. + If your organization already *has* a big Kubernetes cluster running many containers, + and your organization has people who know Kubernetes, + then this Kubernetes deployment template might be helpful. + +This page describes how to deploy a BigchainDB node using `Kubernetes `_. It assumes you already have a running Kubernetes cluster. diff --git a/docs/server/source/production-deployment-template/tectonic-azure.rst b/docs/server/source/production-deployment-template/tectonic-azure.rst index f9d58074..03cfb433 100644 --- a/docs/server/source/production-deployment-template/tectonic-azure.rst +++ b/docs/server/source/production-deployment-template/tectonic-azure.rst @@ -1,6 +1,17 @@ Walkthrough: Deploy a Kubernetes Cluster on Azure using Tectonic by CoreOS ========================================================================== +.. note:: + + A highly-available Kubernetes cluster requires at least five virtual machines + (three for the master and two for your app's containers). + Therefore we don't recommend using Kubernetes to run a BigchainDB node + if that's the only thing the Kubernetes cluster will be running. + Instead, see **How to Set Up a BigchainDB Network**. + If your organization already *has* a big Kubernetes cluster running many containers, + and your organization has people who know Kubernetes, + then this Kubernetes deployment template might be helpful. + A BigchainDB node can be run inside a `Kubernetes `_ cluster. This page describes one way to deploy a Kubernetes cluster on Azure using Tectonic. diff --git a/docs/server/source/production-deployment-template/template-kubernetes-azure.rst b/docs/server/source/production-deployment-template/template-kubernetes-azure.rst index 7b6b4e2c..9ff10cac 100644 --- a/docs/server/source/production-deployment-template/template-kubernetes-azure.rst +++ b/docs/server/source/production-deployment-template/template-kubernetes-azure.rst @@ -1,6 +1,17 @@ Template: Deploy a Kubernetes Cluster on Azure ============================================== +.. note:: + + A highly-available Kubernetes cluster requires at least five virtual machines + (three for the master and two for your app's containers). + Therefore we don't recommend using Kubernetes to run a BigchainDB node + if that's the only thing the Kubernetes cluster will be running. + Instead, see **How to Set Up a BigchainDB Network**. + If your organization already *has* a big Kubernetes cluster running many containers, + and your organization has people who know Kubernetes, + then this Kubernetes deployment template might be helpful. + A BigchainDB node can be run inside a `Kubernetes `_ cluster. This page describes one way to deploy a Kubernetes cluster on Azure. diff --git a/docs/server/source/production-deployment-template/upgrade-on-kubernetes.rst b/docs/server/source/production-deployment-template/upgrade-on-kubernetes.rst index 07d63f7b..8d836564 100644 --- a/docs/server/source/production-deployment-template/upgrade-on-kubernetes.rst +++ b/docs/server/source/production-deployment-template/upgrade-on-kubernetes.rst @@ -1,6 +1,17 @@ Kubernetes Template: Upgrade all Software in a BigchainDB Node ============================================================== +.. note:: + + A highly-available Kubernetes cluster requires at least five virtual machines + (three for the master and two for your app's containers). + Therefore we don't recommend using Kubernetes to run a BigchainDB node + if that's the only thing the Kubernetes cluster will be running. + Instead, see **How to Set Up a BigchainDB Network**. + If your organization already *has* a big Kubernetes cluster running many containers, + and your organization has people who know Kubernetes, + then this Kubernetes deployment template might be helpful. + This page outlines how to upgrade all the software associated with a BigchainDB node running on Kubernetes, including host operating systems, Docker, Kubernetes, diff --git a/docs/server/source/production-deployment-template/workflow.rst b/docs/server/source/production-deployment-template/workflow.rst index 61b499ee..793175b9 100644 --- a/docs/server/source/production-deployment-template/workflow.rst +++ b/docs/server/source/production-deployment-template/workflow.rst @@ -3,9 +3,19 @@ Overview ======== -This page summarizes the steps *we* go through +.. note:: + + A highly-available Kubernetes cluster requires at least five virtual machines + (three for the master and two for your app's containers). + Therefore we don't recommend using Kubernetes to run a BigchainDB node + if that's the only thing the Kubernetes cluster will be running. + Instead, see **How to Set Up a BigchainDB Network**. + If your organization already *has* a big Kubernetes cluster running many containers, + and your organization has people who know Kubernetes, + then this Kubernetes deployment template might be helpful. + +This page summarizes some steps to go through to set up a production BigchainDB cluster. -We are constantly improving them. You can modify them to suit your needs. .. _generate-the-blockchain-id-and-genesis-time: From 9b4273a98719adee75a7613c3e2be94b0df56b8d Mon Sep 17 00:00:00 2001 From: Troy McConaghy Date: Tue, 7 Aug 2018 13:20:47 +0200 Subject: [PATCH 03/15] Problem: Docs: 'Production Deployment Template' not a good name anymore (#2440) Solution: Rename that section to 'Kubernetes Deployment Template' --- .../source/appendices/all-in-one-bigchaindb.md | 2 +- docs/server/source/clusters.md | 10 ---------- docs/server/source/index.rst | 2 +- .../architecture.rst | 0 .../bigchaindb-network-on-kubernetes.rst | 0 .../ca-installation.rst | 0 .../client-tls-certificate.rst | 0 .../cloud-manager.rst | 2 +- .../easy-rsa.rst | 0 .../index.rst | 4 ++-- .../log-analytics.rst | 0 .../node-config-map-and-secrets.rst | 0 .../node-on-kubernetes.rst | 12 ++++++------ .../revoke-tls-certificate.rst | 0 .../server-tls-certificate.rst | 0 .../tectonic-azure.rst | 0 .../template-kubernetes-azure.rst | 0 .../troubleshoot.rst | 0 .../upgrade-on-kubernetes.rst | 0 .../workflow.rst | 6 +++--- 20 files changed, 14 insertions(+), 24 deletions(-) rename docs/server/source/{production-deployment-template => k8s-deployment-template}/architecture.rst (100%) rename docs/server/source/{production-deployment-template => k8s-deployment-template}/bigchaindb-network-on-kubernetes.rst (100%) rename docs/server/source/{production-deployment-template => k8s-deployment-template}/ca-installation.rst (100%) rename docs/server/source/{production-deployment-template => k8s-deployment-template}/client-tls-certificate.rst (100%) rename docs/server/source/{production-deployment-template => k8s-deployment-template}/cloud-manager.rst (96%) rename docs/server/source/{production-deployment-template => k8s-deployment-template}/easy-rsa.rst (100%) rename docs/server/source/{production-deployment-template => k8s-deployment-template}/index.rst (90%) rename docs/server/source/{production-deployment-template => k8s-deployment-template}/log-analytics.rst (100%) rename docs/server/source/{production-deployment-template => k8s-deployment-template}/node-config-map-and-secrets.rst (100%) rename docs/server/source/{production-deployment-template => k8s-deployment-template}/node-on-kubernetes.rst (98%) rename docs/server/source/{production-deployment-template => k8s-deployment-template}/revoke-tls-certificate.rst (100%) rename docs/server/source/{production-deployment-template => k8s-deployment-template}/server-tls-certificate.rst (100%) rename docs/server/source/{production-deployment-template => k8s-deployment-template}/tectonic-azure.rst (100%) rename docs/server/source/{production-deployment-template => k8s-deployment-template}/template-kubernetes-azure.rst (100%) rename docs/server/source/{production-deployment-template => k8s-deployment-template}/troubleshoot.rst (100%) rename docs/server/source/{production-deployment-template => k8s-deployment-template}/upgrade-on-kubernetes.rst (100%) rename docs/server/source/{production-deployment-template => k8s-deployment-template}/workflow.rst (96%) diff --git a/docs/server/source/appendices/all-in-one-bigchaindb.md b/docs/server/source/appendices/all-in-one-bigchaindb.md index 601da89e..246b4e50 100644 --- a/docs/server/source/appendices/all-in-one-bigchaindb.md +++ b/docs/server/source/appendices/all-in-one-bigchaindb.md @@ -15,7 +15,7 @@ This image contains all the services required for a BigchainDB node i.e. *This image is to help quick deployment for early adopters, for a more standard approach please refer to one of our deployment guides:* - [BigchainDB developer setup guides](https://docs.bigchaindb.com/projects/contributing/en/latest/dev-setup-coding-and-contribution-process/index.html). -- [BigchainDB with Kubernetes](http://docs.bigchaindb.com/projects/server/en/latest/production-deployment-template/index.html). +- [BigchainDB with Kubernetes](http://docs.bigchaindb.com/projects/server/en/latest/k8s-deployment-template/index.html). ## Prerequisite(s) - [Docker](https://docs.docker.com/engine/installation/) diff --git a/docs/server/source/clusters.md b/docs/server/source/clusters.md index be1d1c9c..23cbf1f3 100644 --- a/docs/server/source/clusters.md +++ b/docs/server/source/clusters.md @@ -2,7 +2,6 @@ A **BigchainDB Cluster** is a set of connected **BigchainDB Nodes**, managed by a **BigchainDB Consortium** (i.e. an organization). Those terms are defined in the [BigchainDB Terminology page](https://docs.bigchaindb.com/en/latest/terminology.html). - ## Consortium Structure & Governance The consortium might be a company, a foundation, a cooperative, or [some other form of organization](https://en.wikipedia.org/wiki/Organizational_structure). @@ -13,13 +12,6 @@ This documentation doesn't explain how to create a consortium, nor does it outli It's worth noting that the decentralization of a BigchainDB cluster depends, to some extent, on the decentralization of the associated consortium. See the pages about [decentralization](https://docs.bigchaindb.com/en/latest/decentralized.html) and [node diversity](https://docs.bigchaindb.com/en/latest/diversity.html). - -## Relevant Technical Documentation - -Anyone building or managing a BigchainDB cluster may be interested -in [our production deployment template](production-deployment-template/index.html). - - ## Cluster DNS Records and SSL Certificates We now describe how *we* set up the external (public-facing) DNS records for a BigchainDB cluster. Your consortium may opt to do it differently. @@ -30,14 +22,12 @@ There were several goals: * There should be no sharing of SSL certificates among BigchainDB node operators. * Optional: Allow clients to connect to a "random" BigchainDB node in the cluster at one particular domain (or subdomain). - ### Node Operator Responsibilities 1. Register a domain (or use one that you already have) for your BigchainDB node. You can use a subdomain if you like. For example, you might opt to use `abc-org73.net`, `api.dynabob8.io` or `figmentdb3.ninja`. 2. Get an SSL certificate for your domain or subdomain, and properly install it in your node (e.g. in your NGINX instance). 3. Create a DNS A Record mapping your domain or subdomain to the public IP address of your node (i.e. the one that serves the BigchainDB HTTP API). - ### Consortium Responsibilities Optional: The consortium managing the BigchainDB cluster could register a domain name and set up CNAME records mapping that domain name (or one of its subdomains) to each of the nodes in the cluster. For example, if the consortium registered `bdbcluster.io`, they could set up CNAME records like the following: diff --git a/docs/server/source/index.rst b/docs/server/source/index.rst index c8f16d02..be9d1a10 100644 --- a/docs/server/source/index.rst +++ b/docs/server/source/index.rst @@ -10,13 +10,13 @@ BigchainDB Server Documentation simple-network-setup production-nodes/index clusters - production-deployment-template/index dev-and-test/index server-reference/index http-client-server-api events/index drivers-clients/index data-models/index + k8s-deployment-template/index release-notes glossary appendices/index diff --git a/docs/server/source/production-deployment-template/architecture.rst b/docs/server/source/k8s-deployment-template/architecture.rst similarity index 100% rename from docs/server/source/production-deployment-template/architecture.rst rename to docs/server/source/k8s-deployment-template/architecture.rst diff --git a/docs/server/source/production-deployment-template/bigchaindb-network-on-kubernetes.rst b/docs/server/source/k8s-deployment-template/bigchaindb-network-on-kubernetes.rst similarity index 100% rename from docs/server/source/production-deployment-template/bigchaindb-network-on-kubernetes.rst rename to docs/server/source/k8s-deployment-template/bigchaindb-network-on-kubernetes.rst diff --git a/docs/server/source/production-deployment-template/ca-installation.rst b/docs/server/source/k8s-deployment-template/ca-installation.rst similarity index 100% rename from docs/server/source/production-deployment-template/ca-installation.rst rename to docs/server/source/k8s-deployment-template/ca-installation.rst diff --git a/docs/server/source/production-deployment-template/client-tls-certificate.rst b/docs/server/source/k8s-deployment-template/client-tls-certificate.rst similarity index 100% rename from docs/server/source/production-deployment-template/client-tls-certificate.rst rename to docs/server/source/k8s-deployment-template/client-tls-certificate.rst diff --git a/docs/server/source/production-deployment-template/cloud-manager.rst b/docs/server/source/k8s-deployment-template/cloud-manager.rst similarity index 96% rename from docs/server/source/production-deployment-template/cloud-manager.rst rename to docs/server/source/k8s-deployment-template/cloud-manager.rst index fb0512df..458a8eeb 100644 --- a/docs/server/source/production-deployment-template/cloud-manager.rst +++ b/docs/server/source/k8s-deployment-template/cloud-manager.rst @@ -41,7 +41,7 @@ Configure MongoDB Cloud Manager for Monitoring * If you have authentication enabled, select the option to enable authentication and specify the authentication mechanism as per your - deployment. The default BigchainDB production deployment currently + deployment. The default BigchainDB Kubernetes deployment template currently supports ``X.509 Client Certificate`` as the authentication mechanism. * If you have TLS enabled, select the option to enable TLS/SSL for MongoDB diff --git a/docs/server/source/production-deployment-template/easy-rsa.rst b/docs/server/source/k8s-deployment-template/easy-rsa.rst similarity index 100% rename from docs/server/source/production-deployment-template/easy-rsa.rst rename to docs/server/source/k8s-deployment-template/easy-rsa.rst diff --git a/docs/server/source/production-deployment-template/index.rst b/docs/server/source/k8s-deployment-template/index.rst similarity index 90% rename from docs/server/source/production-deployment-template/index.rst rename to docs/server/source/k8s-deployment-template/index.rst index fa9d64c1..44a3fc07 100644 --- a/docs/server/source/production-deployment-template/index.rst +++ b/docs/server/source/k8s-deployment-template/index.rst @@ -1,4 +1,4 @@ -Production Deployment Template +Kubernetes Deployment Template ============================== .. note:: @@ -12,7 +12,7 @@ Production Deployment Template and your organization has people who know Kubernetes, then this Kubernetes deployment template might be helpful. -This section outlines a way to deploy a production BigchainDB node (or BigchainDB cluster) +This section outlines a way to deploy a BigchainDB node (or BigchainDB cluster) on Microsoft Azure using Kubernetes. You may choose to use it as a template or reference for your own deployment, but *we make no claim that it is suitable for your purposes*. diff --git a/docs/server/source/production-deployment-template/log-analytics.rst b/docs/server/source/k8s-deployment-template/log-analytics.rst similarity index 100% rename from docs/server/source/production-deployment-template/log-analytics.rst rename to docs/server/source/k8s-deployment-template/log-analytics.rst diff --git a/docs/server/source/production-deployment-template/node-config-map-and-secrets.rst b/docs/server/source/k8s-deployment-template/node-config-map-and-secrets.rst similarity index 100% rename from docs/server/source/production-deployment-template/node-config-map-and-secrets.rst rename to docs/server/source/k8s-deployment-template/node-config-map-and-secrets.rst diff --git a/docs/server/source/production-deployment-template/node-on-kubernetes.rst b/docs/server/source/k8s-deployment-template/node-on-kubernetes.rst similarity index 98% rename from docs/server/source/production-deployment-template/node-on-kubernetes.rst rename to docs/server/source/k8s-deployment-template/node-on-kubernetes.rst index 96f4098a..e9032ec4 100644 --- a/docs/server/source/production-deployment-template/node-on-kubernetes.rst +++ b/docs/server/source/k8s-deployment-template/node-on-kubernetes.rst @@ -40,7 +40,7 @@ If you don't have that file, then you need to get it. **Azure.** If you deployed your Kubernetes cluster on Azure using the Azure CLI 2.0 (as per :doc:`our template -<../production-deployment-template/template-kubernetes-azure>`), +<../k8s-deployment-template/template-kubernetes-azure>`), then you can get the ``~/.kube/config`` file using: .. code:: bash @@ -288,7 +288,7 @@ The first thing to do is create the Kubernetes storage classes. First, you need an Azure storage account. If you deployed your Kubernetes cluster on Azure using the Azure CLI 2.0 -(as per :doc:`our template <../production-deployment-template/template-kubernetes-azure>`), +(as per :doc:`our template <../k8s-deployment-template/template-kubernetes-azure>`), then the `az acs create` command already created a storage account in the same location and resource group as your Kubernetes cluster. @@ -300,7 +300,7 @@ in the same data center. Premium storage is higher-cost and higher-performance. It uses solid state drives (SSD). -We recommend using Premium storage for our production template. +We recommend using Premium storage with our Kubernetes deployment template. Create a `storage account `_ for Premium storage and associate it with your Azure resource group. For future reference, the command to create a storage account is @@ -383,7 +383,7 @@ but it should become "Bound" fairly quickly. $ kubectl patch pv -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}' For notes on recreating a private volume form a released Azure disk resource consult - :doc:`the page about cluster troubleshooting <../production-deployment-template/troubleshoot>`. + :doc:`the page about cluster troubleshooting <../k8s-deployment-template/troubleshoot>`. .. _start-kubernetes-stateful-set-mongodb: @@ -580,7 +580,7 @@ Step 19(Optional): Configure the MongoDB Cloud Manager ------------------------------------------------------ Refer to the -:doc:`documentation <../production-deployment-template/cloud-manager>` +:doc:`documentation <../k8s-deployment-template/cloud-manager>` for details on how to configure the MongoDB Cloud Manager to enable monitoring and backup. @@ -760,4 +760,4 @@ verify that your node or cluster works as expected. Next, you can set up log analytics and monitoring, by following our templates: -* :doc:`../production-deployment-template/log-analytics`. +* :doc:`../k8s-deployment-template/log-analytics`. diff --git a/docs/server/source/production-deployment-template/revoke-tls-certificate.rst b/docs/server/source/k8s-deployment-template/revoke-tls-certificate.rst similarity index 100% rename from docs/server/source/production-deployment-template/revoke-tls-certificate.rst rename to docs/server/source/k8s-deployment-template/revoke-tls-certificate.rst diff --git a/docs/server/source/production-deployment-template/server-tls-certificate.rst b/docs/server/source/k8s-deployment-template/server-tls-certificate.rst similarity index 100% rename from docs/server/source/production-deployment-template/server-tls-certificate.rst rename to docs/server/source/k8s-deployment-template/server-tls-certificate.rst diff --git a/docs/server/source/production-deployment-template/tectonic-azure.rst b/docs/server/source/k8s-deployment-template/tectonic-azure.rst similarity index 100% rename from docs/server/source/production-deployment-template/tectonic-azure.rst rename to docs/server/source/k8s-deployment-template/tectonic-azure.rst diff --git a/docs/server/source/production-deployment-template/template-kubernetes-azure.rst b/docs/server/source/k8s-deployment-template/template-kubernetes-azure.rst similarity index 100% rename from docs/server/source/production-deployment-template/template-kubernetes-azure.rst rename to docs/server/source/k8s-deployment-template/template-kubernetes-azure.rst diff --git a/docs/server/source/production-deployment-template/troubleshoot.rst b/docs/server/source/k8s-deployment-template/troubleshoot.rst similarity index 100% rename from docs/server/source/production-deployment-template/troubleshoot.rst rename to docs/server/source/k8s-deployment-template/troubleshoot.rst diff --git a/docs/server/source/production-deployment-template/upgrade-on-kubernetes.rst b/docs/server/source/k8s-deployment-template/upgrade-on-kubernetes.rst similarity index 100% rename from docs/server/source/production-deployment-template/upgrade-on-kubernetes.rst rename to docs/server/source/k8s-deployment-template/upgrade-on-kubernetes.rst diff --git a/docs/server/source/production-deployment-template/workflow.rst b/docs/server/source/k8s-deployment-template/workflow.rst similarity index 96% rename from docs/server/source/production-deployment-template/workflow.rst rename to docs/server/source/k8s-deployment-template/workflow.rst index 793175b9..668f1a3c 100644 --- a/docs/server/source/production-deployment-template/workflow.rst +++ b/docs/server/source/k8s-deployment-template/workflow.rst @@ -15,7 +15,7 @@ Overview then this Kubernetes deployment template might be helpful. This page summarizes some steps to go through -to set up a production BigchainDB cluster. +to set up a BigchainDB cluster. You can modify them to suit your needs. .. _generate-the-blockchain-id-and-genesis-time: @@ -123,13 +123,13 @@ and set it equal to your secret token, e.g. 3. Deploy a Kubernetes cluster for your BigchainDB node. We have some instructions for how to -:doc:`Deploy a Kubernetes cluster on Azure <../production-deployment-template/template-kubernetes-azure>`. +:doc:`Deploy a Kubernetes cluster on Azure <../k8s-deployment-template/template-kubernetes-azure>`. .. warning:: In theory, you can deploy your BigchainDB node to any Kubernetes cluster, but there can be differences between different Kubernetes clusters, especially if they are running different versions of Kubernetes. - We tested this Production Deployment Template on Azure ACS in February 2018 and at that time + We tested this Kubernetes Deployment Template on Azure ACS in February 2018 and at that time ACS was deploying a **Kubernetes 1.7.7** cluster. If you can force your cluster to have that version of Kubernetes, then you'll increase the likelihood that everything will work in your cluster. From bdfa059046c8e6f4547eb3fd460a6bcc8ccfbced Mon Sep 17 00:00:00 2001 From: codegeschrei Date: Wed, 8 Aug 2018 12:17:44 +0200 Subject: [PATCH 04/15] Problem: some tests are not activated (#2390) * Problem: core.py contains an unused class, `Bigchain` Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain. * Problem: core.py contains an unused class, `Bigchain` Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain. * Fixed flake8 complaint about too many blank lines * Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI... Sorry in advance! * Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI... Sorry in advance! * Updating from master changed BigchainDB.process_post_response to a private method, so I had to align with that. * Fixed a couple stale references to bigchaindb.Bigchain in docstrings * Missed a reference to `Bigchain` in a patch call... * Problem: some tests are not activated Solution: activate and remove or fix tests * Problem: accidentally un-skipped test_get_blocks_status_containing_tx during merge Solution: Replaced the skip --- tests/test_core.py | 25 ++++++------------------- tests/test_txlist.py | 39 ++++++++++----------------------------- 2 files changed, 16 insertions(+), 48 deletions(-) diff --git a/tests/test_core.py b/tests/test_core.py index 7a648d83..45c369ad 100644 --- a/tests/test_core.py +++ b/tests/test_core.py @@ -1,6 +1,5 @@ import pytest - pytestmark = pytest.mark.tendermint @@ -21,6 +20,10 @@ def config(request, monkeypatch): 'connection_timeout': 5000, 'max_tries': 3 }, + 'tendermint': { + 'host': 'localhost', + 'port': 26657, + }, 'CONFIGURED': True, } @@ -29,7 +32,6 @@ def config(request, monkeypatch): return config -@pytest.mark.skipif(reason='will be fixed in another PR') def test_bigchain_class_default_initialization(config): from bigchaindb import BigchainDB from bigchaindb.consensus import BaseConsensusRules @@ -42,8 +44,7 @@ def test_bigchain_class_default_initialization(config): assert bigchain.consensus == BaseConsensusRules -@pytest.mark.skipif(reason='will be fixed in another PR') -def test_bigchain_class_initialization_with_parameters(config): +def test_bigchain_class_initialization_with_parameters(): from bigchaindb import BigchainDB from bigchaindb.backend import connect from bigchaindb.consensus import BaseConsensusRules @@ -54,7 +55,7 @@ def test_bigchain_class_initialization_with_parameters(config): 'name': 'this_is_the_db_name', } connection = connect(**init_db_kwargs) - bigchain = BigchainDB(connection=connection, **init_db_kwargs) + bigchain = BigchainDB(connection=connection) assert bigchain.connection == connection assert bigchain.connection.host == init_db_kwargs['host'] assert bigchain.connection.port == init_db_kwargs['port'] @@ -62,20 +63,6 @@ def test_bigchain_class_initialization_with_parameters(config): assert bigchain.consensus == BaseConsensusRules -@pytest.mark.skipif(reason='will be fixed in another PR') -def test_get_blocks_status_containing_tx(monkeypatch): - from bigchaindb.backend import query as backend_query - from bigchaindb import BigchainDB - blocks = [ - {'id': 1}, {'id': 2} - ] - monkeypatch.setattr(backend_query, 'get_blocks_status_from_transaction', lambda x: blocks) - monkeypatch.setattr(BigchainDB, 'block_election_status', lambda x, y, z: BigchainDB.BLOCK_VALID) - bigchain = BigchainDB(public_key='pubkey', private_key='privkey') - with pytest.raises(Exception): - bigchain.get_blocks_status_containing_tx('txid') - - @pytest.mark.genesis def test_get_spent_issue_1271(b, alice, bob, carol): from bigchaindb.models import Transaction diff --git a/tests/test_txlist.py b/tests/test_txlist.py index 61c31bb8..7b4f4d5d 100644 --- a/tests/test_txlist.py +++ b/tests/test_txlist.py @@ -4,46 +4,27 @@ This test module defines it's own fixture which is used by all the tests. """ import pytest +pytestmark = pytest.mark.tendermint + @pytest.fixture -def txlist(b, user_pk, user2_pk, user_sk, user2_sk, genesis_block): +def txlist(b, user_pk, user2_pk, user_sk, user2_sk): from bigchaindb.models import Transaction - prev_block_id = genesis_block.id - # Create first block with CREATE transactions + # Create two CREATE transactions create1 = Transaction.create([user_pk], [([user2_pk], 6)]) \ .sign([user_sk]) + create2 = Transaction.create([user2_pk], [([user2_pk], 5), ([user_pk], 5)]) \ - .sign([user2_sk]) - block1 = b.create_block([create1, create2]) - b.write_block(block1) + .sign([user2_sk]) - # Create second block with TRANSFER transactions + # Create a TRANSFER transactions transfer1 = Transaction.transfer(create1.to_inputs(), [([user_pk], 8)], create1.id).sign([user2_sk]) - block2 = b.create_block([transfer1]) - b.write_block(block2) - # Create block with double spend - tx_doublespend = Transaction.transfer(create1.to_inputs(), [([user_pk], 9)], - create1.id).sign([user2_sk]) - block_doublespend = b.create_block([tx_doublespend]) - b.write_block(block_doublespend) - - # Vote on all the blocks - prev_block_id = genesis_block.id - for bid in [block1.id, block2.id]: - vote = b.vote(bid, prev_block_id, True) - prev_block_id = bid - b.write_vote(vote) - - # Create undecided block - untx = Transaction.create([user_pk], [([user2_pk], 7)]) \ - .sign([user_sk]) - block_undecided = b.create_block([untx]) - b.write_block(block_undecided) + b.store_bulk_transactions([create1, create2, transfer1]) return type('', (), { 'create1': create1, @@ -54,8 +35,8 @@ def txlist(b, user_pk, user2_pk, user_sk, user2_sk, genesis_block): @pytest.mark.bdb def test_get_txlist_by_asset(b, txlist): res = b.get_transactions_filtered(txlist.create1.id) - assert set(tx.id for tx in res) == set([txlist.transfer1.id, - txlist.create1.id]) + assert sorted(set(tx.id for tx in res)) == sorted( + set([txlist.transfer1.id, txlist.create1.id])) @pytest.mark.bdb From 66fd00131198fb4140e553f6b5fefb66932fb4a8 Mon Sep 17 00:00:00 2001 From: Troy McConaghy Date: Wed, 8 Aug 2018 13:27:48 +0200 Subject: [PATCH 05/15] Problem: Doc re/ handling external PRs has moved (#2444) Solution: Modify the file `HOW_TO_HANDLE_PULL_REQUESTS.md` to point its readers to the new document about how to handle external pull requests: [BEP-16](https://github.com/bigchaindb/BEPs/tree/master/16). --- HOW_TO_HANDLE_PULL_REQUESTS.md | 81 +--------------------------------- 1 file changed, 2 insertions(+), 79 deletions(-) diff --git a/HOW_TO_HANDLE_PULL_REQUESTS.md b/HOW_TO_HANDLE_PULL_REQUESTS.md index b444f9ce..00ab9335 100644 --- a/HOW_TO_HANDLE_PULL_REQUESTS.md +++ b/HOW_TO_HANDLE_PULL_REQUESTS.md @@ -1,80 +1,3 @@ -# How to Handle Pull Requests +# How to Handle External Pull Requests -This document is for whoever has the ability to merge pull requests in the Git repositories associated with BigchainDB. - -If the pull request is from an employee of BigchainDB GmbH, then you can ignore this document. - -If the pull request is from someone who is _not_ an employee of BigchainDB, then: - -A. Have they agreed to the Individual Contributor Agreement in the past? There's a list of them in [a Google Spreadsheet that's accessible to all bigchaindb.com accounts](https://docs.google.com/spreadsheets/d/1VhekO6lgk1ZPx8dSjriucy4UinaU9pIdPQ5JXKcbD_Y/edit?usp=sharing). If yes, then you can merge the PR and ignore the rest of this document. - -B. Do they belong to a company or organization which agreed to the Entity Contributor Agreement in the past, and will they be contributing on behalf of that company or organization? (See the Google Spreadsheet link in A.) If yes, then you can merge the PR and ignore the rest of this document. - -C. Did they make a pull request to one of the bigchaindb repositories on GitHub (e.g. bigchaindb/bigchaindb)? If you're not sure, or you can't find one, then respond with an email of the form: - -Dear [NAME OF PERSON WHO AGREED TO THE CLA] - -According to the email copied below, you agreed to the BigchainDB Contributor License Agreement (CLA). - -Did you intend to do that? If no, then feel free to ignore this email and we'll pretend it never happened. - -If you did intend to do that, then do you intend to make a pull request in a BigchainDB repository? Maybe you already did? If so, can you please point me to the pull request in question? - -Sincerely, -[INSERT YOUR NAME HERE] - -D. Otherwise, go to the pull request in question and post a comment using this template: - -Hi @nameofuser - -Before we can merge this pull request, we need you or your organization to agree to one of our contributor agreements. One of the big concerns for people using and developing open source software is that someone who contributed to the code might claim the code infringes on their copyright or patent. To guard against this, we ask all our contributors to sign a Contributor License Agreement. This gives us the right to use the code contributed and any patents the contribution relies on. It also gives us and our users comfort that they won't be sued for using open source software. We know it's a hassle, but it makes the project more reliable in the long run. Thank you for your understanding and your contribution! - -If you are contributing on behalf of yourself (and not on behalf of your employer or another organization you are part of) then you should: - -1. Go to: https://www.bigchaindb.com/cla/ -2. Read the Individual Contributor Agreement -3. Fill in the form "For Individuals" -4. Check the box to agree -5. Click the SEND button - -If you're contributing as an employee, and/or you want all employees of your employing organization to be covered by our contributor agreement, then someone in your organization with the authority to enter agreements on behalf of all employees must do the following: - -1. Go to: https://www.bigchaindb.com/cla/ -2. Read the Entity Contributor Agreement -3. Fill in the form "For Organizations” -4. Check the box to agree -5. Click the SEND button - -We will email you (or your employer) with further instructions. - -(END OF COMMENT) - -Once they click SEND, we (BigchainDB) will get an email with the information in the form. (Troy gets those emails for sure, I'm not sure who else.) The next step is to send an email to the email address submitted in the form, saying something like (where the stuff in [square brackets] should be replaced): - -Hi [NAME], - -The next step is for you to copy the following block of text into the comments of Pull Request #[NN] on GitHub: - -BEGIN BLOCK - -This is to confirm that I agreed to and accepted the BigchainDB [Entity/Individual] Contributor Agreement at https://www.bigchaindb.com/cla/ and to represent and warrant that I have authority to do so. - -[Insert long random string here. One good source of those is https://www.grc.com/passwords.htm ] - -END BLOCK - -(END OF EMAIL) - -The next step is to wait for them to copy that comment into the comments of the indicated pull request. Once they do so, it's safe to merge the pull request. - -## How to Handle CLA Agreement Emails with No Associated Pull Request - -Reply with an email like this: - -Hi [First Name], - -Today I got an email (copied below) to tell me that you agreed to the BigchainDB Contributor License Agreement. Did you intend to do that? - -If no, then you can ignore this email. - -If yes, then there's another step to connect your email address with your GitHub account. To do that, you must first create a pull request in one of the BigchainDB repositories on GitHub. Once you've done that, please reply to this email with a link to the pull request. Then I'll send you a special block of text to paste into the comments on that pull request. +See [BEP-16](https://github.com/bigchaindb/BEPs/tree/master/16). From 54b81d3ae8bb95c24c2f2094b5497e9975a19d02 Mon Sep 17 00:00:00 2001 From: Vanshdeep Singh Date: Wed, 8 Aug 2018 14:23:07 +0200 Subject: [PATCH 06/15] Problem: Random failure for event handler test (#2423) Solution: Random failure of test `tests/test_events.py::test_event_handler_raises_when_called_after_start` seem to be because of `get_nowait` which returns way to quickly even when the queue has data. Increasing timeout seems to fix the issue. --- bigchaindb/events.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bigchaindb/events.py b/bigchaindb/events.py index 7959d284..e11037cb 100644 --- a/bigchaindb/events.py +++ b/bigchaindb/events.py @@ -67,7 +67,7 @@ class Exchange: """ try: - self.started_queue.get_nowait() + self.started_queue.get(timeout=1) raise RuntimeError('Cannot create a new subscriber queue while Exchange is running.') except Empty: pass From 44be8f572f0205c8866cb0ddae0620d2ec5730ca Mon Sep 17 00:00:00 2001 From: codegeschrei Date: Wed, 8 Aug 2018 15:08:34 +0200 Subject: [PATCH 07/15] Problem: Block parameters are not required anymore (#2374) * Problem: all blocks are valid Solution: remove the status from transactions * Problem: changed the return type of get_transaction_filtered Solution: return tx class instead of dict --- bigchaindb/lib.py | 35 ++++------------------------ bigchaindb/models.py | 3 --- bigchaindb/web/views/transactions.py | 4 ++-- tests/db/test_bigchain_api.py | 32 ------------------------- tests/web/test_transactions.py | 24 ------------------- 5 files changed, 7 insertions(+), 91 deletions(-) diff --git a/bigchaindb/lib.py b/bigchaindb/lib.py index 13d1ab71..12ab9dc8 100644 --- a/bigchaindb/lib.py +++ b/bigchaindb/lib.py @@ -34,18 +34,6 @@ class BigchainDB(object): Create, read, sign, write transactions to the database """ - BLOCK_INVALID = 'invalid' - """return if a block is invalid""" - - BLOCK_VALID = TX_VALID = 'valid' - """return if a block is valid, or tx is in valid block""" - - BLOCK_UNDECIDED = TX_UNDECIDED = 'undecided' - """return if block is undecided, or tx is in undecided block""" - - TX_IN_BACKLOG = 'backlog' - """return if transaction is in backlog""" - def __init__(self, connection=None): """Initialize the Bigchain instance @@ -251,7 +239,7 @@ class BigchainDB(object): return backend.query.delete_unspent_outputs( self.connection, *unspent_outputs) - def get_transaction(self, transaction_id, include_status=False): + def get_transaction(self, transaction_id): transaction = backend.query.get_transaction(self.connection, transaction_id) if transaction: @@ -269,10 +257,7 @@ class BigchainDB(object): transaction = Transaction.from_dict(transaction) - if include_status: - return transaction, self.TX_VALID if transaction else None - else: - return transaction + return transaction def get_transactions_filtered(self, asset_id, operation=None): """Get a list of transactions filtered on some criteria @@ -280,9 +265,7 @@ class BigchainDB(object): txids = backend.query.get_txids_filtered(self.connection, asset_id, operation) for txid in txids: - tx, status = self.get_transaction(txid, True) - if status == self.TX_VALID: - yield tx + yield self.get_transaction(txid) def get_outputs_filtered(self, owner, spent=None): """Get a list of output links filtered on some criteria @@ -421,16 +404,8 @@ class BigchainDB(object): Returns: iter: An iterator of assets that match the text search. """ - objects = backend.query.text_search(self.connection, search, limit=limit, - table=table) - - # TODO: This is not efficient. There may be a more efficient way to - # query by storing block ids with the assets and using fastquery. - # See https://github.com/bigchaindb/bigchaindb/issues/1496 - for obj in objects: - tx, status = self.get_transaction(obj['id'], True) - if status == self.TX_VALID: - yield obj + return backend.query.text_search(self.connection, search, limit=limit, + table=table) def get_assets(self, asset_ids): """Return a list of assets that match the asset_ids diff --git a/bigchaindb/models.py b/bigchaindb/models.py index dd9a210a..58cc9a99 100644 --- a/bigchaindb/models.py +++ b/bigchaindb/models.py @@ -10,15 +10,12 @@ class Transaction(Transaction): def validate(self, bigchain, current_transactions=[]): """Validate transaction spend - Args: bigchain (BigchainDB): an instantiated bigchaindb.BigchainDB object. - Returns: The transaction (Transaction) if the transaction is valid else it raises an exception describing the reason why the transaction is invalid. - Raises: ValidationError: If the transaction is invalid """ diff --git a/bigchaindb/web/views/transactions.py b/bigchaindb/web/views/transactions.py index 299ace5a..cd64f0a9 100644 --- a/bigchaindb/web/views/transactions.py +++ b/bigchaindb/web/views/transactions.py @@ -29,9 +29,9 @@ class TransactionApi(Resource): pool = current_app.config['bigchain_pool'] with pool() as bigchain: - tx, status = bigchain.get_transaction(tx_id, include_status=True) + tx = bigchain.get_transaction(tx_id) - if not tx or status is not bigchain.TX_VALID: + if not tx: return make_error(404) return tx.to_dict() diff --git a/tests/db/test_bigchain_api.py b/tests/db/test_bigchain_api.py index b1ff8594..2a8ace6a 100644 --- a/tests/db/test_bigchain_api.py +++ b/tests/db/test_bigchain_api.py @@ -280,7 +280,6 @@ class TestBigchainApi(object): @pytest.mark.usefixtures('inputs') def test_write_transaction(self, b, user_pk, user_sk): - from bigchaindb import BigchainDB from bigchaindb.models import Transaction input_tx = b.get_owned_ids(user_pk).pop() @@ -294,7 +293,6 @@ class TestBigchainApi(object): tx_from_db, status = b.get_transaction(tx.id, include_status=True) assert tx_from_db.to_dict() == tx.to_dict() - assert status == BigchainDB.TX_IN_BACKLOG @pytest.mark.usefixtures('inputs') def test_read_transaction(self, b, user_pk, user_sk): @@ -315,7 +313,6 @@ class TestBigchainApi(object): response, status = b.get_transaction(tx.id, include_status=True) # add validity information, which will be returned assert tx.to_dict() == response.to_dict() - assert status == b.TX_UNDECIDED @pytest.mark.usefixtures('inputs') def test_read_transaction_invalid_block(self, b, user_pk, user_sk): @@ -342,35 +339,6 @@ class TestBigchainApi(object): # and a copy of the tx is not in the backlog assert response is None - @pytest.mark.usefixtures('inputs') - def test_read_transaction_invalid_block_and_backlog(self, b, user_pk, user_sk): - from bigchaindb.models import Transaction - - input_tx = b.get_owned_ids(user_pk).pop() - input_tx = b.get_transaction(input_tx.txid) - inputs = input_tx.to_inputs() - tx = Transaction.transfer(inputs, [([user_pk], 1)], - asset_id=input_tx.id) - tx = tx.sign([user_sk]) - - # Make sure there's a copy of tx in the backlog - b.write_transaction(tx) - - # create block - block = b.create_block([tx]) - b.write_block(block) - - # vote the block invalid - vote = b.vote(block.id, b.get_last_voted_block().id, False) - b.write_vote(vote) - - # a copy of the tx is both in the backlog and in an invalid - # block, so get_transaction should return a transaction, - # and a status of TX_IN_BACKLOG - response, status = b.get_transaction(tx.id, include_status=True) - assert tx.to_dict() == response.to_dict() - assert status == b.TX_IN_BACKLOG - @pytest.mark.usefixtures('inputs') def test_genesis_block(self, b): from bigchaindb.backend import query diff --git a/tests/web/test_transactions.py b/tests/web/test_transactions.py index 8582ed16..f1e7d672 100644 --- a/tests/web/test_transactions.py +++ b/tests/web/test_transactions.py @@ -402,30 +402,6 @@ def test_transactions_get_list_bad(client): assert client.get(url).status_code == 400 -@pytest.mark.tendermint -def test_return_only_valid_transaction(client): - from bigchaindb import BigchainDB - - def get_transaction_patched(status): - def inner(self, tx_id, include_status): - return {}, status - return inner - - # NOTE: `get_transaction` only returns a transaction if it's included in an - # UNDECIDED or VALID block, as well as transactions from the backlog. - # As the endpoint uses `get_transaction`, we don't have to test - # against invalid transactions here. - with patch('bigchaindb.BigchainDB.get_transaction', - get_transaction_patched(BigchainDB.TX_UNDECIDED)): - url = '{}{}'.format(TX_ENDPOINT, '123') - assert client.get(url).status_code == 404 - - with patch('bigchaindb.BigchainDB.get_transaction', - get_transaction_patched(BigchainDB.TX_IN_BACKLOG)): - url = '{}{}'.format(TX_ENDPOINT, '123') - assert client.get(url).status_code == 404 - - @pytest.mark.tendermint @patch('requests.post') @pytest.mark.parametrize('mode', [ From ab41b463d81eb12f3b0b51c36dcf1afe841b352f Mon Sep 17 00:00:00 2001 From: Dan Acristinii Date: Wed, 8 Aug 2018 15:45:08 +0200 Subject: [PATCH 08/15] fixed validator curl port (#2447) --- docs/server/source/server-reference/bigchaindb-cli.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/server/source/server-reference/bigchaindb-cli.md b/docs/server/source/server-reference/bigchaindb-cli.md index 7280e2e0..16dbee68 100644 --- a/docs/server/source/server-reference/bigchaindb-cli.md +++ b/docs/server/source/server-reference/bigchaindb-cli.md @@ -94,7 +94,7 @@ Example usage, $ bigchaindb upsert-validator B0E42D2589A455EAD339A035D6CE1C8C3E25863F268120AA0162AD7D003A4014 10 ``` -If the command is returns without any error then a request to update the validator set has been successfully submitted. So, even if the command has been successfully executed it doesn't imply that the validator set has been updated. In order to check whether the change has been applied, the node operator can execute `curl http://node_ip:9985/api/v1/validators` which will list the current validators set. Refer to the [validators](/http-client-server-api.html#validators) section of the HTTP API docs for more detail. +If the command is returns without any error then a request to update the validator set has been successfully submitted. So, even if the command has been successfully executed it doesn't imply that the validator set has been updated. In order to check whether the change has been applied, the node operator can execute `curl http://node_ip:9984/api/v1/validators` which will list the current validators set. Refer to the [validators](/http-client-server-api.html#validators) section of the HTTP API docs for more detail. Note: - When `POWER`is set to `0` then the validator will be removed from the validator set. From 65b6040e6b6fa9c2e8d61f7f9f47038256ad043b Mon Sep 17 00:00:00 2001 From: Troy McConaghy Date: Wed, 8 Aug 2018 16:46:23 +0200 Subject: [PATCH 09/15] Problem: Codecov shield commented-out in README.md (#2421) Solution: Uncomment it, since Codecov is working again ![Codecov](https://img.shields.io/codecov/c/github/bigchaindb/bigchaindb/master.svg) --- README.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/README.md b/README.md index 77f26884..d9d907a8 100644 --- a/README.md +++ b/README.md @@ -2,9 +2,8 @@ (including pre-release versions) from PyPI, so show the latest GitHub release instead. ---> - [![Latest release](https://img.shields.io/github/release/bigchaindb/bigchaindb/all.svg)](https://github.com/bigchaindb/bigchaindb/releases) [![Status on PyPI](https://img.shields.io/pypi/status/bigchaindb.svg)](https://pypi.org/project/BigchainDB/) [![Travis branch](https://img.shields.io/travis/bigchaindb/bigchaindb/master.svg)](https://travis-ci.org/bigchaindb/bigchaindb) From 4806b815774473b1fc41f10ca607cbb344b2e539 Mon Sep 17 00:00:00 2001 From: Troy McConaghy Date: Thu, 9 Aug 2018 11:14:48 +0200 Subject: [PATCH 10/15] Problem: Inconsistent HTTP API docs (#2403) - Make the title of the asset search section consistent with the example (and consistent with the metadata search title and example). - Add a note that one can use `asset?search` or `asset/?search`, and the same for metadata. --- docs/server/source/http-client-server-api.rst | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/docs/server/source/http-client-server-api.rst b/docs/server/source/http-client-server-api.rst index b413430b..fd5c121d 100644 --- a/docs/server/source/http-client-server-api.rst +++ b/docs/server/source/http-client-server-api.rst @@ -301,7 +301,7 @@ Assets Currently this endpoint is only supported if using MongoDB. -.. http:get:: /api/v1/assets?search={search} +.. http:get:: /api/v1/assets/?search={search} Return all assets that match a given text search. @@ -310,6 +310,10 @@ Assets The ``id`` of the asset is the same ``id`` of the CREATE transaction that created the asset. + .. note:: + + You can use ``assets/?search`` or ``assets?search``. + If no assets match the text search it returns an empty list. If the text string is empty or the server does not support text search, @@ -425,6 +429,10 @@ Transaction Metadata The ``id`` of the metadata is the same ``id`` of the transaction where it was defined. + .. note:: + + You can use ``metadata/?search`` or ``metadata?search``. + If no metadata objects match the text search it returns an empty list. If the text string is empty or the server does not support text search, From 205e2cf3fd003ab9c54303392383d88a816cdd6f Mon Sep 17 00:00:00 2001 From: Troy McConaghy Date: Thu, 9 Aug 2018 11:16:05 +0200 Subject: [PATCH 11/15] Problem: Immutability page in docs sometimes wrong or controversial (#2435) Solution: Edit that page --- docs/root/source/immutable.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/docs/root/source/immutable.md b/docs/root/source/immutable.md index 8791c04e..c1aeb9f6 100644 --- a/docs/root/source/immutable.md +++ b/docs/root/source/immutable.md @@ -2,13 +2,14 @@ The word _immutable_ means "unchanging over time or unable to be changed." For example, the decimal digits of π are immutable (3.14159…). -The blockchain community often describes blockchains as “immutable.” If we interpret that word literally, it means that blockchain data is unchangeable or permanent, which is absurd. The data _can_ be changed. For example, a plague might drive humanity extinct; the data would then get corrupted over time due to water damage, thermal noise, and the general increase of entropy. In the case of Bitcoin, nothing so drastic is required: a 51% attack will suffice. +The blockchain community often describes blockchains as “immutable.” If we interpret that word literally, it means that blockchain data is unchangeable or permanent, which is absurd. The data _can_ be changed. For example, a plague might drive humanity extinct; the data would then get corrupted over time due to water damage, thermal noise, and the general increase of entropy. It’s true that blockchain data is more difficult to change (or delete) than usual. It's more than just "tamper-resistant" (which implies intent), blockchain data also resists random changes that can happen without any intent, such as data corruption on a hard drive. Therefore, in the context of blockchains, we interpret the word “immutable” to mean *practically* immutable, for all intents and purposes. (Linguists would say that the word “immutable” is a _term of art_ in the blockchain community.) -Blockchain data can achieve immutability in several ways: +Blockchain data can be made immutable in several ways: -1. **Replication.** All data is replicated (copied) to several different places. The replication factor can be set by the consortium. The higher the replication factor, the more difficult it becomes to change or delete all replicas. +1. **No APIs for changing or deleting data.** Blockchain software usually doesn't expose any APIs for changing or deleting the data stored in the blockchain. BigchainDB has no such APIs. This doesn't prevent changes or deletions from happening in _other_ ways; it's just one line of defense. +1. **Replication.** All data is replicated (copied) to several different places. The higher the replication factor, the more difficult it becomes to change or delete all replicas. 1. **Internal watchdogs.** All nodes monitor all changes and if some unallowed change happens, then appropriate action can be taken. 1. **External watchdogs.** A consortium may opt to have trusted third-parties to monitor and audit their data, looking for irregularities. For a consortium with publicly-readable data, the public can act as an auditor. 1. **Economic incentives.** Some blockchain systems make it very expensive to change old stored data. Examples include proof-of-work and proof-of-stake systems. BigchainDB doesn't use explicit incentives like those. @@ -17,5 +18,3 @@ Blockchain data can achieve immutability in several ways: 1. **Full or partial backups** may be recorded from time to time, possibly on magnetic tape storage, other blockchains, printouts, etc. 1. **Strong security.** Node owners can adopt and enforce strong security policies. 1. **Node diversity.** Diversity makes it so that no one thing (e.g. natural disaster or operating system bug) can compromise enough of the nodes. See [the section on the kinds of node diversity](diversity.html). - -Some of these things come "for free" as part of the BigchainDB software, and others require some extra effort from the consortium and node owners. From 4636a489186076846be71380a68431114dac8e9f Mon Sep 17 00:00:00 2001 From: Troy McConaghy Date: Thu, 9 Aug 2018 11:16:56 +0200 Subject: [PATCH 12/15] Problem: Glossary no longer referenced by any docs (#2441) Solution: Remove Glossary section from the server docs --- docs/server/source/glossary.rst | 19 ------------------- docs/server/source/index.rst | 1 - 2 files changed, 20 deletions(-) delete mode 100644 docs/server/source/glossary.rst diff --git a/docs/server/source/glossary.rst b/docs/server/source/glossary.rst deleted file mode 100644 index 8fc03ac2..00000000 --- a/docs/server/source/glossary.rst +++ /dev/null @@ -1,19 +0,0 @@ -Glossary -======== - -.. glossary:: - :sorted: - - associative array - A collection of key/value (or name/value) pairs - such that each possible key appears at most once - in the collection. - In JavaScript (and JSON), all objects behave as associative arrays - with string-valued keys. - In Python and .NET, associative arrays are called *dictionaries*. - In Java and Go, they are called *maps*. - In Ruby, they are called *hashes*. - See also: Wikipedia's articles for - `Associative array `_ - and - `Comparison of programming languages (associative array) `_ diff --git a/docs/server/source/index.rst b/docs/server/source/index.rst index be9d1a10..f8a4cb96 100644 --- a/docs/server/source/index.rst +++ b/docs/server/source/index.rst @@ -18,5 +18,4 @@ BigchainDB Server Documentation data-models/index k8s-deployment-template/index release-notes - glossary appendices/index From 30115483174b12ba9a9291e97cf6e5600401ce9d Mon Sep 17 00:00:00 2001 From: Muawia Khan Date: Thu, 9 Aug 2018 15:17:46 +0200 Subject: [PATCH 13/15] Problem: remove autodraft from download link (#2454) --- Dockerfile-all-in-one | 2 +- .../run-node-as-processes.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Dockerfile-all-in-one b/Dockerfile-all-in-one index 6c50d971..0ccde6bb 100644 --- a/Dockerfile-all-in-one +++ b/Dockerfile-all-in-one @@ -21,7 +21,7 @@ RUN apk --update add sudo bash \ RUN apk --update add mongodb monit # Install Tendermint -RUN wget https://github.com/tendermint/tendermint/releases/download/v${TM_VERSION}-autodraft/tendermint_${TM_VERSION}_linux_amd64.zip \ +RUN wget https://github.com/tendermint/tendermint/releases/download/v${TM_VERSION}/tendermint_${TM_VERSION}_linux_amd64.zip \ && unzip tendermint_${TM_VERSION}_linux_amd64.zip \ && mv tendermint /usr/local/bin/ \ && rm tendermint_${TM_VERSION}_linux_amd64.zip diff --git a/docs/contributing/source/dev-setup-coding-and-contribution-process/run-node-as-processes.md b/docs/contributing/source/dev-setup-coding-and-contribution-process/run-node-as-processes.md index 4c98578a..d39b2b9f 100644 --- a/docs/contributing/source/dev-setup-coding-and-contribution-process/run-node-as-processes.md +++ b/docs/contributing/source/dev-setup-coding-and-contribution-process/run-node-as-processes.md @@ -23,7 +23,7 @@ Find [the version number of the latest Tendermint release](https://github.com/te ```bash $ sudo apt install -y unzip -$ wget https://github.com/tendermint/tendermint/releases/download/v0.22.8-autodraft/tendermint_0.22.8_linux_amd64.zip +$ wget https://github.com/tendermint/tendermint/releases/download/v0.22.8/tendermint_0.22.8_linux_amd64.zip $ unzip tendermint_0.22.8_linux_amd64.zip $ rm tendermint_0.22.8_linux_amd64.zip $ sudo mv tendermint /usr/local/bin From 37608242611000de055b0602b149519a02d92f99 Mon Sep 17 00:00:00 2001 From: Zachary Bowen Date: Thu, 9 Aug 2018 17:29:21 +0200 Subject: [PATCH 14/15] Create dynamic upsert validator commands (#2446) * Problem: Need a method to initiate a new upsert-validator election Solution: Added subcommand `new` to `upsert_validator` * Problem: Changes to upsert-validator needed to be reflected in the docs Solution: Wrote a section for `upsert-validator new` --- Dockerfile-dev | 1 - bigchaindb/commands/bigchaindb.py | 91 ++++++++++++++----- bigchaindb/lib.py | 1 + bigchaindb/utils.py | 16 +++- .../source/server-reference/bigchaindb-cli.md | 23 +++-- tests/commands/test_commands.py | 74 +++++++++++++-- tests/conftest.py | 27 +++++- 7 files changed, 187 insertions(+), 46 deletions(-) diff --git a/Dockerfile-dev b/Dockerfile-dev index 05e4bf98..4425e0d2 100644 --- a/Dockerfile-dev +++ b/Dockerfile-dev @@ -34,4 +34,3 @@ COPY . /usr/src/app/ WORKDIR /usr/src/app RUN pip install --no-cache-dir --process-dependency-links -e .[dev] RUN bigchaindb -y configure - diff --git a/bigchaindb/commands/bigchaindb.py b/bigchaindb/commands/bigchaindb.py index 379b89a2..63394df1 100644 --- a/bigchaindb/commands/bigchaindb.py +++ b/bigchaindb/commands/bigchaindb.py @@ -9,19 +9,19 @@ import copy import json import sys +from bigchaindb.utils import load_node_key from bigchaindb.common.exceptions import (DatabaseAlreadyExists, DatabaseDoesNotExist, - MultipleValidatorOperationError) + OperationError) import bigchaindb -from bigchaindb import backend +from bigchaindb import backend, ValidatorElection, BigchainDB from bigchaindb.backend import schema from bigchaindb.backend import query -from bigchaindb.backend.query import VALIDATOR_UPDATE_ID, PRE_COMMIT_ID +from bigchaindb.backend.query import PRE_COMMIT_ID from bigchaindb.commands import utils from bigchaindb.commands.utils import (configure_bigchaindb, input_on_stderr) from bigchaindb.log import setup_logging -from bigchaindb.tendermint_utils import public_key_from_base64 logging.basicConfig(level=logging.INFO) logger = logging.getLogger(__name__) @@ -95,21 +95,48 @@ def run_configure(args): @configure_bigchaindb def run_upsert_validator(args): - """Store validators which should be synced with Tendermint""" + """Initiate and manage elections to change the validator set""" - b = bigchaindb.BigchainDB() - public_key = public_key_from_base64(args.public_key) - validator = {'pub_key': {'type': 'ed25519', - 'data': public_key}, - 'power': args.power} - validator_update = {'validator': validator, - 'update_id': VALIDATOR_UPDATE_ID} - try: - query.store_validator_update(b.connection, validator_update) - except MultipleValidatorOperationError: - logger.error('A validator update is pending to be applied. ' - 'Please re-try after the current update has ' - 'been processed.') + b = BigchainDB() + + # Call the function specified by args.action, as defined above + globals()[f'run_upsert_validator_{args.action}'](args, b) + + +def run_upsert_validator_new(args, bigchain): + """Initiates an election to add/update/remove a validator to an existing BigchainDB network + + :param args: dict + args = { + 'public_key': the public key of the proposed peer, (str) + 'power': the proposed validator power for the new peer, (str) + 'node_id': the node_id of the new peer (str) + 'sk': the path to the private key of the node calling the election (str) + } + :param bigchain: an instance of BigchainDB + :return: election_id (tx_id) + :raises: OperationError if the write transaction fails for any reason + """ + + new_validator = { + 'public_key': args.public_key, + 'power': args.power, + 'node_id': args.node_id + } + + key = load_node_key(args.sk) + + voters = ValidatorElection.recipients(bigchain) + + election = ValidatorElection.generate([key.public_key], + voters, + new_validator, None).sign([key.private_key]) + election.validate(bigchain) + resp = bigchain.write_transaction(election, 'broadcast_tx_commit') + if resp == (202, ''): + return election.id + else: + raise OperationError('Failed to commit election') def _run_init(): @@ -208,16 +235,30 @@ def create_parser(): help='The backend to use. It can only be ' '"localmongodb", currently.') + # parser for managing validator elections validator_parser = subparsers.add_parser('upsert-validator', - help='Add/update/delete a validator') + help='Add/update/delete a validator.') - validator_parser.add_argument('public_key', - help='Public key of the validator.') + validator_subparser = validator_parser.add_subparsers(title='Action', + dest='action') - validator_parser.add_argument('power', - type=int, - help='Voting power of the validator. ' - 'Setting it to 0 will delete the validator.') + new_election_parser = validator_subparser.add_parser('new', + help='Calls a new election.') + + new_election_parser.add_argument('public_key', + help='Public key of the validator to be added/updated/removed.') + + new_election_parser.add_argument('power', + type=int, + help='The proposed power for the validator. ' + 'Setting to 0 will remove the validator.') + + new_election_parser.add_argument('node_id', + help='The node_id of the validator.') + + new_election_parser.add_argument('--private-key', + dest='sk', + help='Path to the private key of the election initiator.') # parsers for showing/exporting config values subparsers.add_parser('show-config', diff --git a/bigchaindb/lib.py b/bigchaindb/lib.py index 12ab9dc8..a0b7e727 100644 --- a/bigchaindb/lib.py +++ b/bigchaindb/lib.py @@ -440,6 +440,7 @@ class BigchainDB(object): validators = result['validators'] for v in validators: v.pop('address') + v['voting_power'] = int(v['voting_power']) return validators diff --git a/bigchaindb/utils.py b/bigchaindb/utils.py index 8c4180d0..ba781214 100644 --- a/bigchaindb/utils.py +++ b/bigchaindb/utils.py @@ -2,9 +2,13 @@ import contextlib import threading import queue import multiprocessing as mp +import json import setproctitle +from bigchaindb.tendermint_utils import key_from_base64 +from bigchaindb.common.crypto import key_pair_from_ed25519_key + class ProcessGroup(object): @@ -31,7 +35,8 @@ class ProcessGroup(object): class Process(mp.Process): """Wrapper around multiprocessing.Process that uses setproctitle to set the name of the process when running - the target task.""" + the target task. + """ def run(self): setproctitle.setproctitle(self.name) @@ -167,3 +172,12 @@ class Lazy: self.stack = [] return last + + +# Load Tendermint's public and private key from the file path +def load_node_key(path): + with open(path) as json_data: + priv_validator = json.load(json_data) + priv_key = priv_validator['priv_key']['value'] + hex_private_key = key_from_base64(priv_key) + return key_pair_from_ed25519_key(hex_private_key) diff --git a/docs/server/source/server-reference/bigchaindb-cli.md b/docs/server/source/server-reference/bigchaindb-cli.md index 16dbee68..439e7160 100644 --- a/docs/server/source/server-reference/bigchaindb-cli.md +++ b/docs/server/source/server-reference/bigchaindb-cli.md @@ -80,22 +80,27 @@ configuration file as documented under **This is an experimental feature. Users are advised not to use it in production.** -Add, update, or remove a validator from the validators set of the local node. The command implements [3/UPSERT-VALIDATORS](https://github.com/bigchaindb/BEPs/tree/master/3), check it out if you need more details on how this is orchestrated. +Manage elections to add, update, or remove a validator from the validators set of the local node. The upsert-validator subcommands implement [BEP-21](https://github.com/bigchaindb/BEPs/tree/master/21). Check it out if you need more details on how this is orchestrated. -Below is the command line syntax, +Election management is broken into several subcommands. Below is the command line syntax for each, + +#### upsert-validator new + +Calls a new election, proposing a change to the validator set. + +Below is the command line syntax and the return value, ```bash -$ bigchaindb upsert-validator PUBLIC_KEY_OF_VALIDATOR POWER +$ bigchaindb upsert-validator new E_PUBKEY E_POWER E_NODE_ID --private-key PATH_TO_YOUR_PRIVATE_KEY + ``` +Here, `E_PUBKEY`, `E_POWER`, and `E_NODE_ID` are the public key, proposed power, and node id of the validator being voted on. `--private-key` should be the path to wherever the private key for your validator node is stored, (*not* the private key itself.). For example, to add a new validator, provide the public key and node id for some node not already in the validator set, along with whatever voting power you'd like them to have. To remove an existing validator, provide their public key and node id, and set `E_POWER` to `0`. + Example usage, ```bash -$ bigchaindb upsert-validator B0E42D2589A455EAD339A035D6CE1C8C3E25863F268120AA0162AD7D003A4014 10 +$ bigchaindb upsert-validator new B0E42D2589A455EAD339A035D6CE1C8C3E25863F268120AA0162AD7D003A4014 1 12345 --private-key /home/user/.tendermint/config/priv_validator.json ``` -If the command is returns without any error then a request to update the validator set has been successfully submitted. So, even if the command has been successfully executed it doesn't imply that the validator set has been updated. In order to check whether the change has been applied, the node operator can execute `curl http://node_ip:9984/api/v1/validators` which will list the current validators set. Refer to the [validators](/http-client-server-api.html#validators) section of the HTTP API docs for more detail. - -Note: -- When `POWER`is set to `0` then the validator will be removed from the validator set. -- Upsert requests are handled once per block i.e. the validators set is updated once a new block is committed. So, the node operator is not allowed to submit a new upsert request until the current request has been processed. Furthermore, if Tendermint is started with `--consensus.create_empty_blocks=false`, and there are no new incoming transactions then the validators set update is delayed until any new transactions are received and a new block can be committed. +If the command succeeds, it will create an election and return an `election_id`. Elections consist of one vote token per voting power, issued to the members of the validator set. Validators can cast their votes to approve the change to the validator set by spending their vote tokens. The status of the election can be monitored by providing the `election_id` to the `show` subcommand. diff --git a/tests/commands/test_commands.py b/tests/commands/test_commands.py index a7c6418b..10e41205 100644 --- a/tests/commands/test_commands.py +++ b/tests/commands/test_commands.py @@ -18,7 +18,8 @@ def test_make_sure_we_dont_remove_any_command(): assert parser.parse_args(['init']).command assert parser.parse_args(['drop']).command assert parser.parse_args(['start']).command - assert parser.parse_args(['upsert-validator', 'TEMP_PUB_KEYPAIR', '10']).command + assert parser.parse_args(['upsert-validator', 'new', 'TEMP_PUB_KEYPAIR', '10', 'TEMP_NODE_ID', + '--private-key', 'TEMP_PATH_TO_PRIVATE_KEY']).command @pytest.mark.tendermint @@ -341,15 +342,70 @@ class MockResponse(): return {'result': {'latest_block_height': self.height}} +# @pytest.mark.execute +# @patch('bigchaindb.lib.BigchainDB.get_validators') +# @pytest.mark.abci @pytest.mark.skip -@patch('bigchaindb.config_utils.autoconfigure') -@patch('bigchaindb.backend.query.store_validator_update') +def test_upsert_validator_new_with_tendermint(b, priv_validator_path, user_sk, monkeypatch): + """WIP: Will be fixed and activated in the next PR + """ + from bigchaindb.commands.bigchaindb import run_upsert_validator_new + import time + + time.sleep(3) + + def mock_get(): + return [ + {'pub_key': {'value': 'zL/DasvKulXZzhSNFwx4cLRXKkSM9GPK7Y0nZ4FEylM=', + 'type': 'tendermint/PubKeyEd25519'}, + 'voting_power': 10} + ] + + # b.get_validators = mock_get + # mock_get_validators = mock_get + # monkeypatch.setattr('requests.get', mock_get) + + proposer_key = b.get_validators()[0]['pub_key']['value'] + + args = Namespace(action='new', + public_key=proposer_key, + power=1, + node_id='12345', + sk=priv_validator_path, + config={}) + resp = run_upsert_validator_new(args, b) + time.sleep(3) + + assert b.get_transaction(resp) + + @pytest.mark.tendermint -def test_upsert_validator(mock_autoconfigure, mock_store_validator_update): - from bigchaindb.commands.bigchaindb import run_upsert_validator +@pytest.mark.bdb +def test_upsert_validator_new_without_tendermint(b, priv_validator_path, user_sk, monkeypatch): + from bigchaindb.commands.bigchaindb import run_upsert_validator_new - args = Namespace(public_key='CJxdItf4lz2PwEf4SmYNAu/c/VpmX39JEgC5YpH7fxg=', - power='10', config={}) - run_upsert_validator(args) + def mock_get(): + return [ + {'pub_key': {'value': 'zL/DasvKulXZzhSNFwx4cLRXKkSM9GPK7Y0nZ4FEylM=', + 'type': 'tendermint/PubKeyEd25519'}, + 'voting_power': 10} + ] - assert mock_store_validator_update.called + def mock_write(tx, mode): + b.store_transaction(tx) + return (202, '') + + b.get_validators = mock_get + b.write_transaction = mock_write + + monkeypatch.setattr('requests.get', mock_get) + + args = Namespace(action='new', + public_key='CJxdItf4lz2PwEf4SmYNAu/c/VpmX39JEgC5YpH7fxg=', + power=1, + node_id='12345', + sk=priv_validator_path, + config={}) + resp = run_upsert_validator_new(args, b) + + assert b.get_transaction(resp) diff --git a/tests/conftest.py b/tests/conftest.py index 66d85d3a..7cb33b14 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -4,10 +4,11 @@ Tasks: 1. setup test database before starting the tests 2. delete test database after running the tests """ - +import json import os import copy import random +import tempfile from collections import namedtuple from logging import getLogger from logging.config import dictConfig @@ -668,3 +669,27 @@ def node_keys(): '83VINXdj2ynOHuhvSZz5tGuOE5oYzIi0mEximkX1KYMlt/Csu8JUjA4+by2Pz3fqSLshhuYYeM+IpvqcBl6BEA==', 'PecJ58SaNRsWJZodDmqjpCWqG6btdwXFHLyE40RYlYM=': 'uz8bYgoL4rHErWT1gjjrnA+W7bgD/uDQWSRKDmC8otc95wnnxJo1GxYlmh0OaqOkJaobpu13BcUcvITjRFiVgw=='} + + +@pytest.fixture(scope='session') +def priv_validator_path(node_keys): + (public_key, private_key) = list(node_keys.items())[0] + priv_validator = { + 'address': '84F787D95E196DC5DE5F972666CFECCA36801426', + 'pub_key': { + 'type': 'AC26791624DE60', + 'value': public_key + }, + 'last_height': 0, + 'last_round': 0, + 'last_step': 0, + 'priv_key': { + 'type': '954568A3288910', + 'value': private_key + } + } + fd, path = tempfile.mkstemp() + socket = os.fdopen(fd, 'w') + json.dump(priv_validator, socket) + socket.close() + return path From bbf5310ac85056277e2e4c1cdc1842370674cf45 Mon Sep 17 00:00:00 2001 From: Troy McConaghy Date: Fri, 10 Aug 2018 11:39:44 +0200 Subject: [PATCH 15/15] Problem: community drivers not working w/ latest BDB (#2443) Solution: In the docs page listing the Community-Driven Libraries and Tools, make it clear that they might not work with the latest version of BigchainDB. Make it a warning rather than a note. --- docs/server/source/drivers-clients/index.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/server/source/drivers-clients/index.rst b/docs/server/source/drivers-clients/index.rst index 3cf4d0fa..79954971 100644 --- a/docs/server/source/drivers-clients/index.rst +++ b/docs/server/source/drivers-clients/index.rst @@ -11,10 +11,11 @@ Libraries and Tools Maintained by the BigchainDB Team Community-Driven Libraries and Tools ------------------------------------ -.. note:: +.. warning:: Some of these projects are a work in progress, but may still be useful. + Others might not work with the latest version of BigchainDB. * `Haskell transaction builder `_ * `Go driver `_