From a005cca3b90ef32d559578d5859f8c36d9d8c8ea Mon Sep 17 00:00:00 2001 From: troymc Date: Sun, 18 Dec 2016 10:44:57 +0100 Subject: [PATCH 1/7] Initial edits to tests/README.md --- tests/README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tests/README.md b/tests/README.md index 8e66a708..1c0a734e 100644 --- a/tests/README.md +++ b/tests/README.md @@ -1,14 +1,14 @@ -# Tests +# BigchainDB Server Tests -## Test Structure +## The bigchaindb/tests/ Folder -Generally all tests are meant to be unit tests, with the exception of those in the [`integration/` folder](./integration/). +The `bigchaindb/tests/` folder is where all the tests for BigchainDB Server live. Most of them are unit tests. Integration tests are in the [`bigchaindb/tests/integration/` folder](./integration/). A few notes: -- [`common/`](./common/) contains self-contained tests only testing +- [`bigchaindb/tests/common/`](./common/) contains self-contained tests only testing [`bigchaindb/common/`](../bigchaindb/common/) -- [`db/`](./db/) contains tests requiring the database backend (e.g. RethinkDB) +- [`bigchaindb/tests/db/`](./db/) contains tests requiring the database backend (e.g. RethinkDB) ## Pytest Customizations From dd01dd099fbc5f10acd233c07fbb40459ab44fb1 Mon Sep 17 00:00:00 2001 From: troymc Date: Sun, 18 Dec 2016 11:01:15 +0100 Subject: [PATCH 2/7] Moved tests docs from PYTHON_STYLE_GUIDE.md to tests/README.md --- PYTHON_STYLE_GUIDE.md | 48 +++-------------------------------- tests/README.md | 59 +++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 60 insertions(+), 47 deletions(-) diff --git a/PYTHON_STYLE_GUIDE.md b/PYTHON_STYLE_GUIDE.md index 42f18c85..1c29117e 100644 --- a/PYTHON_STYLE_GUIDE.md +++ b/PYTHON_STYLE_GUIDE.md @@ -65,7 +65,7 @@ x = 'name: {}; score: {}'.format(name, n) we use the `format()` version. The [official Python documentation says](https://docs.python.org/2/library/stdtypes.html#str.format), "This method of string formatting is the new standard in Python 3, and should be preferred to the % formatting described in String Formatting Operations in new code." -## Runnng the Flake8 Style Checker +## Running the Flake8 Style Checker We use [Flake8](http://flake8.pycqa.org/en/latest/index.html) to check our Python code style. Once you have it installed, you can run it using: ```text @@ -75,48 +75,6 @@ flake8 --max-line-length 119 bigchaindb/ ## Writing and Running (Python) Tests -We write unit and integration tests for our Python code using the [pytest](http://pytest.org/latest/) framework. +The content of this section was moved to [`bigchiandb/tests/README.md`](./tests/README.md). -All tests go in the `bigchaindb/tests` directory or one of its subdirectories. You can use the tests already in there as templates or examples. - -You can run all tests using: -```text -py.test -v -``` - -or, if that doesn't work, try: -```text -python -m pytest -v -``` - -or: -```text -python setup.py test -``` - -If you want to learn about all the things you can do with pytest, see [the pytest documentation](http://pytest.org/latest/). - -### Tox - -We use [tox](https://tox.readthedocs.io/en/latest/) to run multiple suites of tests against multiple environments during automated testing. Generally you don't need to run this yourself, but it might be useful when troubleshooting a failing CI build. - -To run all the tox tests, use: -```text -tox -``` - -or: -```text -python -m tox -``` - -To run only a few environments, use the `-e` flag: -```text -tox -e {ENVLIST} -``` - -where `{ENVLIST}` is one or more of the environments specified in the [tox.ini file](tox.ini). - -### Automated testing of pull requests - -We use [Travis CI](https://travis-ci.com/), so that whenever someone creates a new BigchainDB pull request on GitHub, Travis CI gets the new code and does _a bunch of stuff_. You can find out what we tell Travis CI to do in [the `.travis.yml` file](.travis.yml): it tells Travis CI how to install BigchainDB, how to run all the tests, and what to do "after success" (e.g. run `codecov`). (We use [Codecov](https://codecov.io/) to get a rough estimate of our test coverage.) +Note: We automatically run all tests on all pull requests (using Travis CI), so you should definitely run all tests locally before you submit a pull request. See the above-linked README file for instructions. \ No newline at end of file diff --git a/tests/README.md b/tests/README.md index 1c0a734e..d68788bc 100644 --- a/tests/README.md +++ b/tests/README.md @@ -10,9 +10,64 @@ A few notes: [`bigchaindb/common/`](../bigchaindb/common/) - [`bigchaindb/tests/db/`](./db/) contains tests requiring the database backend (e.g. RethinkDB) -## Pytest Customizations -Customizations we've added to `pytest`: +## Writing Tests + +We write unit and integration tests for our Python code using the [pytest](http://pytest.org/latest/) framework. You can use the tests in the `bigchaindb/tests/` folder as templates or examples. + + +## Running Tests + +You can run all tests using: +```text +py.test -v +``` + +or, if that doesn't work, try: +```text +python -m pytest -v +``` + +or: +```text +python setup.py test +``` + +If you want to learn about all the things you can do with pytest, see [the pytest documentation](http://pytest.org/latest/). + + +### Our pytest Customizations + +Customizations we've added to pytest: - `--database-backend`: Defines the backend to use for the tests. Must be one of the backends available in the [server configuration](https://docs.bigchaindb.com/projects/server/en/latest/server-reference/configuration.html) + + + + +## Automated testing of pull requests + +We use [Travis CI](https://travis-ci.com/), so that whenever someone creates a new BigchainDB pull request on GitHub, Travis CI gets the new code and does _a bunch of stuff_. You can find out what we tell Travis CI to do in [the `.travis.yml` file](.travis.yml): it tells Travis CI how to install BigchainDB, how to run all the tests, and what to do "after success" (e.g. run `codecov`). (We use [Codecov](https://codecov.io/) to get a rough estimate of our test coverage.) + + +### Tox + +We use [tox](https://tox.readthedocs.io/en/latest/) to run multiple suites of tests against multiple environments during automated testing. Generally you don't need to run this yourself, but it might be useful when troubleshooting a failing CI build. + +To run all the tox tests, use: +```text +tox +``` + +or: +```text +python -m tox +``` + +To run only a few environments, use the `-e` flag: +```text +tox -e {ENVLIST} +``` + +where `{ENVLIST}` is one or more of the environments specified in the [tox.ini file](tox.ini). \ No newline at end of file From dbe3c15d5177d115c09ba6c53320d34be7d7862c Mon Sep 17 00:00:00 2001 From: troymc Date: Sun, 18 Dec 2016 11:08:32 +0100 Subject: [PATCH 3/7] CONTRIBUTING.md now points to tests/README.md re: tests --- CONTRIBUTING.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index aac4d80d..03d02403 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -28,7 +28,8 @@ To contribute code or documentation, you need a [GitHub account](https://github. Familiarize yourself with how we do coding and documentation in the BigchainDB project, including: -* [our Python Style Guide](PYTHON_STYLE_GUIDE.md) (includes how we write tests) +* [our Python Style Guide](PYTHON_STYLE_GUIDE.md) +* [how we write and run tests](./tests/README.md) * [our documentation strategy](./docs/README.md) (including in-code documentation) * the GitHub Flow (workflow) * [GitHub Guide: Understanding the GitHub Flow](https://guides.github.com/introduction/flow/) @@ -103,7 +104,7 @@ git checkout -b new-branch-name With your new branch checked out locally, make changes or additions to the code or documentation. Remember to: * follow [our Python Style Guide](PYTHON_STYLE_GUIDE.md). -* write and run tests for any new or changed code. There's a section in [our Python Style Guide](PYTHON_STYLE_GUIDE.md) about writing and running tests. +* write and run tests for any new or changed code. There's [a README file in the `tests/` folder](./tests/README.md) about how to do that. * add or update documentation as necessary. Follow [our documentation strategy](./docs/README.md). As you go, git add and git commit your changes or additions, e.g. @@ -119,7 +120,7 @@ git fetch upstream git merge upstream/master ``` -Once you're done commiting a set of new things and you're ready to submit them for inclusion, please be sure to run all the tests (as per the instructions at the end of our [Python Style Guide](PYTHON_STYLE_GUIDE.md)). +Once you're done commiting a set of new things and you're ready to submit them for inclusion, please be sure to run all the tests as per the instructions in [the README file in the `tests/` folder](./tests/README.md). (When you submit your pull request [following the instructions below], we run all the tests automatically, so we will see if some are failing. If you don't know why some tests are failing, you can still submit your pull request, but be sure to note the failing tests and to ask for help with resolving them.) From 8fe0ece0010021e89757d3d89f3c0db9867745b2 Mon Sep 17 00:00:00 2001 From: troymc Date: Sun, 18 Dec 2016 11:13:54 +0100 Subject: [PATCH 4/7] Renamed docs page 'Running Unit Tests' > 'Running All Tests' --- docs/server/source/dev-and-test/index.rst | 2 +- ...{running-unit-tests.md => running-all-tests.md} | 14 +++++++------- 2 files changed, 8 insertions(+), 8 deletions(-) rename docs/server/source/dev-and-test/{running-unit-tests.md => running-all-tests.md} (61%) diff --git a/docs/server/source/dev-and-test/index.rst b/docs/server/source/dev-and-test/index.rst index 4abc56de..f79c20a5 100644 --- a/docs/server/source/dev-and-test/index.rst +++ b/docs/server/source/dev-and-test/index.rst @@ -5,4 +5,4 @@ Develop & Test BigchainDB Server :maxdepth: 1 setup-run-node - running-unit-tests + running-all-tests diff --git a/docs/server/source/dev-and-test/running-unit-tests.md b/docs/server/source/dev-and-test/running-all-tests.md similarity index 61% rename from docs/server/source/dev-and-test/running-unit-tests.md rename to docs/server/source/dev-and-test/running-all-tests.md index c3347d79..f8e48422 100644 --- a/docs/server/source/dev-and-test/running-unit-tests.md +++ b/docs/server/source/dev-and-test/running-all-tests.md @@ -1,10 +1,10 @@ -# Running Unit Tests +# Running All Tests -Once you've installed BigchainDB Server, you may want to run all the unit tests. This section explains how. +Once you've installed BigchainDB Server, you may want to run all the unit and integration tests. This section explains how. -First of all, if you installed BigchainDB Server using `pip` (i.e. by getting the package from PyPI), then you didn't install the tests. **Before you can run all the unit tests, you must [install BigchainDB from source](setup-run-node.html#how-to-install-bigchaindb-from-source).** +First of all, if you installed BigchainDB Server using `pip` (i.e. by getting the package from PyPI), then you didn't install the tests. **Before you can run all the tests, you must [install BigchainDB from source](setup-run-node.html#how-to-install-bigchaindb-from-source).** -To run all the unit tests, first make sure you have RethinkDB running: +To run all the tests, first make sure you have RethinkDB running: ```text $ rethinkdb @@ -16,12 +16,12 @@ then in another terminal, do: $ python setup.py test ``` -(Aside: How does the above command work? The documentation for [pytest-runner](https://pypi.python.org/pypi/pytest-runner) explains. We use [pytest](http://docs.pytest.org/en/latest/) to write all unit tests.) +(Aside: How does the above command work? The documentation for [pytest-runner](https://pypi.python.org/pypi/pytest-runner) explains. We use [pytest](http://docs.pytest.org/en/latest/) to write all unit and integration tests.) ### Using docker-compose to Run the Tests -You can also use `docker-compose` to run the unit tests. +You can also use `docker-compose` to run all the tests. Start `RethinkDB` in the background: @@ -29,7 +29,7 @@ Start `RethinkDB` in the background: $ docker-compose up -d rdb ``` -then run the unit tests using: +then run the tests using: ```text $ docker-compose run --rm bdb py.test -v From ae236d9f232a23538277e1700a6192751c889bf7 Mon Sep 17 00:00:00 2001 From: troymc Date: Sun, 18 Dec 2016 11:41:04 +0100 Subject: [PATCH 5/7] Moved dev-and-test/running-all-tests.md content to tests/README.md --- .../source/dev-and-test/running-all-tests.md | 35 +--------------- tests/README.md | 40 ++++++++++++++----- 2 files changed, 30 insertions(+), 45 deletions(-) diff --git a/docs/server/source/dev-and-test/running-all-tests.md b/docs/server/source/dev-and-test/running-all-tests.md index f8e48422..bc9d32a7 100644 --- a/docs/server/source/dev-and-test/running-all-tests.md +++ b/docs/server/source/dev-and-test/running-all-tests.md @@ -1,36 +1,3 @@ # Running All Tests -Once you've installed BigchainDB Server, you may want to run all the unit and integration tests. This section explains how. - -First of all, if you installed BigchainDB Server using `pip` (i.e. by getting the package from PyPI), then you didn't install the tests. **Before you can run all the tests, you must [install BigchainDB from source](setup-run-node.html#how-to-install-bigchaindb-from-source).** - -To run all the tests, first make sure you have RethinkDB running: - -```text -$ rethinkdb -``` - -then in another terminal, do: - -```text -$ python setup.py test -``` - -(Aside: How does the above command work? The documentation for [pytest-runner](https://pypi.python.org/pypi/pytest-runner) explains. We use [pytest](http://docs.pytest.org/en/latest/) to write all unit and integration tests.) - - -### Using docker-compose to Run the Tests - -You can also use `docker-compose` to run all the tests. - -Start `RethinkDB` in the background: - -```text -$ docker-compose up -d rdb -``` - -then run the tests using: - -```text -$ docker-compose run --rm bdb py.test -v -``` +All documentation about writing and running tests (unit and integration tests) was moved to [the file `bigchaindb/tests/README.md`](https://github.com/bigchaindb/bigchaindb/blob/master/tests/README.md). \ No newline at end of file diff --git a/tests/README.md b/tests/README.md index d68788bc..5521e7eb 100644 --- a/tests/README.md +++ b/tests/README.md @@ -18,7 +18,13 @@ We write unit and integration tests for our Python code using the [pytest](http: ## Running Tests -You can run all tests using: +### Running Tests Directly + +If you installed BigchainDB Server using `pip install bigchaindb`, then you didn't install the tests. Before you can run all the tests, you must install BigchainDB from source. The [`bigchaindb/CONTRIBUTING.md` file](../CONTRIBUTING.md) has instructions for how to do that. + +Next, make sure you have RethinkDB running in the background (e.g. using `rethinkdb --daemon`). + +Now you can run all tests using: ```text py.test -v ``` @@ -33,27 +39,39 @@ or: python setup.py test ``` -If you want to learn about all the things you can do with pytest, see [the pytest documentation](http://pytest.org/latest/). +How does `python setup.py test` work? The documentation for [pytest-runner](https://pypi.python.org/pypi/pytest-runner) explains. + +The `pytest` command has many options. If you want to learn about all the things you can do with pytest, see [the pytest documentation](http://pytest.org/latest/). We've also added a customization to pytest: + +`--database-backend`: Defines the backend to use for the tests. +It must be one of the backends available in the [server configuration](https://docs.bigchaindb.com/projects/server/en/latest/server-reference/configuration.html). -### Our pytest Customizations +### Running Tests with Docker Compose -Customizations we've added to pytest: +You can also use [Docker Compose](https://docs.docker.com/compose/) to run all the tests. -- `--database-backend`: Defines the backend to use for the tests. Must be one of the backends - available in the [server configuration](https://docs.bigchaindb.com/projects/server/en/latest/server-reference/configuration.html) +First, start `RethinkDB` in the background: + +```text +$ docker-compose up -d rdb +``` + +then run the tests using: + +```text +$ docker-compose run --rm bdb py.test -v +``` - - -## Automated testing of pull requests +## Automated Testing of All Pull Requests We use [Travis CI](https://travis-ci.com/), so that whenever someone creates a new BigchainDB pull request on GitHub, Travis CI gets the new code and does _a bunch of stuff_. You can find out what we tell Travis CI to do in [the `.travis.yml` file](.travis.yml): it tells Travis CI how to install BigchainDB, how to run all the tests, and what to do "after success" (e.g. run `codecov`). (We use [Codecov](https://codecov.io/) to get a rough estimate of our test coverage.) ### Tox -We use [tox](https://tox.readthedocs.io/en/latest/) to run multiple suites of tests against multiple environments during automated testing. Generally you don't need to run this yourself, but it might be useful when troubleshooting a failing CI build. +We use [tox](https://tox.readthedocs.io/en/latest/) to run multiple suites of tests against multiple environments during automated testing. Generally you don't need to run this yourself, but it might be useful when troubleshooting a failing Travis CI build. To run all the tox tests, use: ```text @@ -70,4 +88,4 @@ To run only a few environments, use the `-e` flag: tox -e {ENVLIST} ``` -where `{ENVLIST}` is one or more of the environments specified in the [tox.ini file](tox.ini). \ No newline at end of file +where `{ENVLIST}` is one or more of the environments specified in the [tox.ini file](../tox.ini). From 8d0ff25a39d603318b977c433d3db404f0051574 Mon Sep 17 00:00:00 2001 From: troymc Date: Mon, 19 Dec 2016 15:34:58 +0100 Subject: [PATCH 6/7] Don't include the top-level bigchaindb/ folder in path names --- tests/README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/tests/README.md b/tests/README.md index 5521e7eb..ac081de5 100644 --- a/tests/README.md +++ b/tests/README.md @@ -1,26 +1,26 @@ # BigchainDB Server Tests -## The bigchaindb/tests/ Folder +## The tests/ Folder -The `bigchaindb/tests/` folder is where all the tests for BigchainDB Server live. Most of them are unit tests. Integration tests are in the [`bigchaindb/tests/integration/` folder](./integration/). +The `tests/` folder is where all the tests for BigchainDB Server live. Most of them are unit tests. Integration tests are in the [`tests/integration/` folder](./integration/). A few notes: -- [`bigchaindb/tests/common/`](./common/) contains self-contained tests only testing +- [`tests/common/`](./common/) contains self-contained tests only testing [`bigchaindb/common/`](../bigchaindb/common/) -- [`bigchaindb/tests/db/`](./db/) contains tests requiring the database backend (e.g. RethinkDB) +- [`tests/db/`](./db/) contains tests requiring the database backend (e.g. RethinkDB) ## Writing Tests -We write unit and integration tests for our Python code using the [pytest](http://pytest.org/latest/) framework. You can use the tests in the `bigchaindb/tests/` folder as templates or examples. +We write unit and integration tests for our Python code using the [pytest](http://pytest.org/latest/) framework. You can use the tests in the `tests/` folder as templates or examples. ## Running Tests ### Running Tests Directly -If you installed BigchainDB Server using `pip install bigchaindb`, then you didn't install the tests. Before you can run all the tests, you must install BigchainDB from source. The [`bigchaindb/CONTRIBUTING.md` file](../CONTRIBUTING.md) has instructions for how to do that. +If you installed BigchainDB Server using `pip install bigchaindb`, then you didn't install the tests. Before you can run all the tests, you must install BigchainDB from source. The [`CONTRIBUTING.md` file](../CONTRIBUTING.md) has instructions for how to do that. Next, make sure you have RethinkDB running in the background (e.g. using `rethinkdb --daemon`). From ad6f2670555f5d1cc4dc60e1060a69eb930829e0 Mon Sep 17 00:00:00 2001 From: Brett Sun Date: Tue, 20 Dec 2016 13:52:46 +0100 Subject: [PATCH 7/7] Add instructions for rebuilding docker images after upgrading --- docs/server/source/dev-and-test/setup-run-node.md | 3 +++ tests/README.md | 8 ++++++++ 2 files changed, 11 insertions(+) diff --git a/docs/server/source/dev-and-test/setup-run-node.md b/docs/server/source/dev-and-test/setup-run-node.md index db188f71..0cf5334c 100644 --- a/docs/server/source/dev-and-test/setup-run-node.md +++ b/docs/server/source/dev-and-test/setup-run-node.md @@ -50,6 +50,9 @@ Build the images: docker-compose build ``` +**Note**: If you're upgrading BigchainDB and have previously already built the images, you may need +to rebuild them after the upgrade to install any new dependencies. + Start RethinkDB: ```bash diff --git a/tests/README.md b/tests/README.md index ac081de5..e84516a1 100644 --- a/tests/README.md +++ b/tests/README.md @@ -63,6 +63,14 @@ then run the tests using: $ docker-compose run --rm bdb py.test -v ``` +If you've upgraded to a newer version of BigchainDB, you might have to rebuild the images before +being able to run the tests. Run: + +```text +$ docker-compose build +``` + +to rebuild all the images (usually you only need to rebuild the `bdb` image). ## Automated Testing of All Pull Requests