Gitiles
Code Review Sign In
gerrit.lfbroadband.org / voltha-system-tests / ec53a1be69b00bc5c4ac6f6da1192705d00d9744
commitec53a1be69b00bc5c4ac6f6da1192705d00d9744[log] [tgz]
authorZack Williams <zdw@opennetworking.org>Mon Sep 16 15:50:52 2019 -0700
committerAndy Bavier <andy@opennetworking.org>Tue Sep 24 17:04:11 2019 -0700
tree51f3271cbf097e0386b1092cb19c18bc198df332
parent102912a56ac5fcf25a43ac56f9c09b5b9f621d08 [diff]
Build virtualenv using the Makefile

Add tidy target (don't use yet as there's a bug)

Make lint target cover all the files

Start at documentation generation using libdoc/testdoc within robot

Change-Id: I8117baf4bd006588ae9383e0731c49e17102348c
11 files changed
tree: 51f3271cbf097e0386b1092cb19c18bc198df332
  1. .gitignore
  2. .gitreview
  3. Jenkinsfile-voltha-build
  4. Jenkinsfile-voltha-test
  5. Makefile
  6. README.md
  7. VERSION
  8. libraries/
  9. requirements.txt
  10. tests/
  11. variables/
README.md

VOLTHA System Tests

Automated test-suites to validate the stability/functionality of VOLTHA. Tests that reside in here should be written in Robot Framework and Python.

Intended use includes:

  • Functional testing
  • Integration and Acceptance testing
  • Sanity and Regression testing
  • Scale Testing using BBSIM
  • Failure scenario testing

Prerequisites

  • Python virtualenv

  • voltctl - a command line tool to access VOLTHA. Reference - voltctl

  • kubectl - a command line tool to access your Kubernetes Clusers. Reference

  • voltctl and kubectl must be properly configured on your system prior to any test executions

Directory is structured as follows:

├── tests
  └── sanity/           // basic tests that should always pass. Will be used as gating-patchsets
  └── functional/       // feature/functionality tests that should be implemented as new features get developed
└── libraries           // shared test keywords (functions) across various test suites
└── variables           // shared variables across various test suites

Running the sanity tests

To run the the sanity tests using an environment previously set up by kind-voltha, run:

make sanity-kind

This test execution will generate three report files in voltha-system-tests/tests/sanity (output.xml, report.html, log.html). View the report.html page in a browser to analyze the results.

Test variables

The make sanity-kind target is equivalent to the following:

ROBOT_PORT_ARGS="-v ONOS_REST_PORT:8181 -v ONOS_SSH_PORT:8101" \
ROBOT_TEST_ARGS="--exclude notready --critical sanity" \
ROBOT_MISC_ARGS="-v num_onus:1" \
make sanity

If you are running the tests in another environment, you can run make sanity with the arguments appropriate for your environment. Look at variables.robot for a list of variables that you may need to override.

Adding to the tests

Most additions should be done by adding keywords to the libraries, then calling these keywords from the tests. Consult a guide on how to write good Robot framework tests.

Make sure that make lint passes, which runs robotframework-lint on any new code that is created.

WIP:

  • Containerizing test environment so these tests can be run independent of the system.