Tempest: An OpenStack Integration test suite

Integration Testing:

  • Integration Testing is the phase in software testing in which individual software modules are combined and tested as a group.
  • It occurs after unit testing and before system testing.
  • Tests the actual functionality of the application/software you have written.

Why wasting time in integration tests when I have an option to check the functionality manually?

  • Because every time you add a new feature you will have to check and re-check the functionalities manually which is tedious, time-consuming and error-prone.
  • What if you have thousands of features, certainly you can’t afford to check if them one by one.
  • If you claim that your software is platform-independent. How will you make sure that all the functionalities of your software are intact on multiple platforms? That’s where a ‘test-suite’ comes into picture. You can create a dedicated test environment to run your test-suite across different platforms adhering to your scale and stress needs.

I have written unit tests properly. Why should I write integration tests?

  • Because unit-tests test the source code you have written, the semantic and the syntax, not the actual functionality of your software.
  • Integration tests make sure that your source code, when integrated with all your components, serves the real purpose.
  • With addition of a new feature you can just re-trigger the suite and hence make sure that your new feature doesn’t break any existing functionality.

What is tempest?

  • One of Tempest’s prime function is to ensure that your OpenStack cloud works with the OpenStack API as documented. The current largest portion of Tempest code is devoted to test cases that do exactly this.
  • It’s also important to test not only the expected positive path on APIs, but also to provide them with invalid data to ensure they fail in expected and documented ways. Over the course of the OpenStack project Tempest has discovered many fundamental bugs by doing just this.
  • In order for some APIs to return meaningful results, there must be enough data in the system. This means these tests might start by spinning up a server, image, etc., then operating on it.
  • Currently tempest has support to test both APIs and CLIs.

How to run the tempest tests?

  • There are multiple ways to run the tests. You have a choice to run the entire test suite, all the tests in a directory, a test module or a single tests.
  • The documents describes how to run the tempest tests using nosetests.
  • Install the basic dependencies:
  • Go to the directory tempest and execute the tests with the following command;
    • nosetests -sv tempest.tests.identity.admin.test_services.py [ This would run all the tests inside the module test_services.py]
    • nosetests -sv tempest.tests.identity.admin.test_services:ServicesTestJSON.test_create_get_delete_service [This runs a specific tests inside the class ServiceTestJSON of the module test_services.py]

Basic Architecture

  • Tempest tests can be executed from either outside the openstack setup or from the openstack controller itself.
    • The tempest has a rest client implemented in the suite which takes care of the REST operations.
    • All the user inputs which are performing the rest operationon nova apis are read from tempest.conf file.
    • The suite first make a rest call to the keystone api to get the auth token and tenant details for authorization and authentication.
    • Using the auth token obtained from the response, the tests performs the required operations.

More to follow…

  • Debugging using eclipse..
  • Extending tempest framework for your project..
Advertisements

About Abhishek Srivastava
A coder, a gamer, a guitarist, an opensource enthusiast...

3 Responses to Tempest: An OpenStack Integration test suite

  1. Luz Cazares says:

    Hello, what about current tempest coverage percentage. I mean how many methods from the APIs/ CLI are already automated?

    • Abhishek Srivastava says:

      Its one of the biggest craving needs and I don’t think there is something in tempest yet which would show how much of the functionalities have been covered by the integration tests.
      We will keep you posted as and when we find something related.

  2. Shah says:

    Hi, did you guys tried implementing encryption inside host with lets say dm-crypt? If no can you please tell from where I can get this as I was to apply encryption inside the host on openstack. Many thanks!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: