Fork me on GitHub

pytest-scenario: parameterized test case instances and test scenarios.

pytest-scenario is a pytest plugin that aims to extend current test parameterization capabilities. After installing pytest-scenario you will be able run a test suite constructed from a JSON formatted test plan (AKA Test Scenario).

Note: pytest-scenario is currently classified as alpha, feel free to contact me with any issue at: https://github.com/OriMenashe

Features

  • Test parameterization (including fixtures and test arguments).
  • Fixture parameterization on a test level.
  • Test instantiation - run multiple test instances with different parameters.
  • Test ordering - running tests in a thoughtful, user-defined order.
  • Test exclusion - excluding unwanted tests during collection stage.

Installation

Install pytest-scenario by running:

pip install pytest-scenario

Quickstart

  • Test parameterization is done by using a new test_case marker as follows:

    @pytest.fixture
    def not_parametrized_fixture():
        return "I am not parametrized"
    
    
    @pytest.fixture
    def int_parametrized_fixture(request):
        return request.param.num
    
    
    @pytest.fixture
    def string_parametrized_fixture(request):
        return request.param.string
    
    
    class TestParametrize:
    
        @pytest.mark.test_case(fixture_binding=[('first_fixture_place_holder', {'func': 'int_parametrized_fixture',
                                                                                'scope': 'function',
                                                                                'params': [('num', 100)]
                                                                                }),
                                                ('second_fixture_place_holder', {'func': 'string_parametrized_fixture',
                                                                                 'scope': 'class',
                                                                                 'params': [('string', 'Hello World')]
                                                                                 }),
                                                ('third_fixture_place_holder', {'func': 'not_parametrized_fixture',
                                                                                'scope': 'module',
                                                                                })
                                                ],
                               test_params=[('test_param', 1.04)])
        def test_fixture_param(self, first_fixture_place_holder, second_fixture_place_holder, third_fixture_place_holder,
                                test_param):
            print('\n')
            assert isinstance(first_fixture_place_holder, int)
            print('first_fixture_place_holder: %d' % first_fixture_place_holder)
            assert isinstance(second_fixture_place_holder, str)
            print('second_fixture_place_holder: %s' % second_fixture_place_holder)
            assert isinstance(third_fixture_place_holder, str)
            print('third_fixture_place_holder: %s' % third_fixture_place_holder)
            assert isinstance(test_param, float)
            print('test_param: %s' % test_param)
    
    • Output:
    first_fixture_place_holder: 100
    second_fixture_place_holder: Hello World
    third_fixture_place_holder: I am not parametrized
    test_param: 1.04
    PASSED
    ====================================== test_fixture_param_persistency finished ======================================
    

    Another test can benefit from privious parameterization if defined in the same scope, i.e.:

        @pytest.mark.test_case(fixture_binding=[('fixture_place_holder', {'func': 'string_parametrized_fixture'})])
        def test_fixture_param_persistency(self, fixture_place_holder):
            print('\n')
            assert isinstance(fixture_place_holder, str)
            print('fixture_place_holder param: %s' % fixture_place_holder)
    
    • Output:
    fixture_place_holder param: Hello World
    PASSED
    ====================================== test_fixture_param_persistency finished ======================================
    
  • Test scenario is represented by a JSON file located at:

    <projects_root>/sut/scenarios/<scenario_name>.json
    

    Below is an example for a scenario named “main scenario”:

    [
        {
            "id": 1,
            "module_name": "tests.test_parametrize",
            "class_name": "TestParametrize",
            "test_name": "test_scenario_instantiation",
            "fixture_binding": {
                "fixture_place_holder": {
                    "func": "string_parametrized_fixture",
                    "scope": "session",
                    "params": {
                        "string": "Hello"
                    }
                }
            },
            "test_params": {
                "test_param": "World"
            },
            "skip": false,
            "xfail": false
        },
        {
            "id": 2,
            "@ref": "sub scenario"
        }
    ]
    

    “main scenario” is referencing a second scenario named “sub scenario” (Nesting is supported):

    [
        {
            "id": 1,
            "module_name": "tests.test_parametrize",
            "class_name": "TestParametrize",
            "test_name": "test_scenario_instantiation",
            "fixture_binding": {
                "fixture_place_holder": {
                    "func": "string_parametrized_fixture",
                    "scope": "session"
                }
            },
            "test_params": {
                "test_param": "Bob"
            },
            "skip": false,
            "xfail": false
        },
        {
            "id": 2,
            "module_name": "tests.test_parametrize",
            "class_name": "TestParametrize",
            "test_name": "test_scenario_instantiation",
            "fixture_binding": {
                "fixture_place_holder": {
                    "func": "string_parametrized_fixture",
                    "scope": "function",
                    "params": {
                        "string": "Bye"
                    }
                }
            },
            "test_params": {
                "test_param": "Bob"
            },
            "skip": false,
            "xfail": false
        }
    ]
    
    • Invocation of a test scenario would be done as follows:
    ~/workspace/projects_root$ py.test tests/ --scenario="main scenario"
    
    • Output:
    collected 3 items
    selected scenario:
                     _                                       _
     _ __ ___   __ _(_)_ __    ___  ___ ___ _ __   __ _ _ __(_) ___
    | `_ ` _ \ / _` | | `_ \  / __|/ __/ _ \ `_ \ / _` | `__| |/ _ \
    | | | | | | (_| | | | | | \__ \ (_|  __/ | | | (_| | |  | | (_) |
    |_| |_| |_|\__,_|_|_| |_| |___/\___\___|_| |_|\__,_|_|  |_|\___/
    
    
    tests/test_parametrize.py::TestParametrize::test_scenario_instantiation[main scenario-1]
    
    Hello World
    PASSED
    =============================== test_scenario_instantiation[main scenario-1] finished ===============================
    
    
    tests/test_parametrize.py::TestParametrize::test_scenario_instantiation[main scenario-2.sub scenario-1]
    
    Hello Bob
    PASSED
    ======================= test_scenario_instantiation[main scenario-2.sub scenario-1] finished ========================
    
    
    tests/test_parametrize.py::TestParametrize::test_scenario_instantiation[main scenario-2.sub scenario-2]
    
    Bye Bob
    PASSED
    ======================= test_scenario_instantiation[main scenario-2.sub scenario-2] finished ========================
    

License

The project is licensed under the WTFPL license.