Integration testing ensures your Skill works as expected when it is running inside Mycroft in a realistic environment alongside other Skills.
Mycroft has recently adopted a new Behavior Driven Development (BDD) framework called Behave for integration testing. Mycroft's implementation has been code-named Voight Kampff.
Writing tests for your Skill has benefits for both you and Mycroft as a whole. For the project it allows us to automatically check that the Skill is working as intended, and that no other Skill is causing a conflict.
Test-driven development also allows you to work more efficiently and find bugs faster. We are currently working on a Skill Design Guide that outlines the process we go through to design new voice Skills.
Voight Kampff tests are split into
Featuremay have one or more
Scenarios, and each
Scenariowill have multiple
To see it in action let's look at a short example for a Weather Skill.
Scenario: current local weather
Given an English speaking user
When the user says "tell me the weather"
Then "my-weather-skill" should reply with "Right now, it's overcast clouds and 32 degrees."
In this example, we started by defining the name of the Feature, in this case "current-weather", then added a single Scenario "current local weather". Within this Scenario we have three Steps:
Givensteps define the state of the Scenario,
Whensteps describe actions that are taken, and
Thensteps observe the outcome of the test.
When this specific test is run with the provided Steps, the system will: 1. Ensure that it is in the appropriate state awaiting an utterance in English. 2. Send the given utterance to Mycroft as if it had been spoken by a user. 3. Observe Mycroft's response and check that the correct Skill responded with appropriate dialog.
Featurewe define for our Skills test suite should be placed in it's own file inside the
test/behavedirectory of our Skill. For this example we will save the Feature file in: