Prototyping

One way to prototype your skill is to get willing test participants to act as the User of the skill. These tests can be done in person or over a video conferencing tool.

As the skill designer you will act as the Voice Assistant and respond to the user's input with pre-written Example Interations that you have created. You can think of it like a table read for a screenplay except the user is ad-libbing. Make sure you ask your participants if it is alright to record the sessions so that you can take better notes after the test is complete.

As the part of the Voice Assistant you want to stick to the script, but you are actually hoping the tester says something that you haven't thought of yet. The goal of these tests isn't to just validate your assumptions, the goal is to observe interactions that did not occur to you at first.

To begin the test give the user a task. For example, with the Moon Phase skill, we can pull the task directly from our Jobs to be done work.

Task Example:

You're thinking about taking some photos of the moon tonight, and you want to know what phase the moon will be in. What would you ask Mycroft to help you plan your photo session?

The first off the cuff responses from your test are generally the best, but it's always good to ask your testers if they can provide any variations on the way they might ask the question.

Be considerate of your tester's time. A usability test should never exceed 90 minutes, after that the test subject will likely get tired. We think you can get valuable feedback in a 30 to 45 minute session.

If you are creating a Skill as a hobby or as an individual you may not have the resources to provide an incentive to your testers. In that case you might be limited to friends and family for testing. If you are creating a skill for a larger organisation make sure that you think about possible incentives and check if your organisation requires a legal waiver for user testing.