Mycroft AI
  • Documentation
  • About Mycroft AI
    • Why use Mycroft AI?
    • Glossary of terms
    • Contributing
    • FAQ
  • Using Mycroft AI
    • Get Mycroft
      • Mark II
        • Mark II Dev Kit
      • Mark 1
      • Picroft
      • Linux
      • Mac OS and Windows with VirtualBox
      • Docker
      • Android
    • Pairing Your Device
    • Basic Commands
    • Installing New Skills
    • Customizations
      • Configuration Manager
      • mycroft.conf
      • Languages
        • Français (French)
        • Deutsch (German)
      • Using a Custom Wake Word
      • Speech-To-Text
      • Text-To-Speech
    • Troubleshooting
      • General Troubleshooting
      • Audio Troubleshooting
      • Wake Word Troubleshooting
      • Log Files
      • Support Skill
      • Getting more support
  • Skill Development
    • Voice User Interface Design Guidelines
      • What can a Skill do?
      • Design Process
      • Voice Assistant Personas
      • Interactions
        • Intents
        • Statements and Prompts
        • Confirmations
      • Conversations
      • Error Handling
      • Example Interaction Script
      • Prototyping
      • Design to Development
    • Development Setup
      • Python Resources
      • Your First Skill
    • Skill Structure
      • Lifecycle Methods
      • Logging
      • Skill Settings
      • Dependencies
        • Manifest.yml
        • Requirements files
      • Filesystem access
      • Skill API
    • Integration Tests
      • Test Steps
      • Scenario Outlines
      • Test Runner
      • Reviewing the Report
      • Adding Custom Steps
      • Old Test System
    • User interaction
      • Intents
        • Padatious Intents
        • Adapt Intents
      • Statements
      • Prompts
      • Parsing Utterances
      • Confirmations
      • Conversational Context
      • Converse
    • Displaying information
      • GUI Framework
      • Show Simple Content
      • Mycroft-GUI on a PC
      • Mark 1 Display
    • Advanced Skill Types
      • Fallback Skill
      • Common Play Framework
      • Common Query Framework
      • Common IoT Framework
    • Mycroft Skills Manager
      • Troubleshooting
    • Marketplace Submission
      • Skills Acceptance Process
        • Information Review Template
        • Code Review Template
        • Functional Review Template
        • Combined Template
      • Skill README.md
    • FAQ
  • Mycroft Technologies
    • Technology Overview
    • Roadmap
    • Mycroft Core
      • MessageBus
      • Message Types
      • Services
        • Enclosure
        • Voice Service
        • Audio Service
        • Skills Service
      • Plugins
        • Audioservice Plugins
        • STT Plugins
        • TTS Plugins
        • Wake Word Plugins
      • Testing
      • Legacy Repo
    • Adapt
      • Adapt Examples
      • Adapt Tutorial
    • Lingua Franca
    • Mimic TTS
      • Mimic 3
      • Mimic 2
      • Mimic 1
      • Mimic Recording Studio
    • Mycroft GUI
      • Remote STT and TTS
    • Mycroft Skills Kit
    • Mycroft Skills Manager
    • Padatious
    • Precise
    • Platforms
Powered by GitBook
On this page
  • Overview
  • Submission
  • 1. Submitting your Skill
  • 2. Automatic Testing of Skills
  • Review
  • 3. Review by Skill Testing Team
  • 4. Review by Mycroft AI
  • What happens next?
  • 5. Deployment
  • 6. Promotion
  • Frequently Asked Questions
  • What are the labels used on the mycroft-skills repo?
  • What happens if the Skill Testing Team can't test the Skill?
  • I am keen to help test Skills, how do I get involved?

Was this helpful?

  1. Skill Development
  2. Marketplace Submission

Skills Acceptance Process

The Skills Acceptance Process outlines how Skills are tested, reviewed, deployed and promoted through the Mycroft Marketplace.

PreviousMarketplace SubmissionNextInformation Review Template

Last updated 4 years ago

Was this helpful?

It is vital that Mycroft Skills exhibit high quality, and are well tested to encourage adoption, and ongoing development of the platform. This process contributes to that goal by providing a structured framework to follow to ensure repeatable outcomes.

Overview

The process consists of 6 stages:

    • Code Review

    • Information Review

    • Functional Review

Submission

1. Submitting your Skill

Before submitting your Skill we assume you have:

  • Personally tested your Skills functionality to ensure it is working as expected. For a detailed testing overview, see stage three below.

  • Ensure any Python or other packages required to run your Skill are included in your requirements.txt or equivalent.

  • mycroft-skill-testrunner /opt/mycroft/skills/your-skill-directory

When you are satisfied your Skill is ready, submit it for formal review by running:

mycroft-msk submit /opt/mycroft/skills/your-skill-directory

If you don't have the mycroft-msk command available, you can manually activate the Mycroft virtual environment and run msk

source ~/mycroft-core/.venv/bin/activate
msk submit /opt/mycroft/skills/your-skill-directory

2. Automatic Testing of Skills

This automated testing is run as soon as you submit your Skill to the Mycroft-Skills repo, and with each major release of Mycroft-core.

Review

3. Review by Skill Testing Team

The Skill Author will receive feedback and may need to solve issues identified in these reviews before it will be approved.

4. Review by Mycroft AI

The purpose of this step is for a Mycroft AI staff member to do a final check of a Skill. The staff member will be looking for:

  • Profanity – does the Skill contain profanity?

  • Sexually explicit material – does the Skill contain sexually explicit references or language?

  • Offensive or pejorative language – does the Skill contain offensive references or language, for example language that is homo-, trans-, or bi-phobic, racist, religiously intolerant etc.

  • Other content that has the ability to damage the reputation of the Mycroft platform.

What happens next?

5. Deployment

6. Promotion

Once a Skill is deployed, the following measures may be taken by Mycroft Staff promote the Skill to the Mycroft Community. The Skill Author is of course welcome to do additional promotion of their own.

  • Shoutout in Mycroft Newsletter

  • Notification through Mycroft social media

We also encourage the Skill Author to make a short video of the Skill in action to aid in promotion.

Frequently Asked Questions

What are the labels used on the mycroft-skills repo?

While a Skill is undergoing review, there are several labels used on the PR to help indicate status.

  • approved - This Skill is ready to be merged to the current branch, ie 18.02

  • needs validation This Skill needs validating by a Community Member and cannot be validated by Skills Team

  • new This is a new Skill (rather than an update to an existing Skill)

  • override autotester The Skills Team has determined that acceptance of this PR should override the AutoTester

  • under review This Skill is being reviewed by the Skills Team

  • update This Skill is an update to an existing Skill

  • waiting This PR or Issue is waiting on the Skill Author to respond

What happens if the Skill Testing Team can't test the Skill?

Often, Skills will have niche requirements. For example, they may be dependent on third party APIs or the installation of entire software stacks. In this case, it is impractical for the Skill Management Team to test the Skill.

Instead, the Skill Management Team will seek assistance from the Community, and will invite the Community member who volunteers to add comments to the Pull Request. Then, the Skills Management Team will review the comments and merge or decline the PR.

I am keen to help test Skills, how do I get involved?

Create your README.md using the . We also have some .

It is also a great idea to post a link to you Skill in the to let others test your Skill on different platforms. This is encouraged but not required to submit your Skill to the Marketplace.

Run the on your own machine using the command:

This will upload the Skill to Github if it isn’t already, and generate a pull request to add your Skill as a to the .

We have developed an Integration Test Runner that assists you in writing tests that can be executed at both an Intent and a Skill level. .

The Skill Testing Team is made up of experienced Mycroft Community Members and Mycroft AI Staff. One of the team’s responsibilities is to ensure Skills meet minimum quality, technical and functional standards to maintain the user experience with every Skill in the Marketplace. When a Skill Author submits a Skill to the , the Skill Testing Team conducts three reviews.

a. - ensuring the code is stable and secure b. - ensuring all info is accurate and understandable c. - ensuring the experience of using the Skill is intuitive

If any such content is found, the PR for the Skill to be added to the will be tagged as flagged and escalated for review by Mycroft AI senior management, who will then make a determination on whether the Skill will be pulled into the Mycroft repo.

If the Skill passes all reviews, it will then be merged into the . Once it is merged, the Skill becomes available for any Mycroft Device to install using voice commands.

Listing on the

Notification on

Notification in

Mycroft Skills Repo labels

Head to our to submit an application and see the many other ways you can be involved.

Our Skills Testing Team are a fantastic group of community members who ensure that all Skills are secure, stable, accurate, and intuitive. The team has members with a range of skills and communicate through a private channel on .

Skill Meta Editor
additional tips for a great README
Skill Feedback category of the Forums
automated testing
Git submodule
Mycroft Skills Repo
Learn more about Automatic Testing of your Mycroft Skill
Mycroft Skills Repo
Code Review
Information Review
Functional Review
Mycroft Skills Repo
Mycroft Skills Repo
Mycroft Skills Marketplace
Mycroft Forum
~Skills Channel on Mycroft Chat
contribute page
chat.mycroft.ai
Submitting your Skill
Automatic testing
Review by Skill Testing Team
Review by Mycroft AI
Deployment
Promotion