Test suite

The Dragonfly library contains tests to verify its functioning and assure its quality. These tests come in two distinct types:

  • Tests based on unittest.
  • Tests based on doctest; these also serve as documentation by providing usage examples.

See the links below for tests of both types.

Tests in doctests format:

Tests based on the unittest framework reside in the dragonfly.test package.

Running the test suite

This section contains instructions on how to run dragonfly’s test suite with the SR engine backends.

Most tests were written with an English model andvocabulary in mind, so they will not pass if English words are not in the SR engine’s vocabulary. The only exception to this is the text-input (“text”) backend.

Using DNS

Follow the following steps to run the test suite for the DNS backend:

  1. Start DNS. (And ensure that NatLink is also automatically started.)

  2. Extract the Dragonfly source code in a directory <dir>.

  3. Run the follow commands to install test requirements:

    cd <dir>
    pip install -r test_requirements.txt
    
  4. Run the tests with the following command:

    python setup.py test --test-suite=natlink
    

Different tests need to be run for different DNS versions. The natlink test suite will attempt to run the correct tests for the version of DNS you are using.

These tests may fail with certain DNS versions, models and/or vocabularies. It is possible for the test suite to fail even though everything still works in command modules loaded by natlinkmain. This is a natlink issue.

It is not feasible to test dragonfly with all DNS configurations. Please get in touch if the tests fail for you and specific dragonfly functionality doesn’t work in natlink command modules (i.e. .py files in the MacroSystem folder).

Using WSR

Follow the following steps to run the test suite for the WSR backend:

  1. Start WSR.

  2. Wake WSR up, so that it is not in sleeping state, and then turn the microphone off. (It is important to wake the microphone up first, because otherwise it’ll be off and sleeping at the same time. This causes all recognitions to fail. Weird, huh?)

  3. Extract the Dragonfly source code in a directory <dir>.

  4. Run the follow commands to install test requirements:

    cd <dir>
    pip install -r test_requirements.txt
    
  5. Run the tests with the following command:

    python setup.py test --test-suite=sapi5
    

You can also run the test suite for the in process engine class. It has no GUI for the moment. Run the tests with the following command:

python setup.py test --test-suite=sapi5inproc

Some SAPI5 engine tests will fail intermittently, particularly with the shared process engine class ("sapi5" / "sapi5shared"). There is not much we can do about this.

Using the Kaldi engine

Follow the following steps to run the test suite for the Kaldi backend:

  1. Extract the Dragonfly source code in a directory <dir>.

  2. Follow the set up and install instructions for the Kaldi engine on this page.

  3. Run the following commands to install the Kaldi engine and test dependencies:

    cd <dir>
    pip install -e '.[kaldi]'
    pip install -r test_requirements.txt
    
  4. Run the tests with the following command:

    python setup.py test --test-suite=kaldi
    

Using the CMU Pocket Sphinx engine

Follow the following steps to run the test suite for the Sphinx backend:

  1. Extract the Dragonfly source code in a directory <dir>.

  2. Run the following commands to install the Sphinx engine and test dependencies:

    cd <dir>
    pip install -e '.[sphinx]'
    pip install -r test_requirements.txt
    
  3. Run the tests with the following command:

    python setup.py test --test-suite=sphinx
    

Using the text-input engine

Follow the following steps to run the test suite for the “text” backend:

  1. Extract the Dragonfly source code in a directory <dir>.

  2. Run the follow commands to install test requirements:

    cd <dir>
    pip install -r test_requirements.txt
    
  3. Run the tests with the following commands:

    cd <dir>
    python setup.py test
    

This is the default test suite for dragonfly and has no additional dependencies. The --test-suite argument does not need to be specified in this case.