Automated Tests

Automated tests can be added to your Interactive Labs to verify they are working as expected. Each time the Lab is changed, the tests are re-run and the results displayed in the dashboard.

Cypress

Cypress.io is a complete end-to-end testing tool that can be used to verify that your interactive labs are working as expected.

You can learn more about Cypress at https://docs.cypress.io/guides/overview/why-cypress.html.

Cypress version 10.4.0 is currently supported.

Cypress Test Helpers

A collection of Cypress test helpers has been created to take the heavy lifting out of creating tests for your Labs. These helpers capture the core functionality required for a Cypress test to complete an interactive lab and verify its behavior.

These helpers can be combined with the built-in functionality of Cypress to meet your requirements. More details of the built-in helpers can be found at https://docs.cypress.io/api/api/table-of-contents.html

Creating The First Test

1. Within the lab you want to test, add a folder called .cypress.
2. Create a file with the suffix _spec.js (such as test1_spec.js). This will contain your Cypress tests you want to run. The underlying Katacoda platform supports splitting your tests over multiple files, but subdirectories are not supported.
3. Create your Cypress test using the helpers above. Ensure that cy.startScenario() is called to load your lab.
4. Commit the test to Git and push to the remote Git Repository to trigger Katacoda re-processing.
5. Visit the Dashboard to view the results.

A complete example can be found at https://github.com/katacoda/scenario-examples/tree/main/automatedtest

Automatically-Generated Tests

In order to reduce the effort required to get initial test coverage for your labs, we are working towards drafting tests that you may then customize to your needs. They will be delivered to you as a Gitlab Merge Request so that you can adjust them before merging if needed.

These tests are generated based on the commands embedded in your lab. They tend to work best for labs that can by completed solely by running the commands embedded in them. If your lab requires the learner to perform a manual step such as editing a file, you will need to adjust the generated test.

They currently support running commands in a Bash shell or in a Python REPL.

Example

The following is an example of a test to verify the interactive lab:

describe('My First Test', () => {
  before(() => {
    // This would start and visit the Lab being tested. Without
    // this the test will fail.
    cy.startScenario();
  });

  it('finds the content "Start Scenario"', () => {
    cy.contains('Start Scenario');
  });

  it('runs commands', () => {
    cy.terminalType("uname");

    cy.terminalShouldContain('Linux');
  })
})

A complete example can be found at https://github.com/katacoda/scenario-examples/tree/main/automatedtest.

More examples of using Cypress can be found at https://docs.cypress.io/examples/examples/recipes.html#Fundamentals.

View Results

The results of the test execution can be viewed in the Dashboard. You can find test runs for your organization at https://dashboard.katacoda.com/content/testruns. If you have tests in place, a test run will be triggered automatically whenever changes are pushed to your labs’ repository. You can also re-run all of your tests at once by clicking the “Force Retest” button.

When you go into the testruns logs, you will see the output from Cypress execution. This will help you debug the issue if there are failing tests.

Under the Covers

Want to see what’s happening under the covers? Here’s a video of a test execution:

This happens on every change to your repo to ensure your labs and environments are ready for your learners.