In order to automate hundreds or thousands of end-to-end tests, and run them concurrently, we need a testing environment with the following characteristics:

• Stability Applications on the environment do not have frequent restarts or resource consumption issues; and there is some form of change control to prevent unexpected deployments.

• Concurrent user capacity We run each of your tests in a separate container, each of which acts as an individual user. The test environment (and dependent databases, email servers, etc.) must have the resources to support as many concurrent users as you have tests.

  If the environment is under-resourced, we can run smaller batches of tests and reduce the load, but that will increase the length of your QA cycle and slow down your release velocity.

• Configured for automated testing Applications in the environment need to be configured so that the tests can pass. For instance:

  • If the application has feature flags, the flags are set so that tests can pass.
  • There is at least one user account created for QA Wolf, and the ability to programmatically add more as needed to run tests concurrently.
  • The environment accepts the @qawolfworkflows.com email address domain, and the use of email aliases (e.g. [email protected]).
  • Sub-one minute email and SMS delivery.