Test Run Queue
Parallel participant limit
Loadero allows running test in parallel independently from other runs. This gives you the ability to run your tests faster and get results quicker. All your tests will be run on Loadero's infrastructure (except when you are using your own custom AWS). This means that you share the infrastructure with other users who use the same AWS account. Every test mode has its own particular participant limit that allows parallel test execution (except custom AWS queues which have the same queue for all test modes).
All projects that are linked to the same AWS, share the same queue which is not shared with other AWS accounts, e.g., if you use custom AWS to which 2 projects are linked, those projects will share one queue for all test modes.
Assume queue has a limit of 150 parallel participants (real run queue may or
may not have the same parameters) and you run 160 participant test and 5
participant test. It happened that both of your tests were assigned to the same
queue and there are no other runs on it besides yours. Since the 160 participant
test takes up all the concurrent participant limit, all other queued tests will
have status pending. Once the test finishes running, the next queued test will
be run (tests are queued in First-in First-out a.k.a. FIFO method).
Permanent node
There is often a need to run a small test to quickly check certain functionality, e.g., sanity test if a website is even accessible or short 2 attendee WebRTC video call. For such cases Loadero offers permanent node which is an always-on server for certain types of tests, hence it allows to run tests immediately, skipping node initialization phase.
The test must fulfill all of the following conditions to have a chance to be run on a permanent node:
- Total participant count must not exceed 2 participants.
- All participants must be located in Oregon, US region.
- Test mode must be performance.
At the moment permanent node is available only using Loadero infrastructure and it is shared between all Loadero users who qualify for it. That means that if someone is already running a test on a permanent node, all following tests, while the permanent node is occupied, will be run on regular test nodes (node will need initialization phase).
The availability of permanent node depends on several factors, including the number of other tests running in Loadero and whether they qualify to be run on the permanent node.
Test Run statuses
During test execution your test run will go through several stages, which will be represented by the test run status. These are currently possible test run status with descriptions explaining current test run state.
| Status | Description |
|---|---|
| Pending | Test run is still waiting in queue until it can be picked up by a test executor |
| Initializing | Test run has started test environment setup |
| Running | Test run is currently in progress |
| Waiting for results | Test has finished its execution and execution results are being processed |
| Stopping | Test run is currently being stopped by user invocation |
| Done | Test has finished its execution successfully |
| Aborted | Test was aborted before it finished its execution |
| Server error | Test triggered some server-side error that should not have happened |
| Database error | There was an error in database queries or connection during the test |
| AWS error | Some error from AWS was encountered which prevented successful test execution |
| Insufficient resources | Test execution did not start because not enough resource were available |
| Timeout exceeded | Test execution has exceeded a timeout in some test phase |