Parameters #

Test Mode #

Loadero has three test modes to choose from, each of them is tailored to different needs:

  • Performance test - provides all available metrics (WebRTC dump, browser logs, selenium logs, machine statistics), but is very limited in terms of allowed participant count (max 50). This mode provides the most data about the performance of the tested application.
  • Load test - although less metrics are available than in performance tests, load test mode allows significantly more participants. With this mode you can generate a huge load of tens of thousands concurrent users on your application to see if the service is still operational under high usage.
  • Session record test - provides session screen recording video for each participant, with few of the metrics (browser logs, selenium logs), but is extremely limited in terms of allowed participant count (max 10). This test mode is useful to reduce time spent on script debugging and getting a better understanding of the test scenario.

In-depth comparison of available test modes

FeaturePerformance testLoad testSession record test
Max participants5010
Selenium logX (verbose)XX
WebRTC dumpXX
Browser logsXXX
Machine statisticsX
AssertsXX (WebRTC only)
Assert preconditionsXX
File downloadsX
Session recordingX
Audio recordingXXX (with video)
When running up to 50 participants, we recommend choosing the performance test mode to collect more statistics without any drawbacks.

Increment strategy #

Increment strategy specifies participant distribution over start interval when beginning test execution. This is useful because 10k participants joining at the same time would create a DDoS attack instead of load test and for this reason we advise to use incremental load. Linear strategies allow a better control over the order of how participants join the test and delays between the participants or participant groups joining. Random strategies help simulate real users that join randomly.

  • Linear participant - all participants join the test in linear ordering creating a continuous and monotonic increment.
  • Random participant - all participants join the test in random ordering, multiple participants can join at the same time.
  • Linear group - all groups join the test in linear ordering, but each group’s participants join at the same time creating a monotonic increment steps between groups joining.
  • Random group - all groups join the test in random ordering, but each group’s participants join at the same time.

Start interval #

Describes the total amount of time in which all participants start their tests and access the service. The delay between participants depends on how long of a start interval has been set as well as on the increment strategy chosen.

Having a large start interval may cause some participants to finish the test before other participants have started theirs.

Participant timeout #

Specifies the time limit after which the test will be terminated for a test participant, in this case the participant will be marked as timed out in the results report. This is necessary to prevent too long test execution that may happen in cases when defined commands may freeze or require longer execution times. Maximum participant timeout is 8 hours.

Participant timeout is not impacted by start interval as it is added before that, so even if your start interval is set higher than the participant timeout, this in itself will not cause participants to time out.

Keep in mind that all Nightwatch.js and TestUI commands usually take longer to execute than their predefined timeouts.