Parameters #

Test Mode #

Loadero supports these test modes:

  • Performance test - provides all available metrics (WebRTC dump, browser logs, selenium logs, machine statistics), but is very limited in terms of allowed participant count (max 100).
  • Load test - although less metrics are available than in performance tests, load test mode allows significantly more participants.
  • Session record test - provides session screen recording video for each participant, with few of the metrics (browser logs, selenium logs), but is extremely limited in terms of allowed participant count (max 10). This test mode can be useful to reduce time spent on script debugging.

In-depth comparison of available test modes

FeaturePerformance testLoad testSession record test
Max participants5010
Selenium logX (verbose)XX
WebRTC dumpXX
Browser logsXXX
Machine statisticsX
AssertsXX (WebRTC only)
Assert preconditionsXX
File downloadsX
Session recordingX
Audio recordingXXX (with video)
When running up to 50 participants, use performance test mode to collect more statistics without any drawbacks.

Easy Scalability #

Leveraging features of participant groups it is possible to use your existing performance tests also as load tests. Simply setting higher group or participant count will add more participants in test in matter of minutes.

Performance test with a single group that contains 50 participants can easily be scaled by switching test mode to load test and changing group count to 10. That will result in 500 participant test using the same script

Increment strategy #

Increment strategy specifies participant distribution over start interval when beginning test execution. This is useful because 10k participants joining at the same time would create a DDoS attack instead of load test and for this reason we advise to use incremental load.

  • Linear participant - all participants join test in linear ordering creating a continuous and monotonic increment.
  • Random participant - all participants join test in random ordering.
  • Linear group - all groups join test in linear ordering, but each group’s participants join at the same time creating monotonic increment the size of group participant count.
  • Random group - all groups join test in random ordering, but each group’s participants join at the same time.

Start interval #

Describes total amount of time in which all participants start their tests and access the service. Closely related to increment strategy.

Having large start interval may cause some participants to finish test before other participants have joined at all.

Participant timeout #

Specifies the time limit after which the test will be terminated. This is necessary to prevent too long test execution that may happen in cases when defined commands may freeze or require longer execution times. Maximum participant timeout is 8 hours.

Keep in mind that all Nightwatch.js and TestUI commands usually take longer to execute than their predefined timeouts.