Changelog

November 2024

Fixes and improvements

• Added support for Google Chrome 131 and Mozilla Firefox 133.
• Security improvements for instances that are running tests.
• WebRTC charts now take less vertical space, and the tooltips that appear upon hovering over a data point have been visually reworked.
• Reviewed implementation of keyboard actions in the Participants tab of the test builder. Navigating to and interacting with elements via the keyboard should feel more intuitive and accessible.
• Fixed a bug where if you began creating a new participant while editing a test, the "Audio feed" dropdown would incorrectly show the "Silent audio feed" option selected by default. It will now show "Default audio feed".
• Fixed a bug where filtering tests by labels in the Favorites tab of the project would retrieve all tests with those labels, not just favorited ones.
• Fixed a bug where under specific circumstances, a Pay as You Go plan would appear with the "Skip renewal" status, instead of just being cancelled.
• Fixed a bug where failing to retrieve browser logs for a participant for any reason would result in setting that participant's status to "FAIL". This issue will no longer affect the participant's status, and attempting to view browser logs will indicate that there had been an issue in retrieving them. A fix for preventing this issue with browser log retrieval in the first place is still in development.
• A test is now able to display more of its labels before the user has to scroll horizontally to see the rest.

New features

• Labels can now be applied to runs as well. While this feature was initially developed specifically with tests in mind, we decided soon after that the feature would make sense for runs as well. If you run a test with labels, its labels will automatically be applied to the run. If a label is added to or removed from a run, then the original test's labels will not be affected; other runs of that test will also not be affected.

October 2024

Fixes and improvements

• Added support for Google Chrome 130, Mozilla Firefox 131, and Mozilla Firefox 132.
• Removed legacy feature: it is no longer possible to create "Media type" preconditions for asserts. The "Media type" property is being rotated out, as it has been replaced by the "Audio feed" and "Video feed" properties. In all tests that had an assert with this type of precondition, this precondition has been replaced with an equivalent combination of "Audio feed" and "Video feed" preconditions.
• You may now filter tests by test labels. You can do this by using the "Filter" button in the "Tests" or "Favorites" tab. Using this filter you can retrieve a list of only those tests that have the specified label or labels.
• Projects using the Nightwatch.js framework may now use the built-in saveScreenshot() command. The custom takeScreenshot() command is deprecated as of this change.
• In order to resolve an issue where large WebRTC dumps could freeze/crash the page, log preview in participant reports is now limited to files of up to 50 MB in size. Larger files will need to be downloaded and viewed on your own device.
• Improved stability of large tests.
• Various UX fixes to the project upgrade flow.

New features

• Imported packages to support UUID generation for Nightwatch.js and Py-TestUI. Nightwatch.js projects now have access to the crypto module from Node.js. Py-TestUI projects now have access to the uuid library. When developing a script, these packages do not need to be imported within the script, they are already pre-imported and can be referenced in the script immediately. Note that TestUI already had the java.util.UUID package available for this purpose.

September 2024

Fixes and improvements

• Added support for Google Chrome 129 and Mozilla Firefox 130.
• Replaced log preview with a different implementation, to prevent the preview of large logs from causing page freezes.
• When creating or editing a participant, the video feeds are listed in a more intuitive order. Percentiles are now also sorted in numerically ascending order for asserts' aggregator field.
• Various UX improvements for managing labels and applying them to tests.
• UI changes for the project upgrade flow to be more in line with the visual identity of the majority of the app.
• When changing a field in an assert, then subsequent fields in that assert will be cleared only if they are not compatible with the new change. If compatible, subsequent field values will be preserved.
• actions/timeExecution asserts' name field no longer accepts whitespace, since it is impossible for a timeExecution metric's name to have whitespace anyway.
• Fixed misleading information in Loadero API documentation on Swagger for the filter_label_color parameter in the endpoints that support it.

New features

• Users are now allowed to terminate their Monthly and Yearly subscription immediately instead of having to wait until the end of the current billing period. This can be done by cancelling the subscription once to cancel automatic renewal and then cancelling the subscription a second time to terminate it immediately. This can be beneficial for when you want to switch over to a different plan without being forced to wait for your billing period to end or if you simply want to tie up loose ends when you know for a fact you will not be needing to run tests any time soon.

August 2024

Fixes and improvements

• Added support for Google Chrome 128 and Mozilla Firefox 129.
• Tests can now filtered by multiple labels when using the API through repeated usage of the filter_label_name parameter. This is equivalent to a logical AND operation.
• Updated Loadero API documentation on Swagger for participant, runparticipant, and test resources.
• Performance improvements in the tests list to make success rate charts load quicker.
• Various small bug fixes and UX improvements.

New features

• Single test view. Each test now has its own page that you can visit to see the run history for that specific test, alongside a chart that depicts the success rates of runs in a given time period (last 24 hours by default). The single test view can be accessed by clicking anywhere on the test card in the tests list (or favorite tests list), that isn't occupied by a button.

July 2024

Fixes and improvements

• Added support for Google Chrome 127 and Mozilla Firefox 128.
• Updated the behavior of the "Ignore Alert" custom command to be consistent across all 3 frameworks. If no alert is open at the time of execution, then the command will output a non-failing error in the Selenium log that no alert could be found.
• The version of Selenium used in Loadero tests has been upgraded to 4.22.0.
• Improved the stability of test initialization, making test runs less likely to result in a server error.
• Various UI fixes and UX improvements.
• If a test is aborted immediately after it begins, the run report's "Active participants" graph will now correctly display participants as "Aborted" instead of the graph being blank.

June 2024

Fixes and improvements

• Added support for Google Chrome 126 and Mozilla Firefox 127.
• Upgraded AWS instances that are used for test execution. This may result in tests now reporting improved metric values, e.g. especially regarding network metrics, such as higher bitrate and lower jitter. You might also observe reported increases in FPS and decreases in CPU usage.
• The version of Nightwatch.js used in Javascript tests has been upgraded from 2.5.2 to 3.6.1. Any changes Nightwatch.js have made between these two versions are now present in Loadero as well. One unfortunate downside of this is that Selenium logs in Nightwatch.js projects will no longer be as pretty and easy to read as they used to be.
• Changed the way dropdown options are sorted during participant or assert creation. The order is now far more intuitive.
• Fixed an issue where notification messages in the upper right of the app would regularly pop up as a double message. These should now again be appearing only once per event.
• Various UX improvements, most notable of which being that we now allow submitting inputs via ENTER in places where it may have only been possible by interacting with a button before (e.g., when duplicating a test).

New features

• You can now label the tests you have created. This enables the user to categorize their tests as they see fit. You can manage the labels in your project in a dedicated "Labels" page in the "Project settings" tab. Only members with the "Administrator" role can create, edit, and delete labels. Both "Administrator" and "Developer" members can assign them to tests.
• Keyboard shortcuts for saving the test are now enabled in the test builder.

May 2024

Fixes and improvements

• Added support for Google Chrome 125 and Mozilla Firefox 126.
• Various small UX improvements for the test builder.
• Fixed an issue where sometimes it was impossible to change the billing e-mail of a project if the plan had been cancelled.
• Addressed an issue where sometimes a Nightwatch.js test would crash but the test would be marked as a failed test. Such test runs will now be marked as server errors instead, meaning that compute units will not be consumed.

New features

• Invoices for Pay as You Go plans and overuse invoices for Monthly/Yearly plans will now be charged in smaller portions rather than once at the end of the billing period. This does not affect the actual total amount that the customer pays - rather that amount will simply be split up into smaller invoices throughout the billing period. Keep in mind that if such an invoice is generated, you will not be able to run more tests until that invoice is paid.

April 2024

Fixes and improvements

• Added support for Google Chrome 124 and Mozilla Firefox 125.
• Deprecation warning! We are planning to remove the media_type field from participant and run_participant responses. If something you use is dependent on reading this field, then you should stop doing so. This change will take effect after 01.07.2024.
• The success rate chart for each test in the tests list no longer represents a trend across the past 7 days, but instead across the last 20 results, whenever those may have been. Additionally, this graph will no longer take aborted runs and runs that resulted in a server error into account.
• When a user begins creating or editing an assert in the test builder, they will now be forced to finish/cancel what they are doing before being allowed to do anything else.
• Location and filters are now preserved when opening the test builder from the tests list or opening a run report from the recent runs list. If the user chooses to navigate back, they will be directed to the same page with the same applied set of filters as before.
• All information on the Wiki pertaining to Nightwatch.js, TestUI, and Py-TestUI has been completely re-structured. Take a look!
• Fixed an issue where certain Selenium errors in Py-TestUI runs would result in Selenium logs that failed to actually describe anything that happened during the test.
• When upgrading a project, the UI will dynamically update when the subscription is activated, instead of remaining in a static and often misleading state of insisting that the subscription is not active yet.
• Fixed a rare issue where sometimes the run metric aggregator selector (avg, min, max, etc.) would not show up in the run report.
• Dropdown elements (such as the sort strategy selectors in tests/runs lists) now support keyboard inputs to aid option selection.
• All informational banners have undergone a visual re-design.
• Charts in the run report's summary have undergone a visual update.
• machine/cpu/available/total and machine/ram/available/total asserts are now calculated as intended again.
• Various other small bug fixes and UX improvements.

New features

• You can now manage your own list of favorite tests. This feature can be used to keep all the tests you frequently run in one place so that finding them in your project is easier. Favorite tests are on a per-user basis, multiple members of a project can have a different set of favorites - one member favoriting a test will not add that favorite to your own list of favorites. You can favorite a test by hovering over the test in the tests list and pressing the star icon that appears.
• Participants and asserts in the test builder can now be sorted by "Oldest".

March 2024

Fixes and improvements

• Added support for Google Chrome 123 and Mozilla Firefox 124.
• If a test is run with a large quantity of participants (multiple hundreds), the physical locations of their servers will now be more evenly distributed across the selected region. Beforehand the distributon may have been lop-sided and overly concentrated in a specific section of the region. This should improve the realism aspect of load tests.
• Small fixes to make test runs less likely to result in a server error.
• Further improvements for API documentation in Swagger.

February 2024

Fixes and improvements

• Added support for Google Chrome 122 and Mozilla Firefox 123.
• Reduced likelihood of test runs under certain test configurations resulting in a server error.
• When in the midst of creating or editing a participant you accidentally click outside the form, then you will no longer lose the changes you have made. Instead you will be prompted to close either by saving changes or discarding them.
• Deprecation warning for Loadero API users: we plan to deprecate path_id as an order_by option for asserts, having renamed it to path instead. Similarly, we plan to remove the following filters for the run resource: filter_started_from, filter_started_to, filter_finished_from, filter_finished_to. These filters have been renamed to filter_processing_started_from, filter_processing_finished_from, etc. We are still maintaining backwards compatibility for the old names, but we suggest migrating to the new names at your earliest convenience.
• Further improvements for API documentation in Swagger.
• Various small UI changes to achieve a higher degree of consistency in visuals.

New features

• Warning: Loadero has now implemented result expiration, which will cause logs, artifacts, and charts to eventually become unavailable. All runs from before March 4th have a universal 90-day expiration period applied, with the countdown starting on March 4th. All runs on March 4th or later have a plan-specific artifact retention period. If you would like to hold onto this information, we suggest you begin preparations for downloading this content onto your local machine. Note that information such as success rate, metrics, and assertion results will still remain even after expiration. You will be able to see the expiration date if you navigate to any participant result report's logs, artifacts, machine statistics, or WebRTC statistics tab.
• Update to Loadero invoicing is coming soon which means that invoices will be issued based on usage and may be sent more frequently than before.
• Tests now show a success rate chart that displays the general trend of how successful a test's runs have been within the past 7 days.

January 2024

Fixes and improvements

• Added support for Google Chrome 121, Mozilla Firefox 122.
• Performance tests are now able to support up to 200 participants (used to be 50). If you have any load tests with 50-200 participants, we suggest switching these tests to performance test mode instead to make use of more detailed metrics and logs.
• The tests list has been totally revamped, not only visually but functionally as well. Read about the changes in detail in our monthly updates blog post.
• Improved test run stability, reducing the likelihood of users encountering AWS errors.
• Another set of updates for our API documentation.
• In December we had shortly switched over to using Chrome for Testing, however, these browser versions resulted in degraded performance, which resulted in false negatives in users' tests. We have reverted back to using regular Google Chrome.
• Various small fixes to the app's UI to ensure a more consistent and intuitive user experience.

New features

• Projects using the Ultimate or Enterprise plan now have access to G8 and G12 compute units, doubling the maximum amount of compute power you can allocate to a participant.
• Back when we implemented a new header in the run reports, you may have noticed an Audio MOS field that was always blank. E-model MOS calculation has now been implemented for all participants using Google Chrome, and the result of this calculation will be visible in the Audio MOS field of your participants' reports. Additionally, you are able to assert this value in your tests via mos/audio/e-model/avg. Note: Mozilla Firefox does not provide the required WebRTC metrics to support E-model MOS calculation.

December 2023

Fixes and improvements

• Added support for Google Chrome 120, Mozilla Firefox 121.
• Public release of the new test builder to all Loadero users.
• Various small improvements and fixes to the new test builder.
• Improved calculation process for run metrics resulting in more data available in more cases.
• Fixed critical issues for some specific browser versions.

New features

• You can now receive and process email messages within a Loadero test. This functionality allows to process time sensitive information by emails such as verification links, authorization codes and other similar actions.
• Switched to using Chrome for Testing for participants that are using Google Chrome.

November 2023

Fixes and improvements

• Added support for Google Chrome 119, Mozilla Firefox 120.
• Improved app responsiveness when rendering WebRTC charts and the WebRTC dump preview for large amounts of data.
• Various small improvements and fixes to the new test builder.
• The process of changing a project's plan to a different one is now more intuitive.
• Improved Swagger documentation for assert, file, and group resources by adding more informative descriptions on endpoints, their parameters and permitted values.

New features

• Project members with the Administrator role can now set explicit limits for the project in Project Settings -> Plan & Usage - this includes setting how many compute units can be spent in one month, as well as how many compute units are permitted within one test. The maximum permitted compute unit for a participant and a maximum participant timeout value may also be set.

October 2023

Fixes and improvements

• Added support for Google Chrome 118, Mozilla Firefox 119.
• Small fixes to make errors/warnings in the app more informative on what the problem was.
• Various improvements to the UI, such as case-insensitive search functionality for participants/asserts in the test builder, better formatted Time Execution tables in the run report, as well as overall responsiveness improvements and QoL changes.
• Security improvements for authentication processes, including logging into the app.
• Improved the speed at which result statistics are retrieved for large runs, making large-scale run reports load times faster.
• Improved stability of instance start-up, reducing the likelihood of failing to spin up the required instances for a run.

September 2023

Fixes and improvements

• Added support for Mozilla Firefox 118, Google Chrome 117.
• Stability improvements have been made across the whole Loadero platform, improving test launch times and fixing even more potential issues during test launch.
• Updated the UI of recent runs page to include more relevant information as well as generally update the UI.
• Various small improvements to the user interface, including fixes for test script modal and new test builder.
• Deprecation warning! To move towards compliance regarding that the id field for RTC stats should not be predictable, we will be changing them from the current format (e.g., RTCOutBoundRTPAudioStream_4444) to an unpredictable name. The member field which follows the last hyphen will remain as it currently is. Use the internal statsType to determine what values are represented.

New features

• Step time execution is now fully supported and you can get run metrics for the execution times as well as assert that the timings for each participant are within threshold.

August 2023

Fixes and improvements

• Added support for Mozilla Firefox 116, Google Chrome 116, Mozilla Firefox 117.
• Fixed a major issue with large-scale WebRTC tests having instances run out of space to store WebRTC metrics. Long-duration, high-load WebRTC tests are now significantly more stable.
• Additional small fixes to increase the stability of high-load test runs.
• Added a landing page for the wiki and a quick-start guide, as well as implemented a general visual overhaul for the whole documentation.
• Several improvements to the new test builder, most notably support for members with Visitor role to be able to view the test if they have the new test builder enabled.
• Various small improvements to the user interface, including a re-design of the Billing & Invoice tab of project settings.
• Added additional filters to API endpoints: projectrunparticipant, runparticipant, and result endpoints now support filter_compute_unit parameter, to retrieve only participants with a specific compute unit value (G0.5, G1, G2, G4, or G6). You can find more information about our API here.
• Minor accessibility improvements.

New features

• You are now able to set your cookie preferences in the Account Settings view of the web app. You can also do this for the landing page by clicking on Cookie Preferences in the footer of the page.
• The clipboard is now available for participants using Google Chrome, enabling the usage of elements like Copy to Clipboard buttons.

July 2023

Fixes and improvements

• Added support for Mozilla Firefox 115, Google Chrome 115.
• Made even further improvements in result processing speed.
• WebRTC statistics charts and machine statistics charts in run reports now utilize a new library, with the intent to make charts more visually appealing.
• Various bug fixes to the new test builder.
• Fixed an issue where all of a user's projects, where they are the only member, would get deleted if they attempted to delete their account but provided an incorrect password.
• When using Loadero-Python, you no longer need to provide an access token when downloading logs or artifacts, as the access token used to initialize APIClient is now recognized automatically.
• Added the ability to focus elements in the landing page with TAB that were not focusable before in this manner.

New features

• Published fake custom commands for all 3 of Loadero's supported languages (Nightwatch, TestUI, Py-TestUI), to assist local development and debugging.

June 2023

Fixes and improvements

• Added support for Mozilla Firefox 114.
• Made improvements in run result processing speed and concurrency.
• Improved stability for test run initialization.
• Fixed an issue where Javascript and Python test runs wouldn't update browser logs with console output.

May 2023

Fixes and improvements

• Added support for new browser versions: Google Chrome 113, Mozilla Firefox 113, Google Chrome 114.
• Removed deprecated video files as video feed options for participants. This change will not affect your tests, as the options were already getting changed to non-deprecated equivalents server-side. This change simply makes it so that these deprecated options are not selectable in the first place.
• Increased the amount of space available for test runs on instances, decreasing the risk of instance crashes for longer tests with a lot of metric collection.
• Improved Swagger documentation for API, both in performance (size, loading speed) and descriptiveness.
• Added more limitations to trials. Trial projects are now no longer able to set G0.5 compute units for participants, tests are only allowed to contain up to 10 CUs, and only one test can be running at a time.
• Re-designed the run result header in run reports and individual participant reports.
• Some minor UI changes in the project settings and account settings views.

New features

• The new test builder has been released to the public! You may enable it by opening the account settings dropdown in the top right of the app and selecting Enable New Test Builder. This will change the UI and overall flow of test creation and test editing. Try it out and see what has changed! You may switch back to the old test builder if the new test builder is not to your liking. However, the old test builder will be discontinued later down the line and all users will be migrated to the new test builder. New users will start with the new test builder enabled by default.
• Users with Administrator or Developer permissions are now able to create API access tokens for the project. This functionality can be found in the API access tab on the left side of the project settings. Note: This tab is only available for projects with a paid plan enabled.
• For tests where the Time Execution custom command is used, the individual participant report presents the results of these calls in the form of a Gantt chart. You no longer need to look at the Selenium log to find these values.

April 2023

Fixes and improvements

• Added support for new browser versions: Google Chrome 111, Mozilla Firefox 111, Google Chrome 112, Mozilla Firefox 112.
• We have re-designed the project creation process, introducing a brand new project creation form. Note: along with this change we have also removed the projects list view. As such, if you would like to create a new project or navigate between existing projects, you must do so via the project selector in the app's header.
• Improved parallelization of test runs, you should find your tests stuck in Pending status less often. Tip: if your test is stuck in Pending status and participant locations do not matter to you, you may consider changing your participant configurations to a different location - there may be room for new tests in other locations.
• When downloading a log or an artifact, you can now see the download progress.
• Recently you may have noticed that tests take longer to initialize than they used to. We have made improvements to bring this initialization time back down to where it used to be.
• We have also made improvements in the processing speed of results, which is particularly relevant for load tests with hundreds or thousands of participants.

New features

• You may now postpone the use of your account's trial. If you have a trial available and decide not to use it when creating a new project, you can choose to start a trial for this project later, as long as you have not begun upgrading it to a paid plan. Note, that each account is only permitted one trial.
• Since WebRTC metrics have been made available for session recording tests, we have also enabled WebRTC asserts in session recording tests as well.

February 2023

Fixes and improvements

• Added support for new browser versions: Google Chrome 110 and Mozilla Firefox 110.
• We have updated how WebRTC metrics are collected during test runs. This can impact RAM usage measurements since calculations are not done on the participant's instance anymore during the test.

New features

• WebRTC metrics are now available for session recording tests as well, meaning that WebRTC metrics are now available in all three test modes. Note: this change is not retroactive, WebRTC metrics will not be available for test runs from before this change was made.
• WebRTC metrics are now available even if the test run has been aborted, as long as any WebRTC metrics have been collected at all.

January 2023

Fixes and improvements

• Upgraded to NightWatch version to 2.5.2.
• Added support for new browser versions: Google Chrome 109 and Mozilla Firefox 108, 109.
• Changed the in-project navigation layout - the navigation bar that used to reside on the left side of the page is now at the top of it instead.
• Added a statsType field to the WebRTC dump, this field corresponds to the RTCStatsType enum.
• Deprecation warning! To move towards compliance regarding that the id field for RTC stats should not be predictable, we will be changing them from the current format (e.g., RTCOutBoundRTPAudioStream_4444) to an unpredictable name. The member field which follows the last hyphen will remain as it currently is.

New features

• Until now users were able to upload avatars only for their accounts. The project details tab under project settings has been updated, introducing the option for users to upload an avatar for projects as well. Only members with the Administrator role in the project have the ability to do this.
• Users will now be re-directed to the last project they had open upon logging in. You may still access the projects list by clicking on the Loadero logo in the top left corner of the page.
• Warning! We will soon change our WebRTC stats collection engine, which may cause some changes in the formatting of the WebRTC dump. Consumed RAM during the test may also change because of this due to stat calculations no longer being done on the participant's instance.

November 2022

Fixes and improvements

• Improved Loadero's homepage about performance testing. You can check it out at Loadero Performance Testing.
• Added support for new browser versions: Google Chrome 107, 108 and Mozilla Firefox 106, 107.

New features

• Added more result sort options. Find out more about available sort options from our API swagger documentation.
• Added access control to result artifacts (logs, screenshots, downloads, session recordings). OAuth token is now required to download artifacts.
• Published a new blog post about - How to use Loadero Python Client To Store Test Data In GitHub.

September 2022

Fixes and improvements

• Upgraded to NightWatch version to 2.3.0
• Added support for six new browser versions: Google Chrome 104, 105, 106 and Mozilla Firefox 103, 104, 105.
• Warning! API endpoints /projects/{projectID}/tests/{testID}/runs/ and /projects/{projectID}/runs/ to read all runs will have its default result order changed to ascending sorted by id.
• Reduced test start times - on average the tests will now spend about one minute less in the "Initializing Environment" status.
• Fixed issue with Navigator.deviceMemory browser property not beeing initialized properly.

New features

• Loadero-Python a Loadero API client library for python was released.

June 2022

Fixes and improvements

• Added support for five new browser versions: Google Chrome versions 102 and 103 and Mozilla Firefox versions 100, 101, 102.

New features

• TimeExecution custom commands results (called timecards) are now accessible through the API. Available via
  • GET /v2/projects/{projectID}/runs/{runID}/results/{resultID}/
  • GET /v2/projects/{projectID}/tests/{testID}/runs/{runID}/results/{resultID}/
• Added 1st and 5th percentile aggregator functions to post run assets and run and participant result metrics.
• Added a new page about our test creation offer.

April 2022

Fixes and improvements

• Added support for Google Chrome 101 and Mozilla Firefox 99 browsers.
• Enabled /dev/shm usage for tests. If your application utilizes /dev/shm an increase in RAM usage can be expected.
• Improved API performance by upgrading to Go 1.18
• Deprecation warning! We will be fully deprecating profile_parameters field from run participant resource. The information has been moved to the participant_details field. Deprecation will affect these 4 routes:
  1. GET /v2/projects/{projectID}/runs/{runID}/results/
  2. GET /v2/projects/{projectID}/runs/{runID}/results/{resultID}/
  3. GET /v2/projects/{projectID}/tests/{testID}/runs/{runID}/results/
  4. GET /v2/projects/{projectID}/tests/{testID}/runs/{runID}/results/{resultID}/
• Published a new blog post: How We Create Load Tests For Our Customers In Loadero.

March 2022

New features

• New script variables are available. Visit our wiki to learn about the available variables here:
  • compute_unit (String): compute unit of given participant
  • test.participant_timeout (Number): participant timeout in seconds of test
  • test.start_interval (Number): start interval in seconds of test
  • test.total_participants (Number): total participant count in the test
• JSON web token libraries added. Visit out wiki to learn about the all available libraries here.
• Custom SSL certificate support in your tests. Contact support to find out how you can use this feature while it's not yet available in UI!

Fixes and improvements

• Added support for Google Chrome 99, Google Chrome 100 and Mozilla Firefox 98 browsers.
• Blog post about testing screen sharing got published.

February 2022

New features

• Added a custom command to measure the execution time of a function. You may find more information about this function here.

Fixes and improvements

• Updated Selenium to version 4.0.0.
• Added support for Google Chrome 98 and Mozilla Firefox 97 browsers.
• Migrated Loadero application to a new domain app.loadero.com.

January 2022 / December 2021

New features

• Added graph granularity for WebRTC and machine statistics graphs. This allows increasing the graph's delta time value to enable viewing the graph at a larger scale.
• Session recordings filenames now contain a UNIX timestamp - recording start time.
• Machine statistics can now be downloaded as JSON data. The data can be found in the participant's results report under the "Logs" tab.

Fixes and improvements

• Added support for Mozilla Firefox 95, 96, and Google Chrome 97 browsers.
• Two new blog posts got published: CSS Selectors And XPath Locators In Selenium Test Automation Scripts and Test Automation With Python.
• Set a unified screen resolution for tests. Now all tests will have a screen resolution of 1920 x 1080px.
• Added a compute unit usage counter, that keeps track of your usage of Loadero in the current billing period.

November 2021

New features

• Added new video feeds with reduced framerates and video feeds that are made to resemble video content from an online meeting.
• Test results are now made available even if participants didn’t launch in case a test was aborted.

Fixes and improvements

• Improved performance making the results views load faster and optimizing the way AWS instances are run.
• Created a new blog post entry in our blog series - A Beginner’s Guide To Test Automation With Javascript Part 4.
• Fixed an issue that packetLoss/in/percent and packetLoss/out/percent metrics were calculated incorrectly. All existing packet loss percent metrics are re-calculated to be correct.
• Added support for Mozilla Firefox 94 and Google Chrome 96 browsers.

October 2021

New features

• Added a metrics table for WebRTC statistics to the results reports page of performance and load tests.
• Added new asserts for video frame width and height and WebRTC connection count.
• Added static IP for Ohio region. All participants configured to run in US East - Ohio region will have IP: 3.20.19.36.

Fixes and improvements

• Added support for Mozilla Firefox 93, Google Chrome 95 browsers.
• Bug fixes and improvements to Loadero web application.

September 2021

New features

• Added VAT status validation to prevent automatic subscription creation with an invalid VAT ID, which would result in our users paying extra when they should not.

Fixes and improvements

• Deprecation warning! We will be making result start and end times nullable. By the end of October we will be removing machine_stats from GET /projects/{projectID}/tests/{testID}/runs/{runID}/results/statistics/. Instead the data will be available in the same request under the metrics key.
• Added support for Mozilla Firefox 92, Google Chrome 93 and 94 browsers.
• Bugfix for empty classification filters returning no results.
• More information can be found in our blog post.

August 2021

New features

• Added support for choosing audio and video feeds separately. This will allow using different combinations of audio and video inputs.

Fixes and improvements

• Added support for Mozilla Firefox 91 browsers.
• Updated post-run assert value units. Updated assert value units can be found in our wiki page about post run assertions. Existing asserts do not require any changes, their expected values have been updated to the new units accordingly.

July / June 2021

New features

• Added a custom command for closing alerts to all languages supported in Loadero.
• Added WebRTC metrics graphs for test participants with Mozilla Firefox.
• Added new assert paths for WebRTC inbound jitter values and for machine network packets values.
• Created a new blog post entry in our blog series - A Beginner’s Guide To Test Automation With Javascript Part 3.
• Added expected VAT amount in project billing tab.

Fixes and improvements

• Updated Nightwatch.js to v1.7.6.
• Added support for Google Chrome 92 and Mozilla Firefox 90 browsers.
• Updated our application design to unify button styles.

May 2021

New features

• FireFox WebRTC dump key mapping to use the same format as Google Chrome WebRTC dumps have. This allows to use the same code to parse both dumps, simplifying any processing.
• Aborted test run results - we're now collecting the data from aborted test runs and displaying it. This allows to gain more visibility in the test run even if it was stopped during execution.

Fixes and improvements

• Major performance improvements in time to display results for large test runs.
• Added support for Google Chrome 91 browser.

April 2021

New features

• Python language support for test script creation - we've added this language to the 2 existing languages in Loadero. Now you can choose among JavaScript, Java or Python to write your tests in.
• Added new metric paths that will allow creating new post-run asserts.
• FAQ section for pricing related questions in our landing page.
• New media files have been added to run variety of scenarios in your WebRTC tests.

Fixes and improvements

• Improvements in result polling for large load tests.
• Added support for Google Chrome 90 and Mozilla Firefox 88 browser.
• Automatic request retry in our web app.
• Performance improvements in web app result views.
• Bugfixes and improvements in the checkout flow in web app.

March 2021

New features

• In-app payments - We have finally added a long awaited feature of in-app payments. This allows users to upgrade their plan in Loadero app, instead of contacting support to do that.
• We have created a WebRTC testing guide where we have gathered tips and tricks for testing WebRTC applications. Download yours here.

Fixes and improvements

• Added support for Google Chrome 89 and Mozilla Firefox 87 browser.
• We added a safeguard that forbids deleting a test while a run is active for that test.
• We added sort functionality in test results - participants list can now be sorted by a specific property to locate interesting participants faster.
• Deprecation warning! We will be changing the default value of participant compute unit from G2 to G1. This change only affects API users, and only in case compute units are not otherwise specified.

February 2021

Fixes and improvements

• Added support for Mozilla Firefox 86 browser.
• We addressed issue with NightWatch Socket hangup. This is a well-known issue for NightWatch framework. While this is not a complete fix (the issue can be reproduced with too low compute unit assignment), it should only appear in rare cases.
• Performance improvements on test initialization phase. This means that tests will spend less time in "Initializing Environment" status.
• We have limited the maximum number of projects that can be created in a single account. We have seen some users abuse system by creating many trial projects, which will not be possible from now on. Each account can only create one trial project. If that project is paid, a new project can be created.
• We have improved loading times for our frontend app, by adopting CDN network.

January 2021

New features

• New script editor - Script editing experience got way better with the new Monaco editor many of you are already familiar from Visual Studio Code.

Fixes and improvements

• Added support for Google Chrome 88 and Mozilla Firefox 85 browser.
• Now previous position in the list when navigation around the web app is stored, for example, when exiting test edit view for a test that is located on the 2nd page, now will redirect to the 2nd page instead of the 1st.
• Updated the way how error messages are presented in various forms to improve user experience of our web app.
• Fixed a bug when duplicating test, groupless participants are not duplicated.
• Improvements in web app performance and issue tracking to resolve incidents even quicker.
• Renamed assert_paths to metric_paths and assert_key to metric_key (added redirect to the new routes during the transition period).
• Deprecation warning! We will be deprecating machine_stats field in single participant results. The data will be moved to another endpoint, the link to the data will be available through log_paths.rru field in GET /v2/projects/{id}/tests/{id}/runs/{id}/results/{id}/ request response.

December 2020

New features

• Max available CPU - Now you can see max available CPU depending on the selected compute unit. Of course, this allows viewing CPU data in machine statistics graphs as well.
• Filter result participants in web app - A long awaited and finally delivered feature, you can filter result participants based on different criteria, for example, status, browser or region.
• Even more filtering options using API - Loadero API now allows to filter different resources in even finer detail.

Fixes and improvements

• Added support for Mozilla Firefox 84 browser.
• Updated Nightwatch.js to v1.5.1.
• Fixed an issue where setRequestHeader() custom command took into account header value capitalization even though it should be case-insensitive as per RFC 7230.
• Fixed several minor frontend web app issues.

November 2020

New features

• Compute unit setting for participants - We added compute units setting in test participant creation, it allows to optimize cost of running the tests for the applications that don’t use much compute power, and test very resource demanding applications as well. We also updated our pricing, now it is based on the compute units usage instead of participants usage.
• Video autoplay - We enabled video autoplay in both Mozilla Firefox and Google Chrome browsers for the Loadero test participants. If that is something that can impact your existing Loadero tests, make sure to review them and update accordingly.
• Accepting untrusted SSL certificates - Sometimes when testing a website, accepting untrusted SSL certificates is necessary. Running a test in the Session recording mode or adding commands for taking screenshots previously helped to identify the need to accept. Now we enabled acceptSslCerts in our test runner for this.

Fixes and improvements

• Added support for Mozilla Firefox 83 and Google Chrome 87 browsers.
• Fixed a server-error triggered due to syntax errors in JavaScript code.
• Some fixes that improved the test runner stability.
• Some smaller updates in the Loadero application and website pages.

October 2020

New features

• New media files for WebRTC testing - We have updated our built-in media files for simulating microphone and webcam inputs. Now all of them are running at 30 FPS with specific common audio files and frame markers that aid quality assessment.
• Access to location for your tests - We added access to location for tests created in Loadero, now you can test websites that require location access.

Fixes and improvements

• Added support for Mozilla Firefox 81/82 and Google Chrome 85/86 browsers.
• We have refactored project API access - it is simpler to add authorization for the requests you make to Loadero now. We also added an API endpoint for reading all classificator values at once, which will simplify the development of your tests if you are using Loadero API.

August 2020

New features

• Machine graphs - you can now inspect results for each individual test participant to see their machine resource usage over time.
• Optional API parameters - When creating or modifying participant information, it is no longer necessary to provide location, network and media type values in the request, they will always be initialized to defaults (for location Oregon region will be used as the default).
• Adjustable script modal - now you can see more of the script in a single view.

Fixes and improvements

• Added support for Mozilla Firefox 80 and Google Chrome 85 browsers.
• Performance and stability improvements for Loadero API. This should result in improved app loading times as well as API response times.
• We completed a major refactoring of Loadero backend infrastructure meaning that we will be able to deliver new awesome features to you even faster!
• Updated setRequestHeader() and setUserAgent() custom commands so that you can modify User-Agent header value using any of the two commands.
• Minor fixes as well as performance improvements for the Loadero web application.

July 2020

New features

• Export WebRTC charts images - now you can export each WebRTC chart as a PNG image so you can integrate them in your reports more easily.
• A new custom command to change any HTTP request header - override any HTTP header value for the requests your service makes using setRequestHeader() function! This enables testing of different functionality that could only be tested when a custom header value is present. Read more about this command here.
• New API endpoints - allow you to read test run results without specifying the test ID in the request path- only project ID is mandatory. Check out our API documentation to find out technical details about the new endpoints.
• New script variables - a unique identifier of the participant across all tests ever launched in Loadero (run participant ID) as well as a sequential number of the participant within the scope of the test (participant global ID). Read more about these two variables in our wiki.
• WebRTC internals dump in load tests - more data from your load tests including automatic post-run assertions for WebRTC metrics.

Fixes and improvements

• Added support for Mozilla Firefox 78 and 79 as well as Google Chrome 84 browsers.
• Various bug fixes and performance improvements.

June 2020

New features

• Audio recording - allows recording full audio from a participant session.
• Fully customized network conditions - we have added configuration options and functions that allow you to update only a specific parameter of the current network conditions, such as bitrate. Read more about how to do that here.
• Additional filtering options - added more filtering options for Machine Statistics overview in test results. While charts for these statistics are coming in the future, additional filtering options allow you to get a better overview of the resources used during the test.

Fixes and improvements

• We added support for Mozilla Firefox 77 browser
• We have been working on multiple internal systems to reduce the time needed to notice and resolve different issues that can come up. With all these systems in place, we are able to monitor service status with much higher precision and are able to react to events faster than ever before.

May 2020

New features

• Additional aggregators for machine statistics - machine statistics now can be analyzed in more detail filtering by new categories (StdDev, Relative StdDev, 25%, 50%, 75%, 95%, 99%).

Fixes and improvements

• Added support for Mozilla Firefox 76 and Google Chrome 83 browser.
• Fixed timing issue for linear increment strategy where some participants would start the test execution at the same time.
• Improved media stream quality at the start of the test – media streams are at their top quality from the very beginning of test execution.
• Fixed typo in one of our constant names: “asymmetric network”. We’ve stopped supporting the version with the typo and made sure the existing tests keep running even if you had been using it.
• Fixed an issue when sometimes WebRTC statistics would not get recorded.
• General performance updates for Loadero web application.

April 2020

New features

• waitForDownloadFinished() - a new custom command for "Javascript + Nightwatch" and "Java + TestUI" languages. This command will be helpful when downloading files in tests to ensure that the download has finished before the test exits.
• Multiple downloads in Google Chrome - we’ve updated the options for the Google Chrome browser so that it now allows multiple downloads from the same page.
• New media files - There are 3 new media types. These media types all have 1080p video feed with markers for visual verification of media quality. Each of the media types has an audio feed with different sound levels: -20dB, -30dB and -50dB.

Fixes and improvements

• We added support for Mozilla Firefox 75 and Google Chrome 81 browsers.
• We’ve done improvements to Loadero’s test queue by improving the processing of aborted tests that were still in the queue. This means that the waiting time for the test execution to start has gotten even shorter!
• Fixed an issue where WebRTC dump sometimes would not be available.
• Fixed a problem that media type assert preconditions would not be evaluated.
• Fixed an issue where double quotes in test name would cause the test to crash.

March 2020

New features

• New wiki - we’ve updated our documentation site! Check it out as you will definitely find something that is helpful to you. If you notice anything missing or unusual, don’t hesitate to let us know!
• Multiple parallel tests - we have completed a major upgrade of the test run scheduling mechanism, now you can run tens or even hundreds of tests in parallel. This update applies to all test modes and, of course, to tests launched on custom AWS accounts.
• Assert preconditions - we have added assert preconditions to our web application! Head to loadero.com to add preconditions to your existing asserts!
• Resource ID - added resource ID subheader in our web app. This will help you to start using our API – just copy and paste ID where resource ID is needed.

Fixes and improvements

• Improved reliability of session record test mode for short tests – sometimes the recording was not stored.
• Fixes and improvements for app loading screens, now when fetching data for logs and WebRTC statistics you will see a progress bar of how much of your data has been downloaded so far.
• We added support for Mozilla Firefox 74 browser. Due to the current global situation, Google has decided to postpone the release of Google Chrome 81 browser hence new Google Chrome releases in Loadero will be postponed as well.