Changelog

Changelog #

March 2021 #

New features #

  • In-app payments - We have finally added a long awaited feature of in-app payments. This allows users to upgrade their plan in Loadero app, instead of contacting support to do that.
  • We have created a WebRTC testing guide where we have gathered tips and tricks for testing WebRTC applications. Download yours here.

Fixes and improvements #

  • Added support for Google Chrome 89 and Mozilla Firefox 87 browser.
  • We added a safeguard that forbids deleting a test while a run is active for that test.
  • We added sort functionality in test results - participants list can now be sorted by a specific property to locate interesting participants faster.
  • Depracation warning! We will be changing the default value of participant compute unit from G2 to G1. This change only affects API users, and only in case compute units are not otherwise specified.

February 2021 #

Fixes and improvements #

  • Added support for Mozilla Firefox 86 browser.
  • We adressed issue with NightWatch Socket hangup. This is a well-known issue for NightWatch framework. While this is not a complete fix (the issue can be reproduced with too low compute unit assignment), it should only appear in rare cases.
  • Performance improvements on test initialization phase. This means that tests will spend less time in “Initializing Environment” status.
  • We have limited the maximum number of projects that can be created in a single account. We have seen some users abuse system by creating many trial projects, which will not be possible from now on. Each account can only create one trial project. If that project is paid, a new project can be created.
  • We have improved loading times for our frontend app, by adopting CDN network.

January 2021 #

New features #

  • New script editor - Script editing experience got way better with the new Monaco editor many of you are already familiar from Visual Studio Code.

Fixes and improvements #

  • Added support for Google Chrome 88 and Mozilla Firefox 85 browser.
  • Now previous position in the list when navigation around the web app is stored, for example, when exiting test edit view for a test that is located on the 2nd page, now will redirect to the 2nd page instead of the 1st.
  • Updated the way how error messages are presented in various forms to improve user experience of our web app.
  • Fixed a bug when duplicating test, groupless participants are not duplicated.
  • Improvments in web app performance and issue tracking to resolve incidents even quicker.
  • Renamed assert_paths to metric_paths and assert_key to metric_key (added redirect to the new routes during the transition period).
  • Depracation warning! We will be deprecating machine_stats field in single participant results. The data will be moved to another endpoint, the link to the data will be available through log_paths.rru field in GET /v2/projects/{id}/tests/{id}/runs/{id}/results/{id}/ request response.

December 2020 #

New features #

  • Max available CPU - Now you can see max available CPU depending on the selected compute unit. Of course, this allows viewing CPU data in machine statistics graphs as well.
  • Filter result participants in web app - A long awaited and finally delivered feature, you can filter result participants based on different criteria, for example, status, browser or region.
  • Even more filtering options using API - Loadero API now allows to filter different resources in even finer detail.

Fixes and improvements #

  • Added support for Mozzila Firefox 84 browser.
  • Updated Nightwatch.js to v1.5.1.
  • Fixed an issue where setRequestHeader() custom command took into account header value capitalization even though it should be case-insensitive as per RFC 7230.
  • Fixed several minor frontend web app issues.

November 2020 #

New features #

  • Compute unit setting for participants - We added compute units setting in test participant creation, it allows to optimize cost of running the tests for the applications that don’t use much compute power, and test very resource demanding applications as well. We also updated our pricing, now it is based on the compute units usage instead of participants usage.
  • Video autoplay - We enabled video autoplay in both Mozilla Firefox and Google Chrome browsers for the Loadero test participants. If that is something that can impact your existing Loadero tests, make sure to review them and update accordingly.
  • Accepting untrusted SSL certificates - Sometimes when testing a website, accepting untrusted SSL certificates is necessary. Running a test in the Session recording mode or adding commands for taking screenshots previously helped to identify the need to accept. Now we enabled acceptSslCerts in our test runner for this.

Fixes and improvements #

  • Added support for Mozilla Firefox 83 and Google Chrome 87 browsers.
  • Fixed a server-error triggered due to syntax errors in JavaScript code.
  • Some fixes that improved the test runner stability.
  • Some smaller updates in the Loadero application and website pages.

October 2020 #

New features #

  • New media files for WebRTC testing - We have updated our built-in media files for simulating microphone and webcam inputs. Now all of them are running at 30 FPS with specific common audio files and frame markers that aid quality assessment.
  • Access to location for your tests - We added access to location for tests created in Loadero, now you can test websites that require location access.

Fixes and improvements #

  • Added support for Mozilla Firefox 81/82 and Google Chrome 85/86 browsers.
  • We have refactored project API access - it is simpler to add authorization for the requests you make to Loadero now. We also added an API endpoint for reading all classificator values at once, which will simplify the development of your tests if you are using Loadero API.

August 2020 #

New features #

  • Machine graphs - you can now inspect results for each individual test participant to see their machine resource usage over time.
  • Optional API parameters - When creating or modifying participant information, it is no longer necessary to provide location, network and media type values in the request, they will always be initialized to defaults (for location Oregon region will be used as the default).
  • Adjustable script modal - now you can see more of the script in a single view.

Fixes and improvements #

  • Added support for Mozilla Firefox 80 and Google Chrome 85 browsers.
  • Performance and stability improvements for Loadero API. This should result in improved app loading times as well as API response times.
  • We completed a major refactoring of Loadero backend infrastructure meaning that we will be able to deliver new awesome features to you even faster!
  • Updated setRequestHeader() and setUserAgent() custom commands so that you can modify User-Agent header value using any of the two commands.
  • Minor fixes as well as performance improvements for the Loadero web application.

July 2020 #

New features #

  • Export WebRTC charts images - now you can export each WebRTC chart as a PNG image so you can integrate them in your reports more easily.
  • A new custom command to change any HTTP request header - override any HTTP header value for the requests your service makes using setRequestHeader() function! This enables testing of different functionality that could only be tested when a custom header value is present. Read more about this command here for JavaScript and here for Java.
  • New API endpoints - allow you to read test run results without specifying the test ID in the request path- only project ID is mandatory. Check out our API documentation to find out technical details about the new endpoints.
  • New script variables - a unique identifier of the participant across all tests ever launched in Loadero (run participant ID) as well as a sequential number of the participant within the scope of the test (participant global ID). Read more about these two variables in our wiki for JavaScript and Java.
  • WebRTC internals dump in load tests - more data from your load tests including automatic post-run assertions for WebRTC metrics.

Fixes and improvements #

  • Added support for Mozilla Firefox 78 and 79 as well as Google Chrome 84 browsers.
  • Various bug fixes and performance improvements.

June 2020 #

New features #

  • Audio recording - allows recording full audio from a participant session.
  • Fully customized network conditions - we have added configuration options and functions that allow you to update only a specific parameter of the current network conditions, such as bitrate. Read more in our wiki about how to do that using JavaScript + Nightwatch and Java + TestUI.
  • Additional filtering options - added more filtering options for Machine Statistics overview in test results. While charts for these statistics are coming in the future, additional filtering options allow you to get a better overview of the resources used during the test.

Fixes and improvements #

  • We added support for Mozilla Firefox 77 browser
  • We have been working on multiple internal systems to reduce the time needed to notice and resolve different issues that can come up. With all these systems in place, we are able to monitor service status with much higher precision and are able to react to events faster than ever before.

May 2020 #

New features #

  • Additional aggregators for machine statistics - machine statistics now can be analyzed in more detail filtering by new categories (StdDev, Relative StdDev, 25%, 50%, 75%, 95%, 99%).

Fixes and improvements #

  • Added support for Mozilla Firefox 76 and Google Chrome 83 browser.
  • Fixed timing issue for linear increment strategy where some participants would start the test execution at the same time.
  • Improved media stream quality at the start of the test – media streams are at their top quality from the very beginning of test execution.
  • Fixed typo in one of our constant names: “asymmetric network”. We’ve stopped supporting the version with the typo and made sure the existing tests keep running even if you had been using it.
  • Fixed an issue when sometimes WebRTC statistics would not get recorded.
  • General performance updates for Loadero web application.

April 2020 #

New features #

  • waitForDownloadFinished() - a new custom command for “Javascript + Nightwatch” and “Java + TestUI” languages. This command will be helpful when downloading files in tests to ensure that the download has finished before the test exits.
  • Multiple downloads in Google Chrome - we’ve updated the options for the Google Chrome browser so that it now allows multiple downloads from the same page.
  • New media files - There are 3 new media types. These media types all have 1080p video feed with markers for visual verification of media quality. Each of the media types has an audio feed with different sound levels: -20dB, -30dB and -50dB.

Fixes and improvements #

  • We added support for Mozilla Firefox 75 and Google Chrome 81 browsers.
  • We’ve done improvements to Loadero’s test queue by improving the processing of aborted tests that were still in the queue. This means that the waiting time for the test execution to start has gotten even shorter!
  • Fixed an issue where WebRTC dump sometimes would not be available.
  • Fixed a problem that media type assert preconditions would not be evaluated.
  • Fixed an issue where double quotes in test name would cause the test to crash.

March 2020 #

New features #

  • New wiki - we’ve updated our documentation site! Check it out as you will definitely find something that is helpful to you. If you notice anything missing or unusual, don’t hesitate to let us know!
  • Multiple parallel tests - we have completed a major upgrade of the test run scheduling mechanism, now you can run tens or even hundreds of tests in parallel. This update applies to all test modes and, of course, to tests launched on custom AWS accounts.
  • Assert preconditions - we have added assert preconditions to our web application! Head to loadero.com to add preconditions to your existing asserts!
  • Resource ID - added resource ID subheader in our web app. This will help you to start using our API – just copy and paste ID where resource ID is needed.

Fixes and improvements #

  • Improved reliability of session record test mode for short tests – sometimes the recording was not stored.
  • Fixes and improvements for app loading screens, now when fetching data for logs and WebRTC statistics you will see a progress bar of how much of your data has been downloaded so far.
  • We added support for Mozilla Firefox 74 browser. Due to the current global situation, Google has decided to postpone the release of Google Chrome 81 browser hence new Google Chrome releases in Loadero will be postponed as well.