Added support for new browser versions: Google Chrome 110 and Mozilla
Firefox 110.
We have updated how WebRTC metrics are collected during test runs. This can
impact RAM usage measurements since calculations are not done on the
participant’s instance anymore during the test.
WebRTC metrics are now available for session recording tests as well,
meaning that WebRTC metrics are now available in all three test modes.
Note: this change is not retroactive, WebRTC metrics will not be
available for test runs from before this change was made.
WebRTC metrics are now available even if the test run has been aborted, as
long as any WebRTC metrics have been collected at all.
Added support for new browser versions: Google Chrome 109 and Mozilla
Firefox 108, 109.
Changed the in-project navigation layout - the navigation bar that used to
reside on the left side of the page is now at the top of it instead.
Added a statsType field to the WebRTC dump, this field corresponds to the
RTCStatsType enum.
Deprecation warning! To move towards compliance regarding that the id
field for RTC stats
should not be predictable,
we will be changing them from the current format (e.g.,
RTCOutBoundRTPAudioStream_4444) to an unpredictable name. The member field
which follows the last hyphen will remain as it currently is.
Until now users were able to upload avatars only for their accounts. The
project details tab under project settings has been updated, introducing the
option for users to upload an avatar for projects as well. Only members with
the Administrator role in the project have the ability to do this.
Users will now be re-directed to the last project they had open upon logging
in. You may still access the projects list by clicking on the Loadero logo
in the top left corner of the page.
Warning! We will soon change our WebRTC stats collection engine, which
may cause some changes in the formatting of the WebRTC dump. Consumed RAM
during the test may also change because of this due to stat calculations no
longer being done on the participant’s instance.
Added support for six new browser versions: Google Chrome 104, 105, 106 and
Mozilla Firefox 103, 104, 105.
Warning! API endpoints /projects/{projectID}/tests/{testID}/runs/ and
/projects/{projectID}/runs/ to read all runs will have its default result
order changed to ascending sorted by id.
Reduced test start times - on average the tests will now spend about one
minute less in the “Initializing Environment” status.
Fixed issue with Navigator.deviceMemory browser property not beeing
initialized properly.
Added support for Google Chrome 101 and Mozilla Firefox 99 browsers.
Enabled /dev/shm usage for tests. If your application utilizes /dev/shm
an increase in RAM usage can be expected.
Improved API performance by upgrading to Go 1.18
Deprecation warning! We will be fully deprecating profile_parameters
field from run participant resource. The information has been moved to the
participant_details field. Deprecation will affect these 4 routes:
GET /v2/projects/{projectID}/runs/{runID}/results/
GET /v2/projects/{projectID}/runs/{runID}/results/{resultID}/
GET /v2/projects/{projectID}/tests/{testID}/runs/{runID}/results/
GET /v2/projects/{projectID}/tests/{testID}/runs/{runID}/results/{resultID}/
Added graph granularity for WebRTC and machine statistics graphs. This
allows increasing the graph’s delta time value to enable viewing the graph
at a larger scale.
Session recordings filenames now contain a UNIX timestamp - recording start
time.
Machine statistics can now be downloaded as JSON data. The data can be found
in the participant’s results report under the “Logs” tab.
Fixed an issue that packetLoss/in/percent and packetLoss/out/percent
metrics were calculated incorrectly. All existing packet loss percent
metrics are re-calculated to be correct.
Added support for Mozilla Firefox 94 and Google Chrome 96 browsers.
Added VAT status validation to prevent automatic subscription creation with
an invalid VAT ID, which would result in our users paying extra when they
should not.
Deprecation warning! We will be making result start and end times
nullable. By the end of October we will be removing machine_stats from
GET/projects/{projectID}/tests/{testID}/runs/{runID}/results/statistics/.
Instead the data will be available in the same request under the metrics
key.
Added support for Mozilla Firefox 92, Google Chrome 93 and 94 browsers.
Bugfix for empty classification filters returning no results.
Updated post-run assert value units. Updated assert value units can be found
in our wiki page about
post run assertions.
Existing asserts do not require any changes, their expected values have been
updated to the new units accordingly.
FireFox WebRTC dump key mapping to use the same format as Google Chrome
WebRTC dumps have. This allows to use the same code to parse both dumps,
simplifying any processing.
Aborted test run results - we’re now collecting the data from aborted test
runs and displaying it. This allows to gain more visibility in the test run
even if it was stopped during execution.
Python language support for test script creation - we’ve added this
language to the 2 existing languages in Loadero. Now you can choose among
JavaScript, Java or Python to write your tests in.
Added new metric paths that will allow creating new post-run asserts.
FAQ section for pricing related questions in our landing page.
New media files have been added to run variety of scenarios in your WebRTC
tests.
In-app payments - We have finally added a long awaited feature of in-app
payments. This allows users to upgrade their plan in Loadero app, instead of
contacting support to do that.
We have created a WebRTC testing guide where we have gathered tips and
tricks for testing WebRTC applications. Download yours
here.
Added support for Google Chrome 89 and Mozilla Firefox 87 browser.
We added a safeguard that forbids deleting a test while a run is active for
that test.
We added sort functionality in test results - participants list can now be
sorted by a specific property to locate interesting participants faster.
Deprecation warning! We will be changing the default value of
participant compute unit from G2 to G1. This change only affects API users,
and only in case compute units are not otherwise specified.
We addressed issue with NightWatch Socket hangup. This is a well-known issue
for NightWatch framework. While this is not a complete fix (the issue can be
reproduced with too low compute unit assignment), it should only appear in
rare cases.
Performance improvements on test initialization phase. This means that tests
will spend less time in “Initializing Environment” status.
We have limited the maximum number of projects that can be created in a
single account. We have seen some users abuse system by creating many trial
projects, which will not be possible from now on. Each account can only
create one trial project. If that project is paid, a new project can be
created.
We have improved loading times for our frontend app, by adopting CDN
network.
Added support for Google Chrome 88 and Mozilla Firefox 85 browser.
Now previous position in the list when navigation around the web app is
stored, for example, when exiting test edit view for a test that is located
on the 2nd page, now will redirect to the 2nd page instead of the 1st.
Updated the way how error messages are presented in various forms to improve
user experience of our web app.
Fixed a bug when duplicating test, groupless participants are not
duplicated.
Improvements in web app performance and issue tracking to resolve incidents
even quicker.
Renamed assert_paths to metric_paths and assert_key to metric_key
(added redirect to the new routes during the transition period).
Deprecation warning! We will be deprecating machine_stats field in
single participant results. The data will be moved to another endpoint, the
link to the data will be available through log_paths.rru field in
GET /v2/projects/{id}/tests/{id}/runs/{id}/results/{id}/ request response.
Max available CPU - Now you can see max available CPU depending on the
selected compute unit. Of course, this allows viewing CPU data in machine
statistics graphs as well.
Filter result participants in web app - A long awaited and finally
delivered feature, you can filter result participants based on different
criteria, for example, status, browser or region.
Even more filtering options using API - Loadero API now allows to filter
different resources in even finer detail.
Fixed an issue where setRequestHeader() custom command took into account
header value capitalization even though it should be case-insensitive as per
RFC 7230.
Compute unit setting for participants - We added compute units setting
in test participant creation, it allows to optimize cost of running the
tests for the applications that don’t use much compute power, and test very
resource demanding applications as well. We also updated our pricing, now it
is based on the compute units usage instead of participants usage.
Video autoplay - We enabled video autoplay in both Mozilla Firefox and
Google Chrome browsers for the Loadero test participants. If that is
something that can impact your existing Loadero tests, make sure to review
them and update accordingly.
Accepting untrusted SSL certificates - Sometimes when testing a website,
accepting untrusted SSL certificates is necessary. Running a test in the
Session recording mode or adding commands for taking screenshots previously
helped to identify the need to accept. Now we enabled acceptSslCerts in
our test runner for this.
New media files for WebRTC testing - We have updated our built-in media
files for simulating microphone and webcam inputs. Now all of them are
running at 30 FPS with specific common audio files and frame markers that
aid quality assessment.
Access to location for your tests - We added access to location for
tests created in Loadero, now you can test websites that require location
access.
Added support for Mozilla Firefox 81/82 and Google Chrome 85/86 browsers.
We have refactored project API access - it is simpler to add authorization
for the requests you make to Loadero now. We also added an API endpoint for
reading all classificator values at once, which will simplify the
development of your tests if you are using Loadero API.
Machine graphs - you can now inspect results for each individual test
participant to see their machine resource usage over time.
Optional API parameters - When creating or modifying participant
information, it is no longer necessary to provide location, network and
media type values in the request, they will always be initialized to
defaults (for location Oregon region will be used as the default).
Adjustable script modal - now you can see more of the script in a single
view.
Export WebRTC charts images - now you can export each WebRTC chart as a
PNG image so you can integrate them in your reports more easily.
A new custom command to change any HTTP request header - override any
HTTP header value for the requests your service makes using
setRequestHeader() function! This enables testing of different
functionality that could only be tested when a custom header value is
present. Read more about this command
here for JavaScript and
here for Java.
New API endpoints - allow you to read test run results without
specifying the test ID in the request path- only project ID is mandatory.
Check out our
API documentation to
find out technical details about the new endpoints.
New script variables - a unique identifier of the participant across all
tests ever launched in Loadero (run participant ID) as well as a sequential
number of the participant within the scope of the test (participant global
ID). Read more about these two variables in our wiki for
JavaScript and
Java.
WebRTC internals dump in load tests - more data from your load tests
including automatic post-run assertions for WebRTC metrics.
Audio recording - allows recording full audio from a participant
session.
Fully customized network conditions - we have added configuration
options and functions that allow you to update only a specific parameter of
the current network conditions, such as bitrate. Read more in our wiki about
how to do that using
JavaScript + Nightwatch and
Java + TestUI.
Additional filtering options - added more filtering options for Machine
Statistics overview in test results. While charts for these statistics are
coming in the future, additional filtering options allow you to get a better
overview of the resources used during the test.
We have been working on multiple internal systems to reduce the time needed
to notice and resolve different issues that can come up. With all these
systems in place, we are able to monitor service status with much higher
precision and are able to react to events faster than ever before.
Additional aggregators for machine statistics - machine statistics now
can be analyzed in more detail filtering by new categories (StdDev,
Relative StdDev, 25%,50%, 75%, 95%, 99%).
Added support for Mozilla Firefox 76 and Google Chrome 83 browser.
Fixed timing issue for linear increment strategy where some participants
would start the test execution at the same time.
Improved media stream quality at the start of the test – media streams are
at their top quality from the very beginning of test execution.
Fixed typo in one of our constant names: “asymmetric network”. We’ve stopped
supporting the version with the typo and made sure the existing tests keep
running even if you had been using it.
Fixed an issue when sometimes WebRTC statistics would not get recorded.
General performance updates for Loadero web application.
waitForDownloadFinished() - a new custom command for “Javascript +
Nightwatch” and “Java + TestUI” languages. This command will be helpful when
downloading files in tests to ensure that the download has finished before
the test exits.
Multiple downloads in Google Chrome - we’ve updated the options for the
Google Chrome browser so that it now allows multiple downloads from the same
page.
New media files - There are 3 new media types. These media types all
have 1080p video feed with markers for visual verification of media quality.
Each of the media types has an audio feed with different sound levels:
-20dB, -30dB and -50dB.
We added support for Mozilla Firefox 75 and Google Chrome 81 browsers.
We’ve done improvements to Loadero’s test queue by improving the processing
of aborted tests that were still in the queue. This means that the waiting
time for the test execution to start has gotten even shorter!
Fixed an issue where WebRTC dump sometimes would not be available.
Fixed a problem that media type assert preconditions would not be evaluated.
Fixed an issue where double quotes in test name would cause the test to
crash.
New wiki - we’ve updated our documentation site! Check it out as you
will definitely find something that is helpful to you. If you notice
anything missing or unusual, don’t hesitate to let us know!
Multiple parallel tests - we have completed a major upgrade of the test
run scheduling mechanism, now you can run tens or even hundreds of tests in
parallel. This update applies to all test modes and, of course, to tests
launched on custom AWS accounts.
Assert preconditions - we have added assert preconditions to our web
application! Head to
loadero.com to add
preconditions to your existing asserts!
Resource ID - added resource ID subheader in our web app. This will help
you to start using our API – just copy and paste ID where resource ID is
needed.
Improved reliability of session record test mode for short tests – sometimes
the recording was not stored.
Fixes and improvements for app loading screens, now when fetching data for
logs and WebRTC statistics you will see a progress bar of how much of your
data has been downloaded so far.
We added support for Mozilla Firefox 74 browser. Due to the current global
situation, Google has decided to postpone the release of Google Chrome 81
browser hence new Google Chrome releases in Loadero will be postponed as
well.