NEW! Synthetic test agent updates: Chrome, Firefox and Lighthouse

This month, we've made some updates to our synthetic testing agents. In addition to upgrading the underlying operating system, we've added support for:

• Lighthouse 12.3.0 (previously 10.4.0)
• Chrome 133 (previously 126)
• Firefox 135 (previously 128)

What has changed?

We understand the sensitivity related to changes in your performance data.

Synthetic updates are known to cause baseline changes due to hardware changes, browser optimization or in the case of Lighthouse, changes to the methodology.

Here is a rundown of what's changed in this update:

Chrome

Moving from Chrome 126 to 133 should not have a huge impact on your metrics.

As of Chrome 130, transparent text is no longer eligible to be considered for Largest Contentful Paint (LCP) but this change doesn't appear to affect a larger number of sites. 

There were a number of updates to Chrome, which may affect Interaction to Next Paint (INP) introduced between 126 and 130, but as we don't measure INP with synthetic, there is no impact to your metrics (only your RUM data).

scheduler.yield was introduced in Chrome 129 and this may reduce Total Blocking Time (TBT) for some sites.

Firefox

There was no identified impact to metrics between versions 128-135.

See full release history here. 

Lighthouse

There were no significant changes to the Performance category of Lighthouse.

However, there were several updates to the Accessibility and SEO categories that included new audits and improved weighting. Additionally, the PWA category has been removed entirely.

You can learn more about those changes from the resources below:

• What's new in Lighthouse 11
• Lighthouse 12.3.0 release notes
• Change log comparison between 12.3.0 and 10.4.0

OS update

The operating system our agents use has been upgraded to Ubuntu 24.04. This upgrade was overdue and will allow us to rollout updates to our agent more quickly in the future.

The OS update has had an impact on some metrics, and during extensive testing, we've observed that Total Blocking Time (TBT) has improved due to this change.

Impact on metrics

To provide a baseline when upgrading browsers, Lighthouse, or other components of our agents, we regularly measure the speed of more than 200 sites in both our production and pre-production environments. 

During this upgrade, we've found that most metrics have remained reasonably consistent between the new and existing versions of the agent, but that Total Blocking Time (TBT) has improved significantly.

Across the corpus of sites we're testing, we've observed that Total Blocking Time (TBT) has improved by approximately 16% at the 75th percentile. 

After investigating, we've identified this improvement is due to the OS upgrade to Ubuntu 22.04

Largest Contentful Paint (LCP) and Cumulative Layout Shift (CLS) have remained largely unchanged.

What should you do next?

Our observations on how the upgrade affects metrics are based on the sites in our test corpus, and every site is different.

While Total Blocking Time might be the only expected change, there is always a chance your metrics will change based on the optimizations discussed above.

Over the next few weeks, we recommend reviewing your performance budgets and checking that they are still appropriate for your site and then adjusting them if necessary.

This is a practice we recommend doing on a regular basis as part of a 'get fast, stay fast' methodology, which you can learn more about in our Web Performance Guide to Performance Budgets. 

Private agents

For customers who host their own private SpeedCurve agents: We're planning to release an updated version of the Docker container in the next few weeks.

If you have any questions about the upgrade, its impact on your metrics, or any questions about SpeedCurve in general, you can reach us at support@speedcurve.com.