"metrics"
Cumulative Layout Shift: What it measures, when it works (and doesn't), and how to use it
Back in May, we shared that SpeedCurve supports Google's Core Web Vitals in both our synthetic monitoring and LUX real user monitoring tools. Two of the Web Vitals – Largest Contentful Paint (LCP) and First Input Delay (FID) – were actually available in SpeedCurve for quite a while prior to the announcement. The newcomer to the scene was Cumulative Layout Shift (CLS), and, not surprisingly, it's the metric that's gotten the most questions.
A few of the questions I've been asked (or asked myself) about Cumulative Layout Shift:
- What does CLS measure?
- How is it calculated?
- What does it mean in terms of actual user experience?
- Does it correlate to user behaviour or business metrics in any measurable way?
- What are the (inevitable) gotchas?
- Ultimately, how much should we care about CLS?
Six months in, I've had a chance to gather and look at a lot of data, talk with customers, and learn from our friends in the performance community. Here's what I've learned so far.
Visualizing Layout Shifts
One of the big challenges with Google's new Cumulative Layout Shift (CLS) metric is understanding which elements actually moved on the page, when they moved, and by how much. To help with debugging your CLS scores, we've added a new visualization to SpeedCurve that shows each layout shift and how each individual shift adds up to the final cumulative metric.
For each layout shift, we show you the filmstrip frame right before and right after the shift. We then draw a red box around the elements that moved, highlighting exactly which elements caused the shift. The Layout Shift Score for each shift also helps you understand the impact of that shift and how it adds to the cumulative score.
Fast badging for better UX
Is your site fast? Adding to a string of recent announcements including Lighthouse v6 and Core Web Vitals, Google has introduced Fast page labelling in Chrome 85 for Android. If you are curious about what this means for your site and how you can get in front of it, read on!
NEW: Lighthouse v6 support!
Lighthouse v6 has arrived! The much-anticipated update to Lighthouse is now available to SpeedCurvers as part of our latest test agent updates. Keep reading to find out what this means and how it may affect your performance metrics.
NEW: Compare ALL the things!
One of the huge benefits of tracking web performance over time is the ability to see trends and compare metrics. While you've always had the ability to do this in SpeedCurve, we recently added new functionality that makes it much easier for you to bookmark and compare different Synthetic tests in your test history.
With the new 'Compare' feature, you can now generate side-by-side comparisons that let you not only spot regressions, but easily identify what caused them:
- Compare the same page at different points in time
- Compare two versions of the same page – for example, one with ads and one without
- Understand which metrics got better or worse
- Identify which common requests got bigger/smaller or slower/faster
- Spot any new or unique requests – such as JavaScript and images – and see their impact on performance
Exciting stuff, right? But wait, there's more! Along the way, we've also made it much more intuitive for you to drill down into your detailed Synthetic test results.
Let's take a look...
Tracking Web Vitals for a better user experience
Google recently announced an initiative called 'Web Vitals', which focuses on three performance metrics they consider essential for improving the user experience:
- Largest Contentful Paint (LCP)
- First Input Delay (FID)
- Cumulative Layout Shift (CLS)
With the exception of FID, all of these metrics are available in both LUX (our RUM tool) and Synthetic. FID requires a real user for calculation and therefore is only available in LUX. In place of FID for Synthetic, we recommend tracking JS Long Tasks or Total Blocking Time as an alternative CPU metric.
User Timing Level 3: Set a value!
The Chrome team has snuck a fundamental change to how user timing marks work into Chrome 78. For many years now, our own Steve Souders has championed user timing, and he has always believed you should be able to set a value for a user timing mark or measure.
With User Timing Level 3, now you can! Level 3 lets you explicit set a startTime whereas previously you could only pass in the name of a mark.
performance.mark("name", {startTime: 1000})
Introducing Page Speed Benchmarks – a new resource for the performance community
Here are some common questions I’m asked when I talk with people about performance:
- Which metrics should I care about?
- What types of devices and connections should I test on?
- Which third parties should I be most concerned about?
- How fast should I be?
- What are some good sites I can use for benchmarking?
Today, I’m very excited to announce the release of a new project that helps answer those questions – and more!
Page Speed Benchmarks is an interactive dashboard that lets you explore and compare web performance data for leading websites across several industries – from retail to media.
With Page Speed Benchmarks, you can do things like:
- See what the different metrics actually mean in terms of user-perceived performance
- Compare how the same page renders on fast vs slow devices and connections
- Understand what makes fast sites fast (and slow sites slow)
- Get insights into how third parties can perform on different sites
- Identify sites you can use for your own competitive benchmarking
If you already like tools like the HTTP Archive, I think you'll love how you can use Page Speed Benchmarks to complement the insights you're already getting. Keep reading to find out how we set up these benchmarks, and how you can mine our test data – even if you're not a SpeedCurve user – for your own performance research.
Six web performance resolutions for the new year
For the past two years, the performance.now() conference has been the most valuable performance event I've attended. So valuable, in fact, that I've made some of the talks the cornerstone of this list of performance resolutions for 2020. I'd love to know how many – if any – of these are on your list. As always, I'd love people's feedback!
New! User Happiness metric, CI plugin, and an inspiring third-party success story
Here at SpeedCurve, the past few months have found us obsessing over how to define and measure user happiness. We've also been scrutinizing JS performance, particularly as it applies to third parties. And as always, we're constantly working to find ways to improve your experience with using our tools. See below for exciting updates on all these fronts.
As always, we love hearing from you, so please send your feedback and suggestions our way!
New LUX metrics
Over the winter holiday we added a bunch of new metrics to LUX:
- First Contentful Paint
- First CPU Idle
- Longest Long Task
- Number of Long Tasks
- Connection type
- HTML transfer size
- Total # of image requests
Metrics from 1M sites
The number of performance metrics is large and increases every year. It's important to understand what the different metrics represent and pick metrics that are important for your site. Our Evaluating rendering metrics post was a popular (and fun) way to compare and choose rendering metrics. Recently I created this timeline of performance metric medians from the HTTP Archive for the world's top ~1.3 million sites:
Lighthouse scores now available in your test results
In the year since Google rolled out Lighthouse, it's safe to say that "Will you be adding Lighthouse scoring?" is one the most common questions we've fielded here at SpeedCurve HQ. And since Google cranked up the pressure on sites to deliver better mobile performance (or suffer the SEO consequences) earlier this month, we've been getting that question even more often.
We take a rigorous approach to adding new metrics. We think the best solution is always to give you the right data, not just more data. So we're very happy to announce that after much analysis and consideration, we've added Lighthouse scores to SpeedCurve. Here's why – as well as how you can see your scores if you're already a SpeedCurve user.
First Input Delay shows how quickly your site responds to user interaction
We're excited to announce the availability of the First Input Delay metric as part of LUX, SpeedCurve's RUM product.
Introducing Last Painted Hero
We're excited to announce that we've launched Last Painted Hero as an official metric. Last Painted Hero is a synthetic metric that shows you when the last piece of critical content is painted. Keep reading to learn how Last Painted Hero works, why (and how) we created it, and how it can help you understand how your users perceive the speed of your pages.
The case for smarter heuristics
When choosing the right performance metric, my soapbox for the last few years has been "not every pixel has the same value". In other words, rather than chase dozens of different performance metrics, focus on the metrics that measure what's critical in your page.
Here at SpeedCurve, we think it's good to focus on rendering metrics, because they're a closer approximation to what the user experiences. There are some good rendering metrics out there, like start render and Speed Index, but the downside to these metrics is that they give every pixel the same value. For example, if the background renders and some ads render, that could improve your start render time and Speed Index score, but it might not have a big impact on the user's experience. Instead, it's better to measure the parts of the page that matter the most to users. We call those parts of the page the "hero elements".
More RUM metrics in your Favorites dashboards
SpeedCurve comes with a great set of dashboards for synthetic and RUM. But we know that one size does not fit all when it comes to data charts, which is why we've invested so much work into the Favorites dashboards. For customers who use LUX, it provides a place to create custom charts that combine metrics from synthetic and RUM.
We just added some new RUM metrics from LUX in Favorites to allow for even more customized monitoring:
- Page Views – The number of page views, including Single-Page-App page transitions
- Sessions – The number of unique sessions
- Session Length – The number of page views per session
- Bounced Sessions – The number of sessions that only have one page view
- Bounce Rate – The percentage of bounced sessions out of the total number of sessions
Engagement charts: See correlations between performance and user engagement
One of the best – and worst – things about real user monitoring is that it gives you unparalleled access to massive amounts of user data. The problem is when all this data leads to data indigestion. How do you know where to begin? And how do you know what to leave out in order to present a clear case for performance?
At SpeedCurve, we care about more than just showing you all your data. We want to show you the most important data. And we want to make it easy for you to share that data with people throughout your organization. That’s why we’re excited about the newest addition to our family of visualizations: engagement charts.
Waterfall with browser events
We've improved our already fantastic interactive waterfall chart with a new collapsed mode that highlights all the key browser events. This lets you quickly scan all the events that happen as the page loads and if you scrub your mouse across the waterfall you can easily correlate each event to what the user could see at that moment.
Along with all the browser metrics you also get to see our new hero rendering times in context. Click on any event to see a large version of that moment in the filmstrip.
Hero Rendering Times: New metrics for measuring UX
The key to a good user experience is quickly delivering the content your visitors care about the most. This is easy to say, but tricky to do. Every site has unique content and user engagement goals, which is why measuring how fast critical content renders has historically been a challenging task.
That's why we're very excited to introduce Hero Rendering Times, a set of new metrics for measuring the user experience. Hero Times measure when a page's most important content finishes rendering in the browser. These metrics are available right now to SpeedCurve users.
More on how Hero Rendering Times work further down in this post. But first, I want to give a bit of back story that explains how we got to here.
The average web page is 3MB. How much should we care?
A couple of month ago, someone asked if I'd written a page bloat update recently. The answer was no. I've written a lot of posts about page bloat, starting way back in 2012, when the average page hit 1MB. To my mind, the topic had been well covered. We know that the general trend is that pages are getting bigger at a fairly consistent rate of growth. It didn't feel like there was much new territory to cover.
Also: it felt like Ilya Grigorik dropped the mic on the page bloat conversation with this awesome post, where he illustrated why the "average page" is a myth. Among the many things Ilya observed after analyzing HTTP Archive data for desktop sites, when you have outliers that weigh in at 30MB+ and more than 90% of your pages are under 5MB, an "average page size" of 2227KB (back in 2016) doesn't mean much.
The mic dropped. We all stared at it on the floor for a while, then wandered away. And now I want to propose we wander back. Why? Because the average page is now 3MB in size, and this seems like a good time to pause, check our assumptions, and ask ourselves:
Is there any reason to care about page size as a performance metric? And if we don't consider page size a meaningful metric, then what should we care about?
