"metrics"
An Opinionated Guide to Performance Budgets
Performance budgets are one of those ideas that everyone gets behind conceptually, but then are challenged to put into practice – and for very good reason. Web pages are unbelievably complex, and there are hundreds of different metrics available to track. If you're just getting started with performance budgets – or if you've been using them for a while and want to validate your work – this post is for you.
What is a performance budget?
A performance budget is a threshold that you apply to the metrics you care about the most. You can then configure your monitoring tools to send you alerts – or even break the build, if you're testing in your staging environment – when your budgets are violated.
Understanding the basic premise of performance budgets is pretty easy. The tricky part comes when you try to put them into practice. This is when you run into three important questions:
- Which metrics should you focus on?
- What should your budget thresholds be?
- How do you stay on top of your budgets?
Depending on whom you ask, you could get very different answers to these questions. Here are mine.
New! Vitals Dashboard
Getting up to speed on Core Web Vitals seems to be at the top of everyone's to-do list these days. Just in time for the holidays, we are happy to bring you our new Vitals dashboard to help you get a huge jumpstart.
We love to visualize performance data (in case you haven't heard). We love it even more when we can be true to one of our biggest motivations at SpeedCurve: leveraging both RUM and Synthetic data to help you take action on what matters most.
First Input Delay: How vital is it?
We’ve been pretty vocal about Core Web Vitals since Google announced this initiative last spring. We love the idea of having a lean, shared set of metrics that we can all rally around – not to mention having a broader conversation about web performance that includes teams throughout an organization.
For many site owners, the increased focus on Core Web Vitals is driven by the fact that Google will be including them as a factor in search ranking in May 2021. Other folks are more interested in distilling the extremely large barrel of performance metrics into an easily digested trinity of guidelines to follow in order to provide a delightful user experience.
We’ve had some time to evaluate and explore these metrics, and we're committed to transparently discussing their pros and cons.
The purpose of this post is to explore First Input Delay (FID). This metric is unique among the three Web Vitals in that it is can only be measured using real user monitoring (RUM), while the other two (Largest Contentful Paint and Cumulative Layout Shift) can be measured using both RUM and synthetic monitoring.
In this post we'll cover:
- What is FID?
- What does FID look like across the web?
- The importance of measuring user interactions
- How JavaScript affects user behavior
- Suggestions for how you can look at FID in relation to your other key metrics
Let's dig in!
Cumulative Layout Shift: What it measures, when it works (and doesn't), and how to use it
Back in May, we shared that SpeedCurve supports Google's Core Web Vitals in both our synthetic monitoring and LUX real user monitoring tools. Two of the Web Vitals – Largest Contentful Paint (LCP) and First Input Delay (FID) – were actually available in SpeedCurve for quite a while prior to the announcement. The newcomer to the scene was Cumulative Layout Shift (CLS), and, not surprisingly, it's the metric that's gotten the most questions.
A few of the questions I've been asked (or asked myself) about Cumulative Layout Shift:
- What does CLS measure?
- How is it calculated?
- What does it mean in terms of actual user experience?
- Does it correlate to user behaviour or business metrics in any measurable way?
- What are the (inevitable) gotchas?
- Ultimately, how much should we care about CLS?
Six months in, I've had a chance to gather and look at a lot of data, talk with customers, and learn from our friends in the performance community. Here's what I've learned so far.
Visualizing Layout Shifts
One of the big challenges with Google's new Cumulative Layout Shift (CLS) metric is understanding which elements actually moved on the page, when they moved, and by how much. To help with debugging your CLS scores, we've added a new visualization to SpeedCurve that shows each layout shift and how each individual shift adds up to the final cumulative metric.
For each layout shift, we show you the filmstrip frame right before and right after the shift. We then draw a red box around the elements that moved, highlighting exactly which elements caused the shift. The Layout Shift Score for each shift also helps you understand the impact of that shift and how it adds to the cumulative score.
Fast badging for better UX
Is your site fast? Adding to a string of recent announcements including Lighthouse v6 and Core Web Vitals, Google has introduced Fast page labelling in Chrome 85 for Android. If you are curious about what this means for your site and how you can get in front of it, read on!
NEW: Lighthouse v6 support!
Lighthouse v6 has arrived! The much-anticipated update to Lighthouse is now available to SpeedCurvers as part of our latest test agent updates. Keep reading to find out what this means and how it may affect your performance metrics.
NEW: Compare ALL the things!
One of the huge benefits of tracking web performance over time is the ability to see trends and compare metrics. While you've always had the ability to do this in SpeedCurve, we recently added new functionality that makes it much easier for you to bookmark and compare different Synthetic tests in your test history.
With the new 'Compare' feature, you can now generate side-by-side comparisons that let you not only spot regressions, but easily identify what caused them:
- Compare the same page at different points in time
- Compare two versions of the same page – for example, one with ads and one without
- Understand which metrics got better or worse
- Identify which common requests got bigger/smaller or slower/faster
- Spot any new or unique requests – such as JavaScript and images – and see their impact on performance
Exciting stuff, right? But wait, there's more! Along the way, we've also made it much more intuitive for you to drill down into your detailed Synthetic test results.
Let's take a look...
Tracking Web Vitals for a better user experience
Google recently announced an initiative called 'Web Vitals', which focuses on three performance metrics they consider essential for improving the user experience:
- Largest Contentful Paint (LCP)
- First Input Delay (FID)
- Cumulative Layout Shift (CLS)
With the exception of FID, all of these metrics are available in both LUX (our RUM tool) and Synthetic. FID requires a real user for calculation and therefore is only available in LUX. In place of FID for Synthetic, we recommend tracking JS Long Tasks or Total Blocking Time as an alternative CPU metric.
User Timing Level 3: Set a value!
The Chrome team has snuck a fundamental change to how user timing marks work into Chrome 78. For many years now, our own Steve Souders has championed user timing, and he has always believed you should be able to set a value for a user timing mark or measure.
With User Timing Level 3, now you can! Level 3 lets you explicit set a startTime whereas previously you could only pass in the name of a mark.
performance.mark("name", {startTime: 1000})
Introducing Page Speed Benchmarks – a new resource for the performance community
Here are some common questions I’m asked when I talk with people about performance:
- Which metrics should I care about?
- What types of devices and connections should I test on?
- Which third parties should I be most concerned about?
- How fast should I be?
- What are some good sites I can use for benchmarking?
Today, I’m very excited to announce the release of a new project that helps answer those questions – and more!
Page Speed Benchmarks is an interactive dashboard that lets you explore and compare web performance data for leading websites across several industries – from retail to media.
With Page Speed Benchmarks, you can do things like:
- See what the different metrics actually mean in terms of user-perceived performance
- Compare how the same page renders on fast vs slow devices and connections
- Understand what makes fast sites fast (and slow sites slow)
- Get insights into how third parties can perform on different sites
- Identify sites you can use for your own competitive benchmarking
If you already like tools like the HTTP Archive, I think you'll love how you can use Page Speed Benchmarks to complement the insights you're already getting. Keep reading to find out how we set up these benchmarks, and how you can mine our test data – even if you're not a SpeedCurve user – for your own performance research.
Six web performance resolutions for the new year
For the past two years, the performance.now() conference has been the most valuable performance event I've attended. So valuable, in fact, that I've made some of the talks the cornerstone of this list of performance resolutions for 2020. I'd love to know how many – if any – of these are on your list. As always, I'd love people's feedback!
New! User Happiness metric, CI plugin, and an inspiring third-party success story
Here at SpeedCurve, the past few months have found us obsessing over how to define and measure user happiness. We've also been scrutinizing JS performance, particularly as it applies to third parties. And as always, we're constantly working to find ways to improve your experience with using our tools. See below for exciting updates on all these fronts.
As always, we love hearing from you, so please send your feedback and suggestions our way!
New LUX metrics
Over the winter holiday we added a bunch of new metrics to LUX:
- First Contentful Paint
- First CPU Idle
- Longest Long Task
- Number of Long Tasks
- Connection type
- HTML transfer size
- Total # of image requests
Metrics from 1M sites
The number of performance metrics is large and increases every year. It's important to understand what the different metrics represent and pick metrics that are important for your site. Our Evaluating rendering metrics post was a popular (and fun) way to compare and choose rendering metrics. Recently I created this timeline of performance metric medians from the HTTP Archive for the world's top ~1.3 million sites:
Lighthouse scores now available in your test results
In the year since Google rolled out Lighthouse, it's safe to say that "Will you be adding Lighthouse scoring?" is one the most common questions we've fielded here at SpeedCurve HQ. And since Google cranked up the pressure on sites to deliver better mobile performance (or suffer the SEO consequences) earlier this month, we've been getting that question even more often.
We take a rigorous approach to adding new metrics. We think the best solution is always to give you the right data, not just more data. So we're very happy to announce that after much analysis and consideration, we've added Lighthouse scores to SpeedCurve. Here's why – as well as how you can see your scores if you're already a SpeedCurve user.
First Input Delay shows how quickly your site responds to user interaction
We're excited to announce the availability of the First Input Delay metric as part of LUX, SpeedCurve's RUM product.
Introducing Last Painted Hero
We're excited to announce that we've launched Last Painted Hero as an official metric. Last Painted Hero is a synthetic metric that shows you when the last piece of critical content is painted. Keep reading to learn how Last Painted Hero works, why (and how) we created it, and how it can help you understand how your users perceive the speed of your pages.
The case for smarter heuristics
When choosing the right performance metric, my soapbox for the last few years has been "not every pixel has the same value". In other words, rather than chase dozens of different performance metrics, focus on the metrics that measure what's critical in your page.
Here at SpeedCurve, we think it's good to focus on rendering metrics, because they're a closer approximation to what the user experiences. There are some good rendering metrics out there, like start render and Speed Index, but the downside to these metrics is that they give every pixel the same value. For example, if the background renders and some ads render, that could improve your start render time and Speed Index score, but it might not have a big impact on the user's experience. Instead, it's better to measure the parts of the page that matter the most to users. We call those parts of the page the "hero elements".
More RUM metrics in your Favorites dashboards
SpeedCurve comes with a great set of dashboards for synthetic and RUM. But we know that one size does not fit all when it comes to data charts, which is why we've invested so much work into the Favorites dashboards. For customers who use LUX, it provides a place to create custom charts that combine metrics from synthetic and RUM.
We just added some new RUM metrics from LUX in Favorites to allow for even more customized monitoring:
- Page Views – The number of page views, including Single-Page-App page transitions
- Sessions – The number of unique sessions
- Session Length – The number of page views per session
- Bounced Sessions – The number of sessions that only have one page view
- Bounce Rate – The percentage of bounced sessions out of the total number of sessions
Engagement charts: See correlations between performance and user engagement
One of the best – and worst – things about real user monitoring is that it gives you unparalleled access to massive amounts of user data. The problem is when all this data leads to data indigestion. How do you know where to begin? And how do you know what to leave out in order to present a clear case for performance?
At SpeedCurve, we care about more than just showing you all your data. We want to show you the most important data. And we want to make it easy for you to share that data with people throughout your organization. That’s why we’re excited about the newest addition to our family of visualizations: engagement charts.
