We're excited to announce SpeedCurve's RUM product, LUX.
The name LUX is a play on "Live User eXperience" and reflects how we've taken a different approach compared to other Real User Monitoring products. SpeedCurve's mission is to help designers and developers create joyous, fast user experiences. To do that, we focus on metrics that do a better job of revealing what the user's experience is really like.
In addition to standard RUM metrics like page load time and total size, LUX includes innovative new metrics that have more to do with the user experience like start render time, number of critical blocking resources, images above the fold, and viewport size. LUX's RUM metrics help you figure out which design and development improvements will make your users happier and your business more successful.
We put a lot of thought into curating a thematic set of dashboards that help you understand the performance of your front-end, but sometimes you just want to play with the data yourself and slice 'n' dice the data in all sorts of different ways. We've added a new "Favorites" dashboard that lets you do just that. You can explore the data and build your own charts, then rearrange them and share them with the team to help demonstrate the performance issues you're focused on right now.
Here's a walkthrough showing you how to slice the available data in different ways:
We also measure the CPU usage to different key events in the rendering of the page. SpeedCurve's focus is on the user experience and getting content in front of people as fast as possible, so we show you what the CPU is doing up till the page starts to render. This reflects CPU usage during the browser critical rendering path and can highlight various issues. If there's lots of CPU idle time then you're not delivering your resources efficiently. You want to get the CPU busy nice and early rendering the page, rather than sitting idle waiting for slow resources.
In the test below we see in the first pie chart that the CPU is spending a lot of time on layout up to the start render event, which is quite a different picture from the Fully Loaded CPU usage.
If you're a performance engineer, then you're familiar with waterfall charts. They are found in browser dev tools as well as other performance services. I use multiple waterfall tools every day, but the waterfall chart I love the most is the one we've built at SpeedCurve:
Progressive Web Apps (PWAs) combine the best and newest features of the Web to deliver an experience that rivals native applications on mobile. Even better, they work on desktop, too. In fact, they work everywhere that the Web works! "Ah", you say, "that's not true! They require features that don't exist in all browsers." Because PWAs are "progressive", they can adapt to older browsers to deliver the best experience possible given the features that are available.
Given these winning attributes, it's no surprize that PWAs are the most popular movement in web development today. And while there are already numerous conferences, videos, and evangelists for PWAs, there's less focus on testing and tracking the performance of PWAs. Mark and I discussed this gap in PWA performance. In response, we added some new features to SpeedCurve that, coupled with some existing features, provide a great toolkit for evaluating the performance of PWAs.
SpeedCurve’s sweet spot is the intersection of design and performance - where the user experience lives. Other monitoring services focus on network behavior and the mechanics of the browser. Yet users rarely complain that “the DNS lookups are too slow” or “the load event fired late”. Instead, users get frustrated when they have to wait for the content they care about to appear on the screen.
The key to a good user experience is quickly delivering the critical content.
SpeedCurve users may have noticed some changes recently. At the beginning of March we released a major redesign of our Settings UI that gives users more flexibility to get the exact test results they want. The two biggest changes were to the way we emulate devices and to allow different testing configurations for different sites.
At SpeedCurve, we focus on metrics that capture the user experience. A big part of the user experience is when content actually appears in front of the user. Since stylesheets and synchronous scripts are the culprits when it comes to blocking rendering, we've rolled out some new metrics that focus on these critical blocking resources.
The most helpful innovation we made is to highlight the critical blocking stylesheets and synchronous scripts in our waterfall charts. In the following example waterfall chart for ESPN, notice how the critical stylesheets (green) and synchronous scripts (orange) have a red hash pattern. Not surprisingly, the Start Render metric is delayed until after the last of these critical blocking resources is done loading. The "scrubber" at the bottom of the waterfall (showing the screenshot at that point in time) confirms that rendering has been blocked up to this point. Explore an example of a interactive waterfall chart.
If you want to improve performance, you must start by measuring performance. But what should you measure?
Across the performance industry, the metric that's used the most is "page load time" (i.e, "window.onload" or "document complete"). Page load time was pretty good at approximating the user experience in the days of Web 1.0 when pages were simpler and each user action loaded a new web page (multi-page websites). In the days of Web 2.0 and single-page apps, page load time no longer correlates well with what the user sees. A great illustration is found by comparing Gmail to Amazon.
In the last few years some better alternatives to page load time have gained popularity, such as start render time and Speed Index. But these metrics suffer from the same major drawback as page load time: they are ignorant of what content the user is most interested in on the page.
Any performance metric that values all the content the same is not a good metric.
Users don't give equal value to everything in the page. Instead, users typically focus on one or more critical design elements in the page, such as a product image or navbar. In searching for a good performance metric, ideally we would find one that measures how long the user waits before seeing this critical content. Since browsers don't know which content is the most important, it's necessary for website owners to put these performance metrics in place. The way to do this is to create custom metrics with User Timing.
Performance budgets are an important tool for ensuring your site is delivering a great user experience. Steve first experienced performance budgets while Head Performance Engineer at Google. The practice of using budgets to track performance took off with Tim Kadlec's blog post Setting a Performance Budget. The idea is to identify your performance goals and track the metrics that help you achieve your goals.
At SpeedCurve, we give performance budgets first-class status by tracking them in the Site dashboard. Here's an example of tracking a budget for image size.
Before setting your performance budgets, you first have to be monitoring your user experience. Only then can you set budgets and thresholds for improving your baseline user experience. This also allows you to quantify the improvements you're making and share success stories across the organization like "We just improved start render by 32% by reducing image requests to half the budgeted amount".
SpeedCurve now provides a visual diff of every deploy. A full resolution PNG is captured for each URL and each pixel is diffed with the previous deploy allowing you to easily spot any visual changes you may or may not have expected.
The key to practising safe continuous deployment is to have a robust set of tools that give you immediate feedback on how your code has changed between deploys and its effect on the user experience. It's often very hard to spot all the visual changes in each deploy, especially in fast moving teams where a lot of the focus is on unit tests and other automated pass/fail systems. Visual diffs bring an increased level of tracking and confidence when you're able to compare any two deploys and see exactly what has visually changed.
We do a visual diff every time you click "Test Now" on the Deploy dashboard or you use the SpeedCurve API to trigger a round of deploy testing. Integrating with the Deploy API is super easy and provides a robust set of metrics and before/after screenshots, visual diffs, waterfall charts, filmstrips and videos for each deploy. You can then compare any two deploys to see exactly what's changed over time.
This week we added support for organizations with multiple teams and multiple users.
One of the toughest challenges was simply working out what to call the different layers within the SpeedCurve app. Developers can spend years buried inside a data model but at the end of the day the UI has to be intuitive and easy to use! I hope we’ve done that and if not we’d love your feedback.
At SpeedCurve, we want to help designers and developers have better insight into the user experience they're delivering. For websites, this means understanding when the critical parts of the page render and what might be blocking rendering.
Originally published on the O'Reilly Radar Blog
I’ve attended four O’Reilly Velocity conferences over the last year, and I was struck by a notable shift in the conversations at Velocity in Santa Clara, Calif. Many speakers and attendees have started to change their language and describe the experience of their websites and apps from the user’s perspective.
The balance has shifted from just talking about how fast or reliable a particular system is to the overall experience a user has when they interact with and experience a product. Many people are now looking at themselves from the outside in and developing more empathy for their users. The words “user” and “user experience” were mentioned again and again by speakers.
Here are recent talks from Velocity and other events that highlight this shift to UX concerns.
The dramatic growth of mobile traffic has created an urgency for websites to adapt to different screen sizes. Responsive design is the preferred technique for making web content adapt to the device on which it's viewed. This approach ensures that the appropriate content is rendered in an appropriate way on phones, tables, laptops, and any other screen being used.
A corollary to responsive design is the need for performance. Delivering content that's customized for a particular device isn't enough if its delivery is slow and frustrating. Indeed, last week Google announced that websites need to deliver their design appropriately and quickly on mobile devices or they will be demoted in Google's search results.
How can you know if your website matches these standards for adaptive content and speed? SpeedCurve's Responsive Design Dashboard answers both questions in one view.
I'm excited to announce that I've joined SpeedCurve!
When SpeedCurve was just a twinkle in Mark's eye, he contacted me about the concept and I encouraged him that a commercial version of WebPageTest was needed. When I saw the early versions of SpeedCurve, I was blown away. Mark presents traditional performance data in a way that is more compelling, revealing his strong design background.
Mark has pioneered this new territory where performance and design overlap. It's exciting to say "overlap". Many times there's little interaction between designers and performance engineers. When there is interaction, it can feel adversarial with no one wanting to give any ground. And yet, designers and performance engineers are after the same thing: creating a great user experience!
Design and performance are connected, like the yin and yang. They aren't opposing forces, but instead complement each other. Users want an experience that is rich and fast. The trick is figuring out how to do that. That's where SpeedCurve comes in.
Improve web performance by improving your design process… it needs to be iterative, mindful, principled and visual.
At my third Velocity conference for the year (this time in beautiful Barcelona) my keynote presentation explored the ways in which a thoughtfully developed design process can lead to higher functioning teams and better web performance.
The way we visualize performance data can have an impact on how we interpret and communicate performance issues within our teams.
In this talk from Velocity New York I explored the importance of data visualization and presented some of my own explorations into re-imagining the classic waterfall chart which is the mainstay of front-end performance analysis.
Skip to 15:30 if you just want to see the data visualization experiments, and you can play with them directly on lab.speedcurve.com
One of these experiments was also turned into a performance heatmap bookmarklet.
To keep inline with browser trends and average connection speeds SpeedCurve will begin updating the test setting every quarter.
For Q2 2014 we're switching to IE11 which is now the most popular version of IE according to Akamai and StatsCounter and the average connection speed has been updated to 9.8Mbps download, 2.5Mbps upload with a 10ms latency. The 9.8Mbps download speed is based on the latest State of the Internet report from Akamai.
The following article was originally published in the 2013 Performance Calendar. There's 31 great articles to explore in the calendar including Steve Souders's browser wishlist and Tim Kadlec's take on what it takes to create a performance culture.
Responsive Web Design (RWD) is now a well established technique yet it’s adoption is still surprisingly low. Guy Podjarny’s recent research shows that only one in eight websites is responsive. Often when responsive sites are designed the approach is primarily from a visual design perspective and teams struggle with the complexity of changing design and navigation patterns let alone any performance concerns, which become secondary. This can lead to serving the same sized assets to all browser widths resulting in a responsive site that is visually scaled but not performance optimised. On average responsive sites at 320 pixels wide are only 8% small than at 1600 pixels wide. Unsurprisingly these performance concerns and a lack of established techniques have made some hesitate when considering RWD adoption.
Responsive page sizes across different screen resolutions. Source: Guy Podjarny
Sort the items in a browser waterfall by Time, Savings and Slowest. Time is the default showing how assets loaded in the browser. Savings places assets at the top that have the greatest optimization potential and Slowest shows you which are the slowest overall requests.
You can now annotate the main graph on SpeedCurve with notes about deployments or specific performance optimizations you may have put live. Just click on the "Annotate" link at the bottom right of the main graph to start adding them and comparing the before and after performance of your website. A vertical line will be placed on the graph where ever a note has been added.
A lot of sites are now using responsive design techniques to create great user experiences across mobile, tablet and desktop. Unfortunately images are often just set to 100% width with the same large image being delivered to all browsers and platforms. Picturefill was an important step in delivering responsive images and David Newton's fork now extends that approach to include WebP and SVG images in the mix you send down the pipe.
Over the last few years the web performance monitoring toolset has expanded dramatically with the introduction of many new services and products. There are two main types of web performance monitoring, uptime monitoring and real user monitoring. SpeedCurve focuses on a third which I like to call web performance benchmarking.
It's important to not obsess over the full load time of your website. The point at which a user can see content and interact with your site is arguably more important. To help keep a focus on this you can now filter the main graph by the different browser events like "start render" and "DOM complete".