SpeedCurve Blog

Find and fix what matters most

Thu, 22 Jun 2017 00:00:00 +1200

Being able to monitor and measure the performance of your pages is crucial. You know that already. You also know that the next step is to quickly find out what’s hurting your pages so you can stop the pain.

You want to know:

Which performance rules is my page breaking?
How do I prioritize my optimization efforts?
How can I communicate this quickly and clearly to my team?

We’re super excited to announce that you can now use SpeedCurve to answer these questions.

Inspired by the great work done by Google’s PageSpeed Insights team, we developed our own curated list of performance rules. We took the rules that are most relevant to developers in the real world, then augmented them with data that’s unique to SpeedCurve – such as critical blocking JavaScript and CSS. And we packaged it with an intuitive UI that makes it easier for you to share your test results with your team, as well as other non-technical teams throughout your organization.

At a glance, you can see our curated list of performance rules, stacked and color-coded in their order of importance for this particular page. (We define “importance” as “the total amount of positive impact that optimizing for this rule would have on performance”.)

You can then expand any of these rules and see a list of specific assets that are causing trouble – again, listed in order of importance.

And here’s a feature I really love: When you look at your list of images that need optimization, you actually see thumbnails of each one, as well as a calculation of how much further each image can be optimized. This is a fantastic, fast, effective way to communicate to other teams in your organization, such as marketing and sales.

At SpeedCurve, we know you don’t have infinite amounts of developer hours to optimize your pages. We want to help you focus on what matters. This new feature is just the beginning.

I've joined SpeedCurve! Here's why

Mon, 15 May 2017 00:00:00 +1200

TL;DR

If Mark and Steve invited you to work with them, what would you say? Exactly.

Long version

Okay, I have to elaborate a bit more about why I’m ridiculously excited about working with Mark and Steve. My first foray into the performance space was at the Velocity Conference in 2009. If you had told me then that someday I’d be working with that tall guy rocking the main stage, I would’ve thanked you for the kind words… while secretly thinking you were nuts. But here I am!

Tammy at International Women's Day Tech Talks in Toronto

And the first time I saw Mark's awesome Perfmap plugin, I knew that it was created by someone who thinks about performance and usability the same way I do... and someone I needed to get to know.

Saying yes to working with Mark and Steve was a no-brainer, of course. Here's why I'm excited not just about working with these guys, but also about working at SpeedCurve.

I spent the past two decades studying how people use the web. As already mentioned, for the past eight years I focused on how web performance intersects with user experience and business metrics. I’ve done everything from studying massive data sets of billions of user experiences to far-out neuroscientific lab studies involving EEG headsets. I even wrote a book about it.

I tend to think of my work as answering three different questions:

How do people use the web?
How do site owners want to understand how people use the web?
How can we build better and better tools that let site owners understand their visitors in ways that are most meaningful to them and their business?

As Chief Experience Officer at SpeedCurve, I get to answer all three of those questions in new and exciting ways. I’ll be marrying two aspects of experience that go together like PB&J:

User experience, as measured and monitored by our synthetic and RUM tools. I’ve said this countless times: Every company is different. Every site is different. Every visitor is different. Every visit is different. I never stop being curious about these differences, and there are always new metrics to create and new things to learn. I love SpeedCurve’s tools and I’m thrilled to have the opportunity to use them every day.

Customer experience, in terms of how our customers use our tools. I’ll be working closely with our customers to understand the specific ways they use SpeedCurve to understand their visitors. I’ll use that understanding to help make our tools – and our own customers’ experience – even better. Full disclosure: While I’ve been involved in a lot of conversations with customers in the past, it’s never been part of my official job description. I’m super excited about this aspect of my new role!

The best tools are driven by listening to what people want. I’m a good listener. Hit me up.

A/B Testing JSONP and the Preloader with RUM

Tue, 02 May 2017 00:00:00 +1200

SpeedCurve is a SPA (Single Page App) so we construct the charts dynamically using JSONP. It works great, but we're always looking for ways to make the dashboards faster. One downside to making requests dynamically is that the browser preloader isn't used. This isn't a factor for later SPA requests, but on the first page view the preloader might still bring some benefits. Or maybe not. We weren't sure, so we ran an A/B test. Long story short, doing the first JSONP request via markup caused charts to render 300 milliseconds faster.

It's easy to run this A/B test using our RUM product, LUX. The first step is to add a User Timing metric that marks when the JSONP response is received. We call this metric OnDataReceived. Some popular browsers don't support User Timing, so we use LUX's shim for mark() and measure() since it works in all browsers:

    function OnDataReceived(jsondata) {
        LUX.measure("OnDataReceived");
        ...

The code above measures the time from NavigationStart to when the JSONP response is processed. All User Timing marks and measures are automatically captured by LUX and displayed in customers' dashboards.

The next step is to split users across two buckets. In the control bucket we continue to fetch the JSONP dynamically. In the test bucket we use markup to make the JSONP request. We use LUX's addData API to segment the RUM data across these buckets. The control bucket has JsonpMarkup=no, while the test bucket has JsonpMarkup=yes. Here's the backend PHP snippet:

    if ( rand(0,1) ) {
        echo "<script>onGo();</script>\n"; // request JSONP dynamically
        echo "<script>LUX.addData('JsonpMarkup', 'no');</script>\n";
    }
    else {
        echo "<script src='/site-data?jsonp=OnDataReceived'></script>\n";
        echo "<script>LUX.addData('JsonpMarkup', 'yes');</script>\n";
    }

Within a few minutes the data shows that OnDataReceived is ~300 milliseconds faster for the JsonpMarkup=yes bucket. We let the A/B test run a few days to confirm that using markup for the initial JSONP request is faster thanks to the browser preloader.

If you make requests dynamically using JavaScript, you might be missing a performance opportunity by letting the preloader do its work. This isn't just true for JSONP requests, it also applies to images. If you're curious about this optimization, run an A/B test to see if making initial requests using markup makes your site faster and your users happier. We'd be happy to set you up with a LUX free trial to make it easy peasy (lemon squeezy).

Improved Favorites

Tue, 18 Apr 2017 00:00:00 +1200

We've improved the "Favorites" dashboard which now lets you build your own charts which:

Combine synthetic tests and LUX (real user monitoring) in one chart.
Choose average, median, or 95th percentile.
Create charts that have multiple metrics.
Select multiple values for a filter, eg, browser = Chrome or Firefox, country = UK or US.
Compare A/B tests in a single chart.

Here's a walkthrough showing you some of the new features:

We've also improved the performance and user experience of the Favorites charts. There are a lot more filter options and each chart loads individually getting charts to you faster.

We'd love your feedback. Let us know what else you'd like to see added to the Favorites dashboard. Here's a few of the things we're considering adding like filmstrips and histograms.

Measuring Blocking Resources

Thu, 02 Feb 2017 00:00:00 +1300

SpeedCurve reports the number of critical blocking resources in the page. These are the resources that block rendering. Since it's important that users see your content as quickly as possible, it's important to know what might be causing your page to render slowly. We recently enhanced the way we measure blocking resources and wanted to share those improvements with our customers as well as the performance community at large.

The main culprits that block rendering are scripts and stylesheets that are loaded synchronously. A great way to avoid this blocking problem is to load your scripts and stylesheets asynchronously. You can do that for scripts by using the async and defer attributes, plus other programmatic techniques. Loading stylesheets asynchronously is less popular but is still possible using techniques like loadCSS.

We recently improved how blocking stylesheets are measured. The first change is to ignore stylesheets loaded via loadCSS; these are asynchronous and thus don't block rendering. Similarly, we ignore stylesheets that are initially loaded with link-rel-preload and later added to the DOM dynamically. Finally, we ignore stylesheets with a "print" media type. If your site uses any of these techniques, then your "Blocking CSS" SpeedCurve metric will decrease.

We also improved how blocking scripts are measured. But before diving into that, I need to explain a subtle difference in how scripts and stylesheets block rendering. Synchronous stylesheets block the entire DOM from rendering. Synchronous scripts, however, only block the elements that follow the SCRIPT tag in the DOM tree. This is illustrated in the figure below where scripts that block rendering are shown in red. Async and defer scripts are always green (non-blocking) no matter where they occur in the DOM. Synchronous scripts, however, block rendering if they occur in the HEAD or in the BODY above any DOM elements that are in the viewport. If synchronous scripts occur below the visible DOM, then they don't block rendering.

This leads to an important optimization in how to measure blocking JS. If a page has a synchronous script at the bottom of the page (below all the visible DOM elements), then it does not block rendering (because all the visible DOM elements precede the SCRIPT tag).

In order to accurately measure blocking scripts, SpeedCurve does more than just track whether scripts are sync or async. We also determine if the script blocks rendering. We do this by determining if the synchronous script occurs before any of the DOM elements in the viewport. If a script is loaded synchronously and occurs before any DOM elements in the viewport, then we count it as "blocking JS". If it occurs after all the viewport's DOM elements, then it is not counted as blocking (even if it's loaded synchronously).

This change may decrease your "Blocking JS" SpeedCurve metric if you have synchronous scripts in the body of the page. It did for us! In the SpeedCurve.com dashboards, all the charts are loaded via synchronous scripts, but all those synchronous scripts are below the visible DOM. So our Blocking JS metric dropped from 4 to 0! This helps us focus on the resources that are most important for improving rendering.

Measuring blocking resources is difficult to do, but it's one of the most important metrics to track if you want to optimize your Start Render time. We're glad we have both of these metrics. We're also glad this code is behind us. Phew!

SpeedCurve RUM: LUX

Tue, 01 Nov 2016 00:00:00 +1300

We're excited to announce SpeedCurve's RUM product, LUX.

The name LUX is a play on "Live User eXperience" and reflects how we've taken a different approach compared to other Real User Monitoring products. SpeedCurve's mission is to help designers and developers create joyous, fast user experiences. To do that, we focus on metrics that do a better job of revealing what the user's experience is really like.

In addition to standard RUM metrics like page load time and total size, LUX includes innovative new metrics that have more to do with the user experience like start render time, number of critical blocking resources, images above the fold, and viewport size. LUX's RUM metrics help you figure out which design and development improvements will make your users happier and your business more successful.

Now that SpeedCurve has both synthetic monitoring and RUM, we can mash those up to give better insights. For example, we compare your synthetic metrics to RUM so you understand how your synthetic performance will translate when staged code is pushed to production. Also, we give you feedback on which URLs, browsers, and geo locations are popular from RUM to help you improve your synthetic test settings to better approximate the real user experience.

You can see SpeedCurve RUM in action by viewing the LUX demo account. The data comes from webpagetest.org where Pat Meenan agreed to inject the LUX snippet. There are four LUX dashboards: Live, Users, Performance, and Design.

The LUX Live dashboard constantly updates to show the users currently visiting your site.

You can hover over a row to highlight all visits for that user's session. Clicking on a row shows the details for that user's visit including Navigation Timing, User Timing, user interaction, and page design metrics.

The LUX Users dashboard shows information about your users including page views broken out by type of user interaction. It also shows the top pages, browsers, viewports, cities, and countries across all your users.

LUX Performance shows the core performance metrics like backend vs frontend time (where everyone should start), User Timing custom metrics (if you have any), and time to first user interaction (often the best reflection of the user's experience). It also contains a great example of mashing up RUM and synthetic for start render and page load time.

The LUX Design dashboard has most of our innovative views of how performance and design impact the user experience. If your start render time is slow, it's good to see how many critical blocking stylesheets and scripts are being served to real users, especially if you have third party tags. Comparing the number of images above-the-fold with the total number of images indicates whether you should be lazy loading images outside of the initial viewport. Similarly, comparing viewport size to document size indicates if you have other content that you should lazy load. LUX also shows the most popular DOM elements that users interact with.

We also make it easy to slice-and-dice the data. Expanding the Dropdown reveals various dimensions that can be used to segment the data. You can choose from the top values that are shown, or enter a value. Wildcards are supported. You can also click on any of the lists of values in the dashboards to filter on that value.

Take LUX for a test drive by going to the LUX demo account. More information, including an FAQ and API, can be found on the LUX features page.

If you'd like to signup for LUX, head over to our pricing page and fill out the signup form. We look forward to helping you create fast, joyous experiences for your users.

Build your own charts

Mon, 10 Oct 2016 00:00:00 +1300

We put a lot of thought into curating a thematic set of dashboards that help you understand the performance of your front-end, but sometimes you just want to play with the data yourself and slice 'n' dice the data in all sorts of different ways. We've added a new "Favorites" dashboard that lets you do just that. You can explore the data and build your own charts, then rearrange them and share them with the team to help demonstrate the performance issues you're focused on right now.

Here's a walkthrough showing you how to slice the available data in different ways:

We'd love your feedback. Let us know what else you'd like to see added to the Favorites dashboard. Here's a few of things we're considering adding:

Select multiple metrics on the same chart
Choose median rather than average along with support for percentiles
Custom metrics
Click through to an individual test for each data point
Filmstrips
Histograms

Track down front-end CPU hogs

Thu, 22 Sep 2016 00:00:00 +1200

Often when monitoring and debugging site performance we focus on network activity and individual resources, but what about the CPU? As more and more sites switch to using large Javascript frameworks and manipulating the page using Javascript, the execution time this code takes and the available CPU can instead become the performance bottleneck.

CPU usage for all Chrome tests

For any of SpeedCurve's Chrome-based tests, including emulated devices, we capture the Chrome Dev Tools timeline. From the browser main thread usage in the timeline we extract how busy the CPU is and what it's spending time on. Is it busy executing Javascript functions or busy laying out elements and painting pixels?

We also measure the CPU usage to different key events in the rendering of the page. SpeedCurve's focus is on the user experience and getting content in front of people as fast as possible, so we show you what the CPU is doing up till the page starts to render. This reflects CPU usage during the browser critical rendering path and can highlight various issues. If there's lots of CPU idle time then you're not delivering your resources efficiently. You want to get the CPU busy nice and early rendering the page, rather than sitting idle waiting for slow resources.

In the test below we see in the first pie chart that the CPU is spending a lot of time on layout up to the start render event, which is quite a different picture from the Fully Loaded CPU usage.

CPU trends and budgets

We've also added trending charts for the CPU time to both the Start Render and Fully Loaded browser events. This allows you to spot any significant changes over time and can highlight both CSS and Javascript changes which are causing CPU issues. In the example below we see a dramatic change in the Layout CPU time for this URL. The browser was being tied up by dodgy Javascript which was causing a huge amount of layout thrashing and reflows. Once fixed the Layout time drops dramatically.

We also support performance budgets for all the CPU metrics allowing you to get email and Slack alerts whenever a threshold is crossed. It's going to be interesting to see what sort of budgets teams put on script execution and layout. Not only can you monitor the size and number of Javascript or CSS scripts but you can ensure it's well written code that's not thrashing the CPU.

Chrome Dev Tools timeline goodness

It's amazing how much performance data is packed into the Chrome Dev Tools timeline and for every single Chrome based test on SpeedCurve we capture the timeline. You can click through to an online timeline viewer at the bottom of the SpeedCurve waterfall chart.

Here's the timeline for the URL above which had a layout thrashing issue, highlighted by the dark red bar between 2-4s. It's amazing to have this level of debugging for every test!

Test agent CPU

It's worth noting that the test agents we currently run at SpeedCurve are fairly low spec and so don't have oodles of available CPU. We do this to help emulate lower spec mobile devices. Over the last year we've seen CPU usage grow and some URLs with a lot of heavy Javascript and CSS are now maxing out the CPUs on our test agents. Although it's fascinating to see well constructed sites (I'm looking at you Smashing) not have this issue at all, with plenty of CPU idle time available as the page renders.

We all need to add CPU usage to our front-end performance toolkits!

We're thinking about introducing new agents with higher CPU specs so that you can test on both a lower spec CPU that represents an emulated device and a high CPU agent that better represents a desktop browser. Sing out in the comments if this is something you're interested in or you've discovered something interesting in your CPU usage charts.

SpeedCurve Waterfall

Tue, 20 Sep 2016 00:00:00 +1200

If you're a performance engineer, then you're familiar with waterfall charts. They are found in browser dev tools as well as other performance services. I use multiple waterfall tools every day, but the waterfall chart I love the most is the one we've built at SpeedCurve:

We've added a number of great features to our waterfall chart. In the screenshot above we see the vertical lines we've added to show Backend, Start Render, and Onload. My favorite feature is the thumbnail "scrubber" at the bottom of the waterfall. As you move your mouse horizontally through the timeline, the scrubber shows the thumbnail that corresponds to that point in time. This makes it easier to correlate the rendering experience with what's happening in the network.

For example, in the waterfall above we see that Start Render occurs at 5208 milliseconds. It's a good thing that we can see the thumbnail at that point in time because the only thing being rendered is a spinner! To find out when the actual page content is displayed, we move the mouse forward in time while watching the scrubber. Eventually we see that the actual content on the page appears at 9139 milliseconds, as shown below. By correlating this thumbnail with the network requests we can track down the cause for this four second delay, most likely the late arrival of a JSON payload containing the search results.

By default we color the network requests to reflect the Content-Type: scripts are orange, images are purple, etc. We also provide options that allow you to use colors that reflect the connection state: DNS lookup (aqua), TCP connection (orange), SSL negotiation (purple), Time-to-First-Byte (green), and content download (blue). You can also choose to have the height of each request reflect the content size. Both of these options are shown here:

I've added a screencast below to give a taste of what these features are like. But the best way to experience these awesome features is to do a free trial. Signup today to scrub your thumbnails and give your users a fast, joyous experience.

PWA Performance

Mon, 12 Sep 2016 00:00:00 +1200

Progressive Web Apps (PWAs) combine the best and newest features of the Web to deliver an experience that rivals native applications on mobile. Even better, they work on desktop, too. In fact, they work everywhere that the Web works! "Ah", you say, "that's not true! They require features that don't exist in all browsers." Because PWAs are "progressive", they can adapt to older browsers to deliver the best experience possible given the features that are available.

Given these winning attributes, it's no surprize that PWAs are the most popular movement in web development today. And while there are already numerous conferences, videos, and evangelists for PWAs, there's less focus on testing and tracking the performance of PWAs. Mark and I discussed this gap in PWA performance. In response, we added some new features to SpeedCurve that, coupled with some existing features, provide a great toolkit for evaluating the performance of PWAs.

PWA Offline Script

To answer the question "What's the key to measuring PWA performance?" let's look at some of the key features of PWAs that have the biggest impact on performance:

responsive - PWAs work across different devices and viewport sizes
progressive - they adapt to different browsers, regardless of missing features
offline - they work offline (after an initial lightweight installation)

The trickiest part is testing the offline experience. SpeedCurve is built on top of WebPageTest, so we use its scripting language to load the PWA while online, and then load it again without any network access. Here's the script using Pokedex as our example PWA:

logData 0
navigate https://www.pokedex.org/
execAndWait document.body.innerHTML=""
logData 1
blockDomainsExcept unused.zz
navigate https://www.pokedex.org/

For the initial load while we're online, we turn off data logging and navigate to the URL. For cleaner filmstrips we clear the body's content between loads. Then we turn logging on and navigate to the URL again, but this time we're offline. The offline experience is created by blocking all domains. This forces the browser to fallback to its offline capabilities.

Responsive, Progressive, & Offline

Let's look at how SpeedCurve measures these key PWA features. Using the PWA offline script shown above, we can verify that Pokedex loads without a network. Looking at the Pokedex test results for Samsung Galaxy S III, we notice something really peculiar. There's a filmstrip, but there's no waterfall chart! What also stands out is that the Start Render time is 0.1 seconds! Pokedex loads instantly because it falls back to its offline experience. This is a web developer's (and user's) dream come true: a fast experience even while offline.

SpeedCurve's Responsive dashboard for Pokedex sheds light on both the PWA's responsive and progressive behavior. Even while offline, the app loads on all browsers except IE 11. The site also behaves in a responsive manner as well. Notice how the left nav only appears on larger viewports. (Note: Since SpeedCurve uses emulation for mobile devices, it's not worth testing iOS since it doesn't support offline.)

Script Templates

Mark added a cool new "script templates" feature in SpeedCurve to make it easier to create this PWA offline script, as well as other important scripts.

In addition to the PWA offline script, we have these other script templates:

Repeat View - See what performance is like when the cache is full.
Block Third Party - This script blocks all third party requests. This is helpful for teams who want to focus on the parts of the page that they control and eliminate variability from external resources.
Don't add "PTST" to user agent - By default, WebPageTest includes the string "PTST" in the User-Agent request header, but that may cause some web services (especially ad networks) to send back a different response. Using this script ensures third parties are behaving the way they do in the real world, but it also removes the ability to filter out the testing in your own analytics.

NOTE: You need a SpeedCurve Enterprise Plan to see the scripting UI. If you don't have an Enterprise Plan you can try the PWA offline script in WebPageTest.

Happy PWA

Thanks to the PWA offline script, you can verify that your PWA loads without any network. SpeedCurve also helps you see that it's progressive across all browsers, and responsive across all viewport sizes. And don't forget about the initial load experience! That's easy to test with SpeedCurve (just enter the URL without any script). It's important to make sure that your PWA fully loads as quickly as possible so that it's ready to work if the user goes offline.

Making sure your PWA is responsive, progressive, and works offline is key. SpeedCurve is a great way to keep track of how your PWA performs and to deliver a fast, enjoyable experience for your users across all devices.

Measuring the User Experience

Tue, 03 May 2016 00:00:00 +1200

SpeedCurve’s sweet spot is the intersection of design and performance - where the user experience lives. Other monitoring services focus on network behavior and the mechanics of the browser. Yet users rarely complain that “the DNS lookups are too slow” or “the load event fired late”. Instead, users get frustrated when they have to wait for the content they care about to appear on the screen.

The key to a good user experience is quickly delivering the critical content.

SpeedCurve helps you do this by focusing on metrics that reflect what the user actually experiences. For web pages, this comes down to when the content gets rendered. In last month’s redesign, we added a new rendering chart. Here’s an example of Smashing’s rendering chart:

To emphasize the importance of rendering, we made this the first chart in the dashboard. The rendering chart includes metrics for Start Render, Speed Index, and Visually Complete. Start Render is the lower bound indicating render time for the first pixel. Visually complete bookends that with a metric for rendering the last pixel. Speed Index sits in-between showing the average render time across all pixels. (Many people were surprised to find out that Speed Index is “expressed in milliseconds”.)

Rendering is important, but not all pixels have equal importance. Instead, most pages have critical design elements (e.g, the call-to-action or hero image). Knowing when this critical content is available is even more important than overall rendering metrics. Measuring critical content is done with Custom Metrics and the User Timing spec. We jazzed up the Custom Metrics chart as part of the recent redesign:

If rendering and critical content get slower, the first place to investigate is the number of critical blocking resources. Since it’s hard for developers to track changes in the number of blocking scripts and stylesheets across releases, we added that as a new metric in SpeedCurve. Here’s the chart of critical blocking resources for NY Times:

We’re excited to find such fertile ground for accelerating how our customers deliver their critical content to users. If you’d like to see these metrics for the user experience on your website, start a free trial today and give SpeedCurve a try.

More emulated devices and improved flexibility

Thu, 24 Mar 2016 00:00:00 +1300

SpeedCurve users may have noticed some changes recently. At the beginning of March we released a major redesign of our Settings UI that gives users more flexibility to get the exact test results they want. The two biggest changes were to the way we emulate devices and to allow different testing configurations for different sites.

Emulated Phones & Tablets

Previously, emulated devices were hidden in the Responsive testing option. Now you can choose the specific phones and tablets you want to emulate. We use the device emulation built into Chrome DevTools so your tests run in the same environment as your local development. SpeedCurve provides nine emulated phones & tablets or you can build your own custom browsers and control settings like viewport size, device pixel ratio, connection speed and even packet rate loss.

Flexible Testing

Previously, geographic regions used average network speeds, but the new version of SpeedCurve allows you to test specific network configurations for different browsers. For example, you can now emulate the experience of a phone on a 2G network in Brazil or a desktop PC on fibre in Germany.

The new SpeedCurve architecture allows you to access all the granularity of WebPageTest via our user interface.

You can also distribute checks more flexibly across your sites and the competitors you are benchmarking. For example, you might currently check your website several times a day but with the new version you can test production and pre-prod hourly to see the impact of each change, and drop your competitor checks to once a day.

We are constantly working on the SpeedCurve codebase and this is the third complete re-write. Our changes are driven by customer feedback. So if you've provided feedback via Github - thank you - we are listening and please keep posting suggestions. We love hearing how we can make SpeedCurve better.

Critical Blocking Resources

Mon, 23 Nov 2015 00:00:00 +1300

At SpeedCurve, we focus on metrics that capture the user experience. A big part of the user experience is when content actually appears in front of the user. Since stylesheets and synchronous scripts are the culprits when it comes to blocking rendering, we've rolled out some new metrics that focus on these critical blocking resources.

The most helpful innovation we made is to highlight the critical blocking stylesheets and synchronous scripts in our waterfall charts. In the following example waterfall chart for ESPN, notice how the critical stylesheets (green) and synchronous scripts (orange) have a red hash pattern. Not surprisingly, the Start Render metric is delayed until after the last of these critical blocking resources is done loading. The "scrubber" at the bottom of the waterfall (showing the screenshot at that point in time) confirms that rendering has been blocked up to this point. Explore an example of a interactive waterfall chart.

Not all scripts and stylesheets necessarily have a big impact on performance, so it's important to be able to tell the blocking requests when trying to improve the user experience. In the waterfall above we see that there's an early script (request #6, "lazysizes.js") that is not marked as a critical resource. That's because this script is loaded asynchronously and thus doesn't block rendering of the page. Since we only want to highlight resources that block rendering, asynchronous scripts are not highlighted.

Similarly, in some cases it's possible to have stylesheets that don't block rendering of the main page. This typically happens with third party widgets that are loaded in an iframe, like the Twitter widget. Since stylesheets in iframes do not block the main page from rendering, they're not marked as critical resources.

In addition to highlighting critical resources in the waterfall, the number of blocking stylesheets and synchronous scripts is shown under Other Metrics.

Once we have a month of historical data for critical requests, we'll add charts to track this metric over time. This new visibility into the number of blocking requests, and which ones they are, will help teams create fast, enjoyable experiences for their users.

The critical rendering path

For more on what the critical rendering path is, why it matters so much and how to optimize asset delivery to ensure you don't block it IIya Grigorik gave a great overview of the topic at Velocity Conf 2014.

User Timing and Custom Metrics

Thu, 12 Nov 2015 00:00:00 +1300

If you want to improve performance, you must start by measuring performance. But what should you measure?

Across the performance industry, the metric that's used the most is "page load time" (i.e, "window.onload" or "document complete"). Page load time was pretty good at approximating the user experience in the days of Web 1.0 when pages were simpler and each user action loaded a new web page (multi-page websites). In the days of Web 2.0 and single-page apps, page load time no longer correlates well with what the user sees. A great illustration is found by comparing Gmail to Amazon.

In the last few years some better alternatives to page load time have gained popularity, such as start render time and Speed Index. But these metrics suffer from the same major drawback as page load time: they are ignorant of what content the user is most interested in on the page.

Any performance metric that values all the content the same is not a good metric.

Users don't give equal value to everything in the page. Instead, users typically focus on one or more critical design elements in the page, such as a product image or navbar. In searching for a good performance metric, ideally we would find one that measures how long the user waits before seeing this critical content. Since browsers don't know which content is the most important, it's necessary for website owners to put these performance metrics in place. The way to do this is to create custom metrics with User Timing.

User Timing Spec

The W3C User Timing spec provides an API for developers to add custom metrics to their web apps. This is done via two main functions: performance.mark and performance.measure.

performance.mark records the time (in milliseconds) since navigationStart
performance.measure records the delta between two marks

There are other User Timing APIs, but mark and measure are the main functions. Once your page is chock full of marks and measures, the question arises as to how to actually collect these metrics so you can track them. Thankfully, because User Timing is an official W3C specification, many metrics services automatically extract and report these custom metrics in their dashboards. This is true for SOASTA's mPulse, WebPageTest, and SpeedCurve.

Sample Custom Metrics

While the User Timing API is simple, it can sometimes be difficult to know where and when to capture these marks and measures. This is due to the complexity of the browser's inner workings, primarily the way that stylesheets and synchronous scripts block parsing and rendering. Let's look at a few examples of likely custom metrics to gain a better understanding.

A quick note about browser compatibility: User Timing is available in most browsers except Safari. In the examples below we use the native API, but in your live code you'll need to check for compatibility and either use a shim or wrapper. A good polyfill is available from Pat Meenan.

example 1: stylesheets done blocking

Loading a stylesheet blocks the entire page from rendering. This is true in all browsers as well as for stylesheets that are loaded dynamically. Therefore, knowing when stylesheets are done blocking is a good metric for understanding why rendering might start later than desired. Here's sample code for capturing this custom metric:

<link rel="stylesheet" href="/sheet1.css">
<link rel="stylesheet" href="/sheet4.css">
<script>
performance.mark("stylesheets done blocking");
</script>

The key to this snippet is making sure that it's placed below all the stylesheet LINK tags in the page. That's a reasonable requirement since most stylesheets are listed at the top of the page. While simple in appearance, some complex performance know-how is behind this snippet:

Inline scripts are not executed until all previous stylesheets are downloaded, parsed, and applied to the page. Therefore, placing the inline script after the LINK tags ensures that the mark is set after all the blocking CSS has been processed.
It's possible that a stylesheet might include @import which would pull in other stylesheets, for example, sheet1.css might cause sheet2.css and sheet3.css to be downloaded, parsed, and applied. Therefore, simply tracking a stylesheet's load time (with Resource Timing for example) isn't enough. Doing so would produce a measurement that was too short and underestimated the impact of stylesheet blocking.

In order to produce a good (fast) user experience, it's important to understand what might be blocking your web app's rendering. The likely culprits are stylesheets and synchronous scripts, but often it's hard to disambiguate which is responsible for delayed rendering. This "stylesheets done blocking" custom metric is handy because if rendering starts well after the "stylesheets done blocking" metric, then the investigation should focus on synchronous scripts, which leads to our next example.

example 2: scripts done blocking

Synchronous scripts block rendering for all following DOM elements, so knowing when script blocking is finished is also important. Here's a snippet for capturing this measurement:

<script src="a.js"></script>
<script src="b.js" async></script>
<script>
performance.mark("scripts done blocking");
</script>

The inline script (which calls performance.mark) is guaranteed to execute after script a.js has been downloaded, parsed, and executed. On the other hand, the mark will be recorded before b.js is done loading because b.js has the async attribute. This behavior matches our goal of measuring script blocking because a.js blocks rendering, but b.js doesn't.

Capturing an accurate "scripts done blocking" metric is a bit more challenging than its stylesheet counterpart. The reason is that, similar to stylesheets, the User Timing mark must be placed below all synchronous scripts. This can be difficult since websites frequently sprinkle scripts throughout the page. If that's the case, the mark measurement may occur after earlier DOM elements have rendered, and thus it doesn't accurately measure when rendering starts.

Therefore, this example is only useful for pages that have their synchronous scripts in the HEAD. Even with this limitation, this custom metric can be very useful for pages that have large scripts where the blocking time is much longer than the download time due to parsing and execution. When looking at waterfalls for these types of pages, all the scripts finish downloading but there's still a long gap before rendering starts. With a "scripts done blocking" mark, it would be possible to identify long parse and execute times as the culprit for render blocking.

example 3: fonts loaded

If your site uses custom fonts, it's important to know that some browsers won't render the text that use those fonts until after the font file is downloaded. In other browsers the text may be rendered in a system default font and then re-rendered ("flash"ed) when the custom font is loaded. Therefore, a "fonts loaded" custom metric would be valuable to indicate when text elements were rendered.

Currently, there is not an "onload" event for fonts. Nevertheless, there are techniques for observing fonts and determining when they load, for example, Font Loading Revisited with Font Events by Scott Jehl from the Filament Group as well as Web Font Loader used by Typekit and Google Fonts. You could use these to track when your fonts are done loading, and include a call to performance.mark to set the "fonts loaded" custom metric at that time.

example 4: hero images

A "hero image" is often the most important element in the page when it comes to user experience. The Hero Image Custom Metrics article describes an innovative technique for measuring when images are rendered. The snippet looks like this:

<img src="hero.jpg" onload="performance.clearMarks('img displayed'); performance.mark('img displayed');">
<script>
performance.clearMarks("img displayed");
performance.mark("img displayed");
</script>

The innovation is to take the greater of the image onload mark and the inline script mark. This accurately captures when the image is actually rendered. The image onload mark reflects the render time when the image download takes longer than any blocking resources. The inline script mark reflects the render time when the image downloads quickly but blocking resources prevent it from being rendered. In many cases, the most critical content in a page involves images, so this snippet is useful for a wide variety of custom metrics.

example 5: paragraph of text

Surprisingly, one of the most challenging custom metrics is determining when text is displayed. In addition to being blocked by stylesheets and synchronous scripts, text elements (P, SPAN, LI, DIV, etc.) can also be blocked from rendering if they use a custom font that takes a long time to download.

If the text element does not use a custom font, then the time at which it renders is captured by simply putting an inline script after the text element:

<p>This is the call to action text element.</p>
<script>
performance.mark("text displayed");
</script>

If the text element DOES use a custom font, then the render time is the maximum of the inline script's mark and the font loading mark (see previous example). The key here is to use a pattern similar to the hero image where we record the "text displayed" mark twice: once in the font-loading snippet and again in the inline script after the text element. Additionally, we clear any previous marks of the same name before setting the new mark, thus ensuring that we get the maximum of the two values which properly reflects when the text is displayed.

example 6: Single Page Apps

One of the drawbacks of relying on page load time as a metric is that it doesn't measure user actions inside a single page app (SPA) where window.onload only fires once but the user might perform multiple actions, each of which should be measured. Custom metrics solve this problem, but it requires taking on the added responsibility of creating an appropriate start time.

In order to show some sample code, lets assume we have a SPA that has an "Update" button. When clicked, an XHR or JSON request is launched to fetch new data. That returned data is passed to the updateData() function which changes the DOM. Here's sample code that measures this SPA user action:

<input type=button value="Update" onclick="fetchData()">

<script>
function fetchData() {
    performance.clearMarks("start update");
    performance.mark("start update");

    // Do an XHR or JSON request that calls updateData() with the new data.
}

function updateData(data) {
    // Update the DOM with the new data.

    performance.clearMarks("finish update");
    performance.mark("finish update");
    perforance.measure("update data", "start update", "finish update");
}
</script>

Notice that the "start update" mark is the first JavaScript in the SPA action, and "finish update" is the last JavaScript executed in the SPA action. This ensures that the entire SPA action is measured, including downloading the XHR or JSON and updating the DOM. Also notice that we used performance.measure for this first time. That's because we want to get the delta relative to a start time other than navigationStart.

Custom Metrics in SpeedCurve

A big benefit of building custom metrics with User Timing is that many performance metric services automatically extract the User Timing marks and measures to show in your performance dashboards. WebPageTest does this, and since SpeedCurve is built on top of WebPageTest we do it as well.

You can see your custom metrics in SpeedCurve by giving them a name in your Settings. In the speedcurve.com website, we use a custom metric called "heromark" to measure when the hero image (typically the chart at the top of the page) is displayed. We can give it a prettier name, "Hero Mark", using the form found under Settings:

You can add multiple custom metrics to SpeedCurve Settings. Since some pages contain dozens of marks and measures, including ones set by third party scripts, this step ensures that only the most important custom metrics are shown.

Custom metrics are visible on various SpeedCurve dashboards. For example, the Site dashboard shows a trendline of our "Hero Mark" custom metric over time.

It's extremely valuable to see custom metrics on a waterfall chart. This allows you to "debug" if your custom metric is measuring the right thing. For example, in the waterfall below for speedcurve.com, we can see that the "Hero Mark" custom metric is blocked until all the preceding scripts (orange) and stylesheets (green) are done downloading. Using the thumbnails at the bottom confirms that this is when the hero image (the chart) actually gets displayed.

No one knows your website as well as you do. Therefore, it's important that you add custom metrics that measure the content that you and your users care about most. This allows you to track the speed of what your user's are experiencing in both synthetic and RUM.

Performance budgets in action

Tue, 20 Oct 2015 00:00:00 +1300

Performance budgets are an important tool for ensuring your site is delivering a great user experience. Steve first experienced performance budgets while Head Performance Engineer at Google. The practice of using budgets to track performance took off with Tim Kadlec's blog post Setting a Performance Budget. The idea is to identify your performance goals and track the metrics that help you achieve your goals.

At SpeedCurve, we give performance budgets first-class status by tracking them in the Site dashboard. Here's an example of tracking a budget for image size.

Before setting your performance budgets, you first have to be monitoring your user experience. Only then can you set budgets and thresholds for improving your baseline user experience. This also allows you to quantify the improvements you're making and share success stories across the organization like "We just improved start render by 32% by reducing image requests to half the budgeted amount".

What should my budget be?

There are various ways to approach setting a performance budget depending on the nature of your team, and what's going to make the most sense to them and motivate change:

Be faster than your competition. Benchmark a variety of metrics and choose a target of 20% faster or less resources.
Be faster than what’s in production right now. This is great when doing a responsive redesign. Fork your code, rip out the old cruft, set new aggressive budgets and watch for alerts as you add new components and content back in.
Adopt Human Computer Interface (HCI) guidelines such as those described in Response Times: The 3 Important Limits. Aim for rendering the first meaningful content in under 1000ms.
Base your budget on user research and design principles, such as Better performance through better design. What are your users actually trying to achieve and how long will they wait?

SpeedCurve's Benchmark dashboard is a great way to see how you compare to your competitors for setting budgets. In the chart below, we can see why Smashing Magazine is so much faster - they're simply delivering a lot less JavaScript. If this was your site you could pick a metric like JS requests, set a lower budget and see if you can beat the competition!

Not all metrics are equal

It’s vitally important you evaluate which metrics you’ll budget against. For a long time now we’ve known that metrics like the page onload event are terrible indicators of user experience and yet many performance tools and organisations still use it as a key indicator of performance.

At SpeedCurve we often audit sites and it’s surprising how even very large sites can be misled into delivering a very poor user experience by focusing on optimising for the wrong metric or taking a performance guideline too literally.

The only metrics you should care about and champion across your organization are the ones that represent what a user is actually seeing as the page renders. Other metrics can be useful secondary measures to track the changing nature of your site but don't necessarily correlate to what makes a good user experience. Your users don't care how your website is constructed. What they really care about is how long it takes before they can engage with your awesome content. You need to focus on letting them see that content as quickly as possible!

Herein lies the problem, the majority of metrics are not visual metrics based on what a user is actually seeing but rather events that reflect the processes running in the browser. These two sets of metrics can paint very different pictures and it’s the reason SpeedCurve is built on WebPageTest the leading open source performance tool.

WebPageTest records video and creates filmstrips of how a page loads in a real web browser leading to unique metrics based on visual analysis from a users perspective rather than what a browser thinks it’s doing. It’s an independent measure and provides two unique metrics we can budget against, start render and speed index. These two metrics are a far better representation of the user experience than any of the built-in browser events.

Better yet define your own custom metrics

While start render and speed index are great, ultimately the best metric is the one you define yourself. Like Twitter’s time to first tweet you should be adding a custom metric that represents when the user can actually view and interact with your content. Custom metrics are easy to add with user timing marks and SpeedCurve automatically picks up any marks added to your site without additional configuration. There can be challenges around instrumenting assets like images though and Steve recently discussed these in hero image custom metrics.

Recommended metrics for performance budgets

In order of preference, here are the metrics we recommend you benchmark your user experience against:

A single UX based custom metric for each template. E.g., the hero image on a product page.
Speed Index. Paul Irish recommends aiming for under 1000.
Start render under 1000ms.

Then for each of these metrics pick a budget, set alerts and monitor it as your codebase evolves and (hopefully) improves. You can also add secondary budgets like the size and number of requests to help correlate the construction of your code base with the user experience but the three above are the ones you should really focus on.

Performance budgets in action

See how a SpeedCurve budget alert led Zillow to discover a bug that increased its image size on mobile by 4x in Bigger, Faster, and More Engaging while on a Budget. Interestingly the increase in images didn’t have a noticeable effect on Speed Index and the user experience. Does this mean Zillow could be delivering even more content than it does now as long as the initial content in the viewport renders just as quick? Breaking your performance budgets can lead to really interesting insights on how users interact with your content.

Vox Media, makers of The Verge and SB Nation, benchmarked themselves against the competition and then declared performance bankruptcy. This is a great way to engage with your community and be transparent about the challenges of delivering a rich and fast user experience.

It's oftentimes hard to give a direct answer when someone asks you, "how fast is your site?" so we're educating our team on how we quantify performance at Vox by explaining what each metric means and how they interplay with one another.
Dan Chilton - Front-End Engineering Director, Vox Media

Etsy have also done a great job of publicly reporting on performance and meeting the budgets they set for themselves.

Performance budgets are a great way to ensure you deliver an enjoyable, fast user experience. But they only work if you pick the right metrics and set the right thresholds. It's best to focus on metrics that more accurately measure the user experience, such as custom metrics, Speed Index, and start render. All of these metrics are available in SpeedCurve. We focus on measuring what users actually see, and tracking that so you can deliver it even faster.

Visual diffs on every deploy

Mon, 12 Oct 2015 00:00:00 +1300

SpeedCurve now provides a visual diff of every deploy. A full resolution PNG is captured for each URL and each pixel is diffed with the previous deploy allowing you to easily spot any visual changes you may or may not have expected.

The key to practising safe continuous deployment is to have a robust set of tools that give you immediate feedback on how your code has changed between deploys and its effect on the user experience. It's often very hard to spot all the visual changes in each deploy, especially in fast moving teams where a lot of the focus is on unit tests and other automated pass/fail systems. Visual diffs bring an increased level of tracking and confidence when you're able to compare any two deploys and see exactly what has visually changed.

We do a visual diff every time you click "Test Now" on the Deploy dashboard or you use the SpeedCurve API to trigger a round of deploy testing. Integrating with the Deploy API is super easy and provides a robust set of metrics and before/after screenshots, visual diffs, waterfall charts, filmstrips and videos for each deploy. You can then compare any two deploys to see exactly what's changed over time.

Spot the difference

Sometimes when just scanning before/after screenshots themselves without a visual diff it can be very hard to spot subtle changes. In the Tommy John example above just one extra item has been added to the main navigation. Do you think you would have spotted it without the visual diff giving you a clue as to what has changed? The visual diff really highlights the pixel level changes and also shows the effect on other nav items when a new item is added.

Quickly spot changes in grid and CSS alignment. Did you mean to move it or change a column width? With pixel by pixel matching it's easy to see exactly what's changed in your layout.

Sometimes you want to see what hasn't changed as well as what has. Here all the foreground elements in white remain in exactly the same place while the background which has been swapped out becomes bright red, really drawing your attention immediately to what's changed.

Surprisingly you can also spot the really small stuff when highlighted in red. Visual diffs are sensitive enough to pick up even single character changes helping you focus in on every single pixel that has changed from one deploy to another.

We also provide side by side filmstrips and video comparisons which are fantastic not only for reviewing the visual differences but also putting in presentations so everyone across the organization can see exactly how long it takes to render the page from a user's perspective.

Make continuous deployment safe

Google uses visual diffs to spot release issues and make continuous deployment a safer practice. Brett Slatkin gave a great presentation at Velocity on the sorts of unexpected bugs you can discover with visual diffs. Check out the video of Brett's talk and learn more about how visual diffs help you track changes to what you're delivering to users.

Multiple teams and multiple users

Wed, 30 Sep 2015 00:00:00 +1300

This week we added support for organizations with multiple teams and multiple users.

One of the toughest challenges was simply working out what to call the different layers within the SpeedCurve app. Developers can spend years buried inside a data model but at the end of the day the UI has to be intuitive and easy to use! I hope we’ve done that and if not we’d love your feedback.

SpeedCurve now has a new “organization” and “team” structure that provides a great deal of flexibility when setting up a new account.

Larger organisations or agencies can now add multiple teams to their Enterprise plan. Billing and budgets are held at the organization level making it easy to manage costs while giving new teams access to SpeedCurve monitoring for their projects.

If you're an agency you can swap the idea of internal teams for clients. This lets you have an isolated set of dashboards for each client while easily swapping between them using the new menu if you belong to the organization. Each client only has access to their dashboards, while you have the ability to browse across dashboards.

Multiple users can be invited to join a team and you control their view or edit permissions for any of the SpeedCurve dashboards. Users can belong to multiple teams and even multiple organizations.

We also recently added new dashboard sharing links so you can either send someone a read-only private url (without them having to login) or invite a user to join your team.

Here’s a walk-through of the new account structure and features. If you're on an Enterprise plan you have access to these features now, otherwise get in touch to upgrade your existing account.

Up next we’re doing a complete redesign of the Settings UI which will provide a great deal more granularity for each Site being monitored.

UX Focus for Waterfalls and Third Parties

Tue, 23 Jun 2015 00:00:00 +1200

At SpeedCurve, we want to help designers and developers have better insight into the user experience they're delivering. For websites, this means understanding when the critical parts of the page render and what might be blocking rendering.

We've redesigned our waterfall chart to really highlight the relationship between the assets on the page and their affect on the user experience. Now as you move you mouse over the waterfall chart we show you exactly what a user is seeing at that millsecond while the page loads. This makes it much easier to identify any Javascript or CSS that might be blocking the page from rendering. I recently used this new combined waterfall and filmstrip view to identify a common issue with hero images being delayed.

If rendering is blocked, it's important to figure out the cause. This isn't always easy, especially with the amount of third party content on websites today. Often the scripts and stylesheets that come with ads, analytics, and social widgets aren't delivered in a high performant way and can block the main content on the page from ever being seen by users. To help identify this possibility, you can use the Start Render line in the Third Party waterfall chart.

The Third Party waterfall chart bundles HTTP requests into groups representing first and third parties. The grouping logic is based on technology developed at SpeedCurve. We draw a vertical line representing the Start Render point on top of the waterfall. Any third parties to the left on the line have the potential of blocking rendering. You can click on each third party to see more detail and go to the full waterfall chart to investigate the filmstrip thumbnails.

Waterfall charts are great. Adding the ability to see when rendering occurs and how it's affected by third parties makes them even an even better tool for designers and developers to create great experiences for their users.

The language and metrics of UX evolve at Velocity 2015

Thu, 11 Jun 2015 00:00:00 +1200

Originally published on the O'Reilly Radar Blog

I’ve attended four O’Reilly Velocity conferences over the last year, and I was struck by a notable shift in the conversations at Velocity in Santa Clara, Calif. Many speakers and attendees have started to change their language and describe the experience of their websites and apps from the user’s perspective.

The balance has shifted from just talking about how fast or reliable a particular system is to the overall experience a user has when they interact with and experience a product. Many people are now looking at themselves from the outside in and developing more empathy for their users. The words “user” and “user experience” were mentioned again and again by speakers.

Here are recent talks from Velocity and other events that highlight this shift to UX concerns.

The next billion users

At Velocity 2015 in Santa Clara, Bruce Lawson described the dramatic growth of connected users in Asia and Africa, while highlighting the challenge faced when trying to deliver great experiences to a broad range of devices. Companies looking to these regions for their next billion users need to keep in mind that these users are on underpowered phones over slow networks with high data costs.

Reaching everyone

The techniques required to reach such a broad audience were covered by Tim Kadlec at Velocity 2015 in Santa Clara, who took us through the process of creating a high-performant responsive site that can effectively reach a diverse global audience. Here’s Kadlec’s related talk from Velocity 2014 in New York:

The path to performance

Successful performance begins far earlier than development. So, how do you get your entire team excited by it? At Velocity 2015 in Santa Clara, Katie Kovalcin discussed everyone’s role in performance and how to get all teammates caring.

Design + performance

Steve Souders really captured the growing crossover of design and performance cultures within organizations and the desperate need to move from purely technical instrumentation to a focus on custom metrics that capture what the user actually sees as the page loads. Below you’ll find Souders’ related talk from Fluent 2015:

Visualizing performance data in engaging ways

In the talk I gave, I focused on how performance data can be visualized in ways that engage the wider team, ensuring that everyone across the organization understands the impact of their work on the user experience:

Coming from a design background, I’m happy discussions at Velocity are focusing on the importance of viewing performance from the user’s perspective. The key is ensuring that the design and development process keeps the user and their experience of a product as a constant focus.

Operations has users, too

Running and maintaining high-performance, highly-available sites and apps necessitates a near obsession with monitoring and metrics. With operational metrics come a host of tools, and the current state of these tools is unfortunately well behind the end-user customer experience. Two talks focused on the need for a UX-focused approach for operations tools, one from the perspective of UX designer Tim Sheiner, the other from Rob Treat, who uses ops tools on a daily basis. Both stressed the importance of understanding the goals, tasks, and mindset of operations teams when building tools, notably for handling stressful situations like site outages.

Velocity was one of the early places where DevOps originated, bringing developers and ops folks closer together. Last week, we witnessed an additional convergence of developers, ops, and designers. It’s exciting that Velocity is helping these siloed groups find ways to share with and learn from each other. Much of this thinking comes from Lara Callender Hogan’s book Designing for Performance. If you want to produce amazing, fast, enjoyable user experiences, I recommend you read Hogan’s book and review the sessions highlighted above. And keep your focus on the user.

Responsive Design Dashboard

Wed, 29 Apr 2015 00:00:00 +1200

The dramatic growth of mobile traffic has created an urgency for websites to adapt to different screen sizes. Responsive design is the preferred technique for making web content adapt to the device on which it's viewed. This approach ensures that the appropriate content is rendered in an appropriate way on phones, tables, laptops, and any other screen being used.

A corollary to responsive design is the need for performance. Delivering content that's customized for a particular device isn't enough if its delivery is slow and frustrating. Indeed, last week Google announced that websites need to deliver their design appropriately and quickly on mobile devices or they will be demoted in Google's search results.

How can you know if your website matches these standards for adaptive content and speed? SpeedCurve's Responsive Design Dashboard answers both questions in one view.

The Responsive Design Dashbard shows the URL's filmstrip across different viewport sizes: 320x568, 568x320, 768x1024, 1024x768, 1280x768, and 1366x768. Viewing these thumbnails over time reveals how quickly the content is rendered. Similarly, you can see how the site's design adapts to different screen sizes.

The Responsive Design Dashboard is created using Chrome's mobile browser emulation feature. In addition to specifying the viewport size, the User-Agent string is also adapted to correspond to the emulated device's default characteristics. (Soon we'll adapt the network settings, as well.)

Asset Breakpoints

The filmstrips show how the website adapts to different visual breakpoints. It's also important to think about asset breakpoints - how the size and number of requests adapts based on the screen size. For example, we know we don't want to send huge desktop images to smaller screens.

The Responsive Design Dashboard shows asset breakpoints by revealing information about HTTP requests at each viewport size. The screenshot below, for example, paints a picture that we like to see where smaller screens receive fewer bytes, while larger screens get a heavier experience.

The Responsive Design Dashboard perfectly captures what SpeedCurve is about: the intersection of design and performance. Understanding how a design adapts across viewports to tracking kilobytes by Content-Type. We're having fun bringing you the information you need to create great user experiences.

Mark+Steve, Performance+Design

Tue, 24 Mar 2015 00:00:00 +1300

I'm excited to announce that I've joined SpeedCurve!

When SpeedCurve was just a twinkle in Mark's eye, he contacted me about the concept and I encouraged him that a commercial version of WebPageTest was needed. When I saw the early versions of SpeedCurve, I was blown away. Mark presents traditional performance data in a way that is more compelling, revealing his strong design background.

Mark has pioneered this new territory where performance and design overlap. It's exciting to say "overlap". Many times there's little interaction between designers and performance engineers. When there is interaction, it can feel adversarial with no one wanting to give any ground. And yet, designers and performance engineers are after the same thing: creating a great user experience!

Design and performance are connected, like the yin and yang. They aren't opposing forces, but instead complement each other. Users want an experience that is rich and fast. The trick is figuring out how to do that. That's where SpeedCurve comes in.

SpeedCurve focuses on measuring the interplay between design and performance. Rather than providing typical performance metrics that focus on resource downloads, SpeedCurve focuses on metrics and visualizations that correspond more closely to the user experience, such as:

the responsive design dashboard that shows pages rendering in different viewports, orientations, and connection speeds
custom metrics that measure what you decide is critical to your users
the impact of third party content on your site
whether you're sticking to your performance budget

I've been working on performance for more than a decade and have witnessed (and instigated) a lot of changes. I'm excited about this movement to bring design & performance closer together. In the end it'll produce what matters the most: a great web experience for users.

Mark and I are a good team to work on this - with his background heavy in design and mine in performance. Today we just re-launched the site with a major design overhaul and many new features. (Plus a new pay-as-you-go pricing plan that actually lowers costs for most of our existing customers!) We hope you'll give SpeedCurve a try, and that you'll find it useful for understanding what your users are experiencing and how to make that experience richer and faster.

Steve and Mark at the SpeedCurve re-launch in New Zealand.

Velocity: Better performance through better design

Mon, 17 Nov 2014 00:00:00 +1300

Improve web performance by improving your design process… it needs to be iterative, mindful, principled and visual.

At my third Velocity conference for the year (this time in beautiful Barcelona) my keynote presentation explored the ways in which a thoughtfully developed design process can lead to higher functioning teams and better web performance.

I start out by suggesting that before a project is launched the team must sit down and design the process that they are going to use to reach a solution. This process needs to be guided by a project specific set of principles, higher level than requirements, identifying the core ‘values’ and priorities of the project. A performance budget should be represented in one of the principles and will guide the team as they navigate through each iteration.

Small multi disciplinary teams are key to executing this better designed approach. An agile style of working results, one which no longer excludes designers but has the entire team - designer, developer, researcher and the client, revolving around a prototype. Iteration after iteration is refined through the team effort.

Once again I talk about the importance of effective visualization for engaging the entire team and beyond, to the wider organization.

What tactics do you use in your organization to improve your design process?

How is performance incorporated into discussions within your project teams?

Are you working in Agile and are designers part of that process?

What metric are you choosing to share with the wider organization around your project - and how?

Velocity: A better waterfall chart

Tue, 16 Sep 2014 00:00:00 +1200

The way we visualize performance data can have an impact on how we interpret and communicate performance issues within our teams.

In this talk from Velocity New York I explored the importance of data visualization and presented some of my own explorations into re-imagining the classic waterfall chart which is the mainstay of front-end performance analysis.

Skip to 15:30 if you just want to see the data visualization experiments, and you can play with them directly on lab.speedcurve.com

One of these experiments was also turned into a performance heatmap bookmarklet.

To date waterfall charts have been a mainstay in visualizing web performance, giving us details on each asset and how it cumulatively effects the load time of a web page. But there are many ways of visualizing performance: histograms, flame charts, heatmaps, frequency tails, and colony graphs to name just a few.

I explored the strengths and weaknesses of various types of charts and how well suited they are at answering a range of performance questions from both the perspective of a manager looking for quick high level answers and a developer needed to dig into the details. It’s important to start with the questions about performance first and then find the tools, metrics, and visualizations to best answer a question rather than inferring problems and answers from an existing metric.

Often charts are rendered along just two dimensions, which can limit the amount of information displayed and hide important detail. I presented several new visualizations that introduce depth, animation, and interaction to reveal performance bottlenecks and insights at both a holistic and detailed level.

These new visualizations make the most of modern browser techniques allowing greater interaction and demonstrating just how powerful the modern browser has become at visualizing it’s own performance.

Let me know which visualizations you enjoyed the most and I'll add them to SpeedCurve!

Velocity: Responsive in the wild

Wed, 25 Jun 2014 00:00:00 +1200

I was lucky enough to give a Lighting Demo at Velocity Conference in Santa Clara. The focus was on research I conducted into 250 responsive websites and how well optimized for performance they were.

Methodology

250 responsive websites were loaded at 17 different viewport widths between 320px and 1920px using WebPagetest. The number of requests and size of assets were captured for each width and used to build a visulization that revealed the cross-section of a site's performance breakpoints rather than it's visual breakpoints.

Results

Averages can hide the patterns in data so after visually sorting and grouping the cross sections of the sites we see that there are distinct patterns and approaches used by sites to tune their assets for performance at different widths.

Faster EC2 testing agents

Sun, 01 Jun 2014 00:00:00 +1200

As of the 1st of June SpeedCurve has switched to using faster testing agents at Amazon EC2 data centers. As web pages become more Javascript and resource heavy I've noticed more and more pages max out the CPU while performance testing.

SpeedCurve now uses EC2 m1.medium instances which have 3.75GB of RAM with faster CPU and network performance. For CPU heavy pages you should see a drop in overall load time. The connection speed for testing agents is still throttled to an average US connection speed so simple pages may not improve as much.

Overall this should provide more consistent results that are more on par with what an average user of your website might experience.

Do note that SpeedCurve's strength is competitive benchmarking and detailed build analysis. If you're after actual on the ground performance numbers then we suggest combining SpeedCurve with Real User Monitoring.

Speed Index now available

Wed, 28 May 2014 00:00:00 +1200

Speed Index is now available on SpeedCurve. Choose "SpeedIndex" from the top right of the main graphs.

Speed Index is an important measure of a user's experience as the page loads. It's based on video analysis of how the page loads over time. Here's a great video from Paul Irish introducing Speed Index at the recent Front End Ops Conference.

IE11 and average speed update

Tue, 01 Apr 2014 00:00:00 +1300

To keep inline with browser trends and average connection speeds SpeedCurve will begin updating the test setting every quarter.

For Q2 2014 we're switching to IE11 which is now the most popular version of IE according to Akamai and StatsCounter and the average connection speed has been updated to 9.8Mbps download, 2.5Mbps upload with a 10ms latency. The 9.8Mbps download speed is based on the latest State of the Internet report from Akamai.

SpeedCurve wins Webstock BNZ Start-Up Alley

Thu, 23 Jan 2014 00:00:00 +1300

I'm stoked that SpeedCurve has been chosen as one of two winners in the 2014 Webstock Start-Up Alley. I'll be using the 10k prize money and two flights to the US to attend Velocity Conf in San Fran in June and New York in September.

Performance tips for building responsive sites

Thu, 26 Dec 2013 00:00:00 +1300

The following article was originally published in the 2013 Performance Calendar. There's 31 great articles to explore in the calendar including Steve Souders's browser wishlist and Tim Kadlec's take on what it takes to create a performance culture.

----

Responsive Web Design (RWD) is now a well established technique yet it’s adoption is still surprisingly low. Guy Podjarny’s recent research shows that only one in eight websites is responsive. Often when responsive sites are designed the approach is primarily from a visual design perspective and teams struggle with the complexity of changing design and navigation patterns let alone any performance concerns, which become secondary. This can lead to serving the same sized assets to all browser widths resulting in a responsive site that is visually scaled but not performance optimised. On average responsive sites at 320 pixels wide are only 8% small than at 1600 pixels wide. Unsurprisingly these performance concerns and a lack of established techniques have made some hesitate when considering RWD adoption.

Responsive page sizes across different screen resolutions. Source: Guy Podjarny

However, we do now have a full set of techniques to effectively deliver highly performative sites that not only visually scale across devices but also deliver code and assets tuned to the width of a device. Sites need to be built utilising these techniques from a mobile first (and performance first) perspective.

Here are three key techniques that you can use to optimise performance using mobile as your starting point. Let’s tackle these in order of the page load…

Inline the initial experience (within 14k if possible)

First impressions do count. While it’s easy to throw jQuery, a framework like Bootstrap/Foundation and web fonts on a page you trade ease of use for performance. It’s increasingly a mobile world and your main aim should be to engage users as quickly as possible.

Ilya Grigorik’s recent Velocity conference presentation explored the browser's critical rendering path and highlighted exactly what it takes to deliver a mobile page within one second. Surprisingly you’ve got just one HTTP request and only 14k of HTML to play with to get content in front of users. One of the only ways to achieve this is by inlining any “above the fold” CSS and JS required to render the first screen of content. Recently Grigorik has been championing this approach and the Google Pagespeed Insight rules have been updated to reflect this best practice with recommendations on how to reduce the size of "above the fold" content.

Control and branch the loading of CSS and JS

There are plenty of automated front-end optimisation tools available like modpagespeed and CloudFlare but being a bit old-school and craft-orientated I prefer the ability to control the load order and optimise not only the individual assets but their sequencing as well.

Often responsive sites combine the desktop and mobile CSS and JS into one set of files but this results in delivering unnecessary code to the width you’re viewing. You can optimise what’s delivered by using javascript to detect the width of the page and then request specific styles and javascript appropriate to that width. There might even be whole features or different navigation patterns introduced or removed at different widths to optimize the size of the user experience.

Github write mobile specific CSS and JS resulting in much smaller and faster pages. Source: Github Blog

Here’s a few libraries you can use to enable the decision making and branching of assets as the page loads: eCSSential, Interchange, and good old Modernizer.

Shrink and art direct your images

Images often make up the largest set of assets on a page. There are two key considerations here - serving the right image to the right width and being able to art direct the image for smaller screen sizes, i.e. crop into the focal point of an image for smaller widths.

If possible eliminate an image altogether by using an icon font for icons and small graphics. Alternatively use SVG images for logos and other simple graphics. Both these techniques use vector data rather than pixels so they are small in file size and scale efficiently across all sizes including high resolution retina screens.

However when it comes to photos you need to stick with pixel based formats and this requires switching between different sized images to save bandwidth. Picturefill is an interim polyfill for responsive images allowing you to curate which image gets served to which width.

Often image generation can be a real chore and rather than manually outputting images in different formats and sizes your backend or CMS should be doing the hard work for you. CDN Connect is a promising new service that allows you to transform a single high quality image into many different formats and has an impressive range of methods to crop, filter and manipulate images. All CMS's should be offering this level of image manipulation and caching.

While you’re outputting your images to different formats consider adding WebP into the mix. It’s a new image format developed by Google that’s 30-40% smaller than a jpeg. Currently WebP is only supported in Chrome and Firefox are considering adding support. David Newton has created a fork of Picturefill that adds WebP and SVG detection so you can not only deliver the right size but the smallest format to different browsers.

Case study: the new responsive Guardian site

Comparison of page size and assets types across different responsive widths. Source: SpeedCurve

A great example using the above techniques is the new Guardian newspaper website, currently in alpha. On average it’s start render time is 3 seconds faster than the current Guardian site and the mobile width inlines CSS and is 42% smaller overall, leading to a 1 second saving over the desktop width.

Comparison of full loaded time for new Guardian responsive site (Guardian NGW) vs current Guardian site and the New York Times. Source: SpeedCurve

The Guardian have released their responsive code base on Github so you can easily dig through their development setup and see these techniques in action.

Establish a performance baseline and then go performance first

Performance optimisations like these are often invisible and it can be hard to demonstrate a return on investment without clear metrics in place. Some of these techniques can require a significant amount of work and it’s critical you establish a performance baseline first and then experiment with the techniques above to clearly show the improvements in user experience. There are great tools available to monitor the actual in browser speed and benchmark your site against others.

Putting all these techniques together you can dramatically improve the performance of your responsive site. There’s really no excuse for serving the same large sized assets across all browser widths. Make your responsive website respond not only to changing design patterns but to the browser environment it’s being served into. Go mobile first and performance first when designing and coding your next responsive website!

Thanks to Guypo for his ongoing responsive research and Patrick Hamann from The Guardian for letting me share their SpeedCurve dashboard.

New weekly emails

Wed, 06 Nov 2013 00:00:00 +1300

I've redesigned the weekly email reports to provide more trending information so you can compare week on week performance improvements. They're nice simple visualizations for forwarding around your whole team.

Let me know what you'd like to see added to these emails.

Share dashboards, basic auth and export HAR's

Tue, 15 Oct 2013 00:00:00 +1300

New features:

Share your dashboard via a private url. Find the link on the "account" page.
During site setup you can now add a user/pass for developments sites using basic authentication.
On the individual test pages below the waterfall are links to the underlying WebPagetest waterfall and ability to export a HAR file.

SpeedCurve dashboard walkthrough

Tue, 08 Oct 2013 00:00:00 +1300

See how powerful the SpeedCurve dashboard is for digging into your website's front-end performance.

Let me know which parts of the dashboard you'd like to learn more about.

Live!

Tue, 08 Oct 2013 00:00:00 +1300

SpeedCurve is now live! To mark the occasion I took the day off work and Helena & Bruno helped send out the launch emails.

All done while stuffing our faces with cake at Vevo cafe in Titirangi, Auckland. We watched the stats explode and 3 people signed up for subscriptions before we finished our hot chocolates. Start how you intend to continue I say!

SpeedCurve elevator pitch

Thu, 26 Sep 2013 00:00:00 +1200

I've been having a blast over the last couple of weeks putting an intro video together for SpeedCurve.

Big thanks to Matt from Assembly for casting an eye over my dodgy 3D AfterEffects camera work.

This is just the elevator pitch and I'll be working on some product walk through videos shortly. Let me know what web performance techniques, strategies or theory you'd like to see explained in video format. I've been inspired by the charming team at Wistia to create a video learning center for SpeedCurve.

Speed Matters talk at Auckland Web Dev Nights

Thu, 05 Sep 2013 00:00:00 +1200

Here's the slides from my presentation at the Auckland Web Dev Nights meetup. It's an overview of why front-end performance matter, how to monitor it and the challenges faced when building for an increasingly mobile world.

Content

Why speed matters, examples of the impact saving a few seconds of load time has had on revenue and engagement. The network constraints and what makes the web slow? Bandwidth, latency and it's fundamental impact on the speed of the web. An overview of tools for measuring performance, uptime monitoring, real user monitoring and performance benchmarking. How to make your website faster. Optimization tools and techniques. Muti-device challenges. Responsive vs Adaptive and delivering to mobile within a second. Drop that donut superman!

Performance Tools

Diagnotic Tools
WebPagetest and how to read a browser waterfall
Google Pagespeed Insights
YSlow

Uptime Monitoring
Pingdom

Competitive Benchmarking
SpeedCurve

Real User Monitoring (RUM)
Pingdom
New Relic (also backend, database and server health monitoring)
Google Analytics
mPulse

Performance Checklists

14 Rules for Faster-Loading Web Sites
25 tips on how to lose weight in the browser

References

Impact of performance improvements
Optimizing the Critical Rendering Path for Instant Mobile Websites (Video)
What are responsive websites made of?
Responsive web design is solid gold
Etsy - Building Resilient User Experiences
Velocity Conference Videos
Browser Diet References

Credits

These slides are heavily derived and mangled from awesome presentations by Andy Davies and Ilya Grigorik. Illustrations by Scott Johnson via Browser Diet. Feel free to steal it, adapt it and spread the word on front-end web performance.

Sort browser waterfalls

Wed, 14 Aug 2013 00:00:00 +1200

Sort the items in a browser waterfall by Time, Savings and Slowest. Time is the default showing how assets loaded in the browser. Savings places assets at the top that have the greatest optimization potential and Slowest shows you which are the slowest overall requests.

Annotate your graphs

Sun, 21 Jul 2013 00:00:00 +1200

You can now annotate the main graph on SpeedCurve with notes about deployments or specific performance optimizations you may have put live. Just click on the "Annotate" link at the bottom right of the main graph to start adding them and comparing the before and after performance of your website. A vertical line will be placed on the graph where ever a note has been added.

Easily add WebP, SVG and Retina responsive images to your site

Sun, 07 Jul 2013 00:00:00 +1200

A lot of sites are now using responsive design techniques to create great user experiences across mobile, tablet and desktop. Unfortunately images are often just set to 100% width with the same large image being delivered to all browsers and platforms. Picturefill was an important step in delivering responsive images and David Newton's fork now extends that approach to include WebP and SVG images in the mix you send down the pipe.

Picturefill

Picturefill is a fantastic browser polyfill by Scott Jehl that mimics the proposed picture element allowing you to deliver responsive images to users. It gives you control over what image is loaded and displayed at each browser width using conditional loading based on media queries. You can then curate each user's experience optimizing for screen proportions and assumed bandwidth. You could send smaller images with different aspect ratios to mobile users while desktop users with HiDPI Retina screens might get images twice as large as normal.

Now with file types

David Newton has forked Picturefill adding detection for WebP and SVG, allowing you to combine the switching of sizes and formats into one easy to implement HTML structure.

30% smaller images with WebP

With WebP detection now in place it's easy to send this new image format to browsers that support it and save 24%-34% in file size. WebP is a new image format from Google that provides lossless and lossy compression for images. The Chrome Web Store recently switched to WebP images and is saving terabytes of bandwidth a day.

If you're looking to experiment with WebP, XnConvert is an easy to use cross platform batch processing tool that supports WebP or grab the latest package from Google.

I've added WebP to the SpeedCurve homepage and now people using Chrome and Opera actually get images 56% smaller.

For more on detail on the WebP format, development status and various implementation options check out the video below and Ilya Grigorik's great overview of WebP.

How SpeedCurve fits into your web performance toolkit

Wed, 03 Jul 2013 00:00:00 +1200

Over the last few years the web performance monitoring toolset has expanded dramatically with the introduction of many new services and products. There are two main types of web performance monitoring, uptime monitoring and real user monitoring. SpeedCurve focuses on a third which I like to call web performance benchmarking.

Uptime Monitoring

Uptime services like Pingdom and Uptime Robot ping your website with an HTTP request every couple of minutes to check that it's online and available. These services often check from various geographic locations to keep an eye on network routes to your server and will send you alerts via email and txt if your website is down. Reporting is often simplistic with just the load time of the initial HTML page reported but it's perfect for knowing if your server is up and running. It's often called synthetic testing as tests are run from servers in a data centre and don't accurately represent what speeds an actual user might get.

Real User Monitoring

Real user monitoring (RUM) sends performance data directly from a user's browser to a cloud service like New Relic or Google Analytics that aggregates and reports on millions of combined measurements. RUM provides the most realistic picture of what users are actually experiencing in their browsers across different platforms and countries. Google Analytics recently updated their Site Speed report, allowing you to measure up to 10,000 page loads a day with a simple change to the sample rate in your tracking code. RUM code must be added to the pages you want to track so you can only get detailed performance insights for your own website.

Real user monitoring dashboard on New Relic.

Web Performance Benchmarking

SpeedCurve is neither an uptime or real user monitoring service. We focus on bringing two important benchmarking techniques together in a way that no other product does.

Firstly we provide detailed analysis of the build of your website over time. We dive right down into the detail of each request a page makes and demonstrate how your front-end code is effecting the performance of your site. Our analysis is based on the industry leading open source project Webpagetest. Importantly we do this analysis every 24 hours which allows you to monitor changes to your front-end code base over time, catching any regression issues and monitoring the ongoing effects of any performance optimization changes.

Secondly we benchmark your website against two of your competitors or category leaders. We do this by monitoring 3 pages on each site so you get a robust across-site comparison rather than just individual pages. It's not just your homepage that needs to be fast. This benchmarking provides you with the relative performance of each website and allows you to see what a competitor might be doing differently to make their site faster than yours. Are they making less requests than you? Are their images more optimised and grouped into sprite sheets making them a full 2 seconds faster to load? SpeedCurve will tell you.

Web performance benchmarking helps you focus on performance problems with your website build and compares you directly with your competitors to ensure you're keeping your users happy with a smokingly fast website.

Recommended Web Performance Toolset

It's vital you combine a real user monitoring tool with the build detail and competitive benchmarking that SpeedCurve provides. This way you get the best of both worlds with robust high level performance metrics from actual users and detailed build analysis and benchmarking from SpeedCurve.

Our favourite RUM tool is New Relic which gives you great insight across your stack from server health, uptime and backend right through to real user monitoring. It does require an agent to be installed on each server, so won't work for shared hosting. In that case use Google Analytics or Pingdom.

Combine New Relic with SpeedCurve and you've got a very detailed picture of performance across your entire backend stack, the front-end of your site and your competitors.

Filter graphs by browser events

Sat, 22 Jun 2013 00:00:00 +1200

It's important to not obsess over the full load time of your website. The point at which a user can see content and interact with your site is arguably more important. To help keep a focus on this you can now filter the main graph by the different browser events like "start render" and "DOM complete".

SpeedCurve launches at Velocity Conf

Wed, 19 Jun 2013 00:00:00 +1200

SpeedCurve is now in beta. Sign up, check it out and then vote for the features you'd like to see.

A huge thanks Steve Souders for encouraging me to build SpeedCurve and introducing it at Velocity Conference 2013.