Page Phase Time: Beyond Page Load

Wouldn’t it be helpful if there was a UI performance measurement technology that:

• Tells when a user, in his/her mind, considers a web page is “done” loading without relying on browser events (e.g. Onload), network timing (e.g. TTLB), or any pre-defined thresholds of pixel changes on the screen or UI element injection/tracking.
• Measures page load time for modern web pages that are dynamically rendered.
• Provides a platform/browser/device-independent performance measurement method that can be universally applied.

Page Phase Time (PPT) is such a technology that measures performance based on user perception.

PLT Measurement: The User Perception Problem

How many times have you seen the onload time of your web page when you know (or at least suspect) that it is nowhere close to what you (or your user) really perceives? Especially on tablets/phones?

The following two images are video captures of common web pages like an Amazon product page and a Google search result page, demonstrating the difference between onload and what a user perceives.

Figure 1: Amazon.com detail page load

Figure 2: Google search result load

You could likely concede that the Amazon product page (figure-1) finished at 2.2 seconds (while Onload is 4.7 seconds), and the Google search result page (figure-2) finishes at 1.8 seconds (while Onload is 1.3 seconds).

Here we can see how important it is to tell when a normal user considers a page being “done,” not an engineer or code. By getting such perceived performance measurement data, it will be easier to make more cost-effective improvements on perceived performance, and a more effective business alignment.

Page Phase Time

Page Phase Time (PPT) is a UI performance measurement methodology that measures performance based on user perception by intelligently categorizing a whole page load cycle into major phases that represent meaningful rendering changes. Additionally, the phase that first renders the most meaningful content is called a “Key Phase” and its corresponding time is called “Key Phase Time.”

For example, according to PPT, your user thinks Amazon is “done” at 2.2 seconds and Google is done at 1.8 seconds in the above examples—these are the Key Phase times for both sites, respectively.

How It Works

PPT is an algorithm that analyzes screenshots or video captures. It first categorizes them into several major phases, and then identifies which phase is the “Key” phase.

Algorithm 1 – Categorizing Phases

The principle behind this algorithm is to detect visual changes that are substantial enough by checking the rendering transformation acceleration’s deviation:

1. Record page load cycle into N screenshots (10 frames per second)
2. Calculate pixel changing velocity — V1, V2, V3, … Vn-1 (where Vi = Pixel change ratio of Screenshot i+1 and screenshot i)
3. Calculate pixel changing acceleration — a1, a2, a3, … an-2 (where ai = Vi+1 – Vi; if ai [avg + σ(a)]}
4. Filter out adjacent outliers

A More Human Explanation

Lots of us in the web performance industry have already been looking into the pixel changing percentage, which is how Above-Fold-Time (AFT) works. Meanwhile, our purpose here is to identify “meaningful” rendering changes. A rendering change is a pixel changing percentage between two different screen captures, which I call “Pixel Changing Velocity.” Essentially, the algorithm finds each of the following:

• Meaningful rendering changes
• Drastic rendering changes
• Fast rendering changes
• Fast pixel changing velocity
• Velocity of that velocity
• Acceleration!

So what we really want is the Pixel Changing Acceleration. Therefore, “meaningful” rendering changes will be outstanding pixel changing accelerations, which are the outliers of the acceleration numbers. Here I am using 1 standard deviation to get those outliers, whose corresponding screenshots will be the phases and corresponding times will be phase times. (See figure-3 below.)

Figure 3: Finding Key Phases and Times

What If We Only Need ONE Number?

This is a very common case that when a performance engineer reports PLT to managers, he/she is allowed to only report one number to the business. Also, it is true that people will ask which “phase time” to pick up from those multiple page phase time numbers.

So next we need to find the Key Phase and its Key Phase Time among those phases.

Algorithm 2 – Finding the Key Phase and Key Phase Time

The idea here is to find the first time that the user thinks the most important content gets rendered. Usually the most important content is located around the golden section (though you could choose other places like center, center top, etc.). We think of UI design like any design, which often adheres to principles of the golden ratio.

The area it locates is called a tagging area. To find the Key Phase, we need to find out which phase’s tagging area renders the most meaningful content for the first time, which means it’s the very first tagging area where the pixel changing velocity exceeds 1 standard deviation:

Get tagging areas of all detected phases (using the Golden Ratio as default, see Fig 4, below)

Calculate *phase* pixel changing velocity — V1, V2, V3, … Vn-1 (where Vi = Pixel change ratio of tag i and tag i-1; for i=1, use first phase’s tagging area to compare with a blank image)

Find the first outlier beyond 1 standard deviation. The corresponding image is called the image tag. The corresponding phase will be the Key Phase and its corresponding timing will be the Key Phase Time — Image Tag = First {ai > [avg + σ(a)]} (see Fig 5, below)

If the image tag found is a blank image (or with only 1 color), loop step 1~3 with other potential tagging area like center, center top, center left, center right, etc.

Figure 4: Finding the tagging areas

Figure 5: Key Phase and Key Phase Time

Business Impact of Page Phase Time

Some Microsoft teams like MSN.com and MSDN.com have been using Page Phase Time for production monitoring and performance troubleshooting. Quite a few rendering bugs have been found via Page Phase Time, which improved user-perceived performance. A typical example is CodePlex.com—they turned out to have a JavaScript blocking issue (in a 3rd party BI call) that made the page blank for about 7 seconds, while the onload was reported as 2 seconds every day in production. When Page Phase Time was introduced, the bug was immediately found and fixed, resulting in a Key Phase Time of 1.5 seconds, a 467% improvement.

Of course there is no perfect solution that fixes 100% of these problems, just like Page Phase Time might not be applicable to every type of UI (including web pages). However, we are continuing to see improvement opportunities using Page Phase Time, especially for popular web pages we commonly visit.
We are constantly evolving Page Phase Time and hope it will be useful for the performance industry to solve perceived (UI) performance measurement issues. For more detail, please attend my talk at Velocity New York later this month.

This is one of a series of posts related to the upcoming Velocity conference in New York City (Oct 14-16). We hope to see you there.