Metrics at a glance
As we overview in [SIGCOMM-QoE-16], there are basically four classes of WebQoE metrics:
|Objective||Time instant||TTFB, TTFP, DOM, PLT, ATF, AATF, etc.|
|Objective||Time integral||SpeedIndex, ByteIndex/ObjectIndex, PSSI, etc.|
|Objective||Compound scores||Yslow, PageSpeed Insights, dynaTrace|
|Subjective||User feedback||MOS, user-PLT, acceptability, etc.|
Metric of the subjective class require human feedback, whereas objective metrics can be directly computed by the browser software. Only in very few cases, some objective metrics can be directly computed by network devices. For instance, user impatience [GLOBECOM-03], or the ByteIndex metrics defined in [SIGCOMM-QoE-16], can be also measured from encrypted traffic [SIGCOMM-19].
In our work, we focus on defining objective metrics of the time instant and time integral classes, as well as gathering user feedback, to further develop models of user experience. Below, we explain the main concept in a nutshell, but please go through the publications and code sections to know more.
- W3C Navigation timing metrics
- Approximated Above the Fold (AATF)
However, whereas the onLoad is very often used to assess Web performance, it is quite well know that onLoad is simplistic and does not faithfully represent the quality of Web user experience (WebQoE). As such, Google has proposed the concept of Above-the-Fold (ATF) to measure the time that the visible portion of the page takes to render. As in the example below, out of the 154 images in the page, only 8 are direclty visible ``above the fold'': the time it takes to download and render these images (ATF=5.37s) is much lower than the time it takes to download all the data in the page (onLoad=16.11s). However the user will just notice the above the fold content in a first time.
Since measuring ATF time by inspecting the visual rendering of a Web page is costly, we have proposed and implemented very simple approximated version of the ATF concept. Checkout the illustrative YouTube video in the code section, or read the [PAM-18] paper for more information.
In a nutshell, if
- ByteIndex and ObjectIndex
In a nutshell, we argue that the objects (or bytes) that are received by the browser (or the network card) can provide a good enough approximation of the visual completeness of the rendering process. We test SpeedIndex, ObjectIndex and ByteIndex (along with other metrics) on the Alexa top-100 dataset, finding high levels of correlation among these metrics as shown in the arc diagram above. For more details on the metrics, have a look at [SIGCOMM-QoE-16] or directly at the datasets we make available.
- Integral cutoff at AATF
A final technicality is worth pointing out. While this may seem a tiny detail, Time Instant and Time Integral metrics can be combined. Particularly, going back to the SpeedIndex definition, observe that the integral can be terminated earlier than at the end of the page loading process, -- for instance, at the Above The Fold (ATF) time as illustrated below.