# Simon Hearne | all posts and talks (full text)

Source: https://simonhearne.com/ · Index: https://simonhearne.com/llms.txt · Author: https://simonhearne.com/about/


==============================================================================

# Vector Search, Visualised: Milvus Meetup Paris

> A visual crash course in vector search, built around an ecommerce dress-finder: exact search, ANN indexes, quantisation, and the filters that quietly wreck your recall.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2026-10-07
Canonical: https://simonhearne.com/2026/vector-search-visualised-paris/
Tags: VectorDB, Zilliz, Milvus, talk, video, meetup
Video: https://www.youtube.com/watch?v=rLT1hzIU--Y
Slides: https://talks.simonhearne.com/2026-10-milvus-meetup/

---
## About the talk

This is a refined version of my [Vector Search, Visualised](/2026/visualising-vector-search/) talk, rebuilt around a single ecommerce example: finding the right dress. A shopper searches for a Maison Lune dress for a summer wedding in Provence. Keyword search finds the brand but misses the occasion, vector search finds the occasion but blurs the brand, and hybrid search finds both.

From there, the talk follows that dress-finder from zero to production: how embeddings capture meaning, why exact search doesn't scale, what ANN indexes and quantisation trade away, and why a filter as ordinary as "size 38, under €150" can silently break your results. Vector search has no `EXPLAIN`, so the talk closes with how to measure what you can't see, from index recall all the way up to conversion and revenue per search.

## What's covered

- **Why vector search exists.** Keyword matches tokens, vectors match meaning. How models turn products into numbers, and what "similar" really means for a catalogue.
- **Exact vs. approximate search.** Comparing a query to every product doesn't scale. ANN trades less than 10% recall for search that's over 100× faster and cheaper, framed as a triangle of speed, accuracy and cost.
- **ANN indexes, visualised.** HNSW navigates a graph, IVF partitions the space, DiskANN reaches past RAM.
- **Making vectors smaller.** Scalar, product and RaBitQ quantisation, PCA and Matryoshka embeddings, and refinement to buy precision back. Plus AUTOINDEX for when you'd rather not tune eight build parameters.
- **Filters and silent failure.** Why harder filters destroy more of the graph, three ways out depending on selectivity, and how to catch the missing best match before your customers do.
- **Measure what you can't see.** Golden queries scored against exact search on every deploy, and a layered metrics stack: index recall, relevance (NDCG, MRR), behaviour (CTR, zero-result searches) and business outcomes (conversion, return rate).
- **Is the index worth it?** Break-even benchmarks for Q&A, code search and agent memory, and where to run it: Milvus Lite, Standalone or Distributed, or Zilliz Cloud.

The takeaway: **spend recall on purpose.** Check your model's ceiling, pick an index, shrink, buy back, filter wisely, and measure.

Presented at the [Milvus Meetup Paris](https://luma.com/bfo1andh), hosted by Criteo, on 7 October 2026, alongside talks from Criteo on distributed vector search and Gorgias on RAG design patterns for product indexing.


==============================================================================

# Haystack Europe 2026: the bleeding edge of retrieval engineering

> Agents as search consumers, hybrid retrieval as the default, and how to evaluate search systems - a summary of my takeaways from Haystack Europe 2026.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2026-09-29
Canonical: https://simonhearne.com/2026/haystack-europe-2026/
Tags: search, VectorDB, Milvus, video, conference
Video: https://www.youtube.com/watch?v=Ipqnh4upvrI

---
[Haystack Europe](https://haystackconf.com/) took place in Berlin on 15 & 16 September as part of [Berlin Search Week](https://berlinsearchweek.com/): two days, single track, about 120 people who build search for a living. About half of the audience said they run an LLM in production!

The sessions broadly shared four underlying themes:

- **Agents are a first-class search consumer.** Retrieval stacks are being rebuilt to serve two very different query distributions: human users and agents writing long, precise, operator-rich queries.
- **Hybrid search is the default.** Netflix, S&P Global, Leroy Merlin and Coveo all described dense plus sparse retrieval fused with reciprocal rank fusion (RRF) in production.
- **Evaluation is key, and difficult.** A lot of the agenda was dedicated to the difficulty in proving that (& how) a retrieval system works.
- **New embedding models.** Two novel embedding models were presented - one to embed hierarchy and another that has deterministic dimensions to enable query-time composability.

I've written up my notes from the conference below (unfortunately I couldn't attend all of them). All talk recordings are in the [Haystack Europe 2026 playlist](https://www.youtube.com/playlist?list=PLM1qgHLmgOk0) and the full schedule is on the [Haystack website](https://haystackconf.com/schedule/).

## Agents as search consumers

### The unreasonable effectiveness of BM25 for agentic search

[Event page](https://haystackconf.com/session/the-unreasonable-effectiveness-of-bm25-for-agentic-search/) · [Recording](https://www.youtube.com/watch?v=ZnmtJKFonLk)

*[Jo Kristian Bergum](https://www.linkedin.com/in/jo-bergum/), [Hornet.dev](https://hornet.dev/)*

![bar chart comparing query length: human queries median 2, p99 8 terms; agent queries median 10, p90 17 terms](https://simonhearne.com/images/haystack-europe-2026/bm25_agents_hornet.png)
*The median agent query is longer than 99% of human queries (Hornet)*

I was nodding along throughout this talk. Jo's first point was that BM25's weak reputation in benchmarks mostly comes from untuned default parameters, not the algorithm. Also that agents are better searchers than people: they know entity names, dates and jargon, and they happily write long, operator-rich queries which are perfect for lexical matching. He also showed agents working over retrieved documents as a workspace or file system, pulling in detail progressively rather than stuffing everything into context.

On [BrowseComp-Plus](https://arxiv.org/abs/2508.06600), GPT-5 with a BM25 retriever scores 55.9% accuracy; swap in Qwen3-Embedding-8B and it reaches 70.1% with fewer search calls. Jo showed that this benchmark is flawed - and that a tuned BM25 algorithm can achieve 95.7% answer-presence. Better retrieval is always relevant, and fewer calls is also a cost and latency win. So basically - test it on your systems, don't be afraid to change the defaults, and choose the approach that best balances accuracy and performance (like all search implementations!).

I've added this to my (growing) list of experiments to run - if you give an agent access to lexical and dense search plus weighted RRF, which gives the best answer in the fewest turns? I avoided the temptation to try to do this in the evening before my talk the next day!

### The RAG Cost Curve: When an index beats live search

[Event page](https://haystackconf.com/session/the-rag-cost-curve-when-an-index-beats-live-search/) · [Slides](https://talks.simonhearne.com/rag-cost-curve/) · [Recording](https://www.youtube.com/watch?v=Ipqnh4upvrI)

*[Simon Hearne](https://www.linkedin.com/in/simonhearne/), Zilliz - that's me ([full write-up](/2026/rag-cost-curve/))*

![chart of cost per correct answer for no retrieval, agentic and indexed retrieval on known and unseen code](https://simonhearne.com/images/haystack-europe-2026/rag_cost_curve.png)
*Cost per correct answer: indexes win on code the model has never seen*

Mine was the only talk that put a price on the index versus live-search decision, which surprised me given how much of the agenda was about agents. The argument: "RAG or agentic search" is a false binary. It's a cost model, decided by corpus size, update frequency and query volume.

I grounded it in two open-source systems, [memsearch](https://github.com/zilliztech/memsearch) (agent memory over Markdown) and [claude-context](https://github.com/zilliztech/claude-context) (semantic code search), and in benchmarks of what compression really costs across scalar, product and [RaBitQ](https://arxiv.org/abs/2405.12497) quantisation.

During the benchmarks I found a few interesting things:

- Agents don't need indexes for code that models have been trained on (e.g. long-running public GitHub repositories)
- For private code, the break-even point for an index is in the tens to hundreds of daily queries
- If the index is ~free (local, serverless, on an existing box) then the break-even point is on the first query!

### Agent memory that works with your existing search stack

[Event page](https://haystackconf.com/session/agent-memory-that-works-with-your-existing-search-stack/) · [Slides](https://plnech.github.io/talks/haystack26/talk.html#1) · [Recording](https://www.youtube.com/watch?v=3aKXXRrCkk8)

*[Paul-Louis Nech](https://www.linkedin.com/in/plnech/), Algolia*

![table comparing four memory strategies by seconds, input tokens and facts recalled; memory as a tool uses 9,184 tokens to recall one fact](https://simonhearne.com/images/haystack-europe-2026/agent_memory.png)
*Four approaches to agent memory: memory as a tool uses 27× the tokens yet gives less context (Algolia)*

Algolia added memory to its [Agent Studio](https://www.algolia.com/products/ai/agent-studio) under one constraint: customers run keyword search, neural search or both, and nobody should need vector-only infrastructure to use it. Paul-Louis showed how far classic search features go for agent memory: tie-breaking ranking, numeric filters, optional word boosting and LLM-generated tags that are boosted at query time. He also covered compressing conversations into multi-query payloads, keyword extraction across 35 languages, and [LongMemEval](https://arxiv.org/abs/2410.10813) results, including why benchmarks are mostly useless. Again - benchmark on your own data and use real customer feedback.

Paul-Louis' thesis is different to mine: we agree that agent memory is a retrieval problem, not a transcript-replay problem. Where our talks differed is on what the index should hold. Keyword memory is a measurable improvement on no memory for answer-presence, and on context stuffing for latency & token budgets. One of the most interesting moments in the talk (for me) was that removing stop-words degraded answer-presence - it makes sense on reflection: English stop-words include `my` and `their`, `not` and `is`, `few` and `most`. Stripping those words can totally change the meaning of a sentence.

This was the second talk (after [Jo's](#the-unreasonable-effectiveness-of-bm25-for-agentic-search)) that made me want to evaluate BM25 vs. vector search for memory infrastructure. Both claude-context and memsearch use hybrid search so I need to separate the queries to determine answer-presence for each individually compared to the default RRF.

## Hybrid search as the default

### From Tries to Transformers: Scaling Semantic Search

[Event page](https://haystackconf.com/session/from-tries-to-transformers-scaling-semantic-search/) · [Slides (PDF)](https://pretalx.com/media/haystackeu26/submissions/7F3XA8/resources/From_Tries_to_gXhR9ny.pdf) · [Recording](https://www.youtube.com/watch?v=dW2vP_Jl7MM)

*[Ivan Provalov](https://www.linkedin.com/in/provalov/), Netflix*

![diagram of a five-step pipeline: query logs, human annotations, LLM-as-judge, quality gate, train and deploy](https://simonhearne.com/images/haystack-europe-2026/netflix.png)
*The data engine behind Netflix's query understanding model*

Ivan shared some great statistics from Netflix that were new (and surprising) to me: only 16% of Netflix discovery starts with search, and the average query is three characters long (typing on TVs is hard). At that length there are no semantics to embed: a user typing `wes` could be looking for Wes Anderson, West Side Story or Westerns.

Netflix's answer is to stop picking one interpretation. They generate every plausible facet mapping, retrieve for each, and merge the lists with reciprocal rank fusion (plus personalisation), with a "did you mean" helper on top. The stack has moved from trie lookups to a [Joint-BERT](https://arxiv.org/abs/1902.10909) intent model and a hybrid lexical plus embedding engine, running at 25k QPS (!). Ivan also shared Netflix's open source query testing framework [Netflix/q](https://github.com/Netflix/q), which he demoed in a [lightning talk](https://www.youtube.com/watch?v=HJJfMubvEcE).

Short-query ambiguity is a query understanding problem (something I looked at for [typeahead search](/2026/fast-typeahead-milvus/)). Fusion and ranking based on inferred intent is a good solution to a complex UX problem. I'd love to learn how Netflix use customer feedback (CTR, re-search rate etc.) and metrics like MRR to measure and refine their search system.

### Query Understanding with Late Interaction & Wormhole Vectors

[Event page](https://haystackconf.com/session/query-understanding-with-late-interaction-wormhole-vectors/) · [Recording](https://www.youtube.com/watch?v=XeA27uhpY_4)

*[Trey Grainger](https://www.linkedin.com/in/treygrainger/), [Searchkernel](https://searchkernel.com/)*

![slide showing wormhole vectors connecting sparse lexical, dense semantic and behavioral vector spaces](https://simonhearne.com/images/haystack-europe-2026/wormhole_vectors.png)
*Wormhole vectors traverse between sparse, dense and behavioural vector spaces*

Trey's opening point was that most teams lean on ranking maths and hope it compensates for never really understanding the query. The talk was then a toolkit for fixing that upstream.

The headline idea is wormhole vectors. Standard hybrid search runs a lexical query and a dense query in parallel, then fuses the lists. A wormhole instead *travels* between spaces: run the lexical query, pool the dense vectors of the documents it matched (synthesising a new, pool-averaged vector), and search the dense space from that point. It also works in reverse, from dense results back into sparse terms via a [semantic knowledge graph](https://arxiv.org/abs/1609.00464). Around it he covered [late interaction](https://arxiv.org/abs/2004.12832) models and collaborative filtering for learning latent features from user behaviour.

Trey reports benchmarks beating conventional hybrid search (Wormhole precision@10 of 0.862 vs RRF precision@10 of 0.795). If that holds up broadly, it changes what a hybrid search API should expose: not just two queries and a fusion weight, but result-set pooling as a first-class operation. Another thing to add to your benchmarks and evaluations!

As both BM25 and dense search have got cheaper and faster, this concept is realistic in production with low double digit millisecond latency budgets. I would like to test it vs RRF with oversampling and semantic re-ranking.

### Sparse encoders: bridging products and knowledge graphs

[Event page](https://haystackconf.com/session/sparse-encoders-bridging-products-and-knowledge-graphs/) · [Slides (PDF)](https://pretalx.com/media/haystackeu26/submissions/MN9CDU/resources/Haystack_EU_2_cZZdGOm.pdf) · [Recording](https://www.youtube.com/watch?v=fwW0ujG_c74)

*[François Gaillard](https://www.linkedin.com/in/fran%C3%A7ois-gaillard-b2874988/), ADEO (Leroy Merlin, Bricoman)*

![sparse search for 'drill' showing weighted knowledge-graph concepts and top three matching drill products](https://simonhearne.com/images/haystack-europe-2026/sparse_encoders.png)
*A readable sparse vector: knowledge-graph concepts and weights for the query 'drill'*

ADEO runs home-improvement stores in Europe such as [Leroy Merlin](https://www.leroymerlin.fr/). ADEO's products have complex hierarchy and concepts (building materials vs. tools etc), so they need every product and query mapped onto its knowledge graph of concepts. Millions of product-concept pairs were poorly aligned, and a meaningful share of queries (18M / 21%) did not map correctly.

Dense embeddings didn't help: off-the-shelf models don't encode the same concepts, and the results couldn't be explained when a business rule misfired. This lack of 'explainability' is an issue I have encountered many times before, especially in e-commerce and legal document search scenarios.

François presented their solution to this challenge: a custom [SPLADE](https://arxiv.org/abs/2107.05720)-style sparse encoder which injects knowledge-graph concepts straight into the model vocabulary as tokens. Every sparse vector is then expressed in the catalogue's own vocabulary. A generative [T5 expansion](https://github.com/castorini/docTTTTTquery) approach lost on inference time (600 ms against 60 ms), but latency was reduced by adding [FLOPS-loss](https://arxiv.org/abs/2004.05665) and tuning the weights to push sparsity past 99.9%, without losing semantic depth.

This is the most practical case for learned sparse retrieval I've seen. You get semantic matching with concept understanding, a vector you can actually read and debug (it's just a list of terms), and it can be supported by any search engine that can index sparse vectors (such as [Milvus](https://milvus.io/docs/sparse_vector.md)).

### SPGE Enhanced Search: relevancy engine for Agentic RAG

[Event page](https://haystackconf.com/session/spge-enhanced-search-relevancy-engine-for-agentic-rag/) · [Slides (PDF)](https://pretalx.com/media/haystackeu26/submissions/7FLXGJ/resources/SPGE_Enhanced_4ejVhQL.pdf) · [Recording](https://www.youtube.com/watch?v=ydCBqb3VEFY)

*[Tomasz Florin](https://www.linkedin.com/in/tomasz-florin-93187553/) & [Artur Barczyński](https://www.linkedin.com/in/arturbarczynski/), S&P Global*

![diagram of one retrieval layer serving web apps, APIs, copilots and AI agents across relevance, experience, discoverability and AI](https://simonhearne.com/images/haystack-europe-2026/spge.png)
*One retrieval layer serving web apps, APIs, copilots and agents (S&P Global)*

S&P Global runs one hybrid relevancy platform for two very different consumers: the search box in its core enterprise product, and the retrieval layer behind agentic RAG workloads. Lexical and semantic retrieval are fused with RRF, and business-driven relevance rules shape the final ranking.

Tomasz and Artur did a good job of calling out the trade-offs in deploying a single retrieval stack. Complex business data needs precision and explainability as much as recall, and agents and analysts want different things from the same index. This is a hint at where large / enterprise use cases are heading: one retrieval platform, several consumers, with relevance rules the business can still control.

## New embedding models

### Hyperbolic embedding models for visual search

[Event page](https://haystackconf.com/session/hyperbolic-embedding-models-for-visual-search/) · [Recording](https://www.youtube.com/watch?v=qD-vNtcJpTc) · [Paper](https://arxiv.org/abs/2608.29313)

*[Matin Mahmood](https://www.linkedin.com/in/matin-mahmood/), [hyper³labs](https://hyper3labs.com/)*

![architecture diagram of an image-text model mapping captions and image regions into a hyperbolic Lorentz space](https://simonhearne.com/images/haystack-europe-2026/hyperbolic.png)
*Hyper3-CLIP embeds image regions and caption phrases in hyperbolic space*

Most image search systems store hierarchy outside the vectors, in tags or a knowledge graph. hyper³labs instead trained an open image-text model ([Hyper3-CLIP](https://huggingface.co/hyper3labs/hyper3-clip-v1)) in [hyperbolic space](https://arxiv.org/abs/1705.08039), where distance from the origin encodes specificity: for example "animal" sits near the centre, and "border collie" near the edge. The hierarchy lives in the geometry, so a single nearest-neighbour query can respect it.

The weights are open, and the results show good gains over the industry-standard [CLIP](https://arxiv.org/abs/2103.00020) multimodal embedding model. I found the concept intriguing and I can see lots of applications for it - from the obvious e-commerce category search to interesting use cases like security incidents or fraud cases, where specificity could encode the maturity or depth of an attack / case.

I was so impressed with the concept that I've built it out on Milvus and published a four part blog series:

1. [Your Vector Database Already Speaks Hyperbolic](https://milvus.io/blog/your-vector-database-already-speaks-hyperbolic.md): what hyperbolic embeddings are, and how ordinary inner-product search can rank them exactly by flipping one sign
2. [Hyperbolic Embeddings in Milvus](https://milvus.io/blog/hyperbolic-embeddings-in-milvus.md): running hyperbolic search on Milvus and measuring recall across index types on ImageNet
3. [Hyperbolic Search on Zilliz Cloud](https://milvus.io/blog/hyperbolic-search-on-zilliz-cloud.md): recall, capacity and latency for 1.28M ImageNet embeddings on a managed cluster, compared with a standard CLIP index
4. [What Hyperbolic Geometry Does to Ranking](https://milvus.io/blog/what-hyperbolic-geometry-does-to-ranking.md): where Hyper3-CLIP beats CLIP on ranking, and where it's not the right option

### Introducing per-document embedding fine-tuning

[Event page](https://haystackconf.com/session/introducing-per-document-embedding-fine-tuning/) · [Recording](https://www.youtube.com/watch?v=4ERYs2F9B4A)

*[Philippe Bouzaglou](https://www.linkedin.com/in/philippe-bouzaglou-77227811b), Vectra*

![product search demo with sliders adjusting blue and red stone features on an earring embedding](https://simonhearne.com/images/haystack-europe-2026/composable_embeddings.png)
*Adjusting individual features of a product embedding at query time (Vectra)*

Embeddings are usually frozen at index time: they hold whatever the model could infer from the input and changing them means fine-tuning the model and re-embedding the whole collection (which isn't cheap!). Vectra's e-commerce model is novel - it has *deterministic dimensions* for attributes such as colour or material, so you can adjust an individual document's embedding at query time. Nothing changes at the index layer - the stored vector stays as it is and the adjusted copy is used as the query, so there's no need to retrain the model or re-generate embeddings.

My favourite demo was a product embedded from its photo, then enriched at query time with facts the photo can't show (organic cotton, a hippie look, festival wear) to find similar products. The same mechanism can tune a search in real time to a user's preferences.

If composable embeddings catch on, an embeddinf doesn't have to be a static snapshot of a document - the index stays the same and all of the flexibility sits in how you build the query vector. That unlocks a huge potential in personalisation, query refinement and operational cost reduction. I'm watching this closely!

## Evaluating search systems

### Beyond LLM Judges: Deep Evaluation for Conversational Search

[Event page](https://haystackconf.com/session/beyond-llm-judges-deep-evaluation-for-conversational-search/) · [Slides (PDF)](https://pretalx.com/media/haystackeu26/submissions/JDZWMV/resources/Beyond_LLM_Ju_eXkLm4A.pdf) · [Recording](https://www.youtube.com/watch?v=1E14GMCjFTQ)

*[Ruchi Juneja](https://www.linkedin.com/in/ruchi-juneja/), [MediaMarktSaturn](https://www.mediamarktsaturn.com/)*

![slide defining the Conversational Search Reliability Score with Excellent, Good, Fair and Poor bands](https://simonhearne.com/images/haystack-europe-2026/conversational_search.png)
*MediaMarktSaturn's Conversational Search Reliability Score*

MediaMarktSaturn's conversational shopping assistant had just gone into an A/B test before the conference. Ruchi walked through why scoring only the final answer with an LLM judge wasn't enough. The framework they presented evaluates each layer separately: whether the agent extracted the right search terms, whether its facets matched the catalogue, and how often it hit zero results.

The most interesting finding (for me) was that implicit filters hurt. If a customer mentioned "256GB", the agent turned it into a hard filter and narrowed the results - even though that shopper might have happily bought the 128GB or 512GB model. I expect this to come up a lot in agentic retrieval: an agent that follows the query too literally will over-constrain it.

Ruchi also described how they distilled their evaluation into a single score graded from Poor (< 50) to Excellent (≥ 90), so they could track progress and report to leadership. I love a simple metric like this. Evaluation is still largely human-graded on recall, accuracy and reliability though, so I'm curious whether the score correlates with business metrics like conversion rate or abandonment. That wasn't covered in the talk.

Ruchi's closing line was: "a good evaluation doesn't just tell you how good the system is, it tells you what to fix next."

### Teaching your shopping assistant to learn from its mistakes

[Event page](https://haystackconf.com/session/teaching-your-shopping-assistant-to-learn-from-its-mistakes/) · [Slides (PDF)](https://pretalx.com/media/haystackeu26/submissions/R8ETJG/resources/HaystackEU26__JUFr2In.pdf) · [Recording](https://www.youtube.com/watch?v=iRytltsoC4E)

*[Jens Kürsten](https://www.linkedin.com/in/jenskuersten/), [OTTO](https://www.otto.de/)*

![slide showing session traces, shop interactions and conversations merged into a conversation timeline, beside the OTTO assistant app and the collect, judge, cluster, review, act loop](https://simonhearne.com/images/haystack-europe-2026/otto_assistant.png)
*Turning session traces into signals: the first step of OTTO's weekly loop*

This was a great companion to [Ruchi's talk](#beyond-llm-judges-deep-evaluation-for-conversational-search). When OTTO launched its product discovery assistant, failures were silent: shoppers just gave up. Jens described the weekly loop they now run over tens of thousands of production sessions:

1. **Collect** sessions and merge in the funnel events the customer actually saw.
2. **Judge** each one with an LLM anchored to those events.
3. **Cluster** failures into buckets with a single root cause each.
4. **Review** clusters against real dialogues to weed out LLM hallucinations.
5. **Act** on retrieval, prompt engineering or guardrails, then confirm in the next run.

The failures OTTO surfaced were interesting (and actionable): over-constrained queries returning zero results (the same problem MediaMarktSaturn saw), gaps in catalogue attributes, and internal users intentionally pushing the guardrails. My key takeaway: use LLMs to generate hypotheses and to do the first pass of classification, but rely on real customer interactions and human review to improve accuracy in production.

## What I'm taking away

Getting vectors into an index is a given, the hard problems have moved up the stack: understanding the query, serving agents and humans from one retrieval layer, and proving that the answer was right. What surprised me was that nobody discussed cost (except me). Nothing is free, and I suspect token and compute costs will become a more important factor as these solutions scale.

I also came away with a (long) list of experiments to run: BM25 vs. dense retrieval for agent memory, wormhole vectors vs. RRF with re-ranking, and hyperbolic embeddings on Milvus. I'll write them up as I go!


==============================================================================

# Fast typeahead search with Milvus

> Milvus can serve typeahead search recommendations in about 4ms, but only one or two of the obvious approaches get the answer right. I tested 36 query shapes on Milvus 3.0 so you don't have to.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2026-09-28
Canonical: https://simonhearne.com/2026/fast-typeahead-milvus/
Tags: Milvus, Zilliz, VectorDB, search

---
Website visitors and application owners alike expect near-instant feedback, especially when typing in a search bar and receiving recommendations. Type "iph" into a big retailer's search box and you'll see "iphone 17", "iphone case" and "iphone charger" before you've lifted your finger. The suggestions tend to be the popular ones, and they arrive between keystrokes. If they don't, the user has typed another character and your response is already stale.

A team I work with runs typeahead on a dedicated search engine today, alongside Milvus for vector search. Their target is under 50ms from keystroke to suggestions. They asked a fair question: can Milvus do the typeahead too, and save them running a second engine?

Milvus is best known as a vector database, but it also has scalar indexes, text analysers, BM25 and, as of 3.0, `ORDER BY` in plain queries. That gives you several plausible ways to build typeahead. I tested them all on Milvus 3.0.2 running in Zilliz Cloud to find out which ones return the right answer and at what speed.

**TL;DR:** keep a lowercased copy of each suggestion in a `VARCHAR` field with an `NGRAM` index. Query it with `LIKE "prefix%"` and sort by a popularity field. That returned exactly the right top 10 every time, with a p95 of about **4ms** measured at the client. That's under 1ms slower than fetching a single row by primary key, and a single 1 CU performance-optimised cluster held that up to about 1,600 requests per second.

## What typeahead actually needs

Search engineers will recognise the requirements, but it's worth being explicit because they rule out a lot of options:

- **Prefix matching.** "sam" should match "samsung tv". Most full-text machinery matches whole tokens, so "sam" matches nothing.
- **Word-start matching (sometimes).** "tv" matching "samsung tv" is useful. Whether it's required depends on the product.
- **Ranking by something other than relevance.** Every candidate matches the prefix equally well. What decides the top 10 is popularity: query frequency, clicks or sales.
- **Case and accent insensitivity.** People type "IPH", "iph" and "Iph".
- **Speed.** The budget is tens of milliseconds end to end, including network and browser time.

Dedicated engines solve this with completion suggesters or edge n-gram analysers. Milvus doesn't have a completion suggester so the question becomes which of Milvus's other building blocks can stand in.

## The candidates

I gave each approach its field and index in a single collection:

| Approach | Field and index | Query |
|---|---|---|
| Unindexed string | `VARCHAR`, no index | `LIKE "p%"` or `LIKE "%p%"` |
| Inverted index | `VARCHAR` + `INVERTED` | `LIKE "p%"` |
| Trie | `VARCHAR` + `TRIE` | `LIKE "p%"` |
| N-gram index | `VARCHAR` + `NGRAM` (2 to 3 characters) | `LIKE "p%"`, or `LIKE "p%" or LIKE "% p%"` for word starts |
| Analysed text | `VARCHAR` with a standard or custom analyser | `text_match`, `phrase_match`, `text_match_fuzzy` |
| BM25 | sparse vector generated from the analysed text | full-text `search()` |
| Client-side edge n-grams | each word expanded at write time ("sam" → "s sa sam") with a whitespace + lowercase analyser | `text_match` |
| Prefix array | `ARRAY<VARCHAR>` of the same edge n-grams + `INVERTED` | `array_contains_all` |
| Vector ranking | `[log(popularity), 0]` as a 2-d vector | ANN search with a `LIKE "p%"` filter |

Filter-only approaches used `query()` with `order_by_fields=["popularity:desc"]` and `limit=10`, so no vector search is involved at all.

## Correctness first

There's no point benchmarking a query that returns the wrong rows, so the first pass was a capability test. I ran ten deliberately awkward probe terms ("iPhone 15 Pro Max", "Sony WH-1000XM5", "Crème brûlée torch", "耳机 蓝牙" and so on) against 36 combinations of approach and query. Then I compared the results with ground truth computed in Python.

The analysed-text approaches fell at the first hurdle. `text_match`, `phrase_match` and BM25 tokenise the stored text into whole words, so "iph" never matches "iphone". They only returned the right rows when the typed characters happened to be a complete word, such as "usb" or "15". Fuzzy matching doesn't rescue this: it corrects a typo within a whole token, it doesn't complete a partial one.

Everything that matches on characters rather than tokens passed: every `LIKE "p%"` variant, the client-side edge n-grams and the prefix array. Two details matter:

- **`LIKE` is case-sensitive.** Store a normalised copy of the text (lowercased, Unicode-normalised, whitespace collapsed) and apply the same normalisation to what the user types. Among the approaches that passed, only the edge n-gram field with a `lowercase` filter in its analyser was case-insensitive without client-side normalisation.
- **Typo tolerance is weak at typeahead lengths.** Fuzzy matching on edge n-grams recovered swapped characters, but at 3 characters it also returned plenty of noise. It only became reliable from about 5 characters.

## Latency

For the benchmark I loaded 10,000 realistic e-commerce search terms and a simulated 'popularity' numeric field:

- **Cluster:** Milvus 3.0.2 on a single 1 CU Zilliz Cloud cluster.
- **Queries:** prefixes of 3 to 6 characters, sampled by popularity so common prefixes come up often, as they do in real traffic.
- **Load:** a steady 20 requests per second, open loop, so queueing shows up as latency.
- **Client:** an EC2 instance in the same AWS region as the cluster, with a TCP round trip of about 1.6ms.
- **Duration:** 60 seconds per configuration after a warm-up, 180,000 requests in total.

For a floor, I measured the minimum round-trip latency by fetching a single row by primary key at the same rate: **3.1ms at p50 and 3.5ms at p95**.

<div class="vega-chart loading" id="vis1" data-spec="/data/fast-typeahead/p95_by_length.json">Loading chart...</div>

Nearly every approach lands within a millisecond of that floor, whichever index it uses and however many characters are typed. The `NGRAM` field with `LIKE "p%"` had a p95 of **3.7 to 4.0ms**, and the edge n-gram field was **3.8 to 3.9ms**. BM25 search took about half a millisecond longer. Vector ranking with a `LIKE` filter was the slowest at 4.7 to 5.8ms, still far inside the budget. There were no errors across all 180,000 requests.

## Speed is easy, the right top 10 is harder

If latency were the only criterion, you could pick almost anything. So I also checked every response against the expected top 10: every term matching the prefix, sorted by popularity. This is where the approaches separate.

<div class="vega-chart loading" id="vis3" data-spec="/data/fast-typeahead/exact_top10.json">Loading chart...</div>

- **`LIKE "p%"` with `ORDER BY`** returned exactly the right top 10, in the right order, on 100% of requests, on every index type.
- **Word-start matching** was also 100% correct with the `NGRAM` field (`LIKE "p%" or LIKE "% p%"`), the edge n-gram field and the prefix array. The edge n-grams only do word starts, though. Typing "tv" returns anything with a word starting "tv", so against a strict whole-string expectation they scored lower.
- **Infix `LIKE "%p%"`** matches the middle of words ("pho" finds "headphones"). That's usually not what someone typing expects.
- **BM25** ranks by term statistics, not popularity, so it almost never produced the expected top 10. Fetching 100 results and re-sorting by popularity only got about half of them right.
- **Vector ranking** was right 89 to 96% of the time. The search is approximate, so neighbouring popularities occasionally swap places.

## What I'd recommend

For most catalogues, one normalised `VARCHAR` field with an `NGRAM` index covers both whole-string and word-start completion. Create the collection like this:

```python
from pymilvus import MilvusClient, DataType

client = MilvusClient(uri=ZILLIZ_URI, token=ZILLIZ_TOKEN)

schema = client.create_schema(auto_id=False, enable_dynamic_field=False)
schema.add_field("id", DataType.INT64, is_primary=True)
schema.add_field("suggestion", DataType.VARCHAR, max_length=512)    # shown to the user
schema.add_field("suggest_norm", DataType.VARCHAR, max_length=512)  # normalised copy for matching
schema.add_field("popularity", DataType.INT64)                      # what decides the top 10
schema.add_field("placeholder_vec", DataType.FLOAT_VECTOR, dim=2, nullable=True)  # Milvus requires one

index_params = client.prepare_index_params()
index_params.add_index(field_name="suggest_norm", index_type="NGRAM", min_gram=2, max_gram=3)
index_params.add_index(field_name="popularity", index_type="STL_SORT")
index_params.add_index(field_name="placeholder_vec", index_type="AUTOINDEX", metric_type="IP")

client.create_collection("typeahead", schema=schema, index_params=index_params)
```

Then query it on each keystroke, after normalising the input the same way as the stored text:

```python
hits = client.query(
    "typeahead",
    filter=f'suggest_norm like "{prefix}%"',
    # word starts too: f'suggest_norm like "{prefix}%" or suggest_norm like "% {prefix}%"'
    limit=10,
    order_by_fields=["popularity:desc"],
    output_fields=["suggestion"],
    consistency_level="Bounded",
)
```

Escape `%`, `_` and `\` in the user's input before building the filter, as you would for any `LIKE`. Also note that the `NGRAM` index is only used when the typed text is at least `min_gram` characters long. Shorter input falls back to a full scan, so either start querying at 2 or 3 characters or set `min_gram` to match.

If you need matching to ignore case on the server, or want every typed word to match ("sony wh" → "sony wh-1000xm5"), use the client-side edge n-gram field instead. Its analyser is `{"tokenizer": "whitespace", "filter": ["lowercase"]}`, and you AND one `text_match` per typed word. You pay for it with a larger field and some extra work at write time.

**Ties need a tiebreaker.** Popularity scores repeat, especially in the long tail, and Milvus breaks ties in no particular order. In an earlier 100,000-term run, 0.7% of responses listed tied suggestions in a different order from my ground truth. The results were still correct, just not stable between calls. Add a second sort key if the order must be stable: `order_by_fields=["popularity:desc", "id:asc"]`.

**No popularity signal?** You still need an `ORDER BY`. Without one, Milvus returns matches in storage order, which is arbitrary and can change after compaction. Sorting by an integer length field and then alphabetically (`order_by_fields=["term_len:asc", "suggest_norm:asc"]`) puts "iphone" ahead of "iphone 15 pro max case", which is a reasonable default. Deriving popularity from search logs is better still.

## How far does 1 CU go?

A few milliseconds at 20 requests per second says nothing about capacity. So I loaded **100,000 terms** and ramped the recommended query from 100 QPS, adding 100 QPS every minute, up to 2,000 QPS. It ran on the same 1 CU cluster, with 7 load-generating processes on an in-region `c7i.2xlarge`.

<div class="vega-chart loading" id="vis4" data-spec="/data/fast-typeahead/ramp_1cu_chart.json">Loading chart...</div>

All 1.26 million requests succeeded, and every step sustained its target rate, up to 2,000 QPS. But the shape of the curve matters more than the pass mark:

- **Up to about 1,600 QPS, latency is improves with throughput.** p50 sits around 3ms and p99 stays under 5ms.
- **From 1,700 QPS the tail climbs.** p99 latency increases to almost 50ms at 2,000 QPS, at this point we're still within budget but the cluster is queueing requests and it's probably time to scale up for the demand.
- **The client wasn't the limit.** The test machine CPU peaked at 15% and requests were submitted on schedule to within 1ms at p99.

So for this query shape, one CU serves roughly **1,500 QPS with a p99 under 5ms**. That's a lot of typeahead: at a request per keystroke, it's hundreds of people typing at the same moment.

## Caveats

- **100,000 rows is still modest.** Correctness doesn't change with scale, but latency and capacity can. `ORDER BY` sorts every match before taking the top 10, so a broad three-character prefix on a catalogue of millions will cost more than it does here. Test at your real size.
- **The capacity figure is for one query shape.** The ramp used whole-string prefixes of 3 to 6 characters, weighted towards popular terms. Word-start queries, bigger result sets or concurrent writes would all need further benchmarking.

Nothing here needed a vector search, a second engine or an exotic index. If Milvus is already in your stack for vectors, the typeahead can live next to them, in the same collection, answering in a few milliseconds.

## Reproduce

I've open sourced the [benchmarking methodology](https://github.com/simonhearne/milvus_typeahead_benchmarks) on GitHub so you can try it yourself.


==============================================================================

# The RAG Cost Curve

> When does a retrieval index actually beat live search? Measured break-even points, not vendor claims: 334 queries a day on a short agent history, never on code the model already knows, and 24 on code it has never seen.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2026-09-15
Canonical: https://simonhearne.com/2026/rag-cost-curve/
Tags: VectorDB, Zilliz, Milvus, talk, video, conference
Video: https://www.youtube.com/watch?v=Ipqnh4upvrI
Slides: https://talks.simonhearne.com/rag-cost-curve/
Link: Reproduce every number: https://github.com/simonhearne/rag-cost-curve

---
I assumed that an index would be more efficient than agentic code search, fewer turns and lower token consumption, with break-even on the order of 10s - 100s of searches per day. It turns out not to be so clear cut.

## About the talk

"Just use RAG" and "RAG is dead" are both answers to a question nobody asked precisely. An index costs money to build, to store and to keep current. Live search costs money every time you run it. Which one wins is not a matter of taste, it is a crossover point, and it moves depending on what you are searching and how often.

This talk puts a number on that crossover for two workloads, conversation memory and code search, and shows the measurements behind it. Every figure comes from a committed CSV, every chart renders from that CSV at build time, and the whole study runs on a laptop with Docker.

The honest finding is that the index does not always win. It loses badly in one of the cases I measured, and the talk spends time there rather than skipping past it.

## What's covered

- **Defining break-even.** What goes into the cost of an index against the cost of live search, and why "worth it" is meaningless without naming what you are comparing against.
- **Quantisation and dimensionality reduction.** Scalar, product and RaBitQ quantisation, PCA and Matryoshka embeddings, what each costs you in recall, and how refinement claws precision back. SQ8 won on its own.
- **Does recall even matter?** Recall barely moves answer quality until it falls off a cliff, mostly because most questions never had a choice. Aim for recall@k of about 0.95 and stop optimising.
- **Two measured workloads.** Conversation memory, and code search against three repositories: one the model already knows, one it has never seen, and the same repository stuffed into context instead of grepped.
- **The cost curves, assembled.** Where each workload crosses over, and where it never does.

## What the numbers said

**Live search wins** when the corpus is small and the model already knows it:

- A short agent history, at 98k tokens of conversation memory, needs **334 queries a day** before a dedicated index pays for itself.
- On `fastapi`, a repository the model already knows, break-even is **infinite at every size**. No query volume repays the index, because grep is cheaper every time.
- Hold the prompt cache at 99% rather than the measured rate and the crossover is still **232 queries a day**, so the cache assumption is not doing all the work.

**The index wins** as the corpus grows, or when the model has never seen it:

- Conversation memory collapses fast: **15 queries a day** at 392k tokens and **10** at 1.2M, against 334 at 98k.
- On code the model has never seen, the index is **40% cheaper per correct answer** than grep, and more accurate with it: 39 of 40 correct against 36.
- Against stuffing the same repository into context rather than grepping it, the index repays at **12 queries a day**: $0.29 a query re-sent against $0.067 indexed.

That last pair is the sharpest thing in the talk. Same repository, same index, same box, same prices. The only thing that changes is what the live arm does, and the answer flips from "no volume repays this" to "twelve queries a day". The question is never "is retrieval worth it", it is "worth it against what".

## Reproduce it

Everything in the talk runs on your own machine, given Docker with 16GB of RAM or more:

```bash
make setup && source .venv/bin/activate
make up            # docker compose up -d + Milvus health wait
make data-1m       # or data-10m
jupyter lab        # notebooks 01 to 06b in order
```

If you want the tooling rather than the study, [memsearch](https://github.com/zilliztech/memsearch) gives Claude semantic recall over its own conversation history, and [claude-context](https://github.com/zilliztech/claude-context) adds semantic code search to coding agents over MCP.

Presented at [Haystack Europe 2026](https://haystackconf.com/speaker/simon-hearne/) in Berlin on 15 September 2026.


==============================================================================

# Migrating to Milvus: Tooling, Deployment Choices and Lessons from Real Migrations

> How to move from Elasticsearch or OpenSearch to Milvus or Zilliz Cloud: migration tooling, a real end-to-end walkthrough, deployment choices, and lessons from real customer migrations.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2026-09-02
Updated: 2026-09-08
Canonical: https://simonhearne.com/2026/migrating-to-milvus-talk/
Tags: VectorDB, Zilliz, Milvus, video, talk
Video: https://www.youtube.com/watch?v=0xU3Ks2vXIM
Slides: https://talks.simonhearne.com/2026-09-milvus-migration/

---
This is the follow-up to [Milvus 3.0: Closing the Search Gap](/2026/milvus_search_gap/). If you are moving self-managed Milvus rather than Elasticsearch, see [Migrating Self-managed Milvus to Managed Milvus](/2026/migrating-milvus/).

## About the webinar

Milvus 3.0 expands the range of search workloads teams can run in Milvus, with stronger full-text retrieval plus in-engine sorting, aggregation and faceting. In [the previous webinar](/2026/milvus_search_gap/) I explored these search improvements and compared Elasticsearch and Milvus with live side-by-side queries.

In this follow-up session I move from comparison to migration. I show how Zilliz Migration Service helps move data from Elasticsearch or OpenSearch into Milvus or Zilliz Cloud, then walk through a real migration workflow. I also compare Elastic Cloud (ESS) and Zilliz Cloud, discuss deployment options across open-source Milvus, Zilliz Cloud and Zilliz BYOC (Bring Your Own Cloud), and share lessons from customer migrations.

If you use Elasticsearch or OpenSearch today and are considering a migration, this is the walkthrough for you.

## What's covered

- **The migration path.** The major stages and decisions in an Elasticsearch or OpenSearch migration to Milvus or Zilliz Cloud.
- **Migration tooling and a real walkthrough.** How VTS (the open-source migration tool) and Zilliz Migration Service (the managed service built on VTS) move Elasticsearch and OpenSearch data, followed by a real migration workflow end to end.
- **Licensing and commercial tradeoffs.** Licensing models and the commercial considerations behind Elastic Cloud (ESS) and Zilliz Cloud.
- **Deployment choices.** Open-source Milvus, Zilliz Cloud and Zilliz BYOC compared across operating model, control and security needs.
- **Customer migration stories.** Why real teams moved, how they approached the migration, and what they learned along the way.
- **Live AMA.** Around 20 minutes of migration, architecture and evaluation questions.

Presented as a [Zilliz webinar](https://zilliz.com/event/migrating-from-elasticsearch-opensearch-to-milvus) on 2 September 2026.


==============================================================================

# Milvus 3.0: Closing the Search Gap

> Milvus 3.0 closes the gap between vector search and full-text search: one engine for both. A customer-backed comparison of Milvus 3.0 and Elasticsearch 9.14, with live side-by-side queries you can reproduce yourself.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2026-08-19
Canonical: https://simonhearne.com/2026/milvus_search_gap/
Tags: VectorDB, Zilliz, Milvus, video, talk
Video: https://www.youtube.com/watch?v=XxcA1NSQnCo
Slides: https://talks.simonhearne.com/2026-08-milvus-search-gap/
Link: Test it yourself!: https://github.com/simonhearne/milvus_es_lab

---
The follow-up webinar, [Migrating to Milvus](/2026/migrating-to-milvus-talk/), moves from comparison to migration: tooling, deployment choices and lessons from real migrations.

## About the webinar

Milvus has long been known for vector search and has supported full-text search since Milvus 2.5, allowing teams to combine lexical and semantic retrieval in one engine. With Milvus 3.0, full-text search takes another major leap:

- **Compressed BM25 indexes** roughly 3× smaller than those in Milvus 2.6 at comparable recall, reducing memory and bandwidth pressure.
- **SINDI**, a new sparse-retrieval algorithm that delivered roughly 5× to 10× the QPS of MaxScore across four learned-sparse benchmarks, and extends the optimised retrieval path to native BM25.
- **Sorting, aggregation and faceting** inside the engine.

With stronger full-text retrieval added to Milvus's established vector-search foundation, teams can reconsider whether AI retrieval still requires a separate full-text system, along with the infrastructure, pipelines and result-fusion logic that come with it. This talk is a customer-backed comparison of Elasticsearch and Milvus, including live side-by-side queries against Milvus 3.0 and Elasticsearch 9.14. Query semantics, results and performance are all reproducible with the open-source lab linked above.

## What's covered

- **Milvus 3.0 full-text search.** BM25, sparse retrieval, index efficiency, server-side sorting, aggregation and faceted search, with a brief look at planned fuzzy matching, multi-phase reranking and JSON-field aggregations.
- **Elasticsearch vs. Milvus.** Search capabilities, performance and operational tradeoffs across the two systems.
- **Live query demo.** The same queries in Elasticsearch and Milvus, with results and latency visible on screen.
- **Hybrid search and ranking.** How full-text and vector results can be combined through ranking and fusion methods such as RRF.
- **Production customer evidence.** How real teams improve retrieval quality, support growing workloads and simplify search operations.
- **Live AMA.** A 20-minute discussion of search patterns and architecture questions.

Presented as a [Zilliz webinar](https://zilliz.com/event/faster-more-powerful-full-text-search-milvus-3-0) on 19 August 2026.


==============================================================================

# Migrating Self-managed Milvus to Managed Milvus to gain >99% Performance

> At small scale, self-managing a vector database is manageable. Once you're in the tens of millions of embeddings you'll probably want to offload the headache to a managed service.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2026-07-14
Canonical: https://simonhearne.com/2026/migrating-milvus/
Tags: Milvus, Zilliz, VectorDB

---
So you've built an application using Milvus as the vector database, using [Standalone](https://milvus.io/docs/install_standalone-docker.md) or  [Distributed](https://milvus.io/docs/install_cluster-milvusoperator.md?tab=helm). You've reached a point where the application is working, customers are using it and the data volume is growing. At some point managing the vector database starts consuming more time, pod failures cause service jitter, you're running out of RAM on the server, or `etcd` becomes a painful bottleneck.

At this stage you're probably exploring a managed service to take the operational burden away. The good news is that migrating from Milvus to Zilliz Cloud is straightforward, and the various options are [well documented](https://docs.zilliz.com/docs/migrate-from-milvus).

In my case, I had built a simple Wikipedia RAG application: 50M embeddings using Cohere's Embed v3 multi-lingual model at 1,024 dimensions (covering the entire Wikipedia English corpus). Initially I hosted this on my laptop using Milvus Standalone, but the container was unstable and restarts took \~20 minutes - not ideal when you want to show a demo!

Query performance was also suffering due to frequent paging (I don't have enough memory on my laptop, so I enabled [mmap](https://milvus.io/docs/mmap.md)). Below is a quick video showing the application in action:

<figure>
  <video style="aspect-ratio: 810 / 785.758;" autoplay loop muted playsinline preload="metadata">
    <source src="/images/migrating-milvus/standalone.webm" type="video/webm">
    <source src="/images/migrating-milvus/standalone.mp4" type="video/mp4">
    <source src="/images/migrating-milvus/standalone.mov" type="video/mov">
  </video>
  <figcaption>The app works, but it could be faster</figcaption>
</figure>

Three second query times aren't great, so without the budget for a new laptop, I planned my migration to managed Milvus on Zilliz Cloud. The following is the step-by-step process I followed - there are a number of methods available, but I chose backup / restore for simplicity.

## Create a backup

Zilliz provides the [milvus-backup](https://github.com/zilliztech/milvus-backup) utility, and installing it is as easy as `brew install milvus-backup`

You then need to create a config file (milvus-backup looks for `backup.yaml` in the current working directory by default). This is a minimal example to create a backup from Standalone running in Docker locally (see the full [yaml options](https://github.com/zilliztech/milvus-backup/blob/main/configs/backup.yaml) on GitHub):

```yml
milvus:
  address: localhost
  port: 19530
  user: "root"
  password: "Milvus"
  tlsMode: 0
  etcd:
    endpoints: 127.0.0.1:2379
    rootPath: "by-dev"
minio:
  storageType: "local"
  rootPath: "/../milvus_wikipedia/volumes/milvus/data"
  backupStorageType: "local"
  backupRootPath: "/../milvus-backup-test/backup"
```

Then run a quick check:

```bash
$ milvus-backup check

Milvus version: 2.6.2
Storage:
  milvus-storage-type: local
  milvus-bucket: a-bucket
  milvus-rootpath: /../milvus_wikipedia/volumes/milvus/data
  backup-storage-type: local
  backup-bucket: a-bucket
  backup-rootpath: /../milvus-backup-test/backup

Success!
```

And finally create the backup (this will take a little time):

```bash
milvus-backup create -n wiki_backup
```

## Create the target instance on Zilliz Cloud

First make sure you have a [Zilliz Cloud](https://cloud.zilliz.com/) account - then create an instance that matches the minimum requirements of your local deployment. For my 50M x 1,024-D embeddings on Tiered-Storage, the [public calculator](https://zilliz.com/pricing#calculator) estimated that I required 2 [Query CU](https://zilliz.com/blog/how-to-choose-the-right-cu-type-and-size):

![screenshot zilliz cloud pricing calculator](https://simonhearne.com/images/migrating-milvus/estimation.png)
*Public pricing calculator estimates Query CUs*

So I navigated to the cloud console, clicked "\+ Cluster" and used these settings:

![screenshot zilliz cloud cluster creation screen](https://simonhearne.com/images/migrating-milvus/creation.png)
*Creating a cluster in Zilliz Cloud*

While it is creating, you can retrieve / generate the API key we will need for the next step:

![screenshot zilliz cloud generate api key](https://simonhearne.com/images/migrating-milvus/api_key.png)
*Creating / retrieving API keys in Zilliz Cloud*

And note the cluster ID of the new instance:

![screenshot zilliz cloud get instance ID](https://simonhearne.com/images/migrating-milvus/connection.png)
*Retrieve Cluster ID from Zilliz Cloud*

## Migrate to Zilliz

Update your `backup.yaml` to include the cloud key you just generated / retrieved:

```yml
cloud:
  address: https://api.cloud.zilliz.com
  apikey: <your-api-key>
```

Then finally we just need to run one migration command, with the backup name and the target cluster ID passed as arguments:

```bash
milvus-backup migrate -n wiki_backup -c <your-cluster-id>
```

In my case it took a few hours to upload the \~120GB backup file to a [Volume](https://docs.zilliz.com/docs/managed-volume) on Zilliz Cloud, then about an hour to create the target cluster from the Volume. You can monitor the migration status in the Zilliz Cloud console under Jobs:

![screenshot zilliz cloud job progress](https://simonhearne.com/images/migrating-milvus/jobs.png)
*View job progress in Zilliz Cloud*

Once the migration is complete, make sure you load the collection into the cluster!

![screenshot zilliz cloud load collection ](https://simonhearne.com/images/migrating-milvus/load_collection.png)
*Load the new collection in the cluster*

## Validation

Now you can simply update the application to use the new cluster endpoint and credentials.

<figure>
  <video style="aspect-ratio: 810 / 785.758;" autoplay loop muted playsinline preload="metadata">
    <source src="/images/migrating-milvus/zilliz.webm" type="video/webm">
    <source src="/images/migrating-milvus/zilliz.mp4" type="video/mp4">
    <source src="/images/migrating-milvus/zilliz.mov" type="video/mov">
  </video>
  <figcaption>The app still works, and now queries are not the bottleneck!</figcaption>
</figure>

We observe a performance improvement compared to Milvus Standalone - 25ms compared to 3,112ms - that's over a *99% latency reduction*!. This is partly due to the increased compute allocated in the cloud service, as well as the proprietary index engine in Zilliz Cloud - [Cardinal](https://zilliz.com/cardinal) - which can achieve 10x faster queries compared to Milvus OSS.

The performance win is huge, but even better I no longer need to worry about the container stopping and I can free up \~20GB RAM on my laptop!

## Other considerations

- If your application isn't running 24x7, your vector database doesn't need to either. [Suspend](https://docs.zilliz.com/docs/manage-cluster#suspend) inactive clusters to drop the compute cost to zero when it is not being used.

- Tiered-Storage is a great cluster type for low (\<10 QPS) requirements, but there are [other options](https://docs.zilliz.com/docs/cu-types-explained) and you can [migrate](https://docs.zilliz.com/docs/offline-migration) data between them:

    - **On-Demand** - only uses compute when there are queries being run. This could drop the cost by 99%, depending on the frequency of queries, at a cost of higher cold-start latency.

    - **Capacity-Optimized** - provides reduced data density compared to Tiered-Storage, but achieves 10x greater throughput and 2-5x faster queries. This would increase the compute cost of my example by about 60%.

    - **Performance-Optimized** - provides even greater throughput (\>1,000 QPS per replica) and performance (10-100x faster) at the cost of further reduced data density.

- Zilliz Cloud supports mounting external volumes from Google Cloud Storage, Amazon S3, Azure Blob Storage etc. So a cleaner approach would be to upload the backup directly to cloud storage and restoring it from there.

- If creating / restoring a backup is not possible for any reason, and the Milvus deployment can be made accessible publicly - you can use the [Migrate from Milvus Endpoint](https://docs.zilliz.com/docs/via-endpoint) feature in Zilliz Cloud.

- Everything we performed in Zilliz Cloud can be managed via [API](https://docs.zilliz.com/reference/restful) and/or [Terraform](https://docs.zilliz.com/docs/terraform-provider) if click-ops is not your preferred methodology.


==============================================================================

# Vector Search, Visualised

> Vector search has no EXPLAIN. Through interactive visualisations, this talk shows what embeddings look like in high-dimensional space, what quantisation does to your recall, and how to catch retrieval failures before your agents do.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2026-06-04
Canonical: https://simonhearne.com/2026/visualising-vector-search/
Tags: VectorDB, Zilliz, Milvus, talk, video
Video: https://www.youtube.com/watch?v=kY-il0pWjdY
Slides: https://talks.simonhearne.com/vector-search-visualised/

---
## About the talk

SQL makes sense. But when it breaks, you reach for `EXPLAIN`. Vector search offers no such comfort. Multi-thousand-dimension embeddings, approximate nearest-neighbour indexes, and quantisation tradeoffs make it hard to know what your system is doing, and harder still to diagnose when results quietly degrade.

Through interactive visualisations, this talk shows what embeddings look like in high-dimensional space, what quantisation does to your recall, and how to catch retrieval failures before your agents do. You'll leave with a sharper mental model and a diagnostic toolkit for the production problems hardest to see.

## What's covered

- **Why vector search exists.** Keyword search matches tokens, vector search matches meaning. How models turn text, images and faces into numbers, and what "similar" really means.
- **Exact vs. approximate search.** Why flat search doesn't scale, the trade-off triangle of speed, accuracy and cost, and how recall and precision measure what you give up.
- **ANN indexes, visualised.** IVF partitions the space, HNSW navigates a graph, DiskANN reaches past RAM. Each with its tuning knobs and where it lands on the triangle.
- **Quantisation and dimensionality reduction.** Scalar, product and RaBitQ quantisation, PCA and Matryoshka embeddings, what each costs you in recall, and how refinement claws precision back.
- **Filters and silent failure.** Why filtering quietly wrecks recall, three ways out by selectivity, and the signals of degradation that your agent will never report.
- **The EXPLAIN you don't get.** How to measure what you can't see, and strategies that actually work in production.

Presented at [TiDB SCaiLE Europe 2026](https://www.pingcap.com/blog/tidb-scaile-europe-2026/) in Stockholm on 4 June 2026 as "When EXPLAIN Isn't Enough: Visualising Vector Search for Engineering and Product Teams".


==============================================================================

# Adding site search to Netlify sites using Elasticsearch

> A guide to implementing static site search using 11ty, Netlify and Elasticsearch

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2024-05-07
Canonical: https://simonhearne.com/2024/netlify-elasticsearch/
Tags: search, elasticsearch, netlify

---
Static site generators (SSGs) such as [11ty](https://www.11ty.dev/) allow us to build super-fast websites with great developer experience. Hosting platforms like [Netlify](https://www.netlify.com/) allow us to rapidly iterate and release directly from GitHub.

SSGs are perfect for (mostly) static sites such as blogs (like this one!), corporate sites, documentation sites etc. Search, however, is not static. You can build static-ish site search using solutions such as [Pagefind](https://pagefind.app/) but these solutions have limitations in hosting environments and flexibility.

SaaS search solutions like Algolia have their place too, but the [official Algolia x Netlify integration](https://www.netlify.com/integrations/algolia/) simply triggers a web crawl of a site rather than indexing content directly. Again, this limits flexibility and results in a delay between build and updating the search index.

As a fan of elasticsearch I thought I would have a go at implementing search on this site using pure 11ty, Netlify and Elastic capabilities:

1. Create a search index at build time with 11ty
2. Update a remote elasticsearch index after build
3. Use an Elastic search application to create a simple search experience
4. Use Elastic Behavioral Analytics to capture search & click data

Sounds simple enough! Let's walk through the steps:

## Create a search index at build time

There are a number of ways to do this, but I took the route that seemed the most simple to me - using nunjucks. I added a new nunjucks filter and a page template to render blog posts as `ndjson`, which will be important later:

```js
/* --- eleventy.js --- */
eleventyConfig.addNunjucksFilter("ndjson", (page)=> {
  let d = new Date();
  return JSON.stringify({
    'url': 'https://simonhearne.com'+page.url,
    'tags': page.data.tags,
    'published': page.date,
    'title': page.data.title,
    'excerpt': page.data.excerpt,
    'image': page.data.image,
    'indexed': d.toISOString()
  });
});
```

```njk
{# --- esindex.ndjson.njk --- #}
---
permalink: /esindex.ndjson
eleventyExcludeFromCollections: true
---
{%- for post in collections.posts %}
  { "index": {"_id": "{{post.url}}"}}
  {{ post | ndjson | safe }}
{%- endfor %}
```

This creates a new file at build time, `esindex.ndjson`, at the root of the published site directory.
It should look a little like this:

```json
{ "index": {"_id": "/2015/web-performance-optimisation-basics/"}}
{"url":"https://simonhearne.com/2015/web-performance-optimisation-basics/","tags":["posts","WebPerf"],"published":"2015-03-11T19:54:00.000Z","title":"Web Performance Optimisation Basics","excerpt":"Website performance is critical to user experience. We need rules to make it easier.","image":"./images/speed_hero.jpg","indexed":"2024-05-07T11:47:22.116Z"}
...
```

Note that this `ndjson` file corresponds to the format required for the [elasticsearch `_bulk` API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html). I've set the `_id` of each document to the path of the page, this is unique in my context and means that further indexing requests to existing documents in elasticsearch will replace the older document.

## Index the data to elasticsearch

We can validate the bulk data created above by POSTing it to an elasticsearch instance (assuming you have the relevant variables set):

```sh
curl -XPOST -H \"Content-Type: application/x-ndjson\" -H \"Authorization: ApiKey $ELASTIC_APIKEY\" $ELASTIC_HOST/$ELASTIC_INDEX/_bulk --data-binary @_site/esindex.ndjson
```

If that works as expected, we can lazily add it to our build command:

```toml
# --- netlify.toml --- #
[build]
  publish = "_site"
  command = "npm run build && curl -XPOST -H \"Content-Type: application/x-ndjson\" -H \"Authorization: ApiKey $EAK\" $ELASTIC_HOST/$ELASTIC_INDEX/_bulk --data-binary @_site/esindex.ndjson"
```

You will need to [add your variables](https://docs.netlify.com/configure-builds/environment-variables/) to your Netlify config of course!

## Create a search application

[Search Applications](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-application-overview.html) allow you to create a public API endpoint against an elasticsearch instance. Our use case is simple but you can do all sorts with search applications!

I've set my index name `$ELASTIC_INDEX` to `search-simonhearne` (original I know) so creating my search application (called `simonhearne`) is as simple as a single `PUT` via Kibana dev tools:

```json
PUT /_application/search_application/simonhearne
{
  "indices": [ "search-simonhearne" ],
  "template": {
    "script": {
      "source": {
        "fields": ["title","tags","url","excerpt","published","image"],
        "query": {
          "bool": {
            "should": [
              {
                "multi_match": {
                  "query": "{{query}}",
                  "fields": ["title^4","tags","excerpt","url"],
                  "fuzziness": "auto"
                }
              },
              {
                "multi_match": {
                  "query": "{{query}}",
                  "fields": ["title^4","tags","excerpt","url"],
                  "type": "phrase_prefix"
                }
              }
            ]
          }
      	},
      	"from": "{{from}}",
      	"size": "{{size}}",
    	  "highlight": {
          "fields": {
            "title": {},
            "excerpt": {}
          },
          "tags_schema" : "styled",
          "fragment_size": 150,
          "number_of_fragments": 1,
          "type": "plain"
        }
      },
      "params": {
        "query": "*",
        "from": 0,
        "size": 10
      }
    }
  }
}
```

This request is doing quite a lot, defining the query I want the search application to run (against the post `title`, `tags`, `excerpt` and `url`) with relevant weightings, also to run a phase prefix query to provide a bit of query completion logic. I've also enabled [highlighting](https://www.elastic.co/guide/en/elasticsearch/reference/current/highlighting.html) to enable me to show highlights in the search results.

We can test this search application with a simple `POST` against the application endpoint:

```json
POST _application/search_application/simonhearne/_search
{
  "params": {
    "query": "velo"
  }
}
```

## Implementing the search client

We need to create an API key for the search application to allow public API requests:

```json

POST /_security/api_key
{
  "name": "public api key",
  "role_descriptors": {
    "my-restricted-role-descriptor": {
      "indices": [
        {
          "names": ["simonhearne"],
          "privileges": ["read"]
        }
      ],
      "restriction":  {
        "workflows": ["search_application_query"]
      }
    }
  }
}
```

With the API key, we can now construct a client-side API call:

```js
const searchEndpoint = "<elasticsearch-host>/_application/search_application/simonhearne/_search"
const apiKey = "<base64 apiKey>"

const body =
  {
    "params": {
      "query": term
    }
  };
let response = await fetch(searchEndpoint, {
  method: 'POST',
  mode: 'cors',
  headers: {
    'Authorization': `apiKey ${apiKey}`,
    'Content-Type': 'application/json'
  },
  body: JSON.stringify(body)
});
```

And render the results (using a template literal function in this case):

```html
const postTemplate = (result,term,position) => `
<li class="postlist-item" data-query="${term}" data-id="${result._id}" data-index="${result._index}" data-position="${position}">
  <div class="hero-img-container">
    <a href="${result.fields.url[0]}" class="no-outline"><img src="${result.fields.image[0].slice(1)}" width="1200" height="675" onclick="sendClick(this)"></img></a>
  </div>
  <div class="meta-container">
    <div class="post-title"><a href="${result.fields.url[0]}" class="postlist-link" onclick="sendClick(this)">${(result.hasOwnProperty('highlight') && result.highlight.hasOwnProperty('title')) ? result.highlight.title[0] : result.fields.title}</a></div>
    <time class="postlist-date">${new Date(result.fields.published[0]).toLocaleDateString()}</time>
    <div class="post-description">${(result.hasOwnProperty('highlight') && result.highlight.hasOwnProperty('excerpt')) ? result.highlight.excerpt[0] : result.fields.excerpt}</div>
  </div>
</li>`;
```

Note that there is a [search application client library](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-application-client.html) which can do this for you, I just prefer hacking away at HTML and JS manually.

## Enable Analytics

Elastic [Behavioral Analytics](https://www.elastic.co/guide/en/elasticsearch/reference/current/behavioral-analytics-start.html) allows you to capture and view simple analytics of search events. The documentation covers the important stuff so I'll just mention how I've implemented it.

I host the analytics locally and load it only on the search page, deferred to prevent blocking render:

```html
<script>
  function bootstrapAnalytics() {
    window.elasticAnalytics.createTracker({
      endpoint: "<elasticsearch-host>",
      collectionName: "<collection-name>",
      apiKey: "<apiKey>",
    });
  }
</script>
<script src="/js/behavioral-analytics-browser-tracker.2.0.0.min.js" defer onload="bootstrapAnalytics()"></script>
```

### Tracking search events

On each search execution I run this function to send the relevant analytics event:

```js
function sendAnalytics(query,results,total) {
  if (window.elasticAnalytics) {
    window.elasticAnalytics.trackSearch({
      search: {
        query: query,
        search_application: "simonhearne",
        results: {
          total_results: total,
          items: results.map(res=> {
            return {
              'document':{
                'id':res._id,
                'index':res._index
              },
              'page':{
                'url':res.fields.url[0]
              }
            }
          })
        }
      }
    });
  }
}
```

### Tracking search clicks

Each search result link has an event listener to send an analytics beacon on click, using data attributes added in the `postTemplate` documented above:

```js
function sendClick(e) {
  if (window.elasticAnalytics) {
    let el = e.closest('[data-index]');
    window.elasticAnalytics.trackSearchClick({
      'document': {
        'id': el.dataset.id,
        'index': el.dataset.index
      },
      'search': {
        'query': el.dataset.query,
        'search_application': 'simonhearne'
      }
    })
  }
}
```

### Results

You should now have a dashboard in Kibana with your search analytics!

![screenshot of kibana search analytics dashboard](https://simonhearne.com/images/search-analytics.png)
*Kibana search analytics dashboard*

## Productionising

In my case I just wanted something that worked and did not take too much care on production readiness. There are a few further tasks I would complete for a 'proper' deployment:

1. delete the source `ndjson` file after indexing
2. configure separate indices for lower environments
3. take [language into account](https://www.elastic.co/guide/en/elasticsearch/guide/master/one-lang-fields.html)
4. tune the search query based on the analytics results

Overall I spent about three hours building this search solution, with an existing elasticsearch cluster ready to use.


==============================================================================

# WebPageTest Live: Third Parties and Request Map with Simon Hearne

> Third parties have always been a challenge. Having resources loading on your site, beyond your control isn't something users sign up for. In 2015, Simon Hearne created a tool called REQUEST MAP to track just that: requests. This great visualization tool allowed you to see where the request as emerging from, and what was taking place afterwards. In today's episode of WebPageTest LIVE!, Simon Hearne joins us to walk us through this workflow using Request Map -- a supported WebPageTest integration, and how you can use it to see what exact resources are loading, possibly unbeknownst to users.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2022-10-11
Canonical: https://simonhearne.com/2022/third-parties-request-map/
Tags: WebPerf, video, talk
Video: https://www.youtube.com/watch?v=TY82pTfhnXY
Link: View Speaker Notes: https://notes.simonhearne.com/WebPageTest-Twitch-Stream-Notes-e812abaf4d3445c0ac70678a29ca9e0f

---
Most WebPageTest walkthroughs stop at the waterfall. This session was about what you do next.

## About the talk

A live WebPageTest session working through real publisher sites, using Request Map to see which third parties are pulling in which other third parties, and where that chain starts costing real time.

Alongside the audits it collects the practical mechanics that rarely make it into tutorials: getting past cookie consent banners without breaking your measurement, and a handful of bookmarklets that make WebPageTest's own interface easier to read.

## What's covered

- **Live audits.** Global Publishers, Harper's Bazaar and OK Magazine, with Request Map used to trace third-party chains back to whoever invited them.
- **Getting past cookie banners, two ways.** Click the button with `execAndWait`, which simulates a first-time visit and can be merged using `combineSteps` or split with `setEventName`; or set the consent cookie directly with `setCookie`, which emulates a repeat visitor and is the better choice for testing holistic performance.
- **Why the cookie method needs maintenance.** Consent cookies are time-limited, must be set on the exact page origin including protocol and without a trailing slash, and often need a combination of cookies before the banner clears.
- **Five bookmarklets.** Widening filmstrips, widening the run column, reversing comparison order, jumping from SpeedCurve to the underlying WebPageTest result, and opening the Request Map beta from any test.
- **A workflow tip.** Leave the Object tab open to skip quickly between images.

Recorded as a WebPageTest Live stream. Note that the WebPageTest and Request Map result links in the accompanying speaker notes have since expired.


==============================================================================

# Optimising Core Web Vitals on SPAs

> Single-page applications present some unique challenges for measuring and optimising core web vitals, here we review the key optimisations you should be making for your SPA

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2022-03-22
Canonical: https://simonhearne.com/2022/core-web-vitals-on-spas/
Tags: SPA, SEO, WebPerf, consultancy

---
Approximately 40% of my clients have single-page applications (SPAs), and approximately 100% of my clients care about core web vitals - the key performance metrics which impact Google search ranking.

Consistently measuring performance of every website and framework is an impossible task, so the core web vitals (largest contentful paint, first input delay and cumulative layout shift) are lowest common denominator metrics. This means that they have been designed to work almost everywhere, but one of the most important aspects of SPAs is missing: route changes (aka in-app navigations).

The core web vital metrics were designed to be better proxies of user experience, but we need to be mindful of how they apply to SPAs. There are more details on the **core** web vitals in this post, but in summary:

- **LCP**, **FID**, FCP, TTFB, TBT and TTI - only measured on session landing pages
- **CLS** (& Responsiveness) - measured throughout a session, the highest / worst value is collected and *attributed to the landing page URL*

## Largest Contentful Paint

LCP is my favourite web performance metric; it captures a simple event in a web experience &mdash; when the biggest thing is painted to screen &mdash; and envelopes all sorts of important elements of the page load such as time to first byte, document size, blocking JS and CSS.

Only the first landing page in a session will generate LCP values on SPA sites. The [specification](https://developer.mozilla.org/en-US/docs/Web/API/LargestContentfulPaint) calls for LCP capture to start at a navigation event and stop *after the first user interaction*. Route-changes / in-app navigations occur after an interaction and thus are not captured by Chrome for the CrUX dataset.

LCP is captured for every individual page load with MPAs / traditional web apps. This means that mid-session visitors will likely generate fast LCP values due to having a warm cache and connection - and if your average session length is greater than 4.0 this will probably result in a faster 75th percentile value. As LCP is only captured on landing pages for SPAs, the results are biased towards the worst case experience of an empty cache. This potentially means that for two identical sites, the SPA will report worse LCP values.

Whilst mid-session experiences are certainly important for UX, we should focus primarily on landing pages to optimise for Google's core web vitals. This means that we should do everything to reduce the critical path to LCP on a first-view. These first experiences are also crucial to improve user experience and reduce bounce rates!

Unfortunately most SPA frameworks don't make this easy out of the box. E.g. if your product image is defined in data then the image element won't even by injected into the DOM until the application JavaScript has downloaded, parsed and executed. This can lead to late asset discovery and correspondingly poor LCP values.

Server-side rendering (SSR) is key here: first-views should deliver the page as HTML and allow the browser to apply its optimisations to download content in the correct order. Just ensure that hydration doesn't cause reflow!

SSR alone won't save us though. The example waterfall chart below shows an SSRd SPA landing page with some common performance anti-patterns: a very large HTML document and a large amount of preloaded JavaScript assets. Note how these delay the download of the image on line 35, which is the LCP element for this page (LCP is marked as the vertical dashed green line at 5.6s).

![annotated web page test waterfall showing large HTML document and 1.5MB of preloaded JS blocking LCP](https://simonhearne.com/images/cwvs-spas/lcp_waterfall.png)
*Use WebPageTest to generate a landing page waterfall and trace back from LCP*

I've previously written some advice on [optimising for LCP](https://simonhearne.com/2020/core-web-vitals/#largest-contentful-paint-lcp) which is a good place to start. Note that LCP varies by viewport (e.g. a hero image on mobile might be smaller than your heading text element on desktop). Another LCP nuance is that cookie consent banners may be your LCP element for first-time visitors, so make sure to test and optimise for multiple user states! [Harry Roberts](https://twitter.com/csswizardry) has written up a great summary of [the gotchas with LCP](https://csswizardry.com/2022/03/optimising-largest-contentful-paint/).

## Cumulative Layout Shift

CLS is my second-favourite web performance metric; it captures the previously intangible feeling of a page being unstable — how much content moves around when you aren't expecting it.

Unlike LCP, CLS can be captured throughout a user's session on a SPA. Unexpected layout shifts are [accumulated in clusters](https://web.dev/evolving-cls/#why-a-session-window): if three shifts of 0.1 occur within one second then your CLS score would be 0.3. CLS clusters close after one second of inactivity or after a maximum of five seconds; importantly this process occurs throughout a user's session on your SPA and the *largest* CLS score by the end of the session *is attributed to the session landing page*.

This attribution issue means that identifying causes of layout shifts can be difficult. If you see high CLS reports in Google Search Console they will be logged against the session landing page, but could have occurred at any point in the user's session.

CLS has some further nuances: layout shifts are only **unexpected** (and thus count towards CLS) if they are not preceded by a discrete user interaction (click, tap or keypress) within 500ms. This can mean that users on slow network connections report a higher CLS: if for example your search API normally responds in under 500ms from a button press, the subsequent layout shifts are **expected**. In scenarios where the search API takes longer than 500ms, the layout shifts caused by rendering the results will exceed the 500ms threshold and thus become **unexpected**. I call these un-unexpected layout shifts, and on SPAs they can have a significant impact when route changes take longer than half a second.

![screenshot of chrome developer tools showing an unexpected layout shift during a route change](https://simonhearne.com/images/cwvs-spas/unexpected-shift.jpg)
*A route change here took longer than 500ms, resulting in an 'unexpected' layout shift*

Another nuance in the *expectedness* of layout shifts is the user input types: scrolls do not (currently) count as a discrete user input, so any layout shifts on scroll will count towards CLS scores - thus it is important to have `width` and `height` attributes on lazy-loaded images! Also beware of lazy-loaded components, Simon Wicki has a [great blog post](https://wicki.io/posts/2022-03-cls-with-lazy-loading-components/) on this topic.

These nuances mean that optimising for CLS on your SPA is more complex than it may seem at first. I suggest three separate areas of focus:

1. Reduce frequency and size of layout shifts on first-view (easy)
2. Remove layout shifts that occur on scroll (moderate)
3. Investigate potential un-unexpected layout shifts (tricky)

This process (especially for 2. and 3.) is manual and time consuming. Where possible, collect web vitals metrics from your route changes using the [web-vitals](https://github.com/GoogleChrome/web-vitals) JS library or a RUM solution like [Akamai mPulse](https://www.akamai.com/products/mpulse-real-user-monitoring) to help identify the layout shifts that your users are experiencing, attributed to the correct URL.

## First Input Delay

[FID](https://web.dev/fid/) is one of my least favourite web performance metrics &mdash; I have yet to work with a client where their FID numbers showed anything of interest. That said, FID is theoretically a bigger issue on SPAs than MPAs.

FID measures the delay between the first user interaction with your page and the browser being able to respond to it. Basically it represents how likely it is that a user is unlucky and tries to interact with the page whilst a long task is consuming the browser main thread.

The reason I say that FID is more important to SPAs than MPAs is simply due to the reliance on JavaScript for rendering or hydrating pages. Whether your application is client-side rendered or SSRd, there will be a significant chunk of JavaScript execution during the page load to set the page up. FID values on SPAs are biased towards the worst part of the experience as FID is only captured on the landing page.

Optimising FID itself is nearly impossible: the values vary by the visitor's device and the relative timing of their interaction. Instead, focus on reducing total blocking time (TBT). TBT is effectively the sum of long tasks during a page load: a high value indicates that it is more likely that users will try to interact during a long task and thus have a high FID.

You can theoretically improve FID by discouraging interaction until JavaScript execution has completed. I have seen this in practice on some very heavy SPAs (e.g. interactive gaming & gambling) where a splash screen is shown until the application is completely ready. Whilst this will improve FID, it could well degrade LCP, introduce [splash-screen SEO penalties](https://www.koozai.com/blog/search-marketing/why-splash-pages-are-bad-for-seo/) and certainly won't have a positive effect on user experience.

Back to TBT — there are four key ways to reduce the impact of long tasks and positively impact FID:

1. Remove expensive JavaScript (still using [moment.js](https://momentjs.com/docs/#:~:text=We%20now%20generally%20consider%20Moment%20to%20be%20a%20legacy%20project%20in%20maintenance%20mode)?!)
2. Choreograph long tasks to occur when users are less likely to interact
3. Reduce long tasks by breaking expensive functions into chunks (see [requestIdleCallback](https://developer.mozilla.org/en-US/docs/Web/API/Window/requestIdleCallback) for a future option here)
4. Remove all synchronous API calls

To find long tasks: set CPU and Network throttling then record a performance profile of a page load. Look for long tasks in the profile and tackle them from longest to shortest. The example below is an innocuous task which depends on a synchronous XHR - it may not even appear as a long task without network throttling!

![screenshot of chrome timeline showing a three second long task due to a synchronous XHR](https://simonhearne.com/images/cwvs-spas/long-task.jpg)
*Long task due to synchronous XHR discovered using network throttling*

## Bonus: Responsiveness

[Responsiveness](https://web.dev/responsiveness/) is a new web vital being tested by Google. First input delay has received some negative feedback due to being difficult to optimise for and hard to understand. Another issue with FID is that [almost all sites](https://almanac.httparchive.org/en/2021/performance#first-input-delay-fid) (100% of desktop and 90% of mobile) pass the FID assessment, meaning it is not actionable for most site owners.

Responsiveness is a potential replacement for FID, and in my clients it often shows a failing result where FID passes. For example a SPA website I'm currently working on:

![screenshot of crux data showing 0ms FID but 400ms responsiveness](https://simonhearne.com/images/cwvs-spas/responsiveness-treo.png)
*0ms First Input Delay but 400ms for Responsiveness*

The boundaries for good / needs improvement / poor are currently the same as FID: <100ms / <300ms / ≥300ms respectively.

My concern for responsiveness on SPAs is that this new metric captures the *full* interaction. FID only measures the time the browser takes to register a user input, whilst responsiveness measures the end-to-end interaction latency; this potentially includes the client-side processing to respond to the user's input and paint the next frame on screen. For SPAs this could include the logic to execute a route change which can often be an expensive process. The metric may in the future also consider scroll events (currently ignored for FID) which could reflect negatively on infinite-scrollers or lazy loaded components.

If your responsiveness value is currently high (I love [Treo](https://treo.sh/sitespeed/) for quickly seeing CrUX data), this indicates that user interactions on your site may feel slow. The metric itself is still in flux and won't be impacting the page ranking SEO factor for a reasonable time — Google has said that there will be a six month warning before any changes to the core web vitals are pushed live — but it still makes sense to monitor and optimise for responsiveness.

A quick method to determine where a high responsiveness value comes from is to record a profile during an interaction. I use Chrome developer tools for this: load a landing page, then hit record in the performance tab and make an interaction. Look for any delays to rendering the first frame after the interaction and focus your optimisation efforts there. In this case, a click triggers a route change which takes a little over 100ms from the first interaction event to the first frame painted to screen:

![screenshot of chrome developer tools showing long task before frame render after user interaction](https://simonhearne.com/images/cwvs-spas/responsiveness-trace.jpg)
*Chrome performance tab shows delays between interaction and next frame paint*

To reduce the reported responsiveness value we need to review the long task, the multiple style recalculations and garbage collection event.

## In Summary

Core web vitals are useful metrics, and important to optimise for. If you run a SPA focus initially on landing page experiences: these are important not only for the page experience ranking factor but also to reduce bounce rate and improve UX.

Don't be dismayed if you have odd results with cumulative layout shift appearing in Google Search Console. Fixing layout shifts is often easy, as long as you can find them! Record a performance timeline whilst navigating your application in multiple device profiles (at least phone, tablet & desktop) and look for unexpected layout shifts with high scores, I tend to focus on any shift with a score of over 0.05.

If you are still struggling to get your core web vitals into the green, feel free to [reach out to me](/about/) for further advice.


==============================================================================

# Fixing empty responses from Cloudflare Workers Sites

> A quick fix for empty response bodies on Cloudflare Workers Sites

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2022-02-22
Canonical: https://simonhearne.com/2022/empty-responses-cloudflare-workers-sites/
Tags: development

---
I'm currently working on a client project using Cloudflare Workers Sites. Mostly development has been a joy, the `wrangler` CLI is easy to use and production deployment happens in seconds.

There was one strange issue reported by the client which left me looking for answers: sometimes the pages wouldn't render, other times no styles were applied. Occasionally images wouldn't render.

These issues were obviously critical, and I managed to reproduce them only on the production deployment but not with `wrangler dev`. The cause of the broken pages was empty responses &mdash; occasionally a response would appear to be successful (`200` etc.) but there was no content in the body. I had some custom code messing with headers in the document handler, but nothing that modified the CSS or image responses.

The biggest clue to the cause of the issue is that it never occurred with a hard reload, only normal navigations or soft reloads.

It turns out that in this Workers Sites deployment a `last-modified` header is added to responses by default (there's no code in [the bundle](https://github.com/cloudflare/worker-sites-template/blob/master/workers-site/index.js) which does this). As described in my [blog post on caching](/2022/caching-header-best-practices/), this header allows the browser to emit a conditional GET request with the `if-modified-since` header. The correct response from the server is a `200` with the full response if the browser's timestamp is earlier than the server's, or a `304` empty response if the browser has the latest version of the asset.

It appears that this Workers Sites deployment does not correctly respond with a `304` &mdash; instead it emits an empty `200` which causes all of the issues we had observed.

![screenshot of developer tools showing a 200 response with an empty body, with last modified response header and if modified since request header](https://simonhearne.com/images/cf_empty_response.png)
*Empty 200 responses when if-modified-since request header set.*

I'm not a fan of `last-modified` as a caching strategy (especially without a `cache-control` directive) so my quick fix was to delete this response header in the event handler function in `index.js`:

```jsx
async function handleEvent(event) {
  const url = new URL(event.request.url)
  let options = {}
  try {
    const page = await getAssetFromKV(event, options)
    const response = new Response(page.body, page)

    response.headers.set('X-XSS-Protection', '1; mode=block')
    response.headers.set('X-Content-Type-Options', 'nosniff')
    response.headers.set('X-Frame-Options', 'DENY')
    response.headers.set('Referrer-Policy', 'unsafe-url')
    response.headers.set('Feature-Policy', 'none')

    // delete the Last-Modified response header
    response.headers.delete('Last-Modified')

    return response
  }
...
```

I still need to implement a correct caching strategy, but at least the site works consistently now!

I've reviewed other Workers Sites properties that I manage and the `last-modified` response header is not set on those. I'm not sure what is different with this property except that it is the most recent new deployment.


==============================================================================

# Survivorship Bias in Web Performance

> Why your analytics shows your site as faster than it is, and why FCP is a key metric to optimise for.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2022-02-16
Canonical: https://simonhearne.com/2022/survorship-bias-in-webperf/
Tags: WebPerf

---
## Introduction

Way back in 2010, an engineer at YouTube kicked off [Project Feather](https://blog.chriszacharias.com/page-weight-matters): the goal was to reduce the weight of the popular video player page to improve performance. The project was a success: the page weight was reduced from 1,200kB to 98kB and the total number of requests was cut by 90%.

The analytics of Project Feather experiences did not reflect the improvements though: the aggregate page load time actually **increased**. Further investigation revealed that traffic from countries with poor connectivity had dramatically increased &mdash; users who previously could not even load the page were now able to watch videos. These new visitors had relatively slow experiences, bringing the aggregate values down.

There are similar stories from a wide range of websites:

- the bank who decided not to invest in mobile web because customers preferred desktop online banking to mobile (the mobile site was so slow it was barely usable)
- the retailer who had no iPad 1 traffic so stopped testing on iPad 1 (the website failed to load on iPad 1)
- the retailer who had very low Android traffic and assumed their target demographic were iPhone users (Android experiences were twice as slow as iPhone)

There is a term for this issue: *Survivorship Bias*. If you search for it term you'll likely see a variation of this image:

![diagram of a plane with red spots to show an aggregate of bullet strikes](https://simonhearne.com/images/survivorship-bias.svg)
*The classic illustration of survivorship bias.*

The [story goes](https://mcdreeamiemusings.com/blog/2019/4/1/survivorship-bias-how-lessons-from-world-war-two-affect-clinical-research-today) that statistician Abraham Wald used damage logs for aircraft returning from sorties in World War II to identify where reinforcing armour should be applied. The red dots show where bullet holes were found, so the logical assumption would be to reinforce those areas. Wald, however, surmised that the lack of red dots on areas of the aircraft show locations where hits would be critical and result in the aircraft not returning &mdash; so the reinforcements should be applied to those areas.

Our decisions are influenced by the data we have, but what about the data we don't have? In the military example above it is clear where the data is missing, but the web is slightly more complex. The users who have the worst experiences are likely to be [phantom bounces](https://simonhearne.com/2017/analytics-lie/#phantom-bounces): they don't appear in your analytics or intelligence tools because they don't hang around long enough for the app to load and analytics to fire.

## Speed Bias

Google is experimenting with a new web performance metric: **abandonment**. Not to be confused with traditional abandonment metrics like cart abandonment, this is to measure how many users leave a site before it even loads. As part of this research, [Nicolás Peña Moreno](https://twitter.com/NicPenaM) in the [Chrome Speed Metrics](https://chromium.googlesource.com/chromium/src/+/master/docs/speed_metrics/README.md) team used Chrome data to measure the impact of performance on page abandonment rate &mdash; i.e. how many visitors leave a page before it loaded.

The metric used for the performance side of this equation was [First Contentful Paint](https://web.dev/fcp/) (FCP), so abandoners were potential visitors who left the site before anything useful was painted on the screen. It's worth noting at this point that it is very difficult for website owners to track this data themselves as it is unlikely that any analytics or tracking code has had a chance to run before FCP (and nor should it!).

I've taken the [data that Nicolás shared](https://calendar.perfplanet.com/2020/abandonment/) and created a visualisation below to show the scale of the issue. Unfortunately no sample sizes were included in the results so we can't measure statistical significance at the higher FCP numbers, but the trends speak for themselves:

<div class="vega-chart loading" id="vis1" data-spec="/data/survivorship-bias/chrome-abandonment.json">Loading chart...</div>

On Chrome Android, every second that FCP is delayed results in an additional 2.6 percentage points of abandonment. To clarify with an example: if your FCP was exactly 4s for all Chrome Android users, your analytics likely under-reports these users by 17.3%. So if your analytics shows one million users, there are another 210k users that abandoned before being logged (`1,000,000 / ( 1 - 0.173 )`). Reducing your FCP by one second would increase your reported visits by 2.9pts or 35,000 (`( 1,000,000 / ( 1 - 0.173 ) ) * ( 1 - 0.144 )`).

It is clear that poor performance will impact your traffic numbers, and means that thousands of potential visitors are leaving your site without a trace. It is also clear that mobile visitors are less tolerant of slow sites.

There is a secondary issue here: the impact on performance metrics.

It is fair to assume from this data that slower websites don't appear as slow as they are (as the slowest experiences will lead to abandonment) and, as we saw with the YouTube Feather example, making performance improvements may not reflect directly in your data. If we take the values from the chart above and apply them to a typical (log-normal) performance distribution we can predict the impact on reported metrics. A sample data set of 10,000 visits is created and the mobile abandonment statistics applied to see the difference in results, try adjusting the goal mean of the distribution and observe the changes in the table below.

<div class="vega-chart loading" id="vis2" data-spec="/data/survivorship-bias/distribution.json">Loading chart...</div>

<style>
  #chartStats th, #chartStats td {
    padding: 2px 6px;
  }
  #chartStats td:first-child {
    font-weight: bold;
  }
</style>
<table border="1" style="font-size: clamp(0.6rem, 2vw, 1.1rem); min-width:90%;" id="chartStats">
<tr><th>Metric</th><th>Actual Visits</th><th>Excluding Abandons</th><th>Difference</th><th>Relative Difference</th></tr>
<tr><td>Visits</td><td></td><td></td><td></td><td></td></tr>
<tr><td>Median FCP</td><td></td><td></td><td></td><td></td></tr>
<tr><td>75% FCP</td><td></td><td></td><td></td><td></td></tr>
<tr><td>90% FCP</td><td></td><td></td><td></td><td></td></tr>
<tr><td>95% FCP</td><td></td><td></td><td></td><td></td></tr>
</table>

Whilst this data is entirely hypothetical, I hope it helps bring to life the statistics on mobile abandonment shared earlier. Slow sites are more affected by this issue, and so are slow visitors &mdash; is your analytics showing a low level of traffic from Android devices because there really are a low number of visitors, or just a high abandonment rate? The difference at high percentiles is worth noting too, your 75th percentile FCP may be reported over a second faster than reality!

## Actions to take

It is extremely difficult to measure the impact of early abandonment as a site owner. It is possible to compare analytics with logs, though, and determine the user agents or regions which have the highest disparity between intent to load a page (from server or CDN logs for documents) and successfully loading the page (analytics or RUM data). This strategy has some issues: some requests in your logs will be for crawlers which don't execute your analytics JavaScript and there will also be some disparity between user agents due to visitors using tracker blockers.

It is possible to beacon data early in a page load in order to better capture visitors who abandon before traditional analytics scripts fire. [Nic Jansma](https://twitter.com/nicj) has written up an [analysis of strategies](https://nicj.net/beaconing-in-practice/#beaconing-reliability) which shows that beacon capture (on a fast site) can be improved from 86.4% to 92.8% by listening to `pagehide` and `visibilitychange` events in addtion to `onload`. This still requires the JavaScript to execute before the visitor abandons and for the beacon to not be blocked by tracker blockers.

Whilst there are strategies to improve our data collection for abandoners, we will always lose some data. The best method to reduce abandonment and increase captured traffic is... improving performance! There are two reasons for this:

- faster page loads mean that analytics events fire earlier and are more likely to capture the visitor
- faster page loads are a better experience, so visitors are more likely to stay on the page

Both of these are positive outcomes!

Whilst the focus in web performance has recently been on the *Core* Web Vitals, I encourage you to analyse your First Contentful Paint with as much scrutiny as Largest Contentful Paint, Cumulative Layout Shift or First Input Delay. FCP was used by Nicolás in the abandonment study because it is such an important indicator of user experience.

![Web Page Test filmstrip showing a gap of blank screen before first contentful paint](https://simonhearne.com/images/fcp_simon.jpg)
*FCP marks when your visitor can see something useful*

FCP marks the context switch from the previous page or a blank screen to showing something that the visitor actually expects to see, so in my analysis it is often the primary metric I focus on improving. If FCP is high, it means that visitors are more likely to think that the page is not working, they have time to reconsider what action they were taking and can easily hit the back button.

Note that FCP [isn't a great cross-browser metric](https://blog.webpagetest.org/posts/why-first-contentful-paint-doesnt-work-as-a-cross-browser-metric/), so your mileage may vary when tracking it over time and across browsers and operating systems. That does not necessarily mean that it is a bad metric, it's just complex to implement consistently across the different browser engines.

The key points for optimising FCP are universal, though:

- reduce time to first byte (TTFB): use a CDN, cache HTML pages
- reduce blocking JS and CSS
- defer non-critical JS
- remove render blocking third-party tags

Expect to see the unexpected in your real user analytics when shipping performance improvements! Technically reducing FCP may in fact increase your reported FCP as more visitors on low-end devices and connections successfully load your pages. Expect to see traffic numbers change for different device types, and combine your performance metrics with business metrics to get a more holistic view of the success of each technical performance improvement.

<script>
  let initialised = false;
  let data = {'stats':{},'sampledStats':{},'sampleSize':10000,'sampledSize':0};

  let checker = setInterval(()=>{
    if (typeof(window.charts) == 'object') {
      if (window.charts.filter(c => c.id == "vis2").length > 0) {
        initilised = true;
        initChartListeners();
        clearInterval(checker);
      }
    }
  },1000);
  function initChartListeners() {
    let view = window.charts.filter(c => c.id == "vis2")[0].view;
    // set up view listeners for stats and sampledStats
    view.addSignalListener('stats', chartListener);
    view.addSignalListener('sampledStats', chartListener);
    view.addSignalListener('sampledSize', chartListener);
    data.stats = view.signal('stats');
    data.sampledStats = view.signal('sampledStats');
    data.sampledSize = view.signal('sampledSize');
    data.sampleSize = view.signal('sampleSize');
    renderStats();
  }
  function chartListener(name,value) {
    data[name] = value;
    renderStats();
  }
  function renderStats() {
    let el = document.getElementById("chartStats");
    el.innerHTML = `
<tr><th>Metric</th><th>Actual Visits</th><th>Excluding Abandons</th><th>Difference</th><th>Relative Difference</th></tr>
<tr><td>Visits</td>     <td>${(data.sampleSize).toLocaleString()}</td> <td>${data.sampledSize.toLocaleString()}</td>  <td>-${(data.sampleSize - data.sampledSize).toLocaleString()}</td><td>-${Math.round(((data.sampleSize - data.sampledSize)/data.sampleSize)*1000)/10}%</td></tr>
<tr><td>Median FCP</td> <td>${(Math.round(data.stats[0].value*100)/100).toLocaleString()}s</td><td>${(Math.round(data.sampledStats[0].value*100)/100).toLocaleString()}s</td><td>${(Math.round((data.sampledStats[0].value - data.stats[0].value)*1000)).toLocaleString()}ms</td><td>${Math.round(((data.sampledStats[0].value - data.stats[0].value)/data.sampledStats[0].value)*1000)/10}%</td></tr>
<tr><td>75% FCP</td>    <td>${(Math.round(data.stats[1].value*100)/100).toLocaleString()}s</td><td>${(Math.round(data.sampledStats[1].value*100)/100).toLocaleString()}s</td><td>${(Math.round((data.sampledStats[1].value - data.stats[1].value)*1000)).toLocaleString()}ms</td><td>${Math.round(((data.sampledStats[1].value - data.stats[1].value)/data.sampledStats[1].value)*1000)/10}%</td></tr>
<tr><td>90% FCP</td>    <td>${(Math.round(data.stats[2].value*100)/100).toLocaleString()}s</td><td>${(Math.round(data.sampledStats[2].value*100)/100).toLocaleString()}s</td><td>${(Math.round((data.sampledStats[2].value - data.stats[2].value)*1000)).toLocaleString()}ms</td><td>${Math.round(((data.sampledStats[2].value - data.stats[2].value)/data.sampledStats[2].value)*1000)/10}%</td></tr>
<tr><td>95% FCP</td>    <td>${(Math.round(data.stats[3].value*100)/100).toLocaleString()}s</td><td>${(Math.round(data.sampledStats[3].value*100)/100).toLocaleString()}s</td><td>${(Math.round((data.sampledStats[3].value - data.stats[3].value)*1000)).toLocaleString()}ms</td><td>${Math.round(((data.sampledStats[3].value - data.stats[3].value)/data.sampledStats[3].value)*1000)/10}%</td></tr>
    `;
  }

</script>


==============================================================================

# The value of an independent web performance consultant

> If you're wondering if you need an independent web performance consultant, this is why I think you do!

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2022-02-04
Canonical: https://simonhearne.com/2022/value-of-a-consultant/
Tags: WebPerf, consultancy

---
I have been employed in web performance roles for about a decade. From load testing and synthetic monitoring to CDN configuration and true consultancy. In 2022 I became an independent web performance consultant.

One thing has stood out from my experience working with hundreds of brands &mdash; from small national websites to massive multi-national ecommerce websites: folks generally understand web performance, why it matters and the core principles, but they struggle to keep on top of industry trends whilst maintaining a complex production codebase. Sometimes all it takes is a call that brings together marketing, product and development teams to talk it through; other situations call for a multi-year project to gradually introduce optimisations and measure change.

Coming in as a consultant allows me to be a catalyst for positive change, even if all of the performance issues had previously been documented! Dedicated, independent consultants (like me, or others in the industry) help to bring focus and passion to a topic that is often under-valued and under-invested. This quote from Rob Kerr on self-employment resonates with me:

<blockquote>I believe that, given the right preparation, self-employment can be empowering for the individual and can solve meaningful problems in the world. Many of these problems would never justify the investment at a corporate level, and nor could a big business deliver the solution to the same standard as the person who is passionate about making that difference.
<p class="align-right small">&mdash; <a href="https://www.scribd.com/book/484449345/Project-Future-6-Steps-to-Success-as-Your-Own-Boss">Project Future</a> by Rob Kerr</p>
</blockquote>

In most organisations it does not make sense to employ a full-time individual who is solely focussed on site speed. Some of the more successful organisations I've worked with have web performance working groups - a cross-functional team who have a few days per month to discuss and prioritise performance improvements. Even in these cases, having an external view on site speed adds incredible value: reviewing the KPIs that are being measured, prioritising backlogged tasks, improving tooling and even identifying new issues that have been undetected by the in-house teams.

Independent performance consultants add further value because of the work we do across multiple clients: working in different codebases on different platforms for different types of organisation. We absorb best practices from multiple teams and can help apply them to each of our clients. If nothing else, an experienced second set of eyes on a website will always find something to improve.

Speaking of improvements: measuring impact is one of the most difficult parts of web performance in general, and is a real challenge when it comes to consultancy. I would love to be able to attribute performance optimisations back to real business impact, but this is near impossible in a world of daily releases, multiple campaigns per week and the rollercoaster of SEO / SEM. We know from [hundreds of case studies](https://wpostats.com) that improving performance improves user experience and business results, so most times we have to rely on that implicit understanding.

In an ideal world performance changes are [A/A/B](https://blog.analytics-toolkit.com/2014/aa-aab-aabb-tests-cro/) tested to determine the impact on business results, using a real-user monitoring (RUM) solution like [Akamai mPulse](https://www.akamai.com/products/mpulse-real-user-monitoring) or [SpeedCurve](https://www.speedcurve.com/features/performance-monitoring/) to track performance + business metrics together. Even then, some performance improvements will impact metrics that are near-impossible to track: e.g. user satisfaction, return rates and likelihood to recommend. Improving performance *will* improve business metrics, it's just nearly impossible to measure the direct impact.

[Katie Sylor-Miller](https://twitter.com/ksylor) recently made the same observation about the difficulties in attributing positive performance changes to business results:

<blockquote class="twitter-tweet" data-dnt="true"><p lang="en" dir="ltr">It feels like it's easy to detect &amp; see business impact of performance *regressions*, but much harder to correctly measure &amp; know when perf improves - especially when it significantly improves user experience👇🏻</p>&mdash; Katie Sylor-Miller (@ksylor) <a href="https://twitter.com/ksylor/status/1488657225744896005?ref_src=twsrc%5Etfw">February 1, 2022</a></blockquote>

Another difficult part of web performance is determining the impact of a technical change at a user experience level. Reducing time to first byte (TTFB) by 500ms will have a very clear benefit: every experience, every pageview, will become a half a second quicker. But most changes &mdash; like reducing CSS bundle size, optimising images or reducing JavaScript execution times &mdash; will not have such a linear relationship.

Improving page load time by 100ms under test conditions may not seem like a lot, but at the tail-end of your user distribution this could result in more than a second of improvement. Users in more remote geographies, on older phones, on poorer quality networks will always see a bigger improvement than users on a flagship phone connected to a 100+Mbps network. Implementing a RUM solution and providing deep analysis of the data can help to illuminate the users having the worst experiences and to identify the most important optimisations to focus on.

**In summary**: performance matters to user experience, even if it is difficult to prove. An independent performance consultant can help to bring teams together, to identify and prioritise performance enhancements and to measure their impact. As a bonus: onboarding a consultant is almost certainly quicker (and cheaper) than developing and maintaining in-house expertise to the same level - all while sharing knowledge internally and helping to build a performance culture.


==============================================================================

# Caching Header Best Practices

> Caching headers are surprisingly complex and often misconfigured. Here we look at some key cache scenarios and recommend the ideal headers to set.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2022-01-27
Updated: 2022-01-28
Canonical: https://simonhearne.com/2022/caching-header-best-practices/
Tags: WebPerf

---
<style>
  .key {
    border-radius: 5px;
    border: 1px solid var(--border);
    padding: 1px 2px;
    font-size: var(--font-size-small);
    font-family: var(--font-family-monospace);
    background-color: var(--surface);
    color: var(--text);
  }
</style>

## Introduction

Caching headers are one of those deceptively complex web technologies which are so often overlooked or misconfigured. The fastest request is the one that is not made, and caching headers allow us to tell browsers when they can reuse an asset that they have already downloaded. The reason that these headers are often misconfigured, or at least configured suboptimally, is often through a descent to lowest risk. Allowing a browser to use a cached asset can be considered risky - JS which falls out of sync with HTML, CSS which persists an old campaign style, personalised assets accidentally being shared between visitors.

In this post we will review what caching headers are available and when they should be used. We'll also talk about invalidating caches and ensuring browsers use the correct assets at the correct time. Note that the focus of this post is on client-side (or downstream) caching - in the client device. Caching in proxies, load balancers and Content Delivery Networks (CDNs) adds some more complexity, and is not covered here.

### The Solution

Use versioned assets wherever possible (e.g. `main.v123.min.css` or `main.min.css?v=123`) and set a single caching header allowing the maximum cache duration of one year:

```http
Cache-Control: max-age=31536000, immutable
```

For non-versioned assets which may change, combine the Cache-Control header with an ETag for asynchronous revalidation in the client:

```http
Cache-Control: max-age=604800, stale-while-revalidate=86400
ETag: "<file-hash-generated-by-server>"
```

For HTML files, set a low TTL and private cache flags:

```http
Cache-Control: max-age:300, private
```

Do not emit unnecessary caching headers (including `ETag` and `Last-Modified`) to prevent unexpected client and server behaviours.

![Flowchart of decisions to determine which caching headers are best in different scenarios. The logic is described fully in the document](https://simonhearne.com/images/caching-header-best-practices/cache-headers.svg)
*A summary of the logic to determine which caching headers are best in different scenarios.*

### Caching Headers

There are a number of response headers set by a web server or CDN which manage client-side caching. Some are more obvious than others!

`Expires` - a date (in GMT) after which this asset may no longer be used from the browsers cache and must be re-fetched ([docs](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Expires))

`Cache-Control` - a combination of features in one header, including how long the resource can be cached by the client (in seconds) as well as whether proxies can cache it, whether to force revalidation and more ([docs](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control))

`ETag` - a string that uniquely identifies an asset version, generally a server-generated hash of the file ([docs](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag))

`Last-Modified` - a timestamp which allows browsers to validate the freshness of cached assets ([docs](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Last-Modified))

`Pragma` - a hangover from HTTP/1.0, this should generally not be used in preference for `Cache-Control` except where HTTP/1.0 clients *must* be supported ([docs](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Pragma))

In general I recommend to not emit an `Expires` header and rely instead on the more comprehensive `Cache-Control` header. I also recommend to not emit a `Last-Modified` header and use `ETag` instead for asset revalidation, this avoids edge cases such as newer files with identical content or clock mismatches between web servers which would cause unnecessary bandwidth consumption.

For revalidation (aka conditional requests) to work, responses must be served with one or both of the `ETag` or `Last-Modified` headers. The server must also understand conditional get requests and respond with a `304 Not Modified` in the case that the cached asset matches that on origin. Note that validation of weak ETags (prefixed by `W/` in the header) are unsupported in some scenarios &mdash; including if using Akamai as a CDN &mdash; so you may want to use strong ETags where possible.

<blockquote class="callout">
  <p>ETags are simply strings which identify a specific version of an asset. Weak ETags (prefixed with <code>W/</code>) should match assets which are semantically the same (e.g. metadata has been updated but the content is the same), whereas strong ETags should change whenever the asset is changed in any way. (See <a href="https://www.rfc-editor.org/rfc/rfc7232#section-2.1">RFC 7232</a> for more info!)</p>
  <p>By default, Apache 2.3.14 and earlier included <code>INode</code> in the ETag - meaning that the ETag for an identical asset would change between servers! This is no longer included by default, and you can <a href="https://httpd.apache.org/docs/2.4/mod/core.html#FileETag">configure what is used to produce the ETag</a> in Apache. The default is to use the last-modified time and the file size, but you can also choose to use a file digest, and manually include the INode back into the calculation.</p>
  <p>If you have your own versioning implemented on the web server you could generate ETags yourself. E.g. <code>ETag: core-js-es6-v13.1234-gzip</code>. But then you might as well rename the file to break the cache on the front-end.</p>
</blockquote>

## Caching Behaviours

There are generally four types of caching behaviour we may want a browser to use when it has downloaded a static asset:

### 1. Not Cacheable

For assets that are dynamically generated, that are unique or that are only valid once.

`Cache-Control: no-cache`

This directive tells the client that it can cache the asset, but it cannot use the cached asset without revalidating with the server. If the asset cannot be revalidated (i.e. there were no `ETag` or `Last-Modified` response headers on the asset) then the cached asset will never be used.

`Cache-Control: no-store`

This directive tells the client that the asset may not be stored in cache at all. Any further requests for this asset will be full requests back to the server.

### 2. Immutable

These assets that can be stored by the browser indefinitely because they never change. Use this for versioned assets (e.g. `main.v123.min.css` or `main.min.css?v=123`).

```http
Cache-Control: max-age=31536000, immutable
```

The `immutable` directive explicitly tells the browser ([where supported](https://caniuse.com/mdn-http_headers_cache-control_immutable)) that the asset can be stored for a year and never needs to be revalidated.

### 3. Time-restricted

For assets which should be stored for the duration of a session (e.g. one day or one week) but should be refreshed or revalidated if the visitor returns later.

```http
Cache-Control: max-age=86400
```

### 4. Revalidated

When combined with #3, this allows browsers to use a cached asset for a period of time (from zero seconds up to a year) and then revalidate the object with origin once that period has expired. `stale-while-revalidate` causes the revalidation request to happen asynchronously, improving performance at the potential risk of using stale content with a second time to live (TTL) value for how long the stale asset may be used.

```http
Cache-Control: max-age=604800, stale-while-revalidate=86400
```

## Optimal Caching Strategy

In general we want browsers to cache everything forever. This can be achieved quite simply by setting a `Cache-Control: max-age=31536000` response header - using the maximum TTL value of one year. The issue is that your web applications likely change more frequently than yearly. This is where the most important feature of your web build pipeline comes in - versioned assets!

With versioned assets, the browser will automatically ignore stale cached assets as the references will be updated. Instead of the HTML document requesting `main.v123.min.css`, the reference will be to `main.v124.min.css` (note the incremented version number, this will be automatically generated at build time and may be an asset hash).

If assets will always be versioned, we can consider them `immutable`. That means that if the content of the file changes we guarantee that the filename is changed. Some browsers support this concept natively in the `cache-control` header, preventing revalidation requests ever being made for these assets:

```http
Cache-Control: max-age=31536000, immutable
```

### Query Strings and Caching

If an asset filename cannot be updated automatically, a query string parameter can be added to the URL to break the cache. For example `main.min.css?v=124`. This method should be robust in most cases, ensuring that your CDN configuration treats query strings as part of the asset cache key for caching at the CDN level:

- Cloudflare [includes query strings by default](https://developers.cloudflare.com/cache/how-to/set-caching-levels)
- Akamai [excludes query strings by default](https://developer.akamai.com/blog/2017/04/14/what-you-need-know-about-caching-part-3#how-query-parameters-affect-cache-keys)
- Fastly [includes query strings by default](https://docs.fastly.com/en/guides/manipulating-the-cache-key)

### Irregular Updates to Unversioned Assets

One of the trickiest caching scenarios to manage is where an asset is unversioned (no unique identifier in the URL) and it is updated on an irregular schedule. This could be, for example, a document containing stock information, transport schedules or feature flags.

In this case, we still want the browser to be able to use the cached asset but also ensure that it remains relatively fresh. This is (in my opinion) the only scenario where entity tags (ETags) make sense. ETags should be used in combination with a valid `Cache-Control` header to ensure you have control over how the browser manages its cache state:

```http
Cache-Control: max-age=86400, must-revalidate
ETag: "a-unique-hash-generated-by-the-server"
```

This example will allow the browser to use the cached asset for up to 24 hours (86,400 seconds), after which it must revalidate with the server. A revalidation request (also known as a conditional GET request) acts just like a normal request for the asset, with the addition of one or two request headers: `If-None-Match` where an ETag was present on the original response, `If-Modified-Since` in the case that a Last-Modified header was present:

```http
GET /main.min.css

Accept: */*
Accept-Encoding: gzip,br
If-None-Match: "a-unique-hash-generated-by-the-server"
```

This `If-None-Match` header is a message to the server that the client has a version of the asset in cache. The server can then check to see whether this is still a valid version of the asset - if so, we will receive an empty `304` response with another ETag which will match the original:

```http
304 Not Modified

ETag: "a-unique-hash-generated-by-the-server"
```

If the asset has changed since the client cached it, we will get a full response with the new ETag:

```http
200 Found

Content-Length: 100000
ETag: "a-NEW-unique-hash-generated-by-the-server"
```

This approach is valuable, but does have drawbacks:

- Generating & maintaining asset hashes has a small compute cost on the server
- Some web servers have bugs with ETag generation and validation, especially with multiple servers behind a load balancer
- A `304` response will add a small front-end delay for each request and incurs a small amount of compute on the web server - the client will not use the (valid) cached asset until the `304` response is received

A similar process occurs when the `Last-Modified` header is present on responses. You may see conditional requests sent with the `If-Modified-Since` header:

```http
GET /main.min.css

If-Modified-Since: Wed, 21 Oct 2015 07:28:00 GMT
```

If both `If-Modified-Since` and `If-None-Match` request headers are present then the server must only return a `304` if **both** conditional fields match the origin content. The `If-Modified-Since` header allows the server to check the last modified time of the requested asset and again return a `304 Not Modified` if it is the equal to or earlier than the timestamp in the request.

Using `Last-Modified` may cause more cache-misses than `ETag` if releases touch all files on the web server, even if they have no updates (updating the last modified time on all files). There is a benefit to using `Last-Modified` over `ETag` in the edge case where servers may hold older versions of an asset - the `ETag` will not match and result in a full `200` response, whereas the `Last-Modified` date will be later than the origin asset, resulting in an empty `304` response.

## What about HTML?

HTML assets are critical to performance in traditional (non-SPA) web applications. A 100ms delay downloading the HTML for a page will make everything else 100ms slower.

Folks are normally very nervous about allowing browsers to cache HTML though, for a number of good reasons:

- Personalisation in the page (e.g. user name, basket contents, geolocation logic)
- Asset versions (e.g. `main.v123.min.css`) are updated in the HTML to purge the client cache on release
- Rapid updates (e.g. a news publication with breaking stories)

Even with all of these considerations, caching HTML can still be achieved and performance can be significantly improved. We just need to be careful!

### Short TTLs & private caches

Allowing an HTML asset to be cached for a short period can improve user experience with minimal risk. Setting a TTL of five minutes (300s) for example should be fine, you can also add `must-revalidate` to ensure that browsers do not use a stale version of the asset after the TTL has expired. This will benefit visitors who click links returning them to previously visited pages, this does not affect the [back/forward cache](https://web.dev/bfcache/).

The `Cache-Control` header offers the `private` attribute, indicating that assets should not be stored by any proxies or CDNs, but may be cached by the client. This attribute enables you to allow browsers to cache personalised content - as the cached asset will not be shared across multiple visitors.

### Remove dynamic elements from the HTML document

Dynamic elements such as a basket count, user name, logged in / out status indicator all make caching HTML more tricky. Externalising these out to API calls, JavaScript and localStorage will mean that the base HTML template can be cached in the browser, with dynamic elements updated as the page is building. This will also allow pages to be cached by CDNs - greatly reducing Time to First Byte (TTFB) and thus improving experience.

This concept can be taken further in some scenarios, such as when anonymous visitors are presented generic / un-personalised pages. In this case, you can detect the anonymous visitor (e.g. by presence of a user cookie) and serve them a cached page from the CDN, whereas logged-in / recognised visitors will be served by origin to allow personalisation code to run. The possibilities here are quite exciting!

## General Recommendations

Explicitly set the `Cache-Control` header on all responses. Do not set the following response headers in most scenarios:

- `Last-Modified`
- `Expires`
- `ETag`
- `Pragma`

Always set the `Cache-Control` header, preferably with the value `max-age=31536000,immutable` alongside unique asset filenames (or `no-cache` for non-cacheable assets).

In some scenarios, use ETags to allow browsers to revalidated cached content using the following headers (e.g. cache for one week, allow stale assets with async revalidation for up to one day after cache expiry):

```http
Cache-Control: max-age=604800,stale-while-revalidate=86400
ETag: "<generated-by-server>"
```

The lack of `Last-Modified` and `ETag` headers on a response prevents the browser from making conditional GET requests, potentially reducing back-end load.

Ensure that your web application server / load-balancer / CDN handles conditional GET requests correctly and returns a `304 Not Modified` response when ETags match. Use `cURL` or [redbot.org](http://redbot.org) to test this.

## How to check your headers

Throughout this post I have detailed headers that you might see when requesting assets - of course these aren't visible on the web application directly! There are two key methods to check the response headers of your assets: using the browser and using command-line tools.

### Browser developer tools

All browsers have developer tools to help analyse network traffic, in Google Chrome you can press <span class="key">⌘</span> + <span class="key">⌥</span> + <span class="key">i</span> (&nbsp;<span class="key">ctrl</span> + <span class="key">shift</span> + <span class="key">i</span> on Windows) to bring up Chrome Developer Tools, it will default to the last open tab so you may need to select the network tab. The network tab only records traffic whilst open, so reload the page to view the requests if none are present.

Once you have some traffic, click on a request to see more details:

![screenshot of chrome developer tools showing response headers](https://simonhearne.com/images/caching-header-best-practices/header-chrome-devtools.png)
*Chrome Developer Tools showing response headers*

If this is a task that you will repeat often, I recommend adding the relevant response headers to the network table. Right-click on the column headers and select `Response Headers`, `Cache-Control` should be there by default and you can also add custom headers here.

![screenshot of chrome developer tools showing response headers in a column](https://simonhearne.com/images/caching-header-best-practices/header-chrome-column.png)
*Chrome Developer Tools showing a column for Cache-Control headers*

### Command-line tools

[cURL](https://curl.se/) is one of the most widely deployed command line tools and is ideal for checking response headers. A simple command such as `curl 'https://simonhearne.com/css/main.css?v=4.1' -Is` will show you all response headers for the object, although we can build on this to focus just on the headers that we care about for caching (note that I have added some required request headers `user-agent` and `accept-language` so that a valid response is provided):

```bash
curl 'https://phoenixnap.com/kb/wp-includes/css/dist/block-library/style.min.css?ver=5.9' \
  -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36' \
  -H 'accept-language: en-GB,en-US;q=0.9,en;q=0.8' \
  --compressed -Is | grep -E 'cache-control|etag|last-modified|expires|pragma'

last-modified: Wed, 26 Jan 2022 05:24:24 GMT
etag: W/"61f0db08-1357b"
expires: Thu, 26 Jan 2023 11:29:02 GMT
cache-control: max-age=31536000
pragma: public
cache-control: max-age=31536000, public
```

For this example, I would recommend that the website owner investigates the various response headers here which may conflict with each other. This set of six response headers could be replaced with a single header: `Cache-Control: max-age=31536000`! Redbot also shows a number of warnings for this implementation:

![screenshot of redbot showing warnings about response headers](https://simonhearne.com/images/caching-header-best-practices/redbot.png)
*Redbot showing potential issues with response headers (view on redbot.org)*

## How to change response headers

Once you have determined your caching header strategy and reviewed the headers that are currently being emitted - it's time to make some changes!

Response headers can be set at multiple stages depending on your application architecture - for example the web server, load balancer and CDN can all set and modify response headers. Generally we should expect response headers to be forwarded transparently through proxies such as load balancers and CDNs unless explicit changes have been made, so the web server is the best place to start. You should be able to find what you need with a search for `<web server name> set response headers` or `<web server name> set caching headers` on your search engine of choice. For convenience, here are the relevant docs pages for some major web servers, PaaS / SaaS and CDN solutions:

### Web Servers

- Apache
  - [ETags in Core](https://httpd.apache.org/docs/2.4/mod/core.html#FileETag)
  - [mod_headers](https://httpd.apache.org/docs/2.4/mod/mod_headers.html)
  - [mod_cache](https://httpd.apache.org/docs/2.4/mod/mod_cache.html)
- [nginx](https://www.digitalocean.com/community/tutorials/how-to-implement-browser-caching-with-nginx-s-header-module-on-ubuntu-16-04)
- [h2o](https://h2o.examp1e.net/configure/headers_directives.html)

### PaaS / SaaS

- [Netlify](https://docs.netlify.com/routing/headers/) - note that there is an [open issue](https://answers.netlify.com/t/setting-response-headers-only-on-documents/6144/40) regarding setting headers by content-type.
- [Wix](https://support.wix.com/en/article/site-performance-caching-pages-to-optimize-loading-speed) - note that Wix does not provide fine-grained control over caching headers
- [WPengine](https://wpengine.com/support/cache-control-headers-wp-engine/)
- [Webflow](https://discourse.webflow.com/t/setting-http-header-fields/180953/4) - it doesn't look like you can control headers and the recommendation is to use a CDN in front of Webflow

### CDNs

- [Akamai](https://techdocs.akamai.com/api-gateway/docs/caching#downstream-caching)
- [Fastly](https://docs.fastly.com/en/guides/controlling-caching)
- [Cloudflare](https://developers.cloudflare.com/cache/how-to/set-browser-ttl) - lets you set a TTL but no fine-grained control over response headers
- [Cloudflare Pages](https://developers.cloudflare.com/pages/platform/headers)
- [Cloudflare Workers](https://developers.cloudflare.com/workers/examples/alter-headers) - I use a worker in front of Netlify to [overcome limitations](https://answers.netlify.com/t/setting-response-headers-only-on-documents/6144/40) in netlify.toml. Provides absolute control over response headers.

## A note on Cache-Control: public

A lot of resources (e.g. on [web.dev](https://web.dev/http-cache/)) recommend setting the `public` attribute on `Cache-Control` headers if resources can be shared between visitors. This is potentially misleading as `public` [is the default behaviour](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#public) where `private` is not set.

The `public` attribute should **not** be set in almost all cases, as it can lead to an issue with authorized requests. Setting the `public` attribute in a `Cache-Control` header for an authorized request (i.e. one which is made with an `Authorization` header) will explicitly allow intermediary servers to store and serve the response to other visitors. In most cases this is not the expected behaviour and could lead to data leakage.

## In closing&#x2026;

Client-side caching is a key technique to improving front-end speed and user experience. Whilst it may appear complex and risky, investing the time to review your content and setting the correct response headers will reduce bandwidth utilisation and improve speed for return visitors as well as mid-session.

There are multiple methods to manage caching, in this article I have presented a preference for `Cache-Control` (and `ETag` where appropriate). Exact implementation is not as important as correctness and consistency, though. Use what works for your application architecture, environment and processes.


==============================================================================

# Fast and Responsive Hero Videos for Great UX

> Responsive landing page hero videos are increasingly popular, here we show how to make them as fast as possible.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2021-11-02
Updated: 2022-06-28
Canonical: https://simonhearne.com/2021/fast-responsive-videos/
Tags: WebPerf

---
Landing page hero videos are increasingly popular, especially in high fashion and luxury goods. Hero videos can have a detrimental effect on user experience though, delaying page load times and causing the page to jump around. In this post we will take an example homepage and see how we can optimise it to deliver fast, responsive videos for great UX.

Jump straight to the [conclusion](#conclusion) to see the results for each optimisation (in total we halve the video load time) and see the final working demo with code!

<figure class="no-shadow">
  <div class="cols-2">
    <img src="/images/fast-responsive-videos/phone.jpg"
      alt="Mobile mockup of the demo homepage playing a tall portrait rain video above placeholder text"
      loading = "lazy"
      clickable = false
      maxwidth = 300
    />
    <img src="/images/fast-responsive-videos/ipad.jpg"
      alt="Tablet mockup of the demo homepage playing a wide landscape waterfall video above placeholder text"
      loading = "lazy"
      clickable = false
      maxwidth = 300
    />
  </div>
  <figcaption>The example website uses different aspect ratio videos for narrow and wide viewports</figcaption>
</figure>

Consider this simple homepage with a hero video and content below the video. The video has been created in two aspect ratios: one for mobile / portrait viewports and another for desktop / landscape viewports. It is common to see this developed along the following lines:

```html
<!-- ==== homepage.html ==== -->

<video
  class="full-width-video" id="hero-video"
  playsinline muted loop autoplay
  src="desktop.mp4"
  data-src-mobile="mobile.mp4"
  data-src-desktop="desktop.mp4">
    <div class="video-error">Uhoh, no video for you.</div>
</video>
```

```css
/* ==== style.css ==== */

.full-width-video {
  background-color: black;
  width: 100%;
}
```

```js
/* ==== app.js ==== */

const DESKTOP_BREAKPOINT = 480;
window.addEventListener("load",function() {
  let heroVideoEl = document.getElementByID("hero-video");
  let heroVideoSrc = heroVideoEl.dataset.srcMobile
  let viewportWidth = window.innerWidth;
  if (viewportWidth >= DESKTOP_BREAKPOINT) {
    heroVideoSrc = heroVideoEl.dataset.srcDesktop;
  }
  heroVideoEl.src = heroVideoSrc;
})
```

Note that the `<video>` element no size attributes and has a `src` which will be replaced for mobile visitors. This causes two types of issue:

**Performance**

1. The desktop video will likely always be requested, even for mobile visitors who will never see it. The browser should cancel the request when the `src` is replaced, at least.
2. The mobile video doesn't start downloading until after app.js has downloaded, parsed and executed - delaying the mobile experience. On a 3G connection we must wait six seconds to see the first frame of the video!

![WebPageTest filmstrip: the original page shows a blank black video box until the first frame appears at 6.0s](https://simonhearne.com/images/fast-responsive-videos/original.png)
*Slow hero videos can detract from landing page experience.*

**Layout**

The browser cannot allocate the correct vertical space for the video until the first chunk is downloaded, causing an unpleasant layout shift at 5.5s in the filmstrip above.

The following four sections will take us from this six second experience to under three seconds, and with no layout shifts!

## Optimisation 1: Preventing the Layout Shift

The layout shift above is caused by the lack of size information on the video element. In this case there are two different video aspect ratios - wide for desktop and narrow for mobile - so we cannot simply add width and height attributes to the `<video>` element.

<blockquote class="success"><p><em>aspect-ratio</em> to the rescue!</p></blockquote>

`aspect-ratio` in CSS has only recently gained [widespread browser support](https://caniuse.com/mdn-css_properties_aspect-ratio), but it is now safe to use across all evergreen browsers and Safari! This makes it simple to resolve the layout shift with just a few lines of CSS:

```css
/* ==== style.css ==== */

/* default to mobile */
.full-width-video {
  width: 100%;
  background-color: black;
  aspect-ratio: 1280 / 960; /* width and height of the mobile video */
}
/* override for desktop */
@media only screen and (min-width: 480px) {
  .full-width-video {
    aspect-ratio: 1280 / 720;
  }
}
```

The result is that the browser knows how much space to allocate for the video, for both aspect-ratios, as soon as it is able to render the first frame. No more layout shift!

![Filmstrip comparison: the aspect-ratio version reserves the black video box at 26% from the first frame, unlike the original's shifting layout](https://simonhearne.com/images/fast-responsive-videos/aspect-ratio.png)
*A few lines of CSS removes layout shift entirely (view filmstrip)*

We haven't yet made the video load faster, though...

## Optimisation 2: Better source selection

We're currently over-downloading on mobile by requesting the desktop video, then switching it out when our application JavaScript runs. Note that the desktop video continues to download until after `app1.js` has completely loaded at 3.6s.

![WebPageTest waterfall showing both the desktop.mp4 and mobile.mp4 video requests downloading in parallel](https://simonhearne.com/images/fast-responsive-videos/over-downloading.png)
*Late-running JavaScript means that both Desktop and Mobile videos download*

Ideally, we would use native browser support to select the desktop or mobile video to avoid using JavaScript and remove the redundant request on mobile. Unfortunately this is not as easy as it should be. Intuitively we would use the familiar media query to determine which video version to load:

```html
<video>
  <source src="desktop.mp4" type="video/mp4" media="all and (min-width:480px)">
  <source src="mobile.mp4"  type="video/mp4" media="all and (max-width:479px)">
</video>
```

Whilst this was once possible, it has now been [removed from the spec](http://blog.greggant.com/posts/2019/08/02/responsive-videos-javascript-solution-video-source-tag.html) and no browsers support it (except WebKit). So for now, we must use JavaScript. But we **can** still improve on the initial implementation!

The initial implementation had application code to set the video source that ran late in the page, well after the first paint. Moving the logic to be adjacent to the video will ensure that the code runs immediately after the video element is parsed by the browser, bringing the video render earlier.

First, we should remove the potentially wasteful desktop `src` value, then move the source selection logic out of app.js and into the HTML. I'm not generally a fan of inline scripts, but this is exactly the type of scenario where it makes sense. We can wrap this logic in a function and call it on resize to ensure that the correct video is always rendered. You will also want to add a `<noscript>` container with a fallback video at this point.

<blockquote class="info"><p>November 4, 2021: Updated based on <a href="https://twitter.com/anthony_ricaud/status/1456327040370716672">Anthony Ricaud's feedback</a>, thanks Anthony!</p></blockquote>

```html
<!-- ==== homepage.html ==== -->

<video
  class="full-width-video" id="hero-video"
  playsinline muted loop autoplay
  data-src-mobile="mobile.mp4"
  data-src-desktop="desktop.mp4">
    <div class="video-error">Uhoh, no video for you.</div>
</video>
<script>
    const WIDE_VIEWPORT = 480;

    function resizeVideo(wideViewport) {
      let videoEl = document.getElementById("hero-video");
      let src = videoEl.dataset.mobileSrc;
      if (wideViewport.matches) {
        src = videoEl.dataset.desktopSrc;
      }
      if (videoEl.src !== src) {
        videoEl.src = src
      }
    }
    const wideViewport = window.matchMedia(`(min-width: ${WIDE_VIEWPORT}px)`)
    wideViewport.addListener(resizeVideo)
    resizeVideo(wideViewport)
</script>
```

The result is a slightly faster render, both of the page and the first frame of the video.

![Filmstrip comparison showing the inline-JS version rendering the video box at 3.0s, earlier than the aspect-ratio-only version](https://simonhearne.com/images/fast-responsive-videos/inline-js.png)
*A few lines of inline JS improves render time slightly (view filmstrip)*

Removing the redundant download of the desktop video also saves some transmitted bytes, 350kB in this test.

![Bar chart: video bytes drop from 869,498 (aspect-ratio CSS) to 514,981 (inline JS), a 350kB reduction](https://simonhearne.com/images/fast-responsive-videos/difference.png)
*350kB of unnecessary video bytes removed with the inline-JS change*

We are seeing some improvement, but 5.5s is still slow!

## Optimisation 3: Poster Images

The HTML5 `<video>` element supports the `poster` attribute, this lets us define an image to use as a placeholder until the browser has downloaded enough of the video to start playing it. This is a more elegant solution than attempting to preload the entire video file, preserving early bandwidth for more important assets such as CSS and JavaScript.

Placeholder images can be heavily optimised as they are only shown for a short period of time. Use a gaussian blur and a high image compression level to get the image size to around 10kB for each video.

We'll add our optimised placeholder images using the same dataset logic that we have used for the video files:

```html
<!-- ==== homepage.html ==== -->

<video
  class="full-width-video" id="hero-video"
  playsinline muted loop autoplay
  data-src-mobile="mobile.mp4"
  data-src-desktop="desktop.mp4"
  data-poster-mobile="mobile.jpg"
  data-poster-desktop="desktop.jpg">
    <div class="video-error">Uhoh, no video for you.</div>
</video>
<script>
    const WIDE_VIEWPORT = 480;

    function resizeVideo(wideViewport) {
      let videoEl = document.getElementById("hero-video");
      let src = videoEl.dataset.mobileSrc;
      let poster = videoEl.dataset.mobilePoster;
      let widthDisplay = 'MOBILE';
      if (wideViewport.matches) {
        src = videoEl.dataset.desktopSrc;
        poster = videoEl.dataset.desktopPoster;
        widthDisplay = 'DESKTOP';
      }
      if (videoEl.src !== src) {
        videoEl.src = src;
        videoEl.poster = poster;
      }
    }
    const wideViewport = window.matchMedia(`(min-width: ${WIDE_VIEWPORT}px)`);
    wideViewport.addListener(resizeVideo);
    resizeVideo(wideViewport);
</script>
```

The result of this is the biggest improvement we've seen so far - almost two whole seconds faster! This is especially important for low bandwidth users who may not see the video play for another ten seconds after the poster image is shown.

![Filmstrip comparison: the poster-image version shows the blurred placeholder photo at 3.0s, almost two seconds before the inline-JS version](https://simonhearne.com/images/fast-responsive-videos/posters.png)
*Valid poster images dramatically improve perceived performance (view filmstrip)*

## Optimisation 4: Preload the Poster Image

If you look carefully at the waterfall chart of the last test, you'll notice that the poster image is requested quite late - after the blocking script has downloaded and executed.

![WebPageTest waterfall showing the mobile poster jpg not requested until after the blocking app1.js script finishes](https://simonhearne.com/images/fast-responsive-videos/late-poster.jpg)
*Poster images delayed*

We can help the browser here by using preload hints, directives which instruct the browser to download files that it hasn't yet discovered:

```html
<!-- ==== homepage.html ==== -->
  <!-- preloads should be at the bottom of the head -->
  <link rel="preload" href="desktop_poster.jpg" as="image" media="all and (min-width:480px)">
  <link rel="preload" href="mobile__poster.jpg"  as="image" media="all and (max-width:479px)">
</head>
```

These allow the browser to request the poster images early, and do support media queries so that we can fetch the correct image! The result is a slightly faster render of the poster image:

![Filmstrip comparison: preloading the poster shows the image at 3.0s versus 3.7s without the preload hint](https://simonhearne.com/images/fast-responsive-videos/preloads.png)
*Preload poster images for the final performance gains!*

Another alternative that I have seen used is to Base64 encode the poster image and insert it directly into the attribute. I am not a fan of this method: it bloats the HTML document and causes the browser to download unnecessary content (at least one of the poster images is not required). I would not recommend this unless the encoded images are under 1kB, for example a simple play icon on a plain background.

It may be possible to further improve performance, specifically by deferring the application JavaScript bundle and further optimising the video files. For now, though, we should be happy with a 50% improvement in render performance!

## Optimisation 5: Don't request video on low-bandwidth connections

Another consideration is for users who are on very poor connections, they probably would prefer to not have the video load and instead just the poster image.

We can extend our example to use `navigator.connection.downlink` to simply toggle the video src based on the connection speed.

```js
  const MIN_DOWNLINK = 1; // Slow 3G ~~ 0.4, Fast 3G ~~ 1.4
  /* ... */
  let downlink = MIN_DOWNLINK;
  try {
    downlink = navigator.connection.downlink;
  } catch (e) {
    console.log(`Unable to determine downlink`)
  }
  if (videoEl.src !== src) {
    // only override values if they differ
    if (downlink >= MIN_DOWNLINK) {
      videoEl.src = src;
      widthDisplay += " - FAST";
      console.log(`Detected bandwidth (${downlink}Mbps) greater than threshold (${MIN_DOWNLINK}Mbps) - showing video`);
    } else {
      widthDisplay += " - SLOW";
      console.log(`Detected bandwidth (${downlink}Mbps) less than threshold (${MIN_DOWNLINK}Mbps) - not showing video`);
    }
    videoEl.poster = poster;
  }
```

You could also add a play button in this context, so users could opt-in to the video if they would like to see it.

## Conclusion

We have applied four simple optimisations to a landing page hero video, removing a layout shift and improving the render time for the video from six seconds to three on a 3G connection. Videos don't have to slow down your site, as long as we pay attention to the details.

![Filmstrip comparing all five optimisation stages side by side, with the final preload-posters version rendering first](https://simonhearne.com/images/fast-responsive-videos/all-tests.jpg)
*Implementing all of these recommendations makes for a much faster user experience (view filmstrip)*

There are five key steps to ensuring your responsive hero videos render quickly:

1. Set the video `src` as soon as possible
2. Use CSS to allocate the correct space for the video
3. Use the `poster` attribute for a fast rendering first frame
4. Use responsive preloads to optimise poster image delivery
5. Optimize your video assets

We didn't cover it in this post, but it is critical that your video assets are optimised! Some simple tricks are to remove the audio track from muted videos, scale the video to the size it will be rendered and use the optimal format for the vistor's browser. Most of these are relatively simple to do, but for responsive videos it can be difficult to determine the optimal widths and formats. Here I would recommend using a video hosting & optimisation service which can automate these tasks:

- [Akamai Image and Video Manager](https://www.akamai.com/products/image-and-video-manager)
- [Cloudinary Media Optimizer](https://cloudinary.com/products/media_optimizer)
- [Cloudflare Stream](https://www.cloudflare.com/products/cloudflare-stream/)

Each of these (and a number of other services) can automatically serve the best format and width for each visitor, dramatically reducing file size and improving render performance.

To get started with your hero video, you can view the full working demo below or [directly on CodePen](https://codepen.io/simonhearne/pen/abyVGRj?editors=1100). This code should be enhanced with error handling and support for browsers with JavaScript disabled: use a `<noscript>` element to contain a fallback video behaviour - perhaps just showing the poster image.

<p class="codepen" data-height="500" data-default-tab="html,result" data-slug-hash="rNdNypB" data-user="simonhearne" style="height: 500px; box-sizing: border-box; display: flex; align-items: center; justify-content: center; border: 2px solid; margin: 1em 0; padding: 1em;">
  <span>See the Pen <a href="https://codepen.io/simonhearne/pen/rNdNypB">
  fast-responsive-video</a> by Simon Hearne (<a href="https://codepen.io/simonhearne">@simonhearne</a>)
  on <a href="https://codepen.io">CodePen</a>.</span>
</p>
<script defer src="https://cpwebassets.codepen.io/assets/embed/ei.js"></script>


==============================================================================

# An Inclusive Web is Fast by Default

> Web performance is critical to inclusive design - users on low-end devices or with poor connectivity are excluded from large and slow applications

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2021-09-23
Canonical: https://simonhearne.com/2021/inclusive-design/
Tags: WebPerf, video, talk
Video: https://www.youtube.com/watch?v=MiXy2x6flww
Slides: https://simonhearne.com/presentations/inclusive-design/#/

---
Performance is not an average, it is a distribution — and the people at the far end of it are being quietly excluded.

## About the talk

When a home connection fails and drops you to 20–50kbps, the web stops being slow and starts being unusable. That is the everyday condition for a large share of the world: around 65% of people in developing nations have no access to fast internet, against roughly 20% in developed ones.

This talk argues that inclusion and performance are the same project, and that the analytics telling you everything is fine are often measuring only the users who managed to stay.

## What's covered

- **Survivorship bias in your own data.** The users who gave up never appear in the numbers — and with something like 88% of Firefox users blocking Google Analytics, the sample is narrower than it looks.
- **YouTube's Project Feather.** A player cut from 1,200kB to under 100kB across 14 requests measured *slower* in the field, because the people who opted in were concentrated in South East Asia, Africa and South America. The deck still counts it a success, crediting it with opening YouTube to millions of new users.
- **Performance as a distribution.** Why the median hides the problem, and why the tail is where exclusion actually lives.
- **Building empathy deliberately.** Chrome throttling, Facebook's "2G Tuesdays", a roughly $300 test handset, and browsing with JavaScript switched off.
- **What to do about it.** Progressive enhancement, lazy-loaded images in WEBP or AVIF, less client-side JavaScript, inline critical CSS, properly sized responsive images, and a performance budget that includes third parties.
- **Tools.** WebPageTest, What Does My Site Cost?, Treo, PageSpeed Insights and YellowLab Tools.

The deck closes on William Gibson: "The future is already here, it's just not evenly distributed."


==============================================================================

# Slow site speed is still the biggest cause of web stress

> It's something we've all felt before: the stress and anger induced when trying to use a frustratingly slow website or app. It's really hard to quantify this feeling though, and even harder to track it in your users.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2021-02-26
Canonical: https://simonhearne.com/2021/web-stress/
Tags: WebPerf, UX

---
Back in 2010 Foviance ran an EEG study <a class="citation" id="citation-1" href="#source-1">[1]</a> which showed that participants had to concentrate 50% harder when websites were slow, as measured by alpha waves.

In 2011 Harris International conducted a study for Tealeaf <a class="citation" id="citation-2" href="#source-2">[2]</a> which found that American respondents got more frustrated by failed mobile transactions (58%) than being stuck in traffic (56%). 35% of respondents had cursed at their phone, screamed at it or even thrown it in anger.

In 2014 Radware conducted an EEG study <a class="citation" id="citation-3" href="#source-3">[3]</a> which showed that a 500ms delay resulted in up to 26% increase in stress levels, as measured by alpha waves.

In 2018 Philip Tellis showed that pages which were not interactive soon after they rendered (30% after they were visually ready) triggered the greatest number of 'rage clicks' from users <a class="citation" id="citation-4" href="#source-4">[4]</a>. Rage clicks indicate that user is frustrated that a page is not responding quickly, causing them to click or tap on an element repeatedly in quick succession.

More recently in 2020 [Cyber-Duck](https://www.digitalinformationworld.com/2020/12/researchers-conduct-study-on-how-bad.html) tracked users' blood pressure whilst navigating sites <a class="citation" id="citation-5" href="#source-5">[5]</a>. The results showed that slow pages caused the greatest increase in systolic blood pressure (an average 21% increase) when compared with other irritations such as popups and auto-playing music.

It's clear that slow pages increase stress, and we know that stress is not the response we are hoping to induce in our visitors! Delivering fast experiences will reduce visitor stress and might even create user delight.

You can track proxies of stress on your pages through metrics like rage clicks, but ultimately the outcome of slow pages will be lower traffic, shorter sessions and poorer business results. [Core Web Vitals](/2020/core-web-vitals/) are the best proxies we have for user experience on the web, so optimising for these should reduce visitor stress and lead to improved business metrics.

## Sources

<ol class="sources-list">
  <li id="source-1">
    <a target="_blank" rel="noopener" href="/files/final_webstress_survey_report_229296.pdf">[PDF] Web Stress. Foviance. 2010.</a> <a class="citation-link" href="#citation-1">(back to text)</a>
  </li>
  <li id="source-2">
    <a target="_blank" rel="noopener" href="http://cdn2.hubspot.net/hub/190519/file-1073485563-pdf/docs/10_-_whitepaper_-_improving_the_customer_experience_for_mobile_consumers.pdf">[PDF] A report on the Mobile Customer Experience. IBM. 2011.</a> <a class="citation-link" href="#citation-2">(back to text)</a>
  </li>
  <li id="source-3">
    <a target="_blank" rel="noopener" href="https://blog.radware.com/applicationdelivery/wpo/2014/06/mobile-web-performance-stress/">Mobile Web Stress: Understanding the Neurological Impact of Poor Performance - Tammy Everts. 2014.</a> <a class="citation-link" href="#citation-3">(back to text)</a>
  </li>
  <li id="source-4">
    <a target="_blank" rel="noopener" href="https://speakerdeck.com/bluesmoon/ux-and-performance-metrics-that-matter-a062d37f-e6c7-4b8a-8399-472ec76bb75e?slide=16">UX & Performance: Metrics that Matter (slide 16). Philip Tellis. 2018.</a> <a class="citation-link" href="#citation-4">(back to text)</a>
  </li>
  <li id="source-5">
    <a target="_blank" rel="noopener" href="https://www.netimperative.com/2020/12/09/blood-pressure-study-which-website-issue-cause-users-the-most-stress/">Blood pressure study: Which website issue cause users the most stress? Net Imperative. 2020.</a> <a class="citation-link" href="#citation-5">(back to text)</a>
  </li>
</ol>


==============================================================================

# Everything we know about Core Web Vitals and SEO

> There has been a lot of uncertainty and confusion since Google announced the Page Experience Update in 2020. In this post we will summarise everything we know about the update and the potential SEO impact

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2021-02-22
Updated: 2021-06-15
Canonical: https://simonhearne.com/2021/core-web-vitals-seo/
Tags: WebPerf, SEO

---
<blockquote class="info"><p><em>June 15th 2021:</em> Multiple updates from <a href="https://events.google.com/io/program/content?lng=en">Google I/O 2021</a>, a <a href="https://www.youtube.com/watch?v=HWm6WNkHs90">Web Vitals Q&amp;A session</a> and another <a href="https://developers.google.com/search/blog/2021/04/more-details-page-experience?hl=en">Google blog post</a>.</p></blockquote>

<blockquote class="info"><p><em>April 19th 2021:</em> Google have <a target="_blank" rel="noopener" href="https://developers.google.com/search/blog/2021/04/more-details-page-experience">announced a delay</a> to the rollout of the Page Experience update. The SEO changes will now gradually roll out from mid-June 2021.</p></blockquote>

<blockquote class="info"><p><em>April 13th 2021:</em> Google have <a target="_blank" rel="noopener" href="https://web.dev/evolving-cls/">updated the CLS metric</a> to better account for complex web applications. This should generally result in a slight improvement in reported CLS values.</p></blockquote>

<blockquote class="info"><p><em>March 30th 2021:</em> Google have updated their <a target="_blank" rel="noopener" href="https://support.google.com/webmasters/thread/104436075">FAQs page</a> to confirm and clarify many of the details in this post!</p></blockquote>

## In summary

The [update](https://developers.google.com/search/blog/2020/11/timing-for-page-experience#:~:text=Today%20we're%20announcing%20that,roll%20out%20in%20May%202021.&text=The%20change%20for%20non%2DAMP,roll%20out%20in%20May%202021.) will roll out between July and August 2021, it will result in a positive ranking signal boost for page experience in **mobile search only** <a class="citation" id="citation-1" href="#source-1">[1]</a>, for pages or **groups of similar URLs** <a class="citation" id="citation-2" href="#source-2">[2]</a>, which perform well against the core web vitals targets. The impact of the signal is expected to be small <a class="citation" id="citation-4" href="#source-4">[4]</a>. Google has stated that the Page Experience update will roll out to desktop search at some point in the future.

![Diagram of search ranking boost dropping from good to poor across LCP, FID and CLS, applied per URL, page group or origin](https://simonhearne.com/images/core-web-vitals-seo/cwv-summary.jpg)
*@fabkru's summary of a Google Q&amp;A session on web vitals.*

"Meeting the targets" means that the aggregate 75th percentile value, for a single URL (or across a 'similar' set of URLs when there is not enough data for an individual URL), meets the thresholds for 'good' shown below (2.5s for <abbr title="Largest Contentful Paint">LCP</abbr>, 100ms for <abbr title="First Input Delay">FID</abbr> and 0.1 for <abbr title="Cumulative Layout Shift">CLS</abbr>). The ranking boost is calculated separately for each metric: with green giving the maximum boost, red giving no boost and amber giving some boost <a class="citation" id="citation-3" href="#source-3">[3]</a>. The data is aggregated from Google's private view of [CrUX](https://developers.google.com/web/tools/chrome-user-experience-report) data (see [Search console is the source of truth](#search-console-is-the-source-of-truth) for details of this assumption).

<div class="resp-cols-3">
  <img src="/images/core-web-vitals/lcp_ux.svg" alt="Largest Contentful Paint icon" width="171" height="150" loading="lazy">
  <img src="/images/core-web-vitals/fid_ux.svg" alt="First Input Delay icon" width="171" height="150" loading="lazy">
  <img src="/images/core-web-vitals/cls_ux.svg" alt="Cumulative Layout Shift icon" width="171" height="150" loading="lazy">
</div>

## URLs can be grouped by similarity

As can be seen in Search Console, URLs are grouped by Google into similar sets of pages. The aggregate Core Web Vital scores shown here will likely be the ones which impact the Page Experience ranking factor, rather than individual URL performance. You can click on the 'Similar URLs' column in the view below to see up to 20 of the URLs Google has included in the group.

![Search Console CLS Issue Report listing page groups like PDP Style 1 and PLP with their aggregate CLS scores](https://simonhearne.com/images/core-web-vitals-seo/cls-issues.jpg)
*Google Search Console CLS Issues report shows URL grouping*

<blockquote class="info"><p><em>June 15th 2021:</em> I now believe that URL groups are only used when there is not enough field data for a single URL. Run your page through <a href="https://developers.google.com/speed/pagespeed/insights/">Pagespeed Insights</a> to see if there is enough data for your URL(s).</p></blockquote>

## Field is more important than Lab

Field data - data collected from real users browsing pages - is the only source which will impact Page Experience <a class="citation" id="citation-5" href="#source-5">[5]</a>. Lab data - test data collected from emulated browsers - may not reflect the real user experience and has no bearing on SEO.

![PageSpeed Insights origin summary showing lab LCP of 4.4s versus field LCP of 4.2s, both failing Core Web Vitals](https://simonhearne.com/images/core-web-vitals-seo/origin-lab.jpg)
*Field and Lab data often differ significantly*

Any difference between lab and field results could be down to many factors relating to your traffic:

- Geographic user distribution and varying network quality (LCP)
- Varying layouts on different viewports (CLS)
- Varying mobile device specifications (FID)

Lab data is still useful when diagnosing an issue and confirming an improvement, but we must rely on field data for feedback on the actual user experience delivered. Note that the Lighthouse Score is calculated from lab data, not field! <a class="citation" id="citation-6" href="#source-6">[6]</a>

## Search Console is the source of truth

Whilst Core Web Vitals values are available in a multitude of tools...

- [Pagespeed Insights](https://developers.google.com/speed/pagespeed/insights/) (lab and field)
- [WebPageTest](https://webpagetest.org) (lab)
- Real User Measurement tools like [mPulse](https://www.akamai.com/uk/en/products/performance/mpulse-real-user-monitoring.jsp) (field)
- [Chrome UX Report](https://web.dev/chrome-ux-report-bigquery/) (field)

...none of these directly reflect the values that will be used by search.

Pagespeed Insights' field data for a page is the closest you can get to the data used in search, outside of Search Console. Pagespeed Insights has the only public source of data at the page level **and** using the same 28-day rolling window as search - but the values shown are for the individual URL under test, not the aggregate of related pages as indicated in Search Console.

![Search Console Core Web Vitals Mobile report with a bar chart trending the count of Poor URLs over time](https://simonhearne.com/images/core-web-vitals-seo/search-console-2.jpg)
*Google Search Console shows page groups which fail Core Web Vitals audits. Data is sourced from CrUX.*

## What we need to act upon now

The Page Experience Update is more of a carrot approach than stick - there is no direct penalty for failing to meet Google's goals. That said, meeting the target values for Core Web Vitals will inevitably result in an improved user experience. Use the tools available to determine which pages on your sites are falling behind, then get to work on improving them! I've written a post on [diagnosing and improving Core Web Vitals](/2020/core-web-vitals/) which may help.

Remember that any changes you make to your page performance will take 28 days to fully reflect in Google's data - so don't expect immediate returns there.

## Do we like Core Web Vitals?

I believe that Core Web Vitals are on balance a Good Thing™️:

- The vitals encourage us to focus on UX metrics, not just technical metrics
- The update builds a bridge between SEO, Tech SEO, Performance Marketing and Engineering
- Using field data increases focus on real user measurement, versus the more simple (and less representative) lab testing
- The targets start a good discussion around percentiles for field data - 75th is a good balance between the median (hard to impact) and the 90th / 95th (potentially subject to volatility)

There are a number of issues with Core Web Vitals though:

- The metrics are only measurable in Chromium-based browsers
- Ranking benefit is only determined by performance of visitors on Chrome browsers
- First Input Delay depends heavily on when (and if!) users interact with a page, making it less actionable than Total Blocking Time or maximum potential FID <a class="citation" id="citation-7" href="#source-7">[7]</a>.

It's worth noting at this point that the targets for Core Web Vitals set out by Google are quite strict. Many Google pages do not achieve 'good' in all three, and very few production websites do. That said, it is possible to achieve green across-the-board without significant engineering work. In my opinion this makes them good targets: difficult but achievable.

## Other SEO factors

AMP will no longer be a requirement for the Top Stories carousel in mobile search after the update, meaning all pages which meet the Google News content policies will be eligible for inclusion, with Core Web Vitals impacting ranking in a similar way to normal search. <a class="citation" id="citation-8" href="#source-8">[8]</a>

## What is yet to be confirmed?

There are still a number of assumptions we are making, which are yet to be confirmed by Google:

- URLs are grouped for the purposes of the ranking signal, and origin summary used for new / ungrouped URLs. Can we control this grouping? It sometimes does not follow seemingly logical patterns (e.g. PDP and PLPs grouped together)
- How many experiences are included in the CrUX dataset - e.g. how many users opt-in to share usage statistics and what (if any) sampling is applied
- Is geography a factor, e.g. does the TTFB measured in a country determine page experience ranking in that country? (June 15th 2021: I believe geography is not a factor. Core Web Vitals are measured by country, but the ranking boost is global).
- CLS unfairly penalises single-page applications, there is a [proposal](https://web.dev/better-layout-shift-metric/) to improve the metric but we do not know if this will be released before May. (June 15th 2021: Google has [confirmed](https://web.dev/evolving-cls/) that they are using the new CLS metric - calculated by windows of up to five seconds throughout a page lifecycle. This is better for SPAs than before, but does not fix the fundamental issue: soft navigations / route changes are not accounted for correctly.)

I hope that these points are confirmed before the May 31st deadline, if nothing else but for clarity and transparency. It's hard to hit a target that you cannot see!

## Am I missing something?

Probably! I have collected this information from videos, blog posts and tweets; trying to find a legitimate source of truth is disappointingly difficult. If you know of something different to the statements above (with a source) please let me know by email or twitter - I'm keen to update this page as we learn more.

## Sources

<ol class="sources-list">
  <li id="source-1">
    "At this time, using page experience as a signal for ranking will apply only to mobile Search" - <a target="_blank" rel="noopener" href="https://support.google.com/webmasters/thread/86521401?hl=en#gbwa:~:text=Q%3A%20Why%20are%20there%20differences%20in%20scores%20between%20mobile%20and%20desktop%3F">Google Search Console Help. 12 March 2020.</a> <a href="#citation-1">(back to text)</a>
  </li>
  <li id="source-2">
    <a target="_blank" rel="noopener" href="https://www.youtube.com/watch?t=848&v=JV7egfF29pI&feature=youtu.be">John Mueller in Google SEO office-hours on URL grouping</a>: <a href="#citation-2">(back to text)</a>
    <blockquote>"... for example, to split out the forum from our site where we can tell, oh, slash forum is everything forum and it's kind of slow, and everything else that's not in slash forum is really fast. If we can recognize that fairly easily, that's a lot easier. Then we can really say, <em>everything here in slash forum is kind of slow</em>, everything here is kind of OK. On the other hand, if we have to do this on a per URL basis where ... we can't tell based on the URL if this is a part of the forum or part of the rest of your site, then we can't really group that into parts of your website. And then we'll be forced to take an aggregate score across your whole site and apply that appropriately. I suspect we'll have a little bit more information on this as we get closer to announcing or closer to the date when we start using Core Web Vitals in search, but it is something you can look at already a little bit in Search Console. There is a Core Web Vitals report there, and if you drill down to Individual Issues, you'll also see this your URL affects so many similar URLs. And based on that, you can already kind of tell, oh, is Google able to figure out that my forum is grouped together or is it not able to figure out that these belong together?"</blockquote>
  </li>
  <li id="source-3">
    <a target="_blank" rel="noopener" href="https://twitter.com/JohnMu/status/1395798952570724352">@JohnMu on Twitter.</a> <a href="#citation-3">(back to text)</a>
  </li>
  <li id="source-4">
    "Ranking wise it's a teeny tiny factor, very similar to https ranking boost." (not directly related to Page Experience, but can we infer similarity) - <a target="_blank" rel="noopener" href="https://twitter.com/methode/status/1255224116648476675">Gary Illyes (Google's Webmaster Trends Analyst). 28 April 2020.</a> <a href="#citation-4">(back to text)</a>
  </li>
  <li id="source-5">
    "Core Web Vitals are a set of <em>real-world</em>, user-centered metrics" - <a target="_blank" rel="noopener" href="https://developers.google.com/search/blog/2020/05/evaluating-page-experience#about-page-experience:~:text=Core%20Web%20Vitals%20are%20a%20set%20of%20real%2Dworld%2C%20user%2Dcentered%20metrics">Google Search Central. 28 May 2020.</a> <a href="#citation-5">(back to text)</a>
  </li>
  <li id="source-6">
    "The Performance score is a weighted average of the <em>metric</em> scores." - <a target="_blank" rel="noopener" href="https://web.dev/performance-scoring/">Lighthouse performance scoring. 12 June 2020.</a> <a href="#citation-6">(back to text)</a>
  </li>
  <li id="source-7">
    "Since FID measures when actual users first interact with your page, it's more inherently variable than typical performance metrics. See Analyzing and reporting on FID data for guidance about how to evaluate the FID data you collect." - <a target="_blank" rel="noopener" href="https://web.dev/lighthouse-max-potential-fid/">Max Potential First Input Delay. 16 October 2019.</a> <a href="#citation-7">(back to text)</a>
  </li>
  <li id="source-8">
    "As part of this update, we'll ... remove the AMP requirement from Top Stories eligibility" - <a target="_blank" rel="noopener" href="https://developers.google.com/search/blog/2020/05/evaluating-page-experience#devsite-thumb-label-header:~:text=incorporate%20the%20page%20experience%20metrics%20into,AMP%20requirement%20from%20Top%20Stories%20eligibility">Evaluating page experience for a better web. 28 May 2020.</a> <a href="#citation-8">(back to text)</a>
  </li>
</ol>


==============================================================================

# How to avoid layout shifts caused by web fonts

> One of the biggest causes of layout shifts for my clients is late-loading web fonts, let's look at how to optimise them!

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2021-01-19
Updated: 2021-01-21
Canonical: https://simonhearne.com/2021/layout-shifts-webfonts/
Tags: WebPerf

---
One of the outcomes of the release of [Core Web Vitals](/2020/core-web-vitals/) (and subsequent [inclusion](https://developers.google.com/search/blog/2020/11/timing-for-page-experience) in Google's page ranking algorithm) is that we have been paying more attention to unexpected layout shifts and the cumulative layout shift (CLS) score.

Some sources of layout shifts have been simple to resolve: pre-allocate the correct space for dynamic elements, use width and height attributes on images and prioritise visible elements in your HTML document. One common cause of layout shift is surprisingly difficult to resolve though: flashes of unstyled text (FOUT).

In this post we will explore the surprisingly complex world of text rendering on the web and some techniques to remove FOUT while not incurring a <abbr title="Cumulative Layout Shift">CLS</abbr> penalty.

In summary we need to prevent the layout shift by letting the browser render in a fallback system font if it doesn't get the web font in time, then optimise our fonts to try to get them to the browser before it needs them:

- use `font-display: optional` to prevent layout shifts
- subset fonts and serve as `woff2`
- use variable fonts or a limited set of weight variations
- preload critical fonts
- host your own fonts on your main domain

## Why fonts cause layout shifts

Unexpected layout shifts (page content moving around without user interaction) are bad for user experience. Fonts cause layout shifts when the size of the containing element (e.g. a `<div>` or paragraph) changes when the web font is downloaded. This occurs when the height of the font or the length of the paragraph is different with the web font compared to the system font. When laying out a page, browsers will use the dimensions and properties of the fallback font to determine the size of the containing elements, even if you have declared a web font to block the system font with `font-display: block`!

<style>
  #demo-text-1 {
    overflow: hidden;
    width: 80%;
    max-width: 500px;
    font-size: smaller;
    border: 1px solid grey;
  }
  .no-font {
    font-family: serif !important;
    border-color: red !important;
  }
  .fout-fig {
    height: 220px;
    position:relative;
  }
  @media only screen and (max-width: 600px) {
    .fout-fig {
      height: 350px;
    }
  }
</style>
<figure class="fout-fig">
  <p id="demo-text-1">Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin aliquam metus eu lacus placerat feugiat. Suspendisse quis elit fringilla, luctus felis ut, ultricies leo. Morbi vel dui non risus aliquam rhoncus in ac justo. Mauris cursus diam id ipsum aliquet tempus. Praesent feugiat consequat risus, ut eleifend neque consequat quis. Maecenas rhoncus faucibus dui quis pretium. Nam ultrices rhoncus dui, at pretium erat rhoncus et. Vestibulum tempus diam vel ex venenatis placerat. Nam sed elementum odio. Suspendisse vel orci turpis. Fusce in maximus ante, non malesuada justo. Duis in tellus erat.</p>
  <figcaption style="position: absolute; width:100%; bottom:0;">Flash of Unstyled Text</figcaption>
</figure>
<script>
    window.setInterval(() => {
      document.getElementById('demo-text-1').classList.toggle("no-font");
    },2000);
</script>

### Detecting layout shifts

There are a number of methods to detect layout shifts due to web fonts, the most simple is to run your page through [WebPageTest](https://webpagetest.org/) and use the filmstrip view. There is a toggle to show layout shifts, combine that with 'huge' screenshot images and a 0.1s interval to get a clear view of what is happening as your page renders (you can simply add this to the end of the URL: `&highlightCLS=1&thumbSize=600&ival=100&end=visual&text=000&bg=fff"`). The Washington Post has a couple of layout shifts due to web fonts, in the example below the headline text is one line shorter with the web font than without which results in a layout shift. Layout shifts are dependent on viewport size so make sure you test a few different devices.

![](https://simonhearne.com/images/layout-shifts-webfonts/wapo-headline.png)
*Headline web font loading causes a layout shift. View test on WPT*

You can also find layout shifts in the Chrome Developer Tools Performance tab, look for layout shifts attributed to text elements.

![](https://simonhearne.com/images/layout-shifts-webfonts/wapo-shift.jpg)
*Chrome developer tools shows attribution of layout shifts*

I've covered general layout shifts in more detail in a post on [How to Improve Core Web Vitals](/2020/core-web-vitals/).

## Prevent layout shifts with font-display

The dirty little secret of this blog post is that you can resolve layout shifts due to fonts with a single line of CSS:

`font-display: optional`

This directive lives in your font-face declaration and tells the browser to use a fallback system font if the web font is not available at the time of rendering text (plus 100ms). This means that on uncached page loads there is a chance that the fallback font will be used, but all subsequent page loads should be rendered with the web font, as it will be downloaded and available in cache. There are other options, shown in the image below:

- **block** - hide text for up to three seconds while waiting for the web font, and always swap in the web font when it loads
- **swap** - show text as soon as possible, and always swap in the web font when it loads
- **fallback** - hide text for up to 100ms, then only swap in the web font if it loads within three seconds
- **optional** - hide text for up to 100ms, then only use the web font if it is available - never swapping

![](https://simonhearne.com/images/layout-shifts-webfonts/font-display.png)
*Optional is the only font-display value which guarantees no layout shift. Image by @notwalforf. Read more on MDN*

If it's that simple, why this whole blog post? Unfortunately system fonts are not necessarily the nicest designs, and they are not consistent between operating systems. Most designers will cringe at the thought of showing users a fallback system font. The rest of this blog post details various optimisations to get the font files to the browser quicker, allowing the use of any font-display option but with minimal risk of showing a system font, or for options other than `optional`: without triggering a layout shift. We will also look at font-face descriptors / f-mods and progressive font enrichment for potential further optimisations in the future.

## Load fewer font files

Downloading one or two font files to render text won't have a massive impact on speed, downloading five or ten font files will! Ensuring that you deliver the minimium viable number of font files is the best way to ensure that the browser has them available at layout, thus reducing the likelihood of <abbr title="Flash of Unstyled Text">FOUT</abbr> layout shifts. Let's look at some tricks to load fewer font files while maintaining your design.

### Use faux bold and italic

> Designers hate this one weird trick!

Font stacks generally include a bunch of different files, from extra-light italic through to extra-bold. Combining nine font weights with normal and italic variants produces 18 separate font files! Fonts that are not used on the page will not be downloaded, but for the odd occasion where only a single word is both bold and italic (for example) the whole font file will be downloaded to render the single word.

Browsers can make bold and italic versions of fonts themselves, this is called faux bold and faux italic. This could mean that a single regular weight font file is all you need! This should be tested and approved by any designers involved, I recommend a blind [Coke vs. Pepsi](https://juiceboxinteractive.com/blog/how-pepsi-won-the-battle-but-lost-the-challenge/) style test to prevent any bias impacting the outcome.

![image showing faux bold of roboto font compared to real bold, there is little difference](https://simonhearne.com/images/layout-shifts-webfonts/faux-bold.png)
*Example of differences between faux and real bold. source*

There may be subtle differences between the browser's version of a bold font and the font creator's version, as shown above. For some fonts the differences will be too great to consider, especially fancy ones! You can check yours by simply removing the @font-face declarations for all but the regular font version and comparing screenshots of rendered text.

### Use a variable font

All modern browsers support variable fonts (notable exceptions are [IE11 and Opera Mini](https://caniuse.com/variable-fonts)), this allows a single font file to include multiple variable axes like font weight and slant. This results in a larger file than a single weight font, but it replaces multiple weight files and provides more flexibility for the variable axes (e.g. font-weight: <span style="font-weight: 457">457</span>).

Variable fonts can be found in multiple places, for example Google Fonts [has a filter](https://fonts.google.com/?vfonly=true) to only show variable fonts. The majority of variable fonts have a single variable axis for weight, but there are a number of possible dimensions including slant, serif and optical size. [Exo](https://fonts.google.com/specimen/Exo?vfonly=true) for example has both weight and a binary italic dimension. [Axis Praxis](https://www.axis-praxis.org/specimens/__DEFAULT__) is a great place to explore the possibilities of variable fonts.

### Death to icon fonts

Icon font sets like [Font Awesome](https://fontawesome.com/) and Bootsrap's [glyphicons](https://getbootstrap.com/docs/3.3/components/) have popularised the use of web fonts for iconography. Unfortunately this means that your icons will not render until a (typically) large font file has downloaded, and sometimes results in an unsightly <span style="font-family:sans-serif">&#8414;</span> symbol instead of your icons when the font file fails to download in time.

Icon fonts are bad practice for performance and accessibility, [replace them with SVG](https://www.wouterbulten.nl/blog/tech/blog-optimization-replacing-font-awesome-with-svg/) as soon as possible.

N.B. this section is named after the great talk given by [Seren Davies](https://twitter.com/ninjanails): [Death to Iconfonts ☠️](https://www.youtube.com/watch?v=9xXBYcWgCHA)

### Use system fonts

Web fonts are popular because they allow designers to maintain a consistent look and feel across browsers. Where this isn't necessary, system fonts will be the fastest method to render text. If your current web font is close to a system font, you can use [Font Style Matcher](https://meowni.ca/font-style-matcher/) by [Monica](https://twitter.com/notwaldorf) to tweak the font settings until you get a near-perfect match. I've worked with a client who surreptitiously replaced their web font stack with a tweaked system font for two weeks - neither designers nor customers apparently noticed the difference!

Using a system font means that text will render at the earliest possible time. We also now have methods to make the font match the operating system, which may be more attractive than previous fallback options like Arial and Helvetica. To do this you'll need to list the system fonts for all operating systems in a specific order:

```css
body {
  font-family: -apple-system, BlinkMacSystemFont,
    "Segoe UI", "Roboto", "Oxygen", "Ubuntu", "Cantarell",
    "Fira Sans", "Droid Sans", "Helvetica Neue",
    sans-serif;
}
```

You can read more about this approach in a Smashing Magazine [article from 2015](https://www.smashingmagazine.com/2015/11/using-system-ui-fonts-practical-guide/). One drawback to this approach is that it cannot co-exist with the [f-mods](#reduce-layout-shift-with-f-mods) approach we describe below, this is to be used *instead* of web fonts. [Let me know](#conversation) if I'm wrong in this assumption!

## Optimise font files

If we must download a font file, we should make it as small as possible to ensure it downloads quickly. There are two key methods to optimise web fonts: subsetting and formats.

### Subset fonts

Many fonts will have glyphs (glyphs are individual characters like `a` or `&`) from multiple alphabets. If your website is only delivered in the latin alphabet (a - Z) and does not use ligatures (like `é`) then these glyphs represent wasted bytes in your font file. Roboto, the most popular Google font, has 277 glyphs.

![image of all glyphs in the roboto font from google, showing many non-latin characters](https://simonhearne.com/images/layout-shifts-webfonts/roboto-glyphs.png)
*Glyphs in the popular Roboto font. source*

Removing non-latin characters from this font results in a `woff2` file that is one sixth the size!

![screenshot of file system showing subset roboto font is 11KB, original is 66KB](https://simonhearne.com/images/layout-shifts-webfonts/subset-roboto.png)
*Subset font is 5x smaller*

If you use Google fonts, you can [specify which subsets](https://developers.google.com/fonts/docs/getting_started#specifying_script_subsets) you require in the CSS request. If you host your own fonts (which I would always recommend, where possible) you will need to check with the font provider if they have subset versions, or where your license permits you can create subsets yourself.

To create subsets you will ideally start with the `ttf` file and work out what glyphs you need to keep. Filament Group have created [glyphhanger](https://github.com/filamentgroup/glyphhanger) to take out the hard work here, read their [blog post](https://www.filamentgroup.com/lab/glyphhanger/) to see how easy it is to create a unique subset font just for your pages. Note that missing glyphs will be rendered in the fallback system font, so they won't just disappear!

It is possible to create multiple subset variations of your fonts if your website supports multiple languages. In this case, use the `unicode-range` declaration in your @font-face rule to let the browser know which characters are in which font file. Read more about this on [MDN](https://developer.mozilla.org/en-US/docs/Web/CSS/@font-face/unicode-range) or take a look at [a response](https://fonts.googleapis.com/css2?family=Open+Sans&display=swap) from Google Fonts for inspiration.

### Use modern formats

There are a set of different formats available to serve web fonts: `ttf`, `otf`, `eot`, `woff` and `woff2`. `ttf` and `otf` are the 'raw' font files, and probably shouldn't be sent to a browser. `eot` supports subsetting and is compressed using LZ compression, whereas `woff` uses gzip compression and `woff2` uses brotli compression.

Font stacks will often have many or all of these formats available, but you really only need `woff2`. The newer format is around 30% smaller than `woff` and is supported in all modern browsers. Supporting only `woff2` will make it more simple to apply some of the other optimisations recommended, such as subsetting.

![can i use dot com shows all browsers except opera and IE11 support woff2](https://simonhearne.com/images/layout-shifts-webfonts/woff2-support.png)
*woff2 is supported on all the browsers you care about. source*

## Deliver your fonts fast

It may seem obvious, but we must make sure that the browser can download our font files as soon as possible! There are a few techniques we should follow to ensure this is always the case. Hosting fonts on your own domain (and ideally on a content delivery network (CDN)) will result in the best performance.

### Host your own fonts

Font foundries such as Google Fonts are popular: the HTTP Archive shows that [80% of sites](https://almanac.httparchive.org/en/2020/fonts#where-are-web-fonts-being-used) use web fonts, and [7.5%](https://almanac.httparchive.org/en/2020/third-parties#third-party-domains) of all web fonts are served by Google. Using a third-party service can incur a performance penalty, though. The Google Fonts embed process uses either a CSS link or @import, with the CSS file returning a dynamic set of @font-face rules. The rules are distinct for the given font, user agent and any additional query parameters (like subsetting and font-display options).

Using a third-party service means that your fonts will be delayed. The best-case scenario is that you are requesting the font file directly from another hostname (e.g. fonts.gstatic.com) which incurs a connection cost - DNS lookup, TCP connection and TLS negotiation. The worst case is multiple hops, like loading a CSS file from fonts.googleapis.com which references files on fonts.gstatic.com, incurring two connection penalties. This cost may be outweighed by the benefit in some cases for Google Fonts though, as the fonts they deliver are well optimised and subset by alphabet automatically.

In general you should serve the fonts from your domain to avoid the cost of connecting to third-party domains, this will be especially important on high latency connections.

### Cache your fonts

Fonts can be cached in two places: the client and the <abbr title="Content Delivery Network">CDN</abbr>. Caching on the client is important for mid-session navigations and should be done in such a way as to avoid revalidation requests. Revalidation requests (`if-not-modified` and `if-modified-since`) will block the browser from using the font file until it has verified that it has not changed on the server. Fonts rarely change, so we should implement a cache header as follows and update the filename if the font changes to break the cache:

`cache-control: max-age=31536000,immutable`

This tells browsers that they can keep the font for up to a year and that it does not need to be revalidated (`immutable` [is supported](https://caniuse.com/mdn-http_headers_cache-control_immutable) in Firefox and Safari, Chrome should avoid revalidation requests automatically). Avoid adding ETags to these responses as they may force revalidation.

Check also that your Content Delivery Network configuration can store the font files in cache, older configurations may not include the `.woff2` extension resulting in origin hits and slowing down the response.

### Use preload hints

Web browsers do not download fonts unnecessarily, they wait until the render tree has been constructed in order to know which fonts are required. This means that web fonts are only requested when the browser has downloaded and parsed the HTML and CSS, right before text is rendered. Note that inline CSS does not require a network request - meaning your fonts could be fetched earlier in the page load.

![flow diagram shows that web font loading is blocked by render tree construction](https://simonhearne.com/images/layout-shifts-webfonts/font-crp.png)
*Render tree construction delays web font request. source*

If we know that a web font is definitely going to be required to render text on a page, we can make a promise to the browser that it needs to be downloaded as soon as possible. This is a preload hint:

```html
<link rel="preload" href="/my-font.woff2" crossorigin="anonymous" as="font" type="font/woff2">
```

When the browser parses this line of HTML it will immediately dispatch a high priority request for the font file. Doing this means that the web font is much more likely to be available when the browser renders text.

<blockquote class="warning"><p>preloaded requests will cannibalise bandwidth from other early requests, use with caution!</p></blockquote>

Preloaded requests are high-priority. In the waterfall section below you can see that five font files are downloaded, one blocks the page CSS (possibly because the `<link rel="preload">` was above the `<link rel="stylesheet">`) and the others block the main JavaScript bundle for the page. Note that this page was served over HTTP/2 on Chrome, other browsers and protocols will vary!

![waterfall section showing fonts blocking main javascript bundle download](https://simonhearne.com/images/layout-shifts-webfonts/preload-slow-script.png)
*Preloaded fonts delay main JS bundle.*

Ensure that your preload tags are below critical content in the head and limit to two or three font files to get the best benefit.

## Reduce layout shift with f-mods

If `font-display: optional` is not possible for your design, we need to attempt to match the fallback font layout to the web font as closely as possible. Enter font-display modifiers (f-mods), also known as font-face descriptors.

F-mods are in a [proposed update](https://docs.google.com/document/d/1PW-5ML5hOZw7GczOargelPo6_8Zkuk2DXtgfOtJ59Eo/edit) to the font-face descriptors specification which includes four new descriptors:

- **ascent-override (%)** - overrides the size allocated for ascenders
- **descent-override (%)** - overrides the line height allocated for descenders
- **line-gap-override (%)** - overrides the gap between lines
- **advance-override (#)** - sets an extra advance for each character, to help match line width and prevent word overflows

The first three all impact the height of a line: line box height = ascent + descent + line gap. Baseline position = line box top + line gap / 2 + ascent. For example, if we have `ascent-override: 80%; descent-override: 20%; line-gap-override: 0%`, then each line box has height 1em (assuming 1em used font size), and the baseline is positioned at 0.8em below the line box top. The `advance-override` descriptor adds a fraction of the font-size as a gap before each character, for example `font-size:16px; advance-override: 0.1;` will add a 1.6px gap before each character.

The combination of these four descriptors allow us to override the layout of the fallback font to match the web font, by telling the browser how much space the characters will take up before the web font is downloaded. Read more in the [proposal](https://docs.google.com/document/d/1PW-5ML5hOZw7GczOargelPo6_8Zkuk2DXtgfOtJ59Eo/edit).

### Implementing f-mods

The trick to implementing f-mods is to manually define your fallback system fonts with `src: local()`. This allows us to override the display of the fallback font to match the web font:

```css
@font-face {
  font-family: custom-font;
  src: url("https://example.com/font.woff2");
}
@font-face {
  font-family: fallback-font;
  src: local(Arial); /* required! */
  advance-override: xx;
  ascent-override: xx;
  descent-override: xx;
  line-gap-override: xx;
}
body {
  font-family: custom-font, fallback-font;
}
```

Calculating the required values might seem complex, but the information we need is already in your web font file. First you'll need the TTF file for your font; if you use Google Fonts then you can check find the TTF in the [GitHub repository](https://github.com/google/fonts/tree/master/ofl/). If you have licensed a font from a foundry then the TTF should have been supplied already.

Once you have the TTF head on over to [FontDrop](https://fontdrop.info/) and upload the file. Open the **Data** tab and scroll to the `hhea - Horizontal Header Table`. There you should find four key values: `ascender`, `descender` , `line-gap` and `advanceWidthMax`.

![screenshot of font drop showing font HHEA table](https://simonhearne.com/images/fontdrop.jpg)
*HHEA table data in FontDrop*

These values are exactly what we need to set the font-face modifiers! In this case the web font has unitsPerEm = 1000 and ascender = 1027, so it's best to set `ascent-override: 102.7%` in the fallback font face. I recommend experimenting with values until you find the right balance, using the playground below.

### F-mods Playground

Use [this codepen](https://codepen.io/simonjhearne/pen/rNMGJyr) to load your custom and fallback fonts, then play with the overrides to get a perfect match! Note that this will be specific to the operating system you are using, it may be worth validating across multiple devices.

<p class="codepen" data-height="852" data-theme-id="light" data-default-tab="result" data-user="simonjhearne" data-slug-hash="rNMGJyr" style="height: 852px; box-sizing: border-box; display: flex; align-items: center; justify-content: center; border: 2px solid; margin: 1em 0; padding: 1em;" data-pen-title="font-face descriptor playground">
  <span>See the Pen <a href="https://codepen.io/simonjhearne/pen/rNMGJyr">
  font-face descriptor playground</a> by Simon Hearne (<a href="https://codepen.io/simonjhearne">@simonjhearne</a>)
  on <a href="https://codepen.io">CodePen</a>.</span>
</p>
<script defer src="https://cpwebassets.codepen.io/assets/embed/ei.js"></script>

### F-mods limitations

F-mods only really modify vertical spacing and positioning. This means that character- and letter-spacing still need to be dealt with otherwise you could have words breaking lines at different points, leading to a change in element heights and thus layout shifts. Unfortunately the `letter-spacing` and `word-spacing` properties are not available in the @font-face declaration so they must be declared on the body or an element. This means we may still have some work to do in order to prevent layout shift.

If the character and letter spacing needs to be modified for the web font, will need to apply CSS rules to all elements with the web font applied for when the fallback font is shown. This style must then be removed when the web font is loaded. This is possible using the [font loading API](https://developer.mozilla.org/en-US/docs/Web/API/CSS_Font_Loading_API), but the whole process feels racy and fragile. Changing layout-critical CSS with JavaScript doesn't sit right with me.

Browser support for font-face descriptors is... well it's only in Chrome, in v87+. There are, however, positive signals from Safari. No matter the browser support, font-face descriptors are a progressive enhancement so there is no reason not to implement them now.

## Guaranteeing fonts load in time

It has been suggested that fonts can be embedded in CSS as Base64 strings, thus removing the need for additional font requests and ensuring that the font is available at the time of rendering text. I have seen this on production websites, and it does achieve these goals.

Embedding fonts as Base64 strings comes with a number of drawbacks, as such I would not recommend this approach:

1. Font files are compressed binary objects, encoding as Base64 strings will inflate the size significantly. gzip or brotli compression of the CSS bundle will not totally make up for this inflation.
2. The fonts will be sent to every browser, even if they can't use them (e.g. Opera Mini and IE11 for woff2, users of [Dyslexia Friendly](https://chrome.google.com/webstore/detail/dyslexia-friendly/miepjgfkkommhllbbjaedffcpkncboeo?hl=en))
3. Fonts rarely change, but CSS changes often - this will reduce cache effectiveness for fonts as every CSS change will invalidate the whole bundle
4. Inflating CSS size will almost certainly delay page render
5. Embedding fonts prevents you from effectively using subsets and unicode-range for different alphabets

In general, embedding fonts in CSS breaks the magic of web fonts: that browsers know what they need and will download it when they need it. There are two other options if you absolutely require the fonts to be available before the page renders:

1. Hide the body until the font is ready, using `opacity: 0` and the [Font Loading API](https://developer.mozilla.org/en-US/docs/Web/API/CSS_Font_Loading_API) to show content once the font loads (with a sensible timeout)
2. Use `display:block` and match the fallback font as closely as possible to your web font

Option 1 would need a lot of testing to ensure that content is always shown eventually, I would avoid it entirely.

## The future of font performance

It is clear that font performance is not yet a solved problem. Thankfully, there a works afoot to improve this in the [W3C Web Fonts Working Group](https://www.w3.org/Fonts/WG/). One of the proposals is progressive font enrichment, this is an incremental font loading concept which allows browsers to request exactly the characters they need to render text, potentially drastically reducing initial font file size and thus improving render speed.

Google [has a demo](https://fonts.gstatic.com/experimental/incxfer_demo) of this incremental transfer concept, and [Jason Pamental](https://twitter.com/jpamental) has written up the concept in [much more detail](https://rwt.io/typography-tips/progressive-font-enrichment-reinventing-web-font-performance) than I could here.

Whilst the incremental font loading concept is interesting, the main use case will be for very large font sets for non-latin languages. There are potential issues of cache dilution to resolve and the overall size of fonts delivered could exceed the optimal subset font file. Watch this space for more developments!

## In conclusion

Layout shifts are bad for user experience and tricky to resolve. Rendering text without layout shifts is much more complex than it should be! You can avoid layout shifts entirely if you can apply `font-display: optional` to your web fonts, otherwise it's a case of racing the browser - trying to get your fonts to the browser before it starts to render text.

Racing the browser is possible by optimising your font files:

- Use `woff2` to minimise file size
- Serve fonts from your own domain
- Preload critical fonts
- Subset the font to required characters
- Limit the number of weight variations used
- Explore variable fonts
- Experiment with system fonts
- Use f-mods to reduce the impact of font swaps

As always, test all changes with real users and only keep them if you see a positive change. On this site I have set the body text to `font-display: optional` but my heading font and the italic variant of the body font to `font-display: swap`. Swapping in the heading font and italic variant do not cause a layout shift in most cases (except multi-line headings) so this is a good compromise between design and performance, in my opinion.


==============================================================================

# Optimistic UI Patterns for Improved Perceived Performance

> Speed is in the eye of the beholder, it is a perception of how fast an experience feels. Optimistic UI patterns can significantly improve perceived performance for relatively little effort.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2021-01-15
Canonical: https://simonhearne.com/2021/optimistic-ui-patterns/
Tags: WebPerf, UX

---
## Introduction

Web performance is often seen as a technical discipline, arranging bits and bytes to shave off milliseconds of load time, applying best practices, using the latest protocols and formats. Users don't perceive milliseconds though, they don't (typically) care if you use HTTP/1.1 vs HTTP/2 or JPEG vs WebP. Web performance optimisation is really optimising the speed of user experiences, Sergey Chernyshev even [thinks we should](https://simonhearne.com/2020/web-performance-rebrand/#comments) rebrand web performance to **UXSpeed**!

If web performance isn't all about technology, it must be about user experience design: orchestrating the web experience to create user delight. There are a number of techniques which aim to improve the speed of UX including [low quality image placeholders](https://cloudinary.com/blog/low_quality_image_placeholders_lqip_explained), [skeleton UI](https://uxdesign.cc/what-you-should-know-about-skeleton-screens-a820c45a571a) and optimistic UI patterns.

I love optimistic UI patterns because they can often have a great return on investment: a big impact on user experience for a relatively small engineering effort. These UI patterns assume the best but prepare for the worst, optimising the UI feedback loop to give a better user experience.

## Feedback First!

![hand drawn image of a close feedback loop to user action](https://simonhearne.com/images/optimistic-ui/optimistic-ui.jpg)
*Decouple the feedback loop from the execution loop*

Optimistic UI patterns have a simple goal: to give the feeling of a robust and instant UI. As humans we start to notice delays around 100ms and frustration increases with the delay, remember the 300ms [mobile tap delay?](https://developers.google.com/web/updates/2013/12/300ms-tap-delay-gone-away) A round-trip to a server might take 50ms in the best case scenario, but a large number of requests will exceed 100ms due to network latency and varying server response times. Waiting for a network request means that your UI is bound by variables outside of your control, like your users' network connectivity and device performance.

<blockquote class="success"><p>Optimistic UI patterns decouple user feedback from the network.</p></blockquote>

Optimistic UI patterns decouple user feedback from the network, giving you more control of the speed of the user experience. There are some great opportunities to improve the perceived performance of most pages by applying these patterns to key user interaction points, let's look at some examples!

### Optimistic Cart UI

Allbirds is a New Zealand-American company which designs and sells footwear. The Allbirds website feels very fast to navigate, they use a number of optimistic techniques to achieve this sensation even on poor connections. One of my favourite uses of optimistic UI is the Allbirds mini-cart. Anyone who has worked on a retail site will tell you that cart calls are expensive - they cannot be cached and typically require multiple database queries, this means that adding to cart is often a slow user experience. Allbirds have made efforts to resolve this delay by optimistically sliding in the mini cart *immediately* after a product is added, even when the API call has not yet hit the server. This does two things: it gives instant feedback to the user that their interaction has been registered, plus the slight animation to slide in the mini-cart buys some active time from the user to make the process feel even faster.

<figure>
  <video style="height: 600px;max-height:80vh;width:auto" autoplay loop muted playsinline preload="metadata">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/h_600,q_auto/v1612776931/videos/allbirds-mobile.webm" type="video/webm">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/h_600,q_auto/v1612776931/videos/allbirds-mobile.mp4" type="video/mp4">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/h_600,q_auto/v1612776931/videos/allbirds-mobile.mov" type="video/mov">
  </video>
  <figcaption>Allbirds optimistically loads the mini-cart on add to cart</figcaption>
</figure>

This subtle animation only takes half a second, but it buys enough time for the API call to be dispatched to the server and perhaps even for the response to be delivered. This combination of effects culminates in an extremely fast experience, prioritising user feedback in the UI.

This principle can be used any time an action triggers a server-bound UI change, such as creating an account, entering delivery details or submitting information for an insurance quote.

### Optimistic Atomic Actions

You may have noticed that Twitter's UI feels remarkably fluid. One of the features which contributes to this experience is the favourite button.

<figure>
  <video style="max-width:500px" autoplay loop muted playsinline preload="metadata">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/w_500,q_auto/v1612776931/videos/twttr_favourite.webm" type="video/webm">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/w_500,q_auto/v1612776931/videos/twttr_favourite.mp4" type="video/mp4">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/w_500,q_auto/v1612776931/videos/twttr_favourite.mov" type="video/mov">
  </video>
  <figcaption>Twitter's favourite button gives instant feedback</figcaption>
</figure>

Note that the animation starts as soon as you click, giving an immediate feedback response. This happens separately and in parallel to the actual API call to log the action. Further to the instant animation, the API call is resilient to network failures. If the API call fails it is added to a queue to be retried later. Only if the API call fails multiple times is the action "undone" in the UI - the heart icon simply fades back to the unchecked state.

This principle can be applied to any atomic action like starring a product, liking a message or saving an article for later.

### Contextual Buttons

Active states have become less popular as [skeuomorphism](https://www.interaction-design.org/literature/topics/skeuomorphism) has lost favour in UI design. Active states give users *immediate* feedback from their action, like a shadow to indicate a depressed button. You can do more than just add a shadow, though. In this example we give immediate physical feedback, like a real button might behave, then provide further feedback related to the user's action right in their point of focus. The proximity of feedback to the user's focus is key here:

<style>:root{--color-blue:dodgerblue;--color-purple:rebeccapurple;}@-webkit-keyframes short-press{0%{-webkit-transform:scale(1);transform:scale(1)}50%{-webkit-transform:scale(.9);transform:scale(.9)}100%{-webkit-transform:scale(1);transform:scale(1)}}@keyframes short-press{0%{-webkit-transform:scale(1);transform:scale(1)}50%{-webkit-transform:scale(.9);transform:scale(.9)}100%{-webkit-transform:scale(1);transform:scale(1)}}.submit-button{display:block;overflow:hidden;margin:0.5em auto;zoom:2;font-family:inherit;font-weight:300;font-size:.75em;height:20px;line-height:20px;position:relative;background:var(--color-blue);border-radius:3px;box-shadow: 0px 1px 2px 0px rgb(60 54 68);border:0;cursor:pointer;-webkit-transition:all .3s ease;transition:all .3s ease}.submit-button:hover{background:var(--color-purple);outline:0}.submit-button.animated{-webkit-animation:.5s short-press cubic-bezier(.77,0,.175,1) forwards;animation:.5s short-press cubic-bezier(.77,0,.175,1) forwards}.submit-button>span{display:block;color:#fff;text-align:center}.submit-button>span.pre-state-msg{margin-top:-2px;-webkit-transition:all .7s cubic-bezier(.77,0,.175,1);transition:all .7s cubic-bezier(.77,0,.175,1);-webkit-transition-delay:.5s;transition-delay:.5s}.submit-button.state-1 .pre-state-msg{margin-top:-22px}.submit-button.state-2 .pre-state-msg{margin-top:-42px}.submit-button.animated{background:var(--color-purple)}.submit-button span{user-select:none}</style>
<p style="text-align:center">click me 👇<br>
  <button class="submit-button state-0"><span class="pre-state-msg">Search</span><span class="current-state-msg">Searching...</span><span class="done-state-msg">Done!</span></button>
</p>
<script>const button=document.querySelector(".submit-button"),stateMsg=document.querySelector(".pre-state-msg"),updateButtonMsg=function(){button.classList.add("state-1","animated"),setTimeout(finalButtonMsg,2e3)},finalButtonMsg=function(){button.classList.add("state-2"),setTimeout(setInitialButtonState,2e3)},setInitialButtonState=function(){button.classList.remove("state-1","state-2","animated")};button.addEventListener("click",updateButtonMsg);</script>

This principle can be applied anywhere a button press results in a server request but doesn't take the user away from the page, like add to cart, search or product configuration buttons.

### Search Filters

Back to Allbirds for this one. Filtering products is often a painful user experience which can impact average order value and conversion rate. Allbirds have added an interstitial state to their filter UI which moves the content down and adds a spinner, making the experience feel more fluid once the results are delivered over the network.

<figure>
  <video style="max-width:80%" muted playsinline autoplay loop preload="metadata">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/w_auto,q_auto/v1612776931/videos/socks.webm" type="video/webm">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/w_auto,q_auto/v1612776931/videos/socks.mp4" type="video/mp4">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/w_auto,q_auto/v1612776931/videos/socks.mov" type="video/mov">
  </video>
  <figcaption>Allbirds has an interstitial state while waiting for filtered results</figcaption>
</figure>

This interstitial state does not need to serve any functional purpose - it just helps to give the user confidence that the website is working while the browser waits for server response of filtered results. This principle can be applied to any filtering UI which results in a server request.

### Page Transitions

Some actions will always add a delay to the user interface, such as navigating to a new page in a traditional web application. This delay between clicking a link and the page context changing can take multiple seconds due to delays outside of your control. The browser's context will not switch to the next page until it can paint something to screen, often meaning multiple assets need to be downloaded first. Even if all blocking CSS & JavaScript is available in cache, the time taken to download and parse the HTML document and then retrieve the cached static assets and render the page can take over a second.

This inter-page delay can be jarring to the user experience, especially if page links do not have good active states. Thoughts of "did I click on that link?" and "is the site broken? I should click again." will start within fractions of a second with no UI feedback. A simple hack to improve the perceived performance in this scenario is to introduce a page transition: a simple spinner or blur effect to indicate that the navigation is occurring.

<figure>
  <video style="max-width: 320px" autoplay loop muted playsinline preload="metadata">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/w_320,q_auto/v1612776931/videos/jimmychoo-spin.webm" type="video/webm">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/w_320,q_auto/v1612776931/videos/jimmychoo-spin.mp4" type="video/mp4">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/w_320,q_auto/v1612776931/videos/jimmychoo-spin.mov" type="video/mov">
  </video>
  <figcaption>Jimmy Choo animates page transition</figcaption>
</figure>

Jimmy Choo show a simple transition on the unload event of a page which fades away the screen and shows an animated logo. They also have a good active state on the button in this case, making it clear that the user has initiated a navigation. The whole process takes about a second, but feels quite seamless. This principle can be applied anywhere that you have a delayed page transition, especially if you know the target page will be slow such as loading search results or a shopping cart.

### Pre-emptive Loading

Users constantly give you hints as to what they are going to do next when they hover on and touch elements. By attaching to the `mouseover` and `touchstart` events on links we can respond to these hints and use them to buy us some time. On my homepage I inject a `<link rel='prefetch'>` into the document if you hover or touch one of the article cards, with the `href` attribute set to the target blog post. This tells the browser to start fetching the document so that the document should be already downloaded by the time the user's click or tap is complete. This should mean that the page can be served almost instantly from cache!

![network tab showing html file served from prefetch cache](https://simonhearne.com/images/optimistic-ui/prefetch-cache.png)
*Prefetch cache should be much faster than the network*

The open source project [instant.page](https://instant.page/) is a generalised application of this principle, although it is trivial to build it yourself in a few lines of JavaScript.

## In Conclusion

Web performance is as much art as it is science. Consider how your UI behaves under less-than-ideal conditions and how you can implement optimistic UI patterns to decouple perceived performance from application performance.

The examples we have seen all apply some optimism - either that the request will succeed or that the user will complete an action. Optimism allows us to preempt an outcome and deliver it faster, resulting in happier users. Plus it's nice to raise tickets with **optimistic** in the title!

Sometimes small changes can have a big impact. Focus on changes which improve the feedback loop to user interactions and aim for under 100ms to deliver a UI response. Make a change, measure the impact on user experience and keep it if business outcomes improve!


==============================================================================

# Alternatives to Spinners on the Web

> Animated spinners are so 1999, there are some obvious and less-obvious alternatives to improve user experience.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2020-12-22
Canonical: https://simonhearne.com/2020/alternatives-to-spinners/
Tags: WebPerf

---
Animated spinners are one of the lingering legacies of the 1990's web.

In this post I will use an insurance quote service as an example, but these techniques can be applied anywhere that a spinner is currently used. First, though, let's look at the psychology of waiting.

## Rules for waiting time

In [The Psychology of Waiting Lines](https://davidmaister.com/articles/the-psychology-of-waiting-lines/), David H. Maister describes the psychology of queueing. His research is based on real-world queues, but the psychology is applicable any time that people are forced to wait to complete a task - except that people have even less patience on the web, with an unlimited number of websites and applications to distract them from the current task.

I've extracted three key concepts which are most applicable to waits on the web:

1. Unexplained waits feel longer than explained waits
2. Unoccupied time feel longer than occupied time
3. Anxiety makes waits feel longer

### Explain the wait

<blockquote class="thought"><p>Unexplained waits feel longer than explained waits</p></blockquote>

Spinners don't explain why a user is waiting, just that they have to wait. Giving the user an explanation of why they have to wait can be a simple step to reduce the perceived duration of the wait; a secondary benefit is that the time taken to read any explanation is active time - this can buy us a second or two of wait time!

For our insurance website, we can humanise the wait with an explanation:

> We are generating your quote, evaluating the criteria specific to your situation and determining the best possible price and coverage. This might take up to a minute.

You may have seen this in action on travel websites or even retail sites. For example Amazon will add a message when calling APIs for stock levels of reseller products and comparison sites will often have text describing how many quotes have been retrieved.

<div class="two-fig-cols">
  ![screenshot of amazon waiting screen which says: 'rummaging around in the stock room'](https://simonhearne.com/images/alternatives-to-spinners/amazon-rummaging.jpg)
*Amazon's humanising wait text*![screenshot of amazon waiting screen which says: 'rummaging around in the stock room'](https://simonhearne.com/images/alternatives-to-spinners/amazon-rummaging.jpg)
*Amazon's humanising wait text*
</div>

### Occupy time

<blockquote class="thought"><p>Unoccupied time feels longer than occupied time</p></blockquote>

In the real world you might pull out your phone whilst waiting in a queue, or pick up a magazine in a waiting room. Users can easily do this on the web: switching tabs or checking in on a social media application. Our goal is to keep the user active on **our** application, so we should aim to occupy the user's time where they are waiting.

This technique has been used for decades in games: in 1998 Namco was [granted a patent](https://www.eff.org/deeplinks/2015/12/loading-screen-game-patent-finally-expires) to add mini-games to the loading screen for large games. This was a reaction to the move from cartridge games to CD-ROM which took much longer to load into memory, leading to frustration.

Whilst users on the web might object to playing games on waiting screens, there are other techniques which might achieve the same goal. For our insurance quote, we could add a simple survey to the waiting screen:

> How easy was this quote process? Answering will not interrupt your quote. 1 - very difficult, 10 - super easy.

It can be even more simple, such as some text body for the user to read. This can be marketed to the user based on the product or service they are waiting for:

> Our insurance products are backed by 50 years of experience, we insure over 2 million people like you every year.

There may be other opportunities to keep the user engaged, such as showing interim results like SkyScanner or custom quotes like Slack.

<div class="two-fig-cols">
  ![screenshot of skyscanner search results page showing results while still searching](https://simonhearne.com/images/alternatives-to-spinners/skyscanner.jpg)
*SkyScanner starts showing results straight away*![screenshot of skyscanner search results page showing results while still searching](https://simonhearne.com/images/alternatives-to-spinners/skyscanner.jpg)
*SkyScanner starts showing results straight away*
</div>

### Create certainty

<blockquote class="thought"><p>Anxiety makes waits feel longer</p></blockquote>

Knowing that you have to wait is bad enough, but not knowing how long you need to wait makes it much more frustrating. Think back to the last time your flight or bus was delayed without an ETA. You cannot make plans without knowing when you will depart or arrive - do you have enough time to use the restrooms or get some lunch? Will you need to change a transfer?

Amusement parks often have estimated wait times in queues, this is done to reduce wait anxiety! It also serves a secondary benefit: the estimated wait times are almost always an over-estimate, so that as you progress through the queue you feel like you are winning back time. If you expected to queue for 45 minutes but got to the end in 30 minutes, you would feel good about the waiting time. Compare that to how it feels to have an unknown wait that ends up taking 30 minutes, like in a queue at a shop or government office.

Doing this on the web can be tricky, but I'm certain that somewhere in your business you have measurements of the duration of complex tasks. For example an insurance company knows that it takes on average 30s to generate a quote, with the 95th percentile task taking 60s (that means that 95% of quotes will be generated within a minute). This data can be used to create messaging on the web:

> Please wait while we generate your quote, this should take thirty seconds

At the point that 30s has passed, update the message:

> This is taking a little longer than normal but the quote should be generated in just a minute

Once the 60s mark has gone, it's time for an apologetic message:

> We're sorry this is taking longer than usual, please bear with us as we generate your quote. If you'd like to speak to us on the phone, please call +1 555 1234

## Alternatives to spinners

Consider the three concepts above: explanation, occupying time and certainty. Spinners don't really solve for any of these! Let's look at three alternatives.

### 1. Show progress indication

One of the frustrations of spinners is that they don't convey any progress, just a perpetual animation that *might* mean that something is happening in the background.

Users want to know that something is really happening, and that there is progress towards completion. It is tempting to replace a spinner with a progress bar - but this only works if you can show real progress. Fake progress bars might be worse than spinners, especially when they get stuck at 99% and cause user frustration!

It is best to complete longer tasks earlier if you can get stages of feedback to update your progress bar, users want to see feedback early but become less patient as they wait.

You can estimate the speed of the progress bar using historical data if you can't get realtime feedback from the application. In our insurance example we know that 95% of quotes are returned in a minute or less, so we can use that as our extent on the progress bar. A trick to make the progress feel a little faster is to use a non-linear progression: accelerate to make the initial few seconds feel faster, and decelerate at the end to buy a little more time. Research on [non-linear progression](https://www.chrisharrison.net/index.php/Research/ProgressBars) shows that users prefer accelerating progress indicators, although this research did not include multi-phase progression (e.g. accelerating and decelerating).

All of these progress indicators have the same total duration, which one feels faster?

<figure>
  <div class="progress_bar progress_bar_linear"><span>linear</span></div>
  <div class="progress_bar progress_bar_decel"><span>decelerates</span></div>
  <div class="progress_bar progress_bar_accel"><span>accelerates</span></div>
  <div class="progress_bar progress_bar_both"><span>accelerates and decelerates</span></div>
  <figcaption>Various easing methods on progress indicators</figcaption>
</figure>

### 2. Remove the spinner

If your spinner is a placeholder for other content - like images or dynamic forms - you might be able to get away with simply removing it!

Spinners appearing and disappearing can be disorienting, so if the wait time is under a second or two it is worth considering removing the spinner. Good alternatives to spinners are placeholder elements such as low quality image placeholders or skeleton UI.

<div class="two-fig-cols">
![simon's logo switching between SVG placeholder and downloaded image](https://simonhearne.com/images/alternatives-to-spinners/simon-sqip.gif)
*SQIP on this site*
![comparison of a page loading with a spinner and a skeleton UI](https://simonhearne.com/images/alternatives-to-spinners/skeleton.gif)
*Simple skeleton UI*
</div>

### 3. Keep a spinner, but make it nice

If there really is no way to determine how long a wait will be, we might need to keep a spinner. In this case, the spinner should be branded and accompanied by some explanatory text.

Animated gif files are bulky and have a limited colour palette - SVGs and CSS animations are the way forward. These will render nearly instantly (assuming inline SVG and CSS) and will look great on retina displays. Take a look at [loading.io](http://loading.io) for some inspiration, or read [Cassie Evans' blog post](cassie.codes/posts/creating-my-logo-animation/) on how they created their animated logo for some advanced techniques!

<p class="codepen" data-height="400" data-theme-id="light" data-default-tab="result" data-user="simonjhearne" data-slug-hash="YzGxmrd" style="height: 265px; box-sizing: border-box; display: flex; align-items: center; justify-content: center; border: 2px solid; margin: 1em 0; padding: 1em;" data-pen-title="Cassie!">
  <span>See the Pen <a href="https://codepen.io/cassie-codes/pen/mNWxpL">
  Cassie!</a> by Cassie Evans (<a href="https://codepen.io/cassie-codes">@cassie-codes</a>)
  on <a href="https://codepen.io">CodePen</a>.</span>
</p>
<script defer src="https://cpwebassets.codepen.io/assets/embed/ei.js"></script>

There are some good examples of custom loaders out in the wild, such as the [Transport for London](https://tfl.gov.uk) travelling bus and [Jimmy Choo's](https://jimmychoo.com/) spinning logo which is shown on page transition.

<div class="two-fig-cols">
  <figure>
    <video title="TfL custom loading spinner" width="620" autoplay loop muted playsinline preload="metadata">
      <source src="https://res.cloudinary.com/simonhearne/video/upload/w_620,q_auto/v1612776931/videos/tfl-anim.webm" type="video/webm">
      <source src="https://res.cloudinary.com/simonhearne/video/upload/w_320,q_auto/v1612776931/videos/tfl-anim.mp4" type="video/mp4">
      <source src="https://res.cloudinary.com/simonhearne/video/upload/w_620,q_auto/v1612776931/videos/tfl-anim.mov" type="video/mov">
    </video>
    <figcaption>Transport for London's custom HTML loader</figcaption>
  </figure>
  <figure>
    <img width="140" height="140" src="/images/alternatives-to-spinners/jc-spinner.gif" style="max-width: 200px;" alt="jimmy choo custom loading spinner"/>
  <figcaption>Jimmy Choo shows a custom spinner between pages</figcaption>
</figure>
</div>

## In summary

Spinners are the most simple option when you have a long task of indeterminate duration, but they may cause user frustration. You should replace ambiguous spinners with better progress indication and add dynamic explanatory text. If you must use a spinner, make sure it uses more modern technologies like CSS and SVG, that it is branded to your site and that it is relevant to the task at hand.


==============================================================================

# Web Performance Predictions for 2021

> 2020 was a wild ride. Here are five key things to watch in the world of web performance in 2021!

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2020-12-10
Canonical: https://simonhearne.com/2020/predictions-2021/
Tags: WebPerf

---
I [wrote an article in 2019](/2019/2020-predictions) where I predicted five top technologies for 2020. Well, 2020 was a strange year! Some things were correct though, for example the growth of JAMStack sites: the [Web Almanac shows](https://almanac.httparchive.org/en/2020/jamstack) that JAMStack now powers 154% more mobile pages than in 2019!

As for 2021, there are some key trends which I expect to have a significant impact on the web ecosystem:

1. [The Page Experience Update](#the-page-experience-update)
2. [Better behaving third-parties](#better-behaving-third-parties)
3. [Logic moving to the edge / FaaS](#logic-moving-to-the-edge)
4. [Browser Monopolies](#browser-monopolies)
5. [HTTP/3](#http3)

## The Page Experience Update

Performance will become an even stronger ranking factor in mobile search [come May 2021](https://developers.google.com/search/blog/2020/11/timing-for-page-experience). You might be sick of hearing about it by now, but there are hundreds of thousands of sites which currently fail the [core web vitals](https://web.dev/vitals/) assessment. [57% of pages have a slow mobile LCP](https://almanac.httparchive.org/en/2020/performance#lcp-by-device) and [44% of pages have a poor CLS on desktop](https://almanac.httparchive.org/en/2020/performance#cls-by-device).

The Page Experience Update is the glue that will bind Product, SEO and engineering teams to deliver great web experiences. There are concerns about the key metrics involved, especially as they can only be collected (at the time of writing) on Chromium-based browsers. There's also a concern about the continuing monopoly of Google in both browser and search, but for now I see this as a positive change. I hope that the 2021 web almanac shows a significant improvement in these key metrics as engineering teams are given more resources to work on performance. You can read more about Core Web Vitals (and how to improve them) in [this blog post](/2020/core-web-vitals/).

## Better Behaving Third-Parties

Third-party content is critical to most websites. The [web almanac reports](https://almanac.httparchive.org/en/2020/third-parties) that the median web page has 24 separate third-party requests! Data from the web almanac also shows that advertising has the greatest impact on browser CPU, and that the most popular third-parties range from webfonts and JavaScript CDNs to advertising and analytics. We also know that some particular forms of third-parties have a huge impact on performance: blocking tag managers and multivariate testing frameworks. Don't get me started on [anti-flicker scripts](https://andydavies.me/blog/2020/11/16/the-case-against-anti-flicker-snippets/)!

The increased focus on performance created by the Page Experience Update will put serious pressure on bad behaving third-parties, at the same time there are a number of web features which will make it easier for them to reduce their performance impact! Take a look at [the import on interaction pattern](https://addyosmani.com/blog/import-on-interaction/) and [Trust Tokens](https://web.dev/trust-tokens/) to read about some of these.

New technologies and techniques allow third-party providers to reduce their impact on user experience, unfortunately we will still be dependent on them to make these changes! If you have a third-party provider who currently impacts performance, please reach out to them and ask them to reduce their impact! You likely have a better understanding of how their code impacts experience than they do, and they might appreciate your input.

## Logic Moving to the Edge

Edge compute has been around for a while. Edge is different from traditional function as a service (FaaS) solutions as logic is executed on a much more distributed set of compute nodes. Hundreds of thousands of servers in the case of Akamai!

This new compute model has a number of limitations: state is minimal (all CDNs offer some kind of key-value storage, but not session storage) and execution time is limited ([100ms on Akamai EdgeWorkers](https://learn.akamai.com/en-us/webhelp/edgeworkers/edgeworkers-user-guide/GUID-F709406E-2D67-4996-B619-91E90F04EDF2.html)). The distributed compute paradigm is not going to replace your origin servers, or even your serverless deployments; where it excels though is in stripping small elements of stateless logic away from origin and clients.

A good example of logic best deployed at the edge is geolocation. Sure you can redirect users at origin, or have the device behave differently based on a known location, but edge logic can do this quicker and more robustly. Another example is stateless, quick compute requirements like generating tokens, QR Codes or signatures. For example you might use edge logic to quickly generate a hash of a file for duplication detection, or add watermarks to images.

Migrating logic from client-side code means a more consistent and predictable processing model, taking logic from origin means lower latency and greater distribution. Another great use case is simple A/B testing: a few lines of logic at the edge can replace heavy client-side scripts or load balancer configurations, with the obvious performance benefits this will bring.

Andrew Betts gave a [fantastic talk](https://ldnwebperf.org/sessions/design-factors-for-decentralised-edge-applications/) on design factors for edge compute at London Web Performance earlier in 2020.

## Browser Monopolies

Do you remember when Chrome was launched? It doesn't seem so long ago but the web has changed a lot since then. Notably, Chrome and Safari together now account for about 83% of all browser usage, according to [Statcounter](https://gs.statcounter.com/browser-market-share#monthly-200901-202011).

![chart of browser usage since 2009 showing Chrome and Safari taking almost all traffic in 2020](https://simonhearne.com/images/predictions-2021/statcounter.png)
*Browser share 2009 - 2020 (source: statcounter.com)*

These browsers split the market share, but have very different support levels for modern features. As developers it makes sense to develop and optimise for the majority of users, this means we tend to develop and test on Chrome, with QA on Safari and (perhaps) a few other key browser families.

As Chrome continues to ship draft features before any other browsers (like QUIC, LayoutInstability API and LargestContentfulPaint API) we may see an increasing performance divide between Chrome and all other browsers. How will this impact the web in 2021? I'll leave that as a question for the reader.

## HTTP/3

HTTP/3 has been slowly creeping across the web for the past few years, led by Google implementing gQUIC both in their browser and applications.

HTTP/3 is a fundamental change to networking from HTTP/1.1 & HTTP/2, swapping the underlying TCP protocol for UDP and moving more logic into user-space. This means that the protocol can evolve faster than H/1.1 or H/2 ever could. I don't expect to see a huge change in performance immediately, Cloudflare even saw some [performance degradation.](https://blog.cloudflare.com/http-3-vs-http-2/) I highly recommend watching Robin Marx's talk [Fixing HTTP/3 and Preparing for HTTP/3 over QUIC](https://www.youtube.com/watch?v=pq_xk_Pecu4) for a great primer on HTTP/3 and QUIC.

You can test the impact for yourself using [WebPageTest](https://www.webpagetest.org/) (assuming your server or CDN supports QUIC): run a test in a browser that supports HTTP/3, then run again but disable HTTP/3. For Chromium-based browsers you can pass the command-line flag `--disable-quic` in the Chromium tab.

![screenshot webpagetest result showing 100ms improvement on simonhearne.com when loaded over HTTP 3](https://simonhearne.com/images/predictions-2021/http3-simonhearne.com.jpg)
*HTTP/3 results in ~100ms first paint improvement on this site, total performance may be slower.*

The key message with HTTP/3 is that it won't necessarily have the immediate benefits that we saw with HTTP/2 over HTTP/1.1, but it gives browser and server developers the opportunity to better implement features like prioritisation and congestion control.

## In Summary...

2020 was an interesting year, for many reasons! 2021 will continue to bring more focus on web performance as well as some great new technologies to support us building better web experiences. I hope that the increased focus on web performance will build more momentum behind our little industry, and enable us to invest the required resources to keep speeding up the web.

Performance is still a business problem. Modern protocols and tools are useless if we don't spend the time to measure and improve user experience.


==============================================================================

# The Impact of Let's Encrypt Changes on Android Users

> Let's Encrypt will use its own root certificate in 2021, this may block access to 12% of Android users globally.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2020-11-09
Updated: 2020-12-21
Canonical: https://simonhearne.com/2020/lets-encrypt-old-android/
Tags: WebPerf

---
Let's Encrypt [recently announced](https://letsencrypt.org/2020/11/06/own-two-feet.html) that it is moving to using its own root certificate on January 11, 2021. This will reduce operating costs and complexity for Let's Encrypt, helping their mission to make all websites secure by default.

<blockquote class="info"><p>December 21, 2020: Let's Encrypt have now <a href="https://letsencrypt.org/2020/12/21/extending-android-compatibility.html">announced a solution</a> to resolve this issue and prevent any impact to Android users!</p></blockquote>

The potential footprint for this change is huge: Let's Encrypt <abbr title="Domain Validation">DV</abbr> certificates are used on 225 million websites at the time of writing. You and I, however, are unlikely to be impacted in our daily browsing: the new root certificate that will be used was added to browsers in 2017. This means that if you have updated your device in the past three years it will have the new root certificate installed.

Devices that have not been updated since the root certificate was released will not trust certificates issued against it, so older devices will simply not load websites that use certificates issued after January 11, 2021. This is an issue for a couple of device categories: Android and set top boxes (like your Smart TV). Android devices older than version 7.1.1 do not have the new root certificate, <abbr title="set top boxes">STBs</abbr> are harder to define. Who doesn't update their device in three years, you might think. Unfortunately, Android device updates must be pushed by individual manufacturers (in the majority of cases) and devices that are carrier-locked further require the carrier to push updates. Some manufacturers and carriers may have decided to simply stop supporting older devices. System updates may also be too large for users on limited data plans to even consider downloading.

## What's the impact

Globally, 12% of hits from Android tracked in [mPulse](https://www.akamai.com/uk/en/products/performance/mpulse-real-user-monitoring.jsp) were from unsupported versions (i.e. <7.1.1) in data sampled in November 2020. This figure varies dramatically by country, though:

<div class="vega-chart loading desktop-only" id="vis" data-spec="/data/old-android/old-android.json"><a target="_blank" href="/data/old-android/chart.svg"><img src="/data/old-android/chart.svg" /></a></div>

<p style="margin-top: 1em;">Some stand out country statistics from the interactive map above:</p>

- **United States:** 8.2%
- **United Kingdom:** 14.2%
- **Brazil:** 16.2%
- **Russia:** 17.0%
- **India:** 14.6%
- **China:** 9.6%

The data is not exhaustive, and is only measuring hits that identify themselves as Android on websites instrumented with mPulse. The outcome is startling nonetheless.

<blockquote class="error"><p>In some countries over 25% of Android devices will not trust Let's Encrypt certificates issued in 2021.</p></blockquote>

## Do you use Let's Encrypt?

If you don't know whether your current certificate is from Let's Encrypt, because you use a third-party service such as a CDN or hosting company, you can check in your browser:

![screenshot of certificate information screen accessed from browser address bar, showing a let's encrypt authority](https://simonhearne.com/images/lets-encrypt-old-android/cert-check-LE.jpg)
*Click the padlock to view certificate information*

## What you can do about it

Let's Encrypt provides [advice to site owners](https://letsencrypt.org/2020/11/06/own-two-feet.html#if-you-are-a-site-owner). The gist is that although certificates will start to use the new, less compatible root certificate in January, you can force the old certificate until September. If you use [certbot](https://certbot.eff.org/) then you just need to use the flag `--preferred-chain "DST Root CA X3"`, for example.

Other than delaying the inevitable until September 2021, there is nothing that can be done except for switching certificate providers or encouraging your users to install Firefox for Android, which uses its own, updated set of root certificates.

As for the other device categories, such as <abbr title="set top boxes">STBs</abbr>, I would love to hear what folks are doing to prepare for the coming TLS-pocalypse.


==============================================================================

# The Performance Cost of EV Certificates

> Extended Validation certificates are expensive and degrade performance. Move to an OV certificate if you can!

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2020-11-04
Updated: 2021-11-22
Canonical: https://simonhearne.com/2020/drop-ev-certs/
Tags: WebPerf, security

---
## Introduction

Extended Validation or EV Certificates were once prized for their impression of increased security. Sites with an EV certificate would be rewarded with a nice green padlock and their company name in the address bar, potentially giving customers an increased sense of security when making purchases or entering sensitive data.

![screenshot of address bar showing paypal inc company name](https://simonhearne.com/images/drop-ev-certs/ev-green-padlock.jpg)
*EV Certificates used to give a benefit in the address bar*

If you load a page on a domain with an EV certificate now, you will notice there is no longer a nice green padlock and the company name. This change [rolled out](https://chromium.googlesource.com/chromium/src/+/HEAD/docs/security/ev-to-page-info.md) in Chrome 77 way back in September 2019. Other browsers have followed suit, now a secure page (using any certificate type) will have a small indicator, normally a padlock, and non-secure pages will be highlighted as `Not Secure`.

![screenshot of address bar showing not secure for http page](https://simonhearne.com/images/drop-ev-certs/insecure-warning.png)
*Non-secure domains are highlighted to the user*

## The rise of free certificates

There are three primary types of certificate used to secure web traffic:

- **Domain Validation (DV)** validates that the certificate requestor owns the domain
- **Organisation Validation (OV)** validates that an organisation owns the domain
- **Extended Validation (EV)** validates that an organisation owns the domain, with rigorous validation

DV certificates just require you to prove that you own a domain: using a DNS record, a redirect or a file on your web server. This process can be completely automated. EV and OV certificates require some manual intervention and further checks at renewal.

The screenshots from Chrome below show the information provided to the user for these different certificate types.

<div class="resp-cols-3">
<figure class="no-shadow">
<img src="/images/drop-ev-certs/dv-cert.png"
    alt = "screenshot of address bar showing not secure for http page"
    maxwidth = 300
    loading = "lazy"
    clickable = true
/>
  <figcaption>DV cert info in Chrome</figcaption>
</figure>
<figure class="no-shadow">
<img src="/images/drop-ev-certs/ov-cert.png"
    alt = "screenshot of address bar showing not secure for http page"
    maxwidth = 300
    loading = "lazy"
    clickable = true
/>
  <figcaption>OV cert info in Chrome</figcaption>
</figure>
<figure class="no-shadow">
<img src="/images/drop-ev-certs/ev-cert.png"
    alt = "screenshot of address bar showing not secure for http page"
    maxwidth = 300
    loading = "lazy"
    clickable = true
/>
  <figcaption>EV cert info in Chrome</figcaption>
</figure>
</div>

Making the web secure by default is an initiative I wholly support. [Let's Encrypt](https://letsencrypt.org/how-it-works/) is a non-profit certificate authority which allows any website owner to create a DV certificate for free. This makes it easier than ever to deliver pages over secure connections - preventing man-in-the-middle attacks and allowing the use of HTTP/2 which only works over <abbr title="Transport Layer Security">TLS</abbr>. Let's Encrypt is already integrated into most hosting providers and <abbr title="Content Delivery Networks">CDNs</abbr>.

Extended Validation certificates are expensive; they require a legal entity to prove that they own the domain which requires some human intervention. At the time of writing, this results in an EV certificate costing about $125 USD per year. This is not a huge cost, but is infinitely more than free!

## So what's the problem?

The performance challenge of EV certificates is that they do not fully support <abbr title="Online Certificate Status Protocol">OCSP</abbr> stapling. The online certificate status protocol is a method which allows clients like web browsers to ensure that certificates are valid, by checking with the certificate authority that it has not been revoked. Stapling allows the web server to perform this validation on the server-side and send the validation with the certificate. Without stapling the client has to do the validation. This can be seen in your web performance waterfalls as one or two (or more!) requests which occur during the TLS negotiation of the connection.

This abridged waterfall shows a [webpagetest run](https://www.webpagetest.org/result/201104_DiZ9_b464f5f612708262336735a850328a25/) at 3G speed against PayPal.com. The OCSP revalidation check takes almost a full second, over half of the TTFB for the response and 42% of the whole response.

![waterfall of paypal.com showing almost one second of time lost to OCSP revalidation](https://simonhearne.com/images/drop-ev-certs/ev-hero.png)
*OCSP checks impact TTFB*

This effect is more pronounced on high-latency connections. The latency here is not just based on the network connection, but also the distance between the user and the OCSP service. Digicert in this case uses Verizon's CDN which has [165 global points of presence](https://www.verizondigitalmedia.com/technology/our-network/) (PoPs). Users in Western Australia, most of Africa, Canada and the former Soviet Union will have worse experiences than those in Europe and the United States.

As pointed out by [Robin Osborne](https://twitter.com/rposbo) - there are [no OCSP resolvers in China](https://twitter.com/rposbo/status/1462795069996548110?s=20). This means that visitors could have up to a one minute delay before they load your page for the first time, and due to the fact that OCSP status is stored for seven days - this delay will occur once per week for each user!

A European client of mine recently switched from EV to OV and saw a significant performance improvement in TLS negotiation and thus Time to First Byte. The median negotiation time improved by 20% and it halved at the 95th percentile. Note the drop across all percentiles in this [mPulse](https://www.akamai.com/products/mpulse-real-user-monitoring) chart!

![graph showing TLS performance improve across all percentiles after switching from EV to OV certificate](https://simonhearne.com/images/drop-ev-certs/tlsperf.jpeg)
*mPulse chart showing TLS negotiation improves after migration from EV to OV certificate.*

<blockquote class="success"><p>It is likely that moving from an EV certificate to organisation validation (OV) certificate will reduce costs and improve performance.</p></blockquote>

Note that OCSP stapling must be enabled on your TLS terminating endpoint (e.g. your firewall or CDN) for it to work on DV and OV certificates. You can check your current certificate using the [Qualys SSL Server Test](https://www.ssllabs.com/ssltest/).

For simplicity I have ignored some of the intricacies of how OCSP works with intermediary certificates, and some of the many edge cases that occur with OCSP stapling. [Matt Hobbs](https://twitter.com/TheRealNooshu) goes into much more detail in [his blog post on certificate revocation](https://nooshu.github.io/blog/2020/01/26/the-impact-of-ssl-certificate-revocation-on-web-performance/) if you crave more technical detail!


==============================================================================

# Getting Fast and Staying There

> Getting fast is difficult, staying fast even more so. How do successful companies ensure that their site speed stays at peak?

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2020-10-27
Canonical: https://simonhearne.com/2020/getting-fast-staying-there/
Tags: WebPerf, opinion

---
If you've ever trained for a big sporting event: a marathon, bike race or similar, you'll know how tough it is to train to get to your peak performance. You'll also know that after the event you feel totally drained, your performance will not be the same for weeks while your body recovers. Training for a tough physical event is a project, a focused effort on getting to a goal. This approach does not work for ultra-marathoners or during the Tour de France, though: their performance must be maintained for days, weeks or months. These elite athletes treat their performance as a feature, something they devote everything to, rather than a project to be executed.

---

Site speed is seen as a technical problem: bits and bytes, microseconds, three-letter-acronyms. Solving site speed is seen as business success, though: faster experiences lead to happier users and better business outcomes.

Site speed is not clear-cut: we know that faster is better, but how much faster and how much better? This varies between sites and there can not be a single answer. If we look at security & accessibility - there are (conceptually) simple end-goals: AA accessibility rating or a successful penetration test. Feature development is similarly simple: did the feature meet the requirements, and did it improve business metrics?

<blockquote class="thought"><p>Site speed is an art: orchestrating technology to deliver a fluid and choreographed user experience.</p></blockquote>

The only way to truly succeed with a site speed project is to make speed a feature: projects end but features survive as long as they deliver value. Features are shared top-down from product to engineering, not the other way around. Features get investment, projects get budgets.

A common challenge when making performance a feature is determining the value that it will bring. Looking at the [relationship](/2020/value-of-site-speed/) between performance and user outcomes helps to build the business case, but it doesn't tell the whole story. Take for example the [YouTube Feather project](https://blog.chriszacharias.com/page-weight-matters): a super lightweight version of the YouTube web app. This project reduced the page weight of the video page from 1.2MB to under 100kB and dropped the number of requests by a factor of ten, resulting in a much faster video experience. The real user data, however, showed that page load time **increased** on the lighter, faster pages. The reason? Users who were previously unable to use the web app due to multi-minute load times could now watch videos! The distribution of traffic from low-end devices and countries with poor connectivity increased, pulling the average load times up.

<blockquote><p>...entire populations of people simply could not use YouTube because it took too long to see anything. Under Feather, despite it taking over two minutes to get to the first frame of video, watching a video actually became a real possibility.</p><aside class="attribution"><a href="https://blog.chriszacharias.com/page-weight-matters#os1i4hGykJcr7VJLS1K6fa:~:text=entire%20populations%20of%20people%20simply%20could,video%20actually%20became%20a%20real%20possibility">source</a></aside></blockquote>

As [Anna Debenham](https://twitter.com/anna_debenham) says: performance is a basic multiplier. Making your applications faster means they are more accessible to users in the long tail of the performance distribution. A performance feature may bring you significant increases in traffic from new users, those that had previously given up on your application due to bad prior experiences. This is an <a href="https://en.wikipedia.org/wiki/There_are_known_knowns#:~:text=Unknown%20unknowns%20(unexpected%20or%20unforeseeable,recognized%20but%20poorly%20understood%20phenomena.">unknown unknown</a>: you don't know how many users don't use your application!

<blockquote class="twitter-tweet" data-dnt="true"><p lang="en" dir="ltr">Performance is a basic multiplier. Want more people to use a feature? Make it faster. <br><br>I was trying to figure out why usage of an old feature had suddenly spiked – we hadn&#39;t changed anything about it. 🤔 But we made the page it was on load 4x faster. Duh. <a href="https://t.co/ZZnQh4pdwT">https://t.co/ZZnQh4pdwT</a></p>&mdash; Anna Debenham (@anna_debenham) <a href="https://twitter.com/anna_debenham/status/1184543959957291008?ref_src=twsrc%5Etfw">October 16, 2019</a></blockquote>

Site speed is technically a solved problem: ship small files, fast. Unfortunately this goes against the trend for JavaScript frameworks, rich media content, rapid feature development and of course the many third-party services which promise to offer increased revenue or insight.

<blockquote class="success"><p>Making speed a feature reduces <a href="https://en.wikipedia.org/wiki/Social_loafing">social loafing</a> by bringing focus to a common goal.</p></blockquote>

Front-end engineers in the most part know how to deliver a fast experience. QA knows when a release feels slower. Product knows that faster speeds result in better user outcomes (see my [blog post on that here](/2020/value-of-site-speed/)). In fact, **anyone** who has anything to do with the website should care about speed, and thus has some responsibility for it. This often creates a [diffusion of responsibility](https://en.wikipedia.org/wiki/Diffusion_of_responsibility#Division_of_labor): when everyone shares responsibility, no-one takes responsibility.

![](https://simonhearne.com/images/not-my-problem.jpg)

<blockquote class="success"><p>Successful speed goals are simple. My favourite: <strong>don't get slower</strong>.</p></blockquote>

Simple goals are the easiest to achieve. Setting a goal as simple as "don't get slower" may seem trivial, but it raises many questions: How fast are we? How do we know? How do we know when it changes? These goals should be shared across the business. In my experience successful site speed initiatives share the following common traits:

- Speed is understood and supported from the top (CTO has a speed dashboard)
- Speed data is socialised and celebrated (live dashboards for all teams, performance budgets)
- Engineering is empowered to focus on speed (tooling, speed week, speed as a feature)
- Speed is measured in CI & QA (CLI tooling)
- Product design accounts for speed (prioritising key visual elements, designing for data-saver mode)
- Speed is measured in-live, for real users on real devices (real user monitoring)

These can only be achieved when the whole business understands the value of speed, invests in it as a feature and celebrates it as a key to success. There's no point monitoring performance if no-one does anything with the data.

One way to affect positive change is through gamification. Folks love gold stars 🌟: creating performance budgets and goals can help drive an initial performance project to a successful outcome. Be careful with performance budgets though, they can encourage complacency once met and often focus on a small subset of static performance measures. I suggest finding the performance metrics which [best correlate](/2020/perf-metrics/) with user outcomes on your application and create a goal based on that, such as `75% of our mobile users achieve a largest contentful paint of less than 2.5 seconds on landing pages`. This metric also happily matches [Google's goals](/2020/core-web-vitals) for optimum SEO benefits.

<blockquote class="warning"><p>There's no point monitoring performance if no-one does anything with the data</p></blockquote>

Investing in site speed might be a leap of faith, but it is one which will always pay off. Start by [measuring what matters](/2020/value-of-site-speed/), then start tracking improvement.


==============================================================================

# What is the best page speed metric?

> Out of the hundreds of available performance metrics, which ones make sense for you?

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2020-10-26
Canonical: https://simonhearne.com/2020/perf-metrics/
Tags: WebPerf

---
The recent updates from Google on [Core Web Vitals](https://web.dev/vitals/) are a great effort to find universal speed metrics that reflect quality of user experience. Google's goals are closely aligned with yours: fast user experiences result in better business outcomes. Where your goals may differ from Google is that you know your users and your pages better. You know that marketing landing pages have to deliver the right content in the right position, that your product images are more important than social media icons and that your checkout flow must be lightning fast to maximise conversion rate.

How, then, do you determine what metrics to track on your site, and how to set performance goals that reflect optimum user outcomes? Real user monitoring gives us a unique insight here: tracking performance and business metrics in the same place gives us the ability to correlate them.

## What's out there?

There are hundreds of performance metrics that we can choose from. The [Navigation Timing API](https://developer.mozilla.org/en-US/docs/Web/API/Navigation_timing_API) defines a bunch, plus there are the Paint Timing API and Chrome-specific metrics like Largest Contentful Paint, synthetic measurements like [Speed Index](https://web.dev/speed-index/) and you can also define your own using the [User Timing API](https://developer.mozilla.org/en-US/docs/Web/API/User_Timing_API).

Google has put emphasis on two key page speed metrics lately: First Contentful Paint and Largest Contentful Paint. Unfortunately, these metrics are available only on a subset of browsers: FCP is in the [Paint Timing API](https://developer.mozilla.org/en-US/docs/Web/API/PerformancePaintTiming) and LCP is in its very own [Largest Contentful Paint API](https://developer.mozilla.org/en-US/docs/Web/API/LargestContentfulPaint).

![screenshot of mozilla developer network showing poor support for paint timing apis in non-chrome browsers](https://simonhearne.com/images/paint-timing-support.png)
*Paint timings are currently only supported in Chromium browsers (source on MDN)*

So whilst FCP and LCP are undoubtedly good performance metrics, relying on them might exclude a lot of devices from your benchmarks; most notably: iOS Safari.

## How to determine what metrics work for you

Every site is different, so some analysis is required to determine which metrics matter the most to your business outcomes. For one website I have correlated six key performance metrics with four key business outcomes. The performance data in this case is collected from the first page in the session, another analysis would need to be performed on mid-session page views.

Here you can play around with different combinations of performance and business metrics. Note that the three device type categories have very different results!

<div class="vega-chart loading" id="vis1" data-spec="/data/perf-metrics/perf-metrics.json">Loading chart...</div>

These charts all show that performance impacts user outcomes, but there are a few stand-out charts worth highlighting:

<script>
  function switchChart(metric,timer) {
    if (window.charts.length > 0) {
      let view = window.charts[0].view;
      let state = view.getState();
      state.signals['metric'] = metric;
      state.signals['timer'] = timer;
      view.setState(state);
    }
  }
</script>
<ol>
  <li>FCP correlates strongly with bounce rate (p=0.8) <button onclick="switchChart('bounce_rate','firstContentfulPaint')">Show chart</button></li>
  <li>Mobile &amp; Tablet LCP correlates strongly with conversion rate (p=-0.7) <button onclick="switchChart('conversion_rate','largestContentfulPaint')">Show chart</button></li>
  <li>Page Load Time correlates strongly with session length (p=-0.9) <button onclick="switchChart('session_length','timerstload')">Show chart</button></li>
</ol>

I encourage you to think about which user outcomes are most important to your website and build this analysis yourself. Determine the top three performance metrics which correlate most strongly with these outcomes then use this data to set your performance objectives. For example in this data we can deduce that the optimal landing page First Paint for mobile devices (optimising for bounce rate) is 1,200ms.

Landing page LCP has shown the strongest correlation to business outcomes (e.g. conversion) across multiple websites I have analysed, and FCP correlates best with session metrics like bounce rate and session length. Where available, these new paint timings give us the best proxy of user experience. In practice, first paint is more widely supported and is often very close to FCP. LCP can not be polyfilled, but a [user timing mark](https://www.html5rocks.com/en/tutorials/webperformance/usertiming/) can be devised for each page template to measure the loading of hero elements, this should provide a good proxy and can be tested against LCP in browsers that support it.

The key takeaway here is to correlate speed metrics with business outcomes, determine which are the best proxies for user experience on *your* pages and use those to set your speed goals. Read my post on [getting fast and staying there](/2020/getting-fast-staying-there/) for more detail on setting goals.


==============================================================================

# Four Charts That Prove the Value of Site Speed

> Site speed is critical to user experience and online success, here are four charts which prove the value!

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2020-10-22
Canonical: https://simonhearne.com/2020/value-of-site-speed/
Tags: WebPerf, UX

---
Site speed is an important but often-overlooked component in user experience. We know intuitively that slow experiences are unpleasant and make us more likely to leave a site, but building evidence of this effect has always been tricky.

The charts below were built with data collected using mPulse RUM.

<div class="chart-sticky-container" id="sticky-container">
  <div class="vega-chart loading chart-animate chart-sticky chart-active" id="vis" data-spec="/data/speed-value/speed-value.json">Loading chart...</div>
  <div class="stage" data-stage="0">
    <h2 id="slow-landing-high-bounce" class="no-link">Slow Landing Pages = High Bounce Rate</h2>
    <p>This chart shows the page speed distribution for an website by device type. The area is the distribution of sessions by average page speed and the lines in this case show the average bounce rate for sessions by the average page speed in the session.</p>
    <p>The majority of traffic is mobile, hence the higher blue distribution. This website is well optimised for mobile browsers resulting in a faster average page speed than on tablet or desktop experiences. The distributions show a peak of session average page speed at around 1.2 seconds for mobile and desktop, with tablet lagging slightly at around 1.5s. Note that the page speed here is the average of all page views in a session: including the landing page, browsing and the checkout flow where appropriate.</p>
    <blockquote class="error"><p>Users are more likely to leave the site immediately when the first page of a session is slow.</p></blockquote>
    <p>The bounce rate lines here show that there is a strong correlation between page speed and bounce rate. The initial drop is due to error pages and bots which result in high bounce rates and very fast pages. There is a settling around the peak of the distribution where we find the optimal bounce rates of 22% on mobile, 14% on desktop and 9% on tablet, these occur at the fastest page speeds of around one second on average!</p>
    <p>Users are more likely to leave the site immediately as their page speed gets worse. The mobile bounce rate increases from 22% to 40% as page speed increases from 1s to 2s! The median bounce rate for mobile is 41% at 2.7s, only 50% of sessions achieve this speed.</p>
    <blockquote class="warning"><p>Your slowest users will not even appear in your data - they will have left before the page loads <span style="font-style:normal">😱</span></p></blockquote>
  </div>
  <div class="stage" data-stage="1">
    <h2 id="slow-landing-low-conversion" class="no-link">Slow Pages = Low Conversion Rate</h2>
    <p>We see a similar pattern when looking at conversion rate. Conversion peaks where page speed is under a second, with conversion rate halving across all device types from experiences of one second to two seconds.</p>
    <blockquote class="error"><p>Users are less likely to convert if their average experience during a session is slow.</p></blockquote>
  </div>
  <div class="stage" data-stage="2">
    <h2 id="fast-pages-higher-revenue" class="no-link">Fast Pages = Higher Revenue</h2>
    <p>It's not just conversion that takes a hit when users have a slow experience. Average order value drops significantly in the first second here from $190 on desktop at 800ms to $140 at 3s. Faster experiences lead to bigger basket sizes!</p>
    <blockquote class="success"><p>Visitors spend more money when the average experience is faster.</p></blockquote>
  </div>
  <div class="stage" data-stage="3">
    <h2 id="fast-pages-longer-sessions" class="no-link">Fast Pages = More Pageviews</h2>
    <p>If you make money through advertising, session length is a key performance indicator. In this graph we see that session length correlates with page speed too!</p>
    <blockquote class="success"><p>Users visit more pages when the average experience is faster.</p></blockquote>
    <p>Here the average session length peaks on desktop at 20 pages per session at a one second average page speed. This drops dramatically to just 13 pages when users have an average 2.25s page speed.</p>
  </div>
</div>

<blockquote class="info small"><p>The charts presented here are generated from real data, the explicit values are examplar.</p></blockquote>

## Conclusions

Page speed is a critical factor in user experience, but most visitors are not even aware of it! Slower experiences are [reported to](https://calendar.perfplanet.com/2013/slow-pages-damage-perception/) feel 'basic', 'confusing' and 'clunky', rather than slow. Delivering fast user experiences gives your website or application the best chance at meeting your business objectives and creating positive user experiences.

<script>
  window.addEventListener('load',() => {
    const sticky_els = document.querySelectorAll("#sticky-container div[data-stage");
    const stages = [
      "bounce",
      "conversion",
      "revenue",
      "sessionlength"
    ];
    const chartID = "vis";
    const signalName = "stage";
    var currentStage = sticky_els[0].dataset.stage;
    var chartId = -1;
    var lastBottom = Infinity;
    sticky_els[0].classList.add('stage-active');

    const options = {
      rootMargin: '-150px',
      threshold: 0
    };

    const updateChart = stage => {
      chartElem = document.getElementById(chartID);
      chartElem.classList.add('active');
      setTimeout(function(){document.getElementById('vis').classList.remove('active')},250)
      let c = window.charts.filter(c => c.id === chartID);
      if (c.length > 0) {
        let chart = c[0].view;
        let state = chart.getState();
        state.signals[signalName] = stages[stage];
        chart.setState(state);
      }
    };

    const toggleActive = stage => {
      sticky_els.forEach(e => {
        if (e.dataset.stage == stage) {
          e.classList.add('stage-active');
        } else {
          e.classList.remove('stage-active');
        }
      })
    };

    const intersector = (entries,observer) => {
      entries = entries.filter(e => e.isIntersecting);
      if (entries.length >= 1) {
        entries = entries.sort((a,b)=>a.intersectionRatio < b.intersectionRatio);
        let entry = entries[0];
        if (entry.intersectionRect.top > 0) {
          let stage = entry.target.dataset.stage;
          if (stage && stage !== currentStage) {
            currentStage = stage;
            updateChart(stage);
            toggleActive(stage);
          }
        }
      }
    };
    tries = 0;
    obsInterval = setInterval(() => {
      if (window.charts.length > 0) {
        console.info("Charts ready, creating observer.");
        const observer = new IntersectionObserver(intersector,options);

        sticky_els.forEach(el => {
          observer.observe(el);
        });
        clearInterval(obsInterval);
      } else {
        tries++;
        if (tries > 20) {
          console.info("Charts not ready after 10 seconds, giving up 😧");
          clearInterval(obsInterval);
        } else {
          console.info("Charts not ready yet, delaying 500ms.");
        }
      }
    },500);
  });

</script>


==============================================================================

# How to Improve Core Web Vitals

> Core Web Vitals are three new performance metrics which will soon impact SEO ranking. Learn how to measure and improve them to maximise user experience!

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2020-10-19
Updated: 2021-01-11
Canonical: https://simonhearne.com/2020/core-web-vitals/
Tags: WebPerf, SEO

---
<blockquote class="warning"><p>10 Nov update: Google <a href="https://webmasters.googleblog.com/2020/11/timing-for-page-experience.html">has announced</a> that the Page Experience update will go live in May 2021!</p></blockquote>

## Introduction

Google [announced](https://webmasters.googleblog.com/2020/05/evaluating-page-experience.html) updates to the Page Experience ranking algorithm in May of 2020. This update includes two key highlights:

1. Three new performance metrics will be used as ranking signals for SEO
2. Top Stories no longer requires an AMPHTML page, and performance will become a ranking factor

The update is great news for site speed fans: Google is putting an even greater emphasis on the speed of user experiences, offering two SEO benefits to site owners who deliver the fast experiences that customers expect.

<blockquote class="info"><p>The Page Experience Update was delayed due to COVID-19. Changes to the ranking algorithm will happen in May 2021</p></blockquote>

To get a feel for your current performance against these metrics, try running a page through [PageSpeed Insights](https://developers.google.com/speed/pagespeed/insights/) or [web.dev/measure](https://web.dev/measure). These Google tools will give you a score for performance and let you know how a page measures up against the new performance goals.

![PageSpeed Insights field data showing LCP failing at 5.4s while FID and CLS pass](https://simonhearne.com/images/core-web-vitals/psi.jpg)
*PageSpeed Insights assesses a URL against the Core Web Vitals*

These performance metrics have been designed to be universal measures of page speed and to be good proxies for perceived user experience, they should be valid measures of how the experience feels for almost all web pages. That said, Google has stated that these metrics are being reviewed constantly and may change in the future.

<div class="resp-cols-3">
  <img src="/images/core-web-vitals/lcp_ux.svg" alt="Largest Contentful Paint icon" width="171" height="150" loading="lazy">
  <img src="/images/core-web-vitals/fid_ux.svg" alt="First Input Delay icon" width="171" height="150" loading="lazy">
  <img src="/images/core-web-vitals/cls_ux.svg" alt="Cumulative Layout Shift icon" width="171" height="150" loading="lazy">
</div>

**Largest Contentful Paint** is a measure of rendering performance, a good replacement for page load time or DOM content ready.

**First Input Delay** is a measure of how long it takes the browser to respond to discrete user input like clicks or taps.

**Cumulative Layout Shift** is a measure of how unstable the page was, a sum of all unexpected layout shifts in the page lifecycle.

The recommended values above are determined from the 75th percentile from real user experiences: at least 75% of your page views should exceed the **Good** value to pass this assessment!

<blockquote class="success"><p>Improving your core web vitals will potentially have a positive impact on SEO ranking, it will definitely have a positive impact on user experience!</p></blockquote>

## How Google Measures Speed

The three new performance metrics are relatively new to the world of web performance, as such they are currently only supported via API in Blink-based browsers (Chrome, Chrome on Android, Chromium Edge). The data that Google will use for the Page Experience update is taken from [Chrome UX Report (CrUX)](https://developers.google.com/web/tools/chrome-user-experience-report), a collection of anonymised performance statistics taken from real page loads in Chrome browsers around the world. CrUX measures all regular page loads, for both landing and mid-session pages, regardless of cache state. It does not measure soft navigations (route changes) within single page applications.

This means that soft navigations will potentially be penalised, with higher than expected CLS scores and missing or unfairly high LCP values. Unfortunately there is not yet a good solution to this problem. This makes it even more important to reduce unexpected layout shifts and optimise LCP as much as possible!

<blockquote class="twitter-tweet" data-dnt="true"><p lang="en" dir="ltr">So SPA landing pages will be penalised as CLS is summed throughout the whole session (and attributed to landing URL).<br>SPA soft navs will be further penalised as LCP is only calculated on hard navigations (so no data or slow data, depending on app).<br><br>Unless I&#39;m misunderstanding?</p>&mdash; Simon Hearne (@simonhearne) <a href="https://twitter.com/simonhearne/status/1316775506222682112?ref_src=twsrc%5Etfw">October 15, 2020</a></blockquote>

The update comes with a set of recommended speed goals, these are measured at the 75th percentile in the CrUX data so at least 75% of the measured experiences should meet or exceed these goals. The cache and session status are not available in the CrUX dataset so the 75th percentile value is taken from all page loads, cached or not.

PageSpeed Insights will give you three sets of performance metrics, where available:

- **Field** - the data from CrUX for the given URL
- **Origin summary** - the aggregate field data of all of your pages
- **Lab** - a test run from a Google server which applies throttling

If your page does not have enough traffic to produce the **Field** data you will only get the origin summary result. Recommendations and diagnostics in PageSpeed Insights are produced from the **Lab** test, performance metrics should be taken from the field results. The recommendations should give you a general idea of areas for improvement which will positively impact your web vital results.

![PageSpeed Insights diagnostics listing large network payloads and main-thread work as issues](https://simonhearne.com/images/core-web-vitals/pagespeed.png)
*PageSpeed Insights provides recommendations based on static analysis on lab tests*

## Optimising for Core Web Vitals

Let's talk about how to measure, diagnose and improve each of the three new metrics: LCP, FID and CLS, in turn. Remember that any optimisations to improve these metrics will almost certainly make your pages feel faster and improve user experience!

![Hand-drawn diagram of FCP, LCP, CLS and FID plotted against page load and CPU activity](https://simonhearne.com/images/core-web-vitals/cwv_hero.jpg)
*How (core) web vitals fit into the page lifecycle*

### Largest Contentful Paint (LCP)

LCP is a timing point measured when the largest above-the-fold element is painted to the screen. In most cases this will be either a hero image / video, or the main block of text on the page.

![WebPageTest filmstrip with a red dashed border marking the Largest Contentful Paint frame at 8.3s](https://simonhearne.com/images/core-web-vitals/wpt1.jpg)
*WebPageTest highlights LCP with a red dashed border on filmstrip frames*

LCP is a good alternative to "Page Load Time": it measures everything required for your site to render the key content on a page - DNS, TLS, HTML, CSS and blocking JavaScript, but does not include asynchronous scripts or lazy-loaded content.

LCP occurs only after three stages of page load have completed:

1. HTML delivered and parsed
2. Critical path assets downloaded, parsed and processed
3. LCP asset downloaded and rendered (image, video, web font etc.)

![WebPageTest waterfall chart marking LCP after dozens of requests have already loaded](https://simonhearne.com/images/core-web-vitals/wpt2.png)
*Abridged WebPageTest waterfall showing that over 50 objects load before LCP on the earlier filmstrip (LCP line added)*

Delivering fast documents is one of the most important things you can do to improve page speed, if you can reduce this by 100ms then every other duration metric will get 100ms faster! Fast document delivery comes down to a few key optimisations:

- **DNS** - DNS is the address book of the internet. Optimise DNS by increasing TTL and using a globally distributed DNS service ([See this blog post for more detail](https://kinsta.com/blog/reduce-dns-lookups/)).
- **TCP** - The constraining factor in establishing a TCP connection is (normally) the round-trip time between user and server. Use a content delivery network to bring this as low as possible.
- **TLS** - Secure websites require one or more additional round-trips to create a secure connection. Ensure you have OCSP stapling enabled on the site's certificate (maybe even downgrade from EV to OV to achieve this!) and that your server or CDN is configured for TLSv1.3 support (see [Is TLS Fast Yet?](https://istlsfastyet.com/)).
- **TTFB** - The time to first byte for your website is constrained by how fast the server can create the response. If possible, this should be cached in a CDN or reverse proxy. If HTML caching is not possible (for example if there is personalisation in the page), ensure that your server environment is able to deliver pages within 100ms.
- **HTML** - It may sound obvious, but the size and structure of your HTML document is critical to performance. Ensure that the HTML document is compressed and under 50kB over the network. Pay attention to the `<head>` of the document to ensure that the `<title>` is first and that there are no blocking third-party `<script>` tags here.

Once the HTML is downloaded, the browser parses the document line-by-line to find resources in the critical path. CSS and JavaScript in the head are given very high priority, then images in the body are downloaded in the order they appear. If the browser parser sees a blocking script tag (i.e. no `async` or `defer` attribute) it will stop everything it is doing while fetching, parsing and executing that script. As such, scripts should always be either async, when order of execution is not important, or deferred when order of execution is important. It also makes sense to review inline scripts to reduce their impact.

![Diagram comparing script parsing and execution timing for default, async and defer script tags](https://simonhearne.com/images/core-web-vitals/async_defer.png)
*Defer scripts where possible. More details.*

Where possible, split your JavaScript bundle by page and by modern JavaScript support. This will allow you to send the smallest possible bundle and use modern technologies where supported. Note that `module` has a defer behaviour by default:

```html
<script type="module" src="/app-homepage.esm.js"></script>
<script nomodule src="/app-homepage.js"></script>
```

The next step in the critical path is loading stylesheets: without CSS the browser does not know how to render the page so it will block render on any `<link rel="stylesheet">` tags. Ensure that CSS is bundled by page to reduce unnecessary weight, you can fetch and cache a full stylesheet later in the page lifecycle using the media="none" hack (ensuring that this file won't cause a layout shift!):

```html
<link rel="stylesheet"
      href="lazy.css"
      media="none" onload="this.media='all'" >
```

LCP is measured independent of cache state, so ensure your static assets (JavaScript, CSS, images and fonts) can be cached in the browser for at least am hour with a response header like `cache-control: max-age: 3600`. Also make sure your text assets are compressed with gzip or brotli!

A common issue I see, especially on ecommerce sites, is a large number of non-critical images that load early in the page, such as in a mega menu definition. Native lazy-loading is a great technique to optimise LCP by reducing bandwidth contention during page load. The `loading` attribute is not yet supported in Safari, but it [is in WebKit](https://bugs.webkit.org/show_bug.cgi?id=196698) and currently available in iOS Safari behind a flag, so we can expect general support soon! It *is* supported in all browsers which send data to CrUX, so implement now to benefit the data which will drive the Page Experience SEO update.

```html
<img src="menu-img.jpg" alt="..."
      width="200" height="200"
      loading="lazy" >

<img src="hero-img.jpg" alt="..."
      width="1024" height="600"
      loading="eager" >
```

Ensure your hero element has a `loading='eager'` attribute and that images below the fold or hidden by default have a `loading='lazy'` attribute. This simple optimisation allows the browser to give priority to downloading important assets sooner, improving LCP and user experience. Read more on [web.dev](https://web.dev/browser-level-image-lazy-loading/).

It should go without saying, but if your hero element is an image or a video then it should be delivered in the most optimised format for the browser. This may mean that a third-party service like Cloudinary or Akamai Image and Video Manager is a good option to dynamically optimise your media content.

The hero element should not be JavaScript-bound, so image carousels and embedded video players are best replaced with static images and native `<video>` elements with an appropriate `poster` image attribute. If you have a video poster, make sure that it is optimised and use a valid `preload` attribute like `none` or `metadata` to reduce bandwidth contention during page load. Read more about these optimisations on [Mozilla Developer Network.](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/video)

<blockquote class="info"><p>A rule of thumb is that a hero image should not be larger than 200kB.</p></blockquote>

### First Input Delay (FID)

FID is a measure of how long the browser was busy with other tasks before it could react to a discrete user input event like a tap or click. This is an indication of how responsive the UI feels to a user, and how busy the CPU is with JavaScript processing.

The only consistent way to improve FID without degrading the user experience is to reduce the time spent executing JavaScript, both in the page load and during the page lifecycle. A simple way to game this metric would be to hide your content (perhaps behind a loading screen / spinner) until your JavaScript has completed executing, so your visitors don't try to interact before your app is completely ready. This  will most probably negatively impact your LCP and CLS metrics though, so tread with caution!

Assuming we're trying to improve FID **and** improve visual performance, we only have a few options:

- Delay or remove third-parties
- Defer non-critical scripts
- Improve JS performance

The first task is to run a performance trace on a key page to see where the main thread time is consumed. Any big chunks are cause for concern and should be investigated.

![Chrome Performance panel summary donut: 1400ms scripting out of 2741ms total, with Total Blocking Time reported as 638ms](https://simonhearne.com/images/core-web-vitals/tbt1.png)
*Performance summary shows where time was spent, and total blocking time*

Focus on **long tasks**, these are singular blocks of execution which are most likely to cause high first input delays. Click on a long task to reveal the call stack in the "Bottom-Up" tab below:

![Chrome Performance panel with a long task selected, grouped by URL to show which script owns the time](https://simonhearne.com/images/core-web-vitals/tbt2.png)
*Selecting a long task shows what code is responsible*

You can group the view by URL to quickly determine which scripts are causing the issues, in this case the `cookielaw` banner SDK is responsible for 200ms of 'self' time, meaning that the time was spent in this script rather than due to calls made by it. This could be due to the script itself being slow, or the implementation being poor resulting in multiple duplicated function calls. Time to start digging!

Another potential cause for high FID is the hydration process. If you have a server-side rendered single page application, the hydration process adds the client-side application functionality to your static HTML. Whilst this process results in a faster first render (and LCP), it may delay interactivity and increase the potential for delay when a user tries to interact. If you are using SSR with client-side hydration [Addy](https://twitter.com/addyosmani) has a [brief write-up](https://addyosmani.com/blog/rehydration/) of the issue with a few resources on how to manage this performance gap.

### Cumulative Layout Shift (CLS)

CLS is a measure of how stable the UI feels as the user loads and interacts with a page. It is a sum of unexpected layout shifts during the page lifecycle, like when an advertisement banner loads and pushes the main content of the page around.

<figure>
  <a href="/images/core-web-vitals/cls1.gif">
  <img
    src="/images/core-web-vitals/cls1.gif"
    alt="animated gif of layout shift causing user to press the wrong button"
    loading = "lazy"
    width = "400" height = "310"
    style="max-width:400px"
  ></a>
  <figcaption>Layout shifts make for bad web experiences</figcaption>
</figure>

Layout shift scores are derived from the impact the shift has on the viewport: a product of how much of the viewport changes and how far the element moves. A perfect cumulative score is zero, 0.1 is good. See the [official documentation](https://web.dev/cls/) for more details.

Unexpected layout shifts mean that they do not occur immediately after a discrete user interaction, so it will likely have a negative impact on user experience.

CLS is measured by CrUX throughout the lifecycle of a page, from the start of the navigation right through to when the user leaves the page. All unexpected layout shifts are summed throughout this time and the total score is used for the measurement; this makes it a tricky metric to measure in a lab setting. This also means that tools such as [WebPageTest](https://www.webpagetest.org/) and [mPulse](https://www.akamai.com/uk/en/products/performance/mpulse-real-user-monitoring.jsp) will report the best-case scenario for CLS, they will only collect this data whilst the page is loading and ignore further shifts such as those that occur during scroll.

Optimising CLS during page load is reasonably simple, we just need to prevent layout shifts! There are multiple causes for layout shifts, so let's look at some of the causes and how to prevent them occuring.

- **Web fonts** - match your web font character and line spacing to the fallback font
- **Ads** - pre-allocate layout slots for ads, use a fallback image if ads fail or are blocked
- **Late-loading CSS** - ensure layout-critical CSS is in the critical path
- **Images** - always add a width and height attribute for images, so the browser can allocate space before the image downloads
- **Dynamic content** - where possible, pre-allocate layout space for dynamic elements

Chrome's developer tools now has a handy method to identify what elements caused a layout shift, and how they contributed to your cumulative layout shift score

![Chrome Performance panel highlighting a Layout Shift event with a 0.01774 score](https://simonhearne.com/images/core-web-vitals/layoutshift1.png)
*Layout shifts are highlighted in the performance profile*

If you hover on the layout shift in the experience track, Chrome will highlight the element on the page. Note that the details show you where the element moved from and to.

![Hovering a layout shift highlights the cookie-consent banner element that moved on the page](https://simonhearne.com/images/core-web-vitals/layoutshift2.png)
*Hover on a shift to highlight the offending element*

I recommend setting your network and CPU throttling options in the Performance tab, this makes it more likely that you will catch the layout shifts that happen in the wild due to network race conditions.

![Chrome DevTools Performance settings with CPU throttled 4x and network set to Fast 3G](https://simonhearne.com/images/core-web-vitals/layoutshift3.png)
*Slow the profile down to make issues more obvious*

Once you've identified the elements that are causing layout shifts, it's time to determine how to reduce or prevent them.

<figure>
  <a href="/images/core-web-vitals/layoutshift3.gif">
  <img
    src = "/images/core-web-vitals/layoutshift3.gif"
    alt="animated gif showing layout shifts on CNN article headline"
    loading = "lazy"
    width = "1038" height = "356"
    style="max-width:700px"
  ></a>
  <figcaption>Layout shifts on a CNN headline</figcaption>
</figure>

In this example from CNN, the headline element shifts multiple times during page load. The major shift is due to the author's image loading and pushing the byline to the right. Another shift is caused by the web font for the headline changing the headline layout. The image shift can be resolved simply by adding width and height attributes to the `<img>` element!

For the text element shift, the font-face has a `font-display:swap` descriptor (which is a [good thing](https://developers.google.com/web/updates/2016/02/font-display)!) but the fallback font has a different character width to the web font. When the web font is loaded, the text element shifts as the characters are more condensed and the element is sized to fit the contents. In this case it is a small layout shift but the impact on body text can be much greater, as text overflow may cause the body element to resize. It *might* be possible to fix this by matching the system font to the web font, using [font-face descriptors](https://www.chromestatus.com/feature/5651198621253632), but the most robust way to avoid font-based layout shifts is to preload the webfont and use `font-display: optional`. This combination gives the browser the best chance to have the webfont(s) available when it needs them, but lets the browser use the fallback font in case the webfont is not available. This ensures that there will be no layout shift due to fonts, both for the initial page load and subsequent page loads which will use the now cached webfont.

## Other Vitals

Whilst the initial Page Experience update defines LCP, FID and CLS as the *core* web vitals, these may change over time! There are a host of other metrics which provide additional value and are worth tracking & optimising. Below are some more key metrics that you might want to track and improve.

<div class="resp-cols-3">
  <img src="/images/core-web-vitals/fcp_ux.svg" alt="First Contentful Paint icon" width="269" height="235" loading="lazy">
  <img src="/images/core-web-vitals/tti_ux.svg" alt="Time to Interactive icon" width="269" height="235" loading="lazy">
  <img src="/images/core-web-vitals/tbt_ux.svg" alt="Total Blocking Time icon" width="269" height="235" loading="lazy">
</div>

### First Contentful Paint (FCP)

Whilst LCP measures the largest paint on the screen, FCP measures when the first paint occurs. This is an important metric for a number of reasons, not least that this is the first time a user knows that their navigation is actually working. This timing point will also correlate well with when the user's browser switches context from the previous page.

FCP is often identical to first paint, or very shortly after. First paint includes non-visible elements in the calculation, whereas FCP only measures content which is visible to the user. In this case, the site logo and header:

![Filmstrip and timeline marking First Paint and First Contentful Paint occurring together](https://simonhearne.com/images/core-web-vitals/firstpaint.png)
*First Paint and First Contentful Paint often occur together*

### Time to Interactive (TTI)

Time to interactive (TTI) is an approximation of when the page will feel interactive if a user tries to interact. This timing point is measured between the First Contentful Paint and when main thread has been free of long tasks for at least five seconds. Getting this metric as low as possible will ensure that your users have a great experience when they try to interact with your pages. Use TTI with TBT to get an overall picture of how busy the browser is when loading your pages.

### Total Blocking Time (TBT)

Total blocking time (TBT) is a measure of how busy the browser's CPU was while loading the page, measured as the sum of long tasks (less 50ms each) between FCP and TTI. Reducing this time will likely improve user experience and may help improve perceived performance too. Look for large long tasks in your JavaScript profiles and try to reduce, remove or delay them.

## How to track performance

Google's tools such as PageSpeed Insights, Lighthouse and [web.dev](http://web.dev) will all give you a measurement of your core web vitals. The 'field' data has some limitations though: the data is collected only from Google Chrome users who are opted in to anonymous usage statistics collection, and is aggregated monthly with a week or two delay. So if it's the first week of October, you may only have data from the whole month of August available.

If you want to track web vitals more closely, look into a real user measurement solution like [Akamai mPulse](https://www.akamai.com/uk/en/products/performance/mpulse-real-user-monitoring.jsp). RUM tools can collect performance data from every browser that supports them and give you real time insight into how your performance is tracking. You can also quickly spot issues with specific pages or devices, making the data actionable.

## Conclusion & Actions

The proposed page experience update will most likely occur in mid-2021. This will have an impact on SEO ranking as well as eligibility criteria for Top Stories in Google SERPs. Google has proposed three new performance metrics which will be used as signals for this SEO update, with selection based on their adjacency to user-perceived performance and the ability to collect the data in the field.

Optimising for these performance metrics, as measured by Google in CrUX, might have a positive impact on ranking in the future, but will certainly have a positive impact on user experience. [We know](https://wpostats.com/) that faster experiences lead to lower bounce rate, higher session length, better satisfaction scores, increased conversion, increased SEO traffic, increased revenue... So why wait?!

The great thing about performance optimisations is that many can be achieved with relatively minor tweaks to code, so go ahead and test your key pages, spot room for improvement and make positive change.


==============================================================================

# The Psychology of Speed

> We intuitively know that faster is better, but what's the psychology and physiology behind it?

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2020-10-15
Canonical: https://simonhearne.com/2020/psychology-of-speed/
Tags: WebPerf, video, talk
Video: https://www.youtube.com/watch?v=7i_yQyHdxUo
Slides: https://simonhearne.com/presentations/psych-speed/#/

---
Feeling fast and being fast are different problems. This talk is about the gap between them.

## About the talk

Performance work starts with a stopwatch, but the number on the stopwatch and the speed a user perceives are not the same quantity. The difference is psychological, it is well studied, and it is largely under your control as a designer.

This talk takes the research on waiting — why an occupied wait feels shorter, why uncertainty hurts more than duration — and maps it onto the concrete decisions you make when building an interface.

## What's covered

- **What the business data shows.** The BBC lost 10% of additional users for every extra second of load time; Pinterest turned a 40% load-time improvement into 15% more SEO traffic and 15% more conversions; the Financial Times saw engagement rise 30%.
- **Three rules of waiting.** Unoccupied time feels longer than occupied time, anxiety stretches a wait, and an unknown wait feels longest of all.
- **Miller's thresholds.** Feedback inside 100ms, updates every second, actions completed within three seconds — published in 1968, and still the budget.
- **Targets worth holding.** Cumulative Layout Shift at 0, First Input Delay between 50 and 100ms, First Contentful Paint at one second, Largest Contentful Paint at three, Time to Interactive at three and a half.
- **Designing the wait.** Occupy it by showing something early and prioritising content; reduce anxiety with visual stability, optimistic UI and feedback; remove uncertainty by anchoring the user, showing progress and explaining the delay.
- **Measuring perception.** RUM over synthetic, custom User Timing marks for the moments that matter to you, and full capture rather than a sample.


==============================================================================

# Giving Great (Remote) Software Demos

> Giving great software demos is both science and art. Here are some tips on making the most out of every demo opportunity, even when you're stuck at home!

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2020-06-15
Canonical: https://simonhearne.com/2020/great-demos/
Tags: work

---
<style>
  .key {
    border-radius: 5px;
    border: 1px solid var(--border);
    padding: 1px 2px;
    font-size: var(--font-size-small);
    font-family: var(--font-family-monospace);
    background-color: var(--surface);
    color: var(--text);
  }
</style>

## Introduction

I've been in technical pre-sales for about ten years, and software demos have been a part of my role since day one. I'm still working on perfecting my demos, but I do have some tips on getting the best results from the short time you have with your clients.

Key takeaways for those in a hurry:

- demo to the *people* in the meeting
- no demo should be the same as another
- start with the "aha!" moment and work back
- only show the features that matter to the customer
- always prepare for a strong five minute open and close

Technical sales / solution engineering / pre-sales is (in my humble opinion) one of the best roles you can have in tech. It combines technical skill, creativity and sales skills and is critical to the success of complex software sales.

Demonstrating software is one of the most fun and high-impact tasks for a pre-sales engineer. You have the opportunity to use your expertize and experience to deliver a powerful, customized demo which could be the difference between a sale and a lost opportunity!

It is easy in life to settle into routines, and we are all guilty of giving the same demo a few times. Spending a little time to think more about your customer and your demo can result in a more engaging, more enjoyable and more successful meeting. In the following sections we will walk through some key steps to help achieve this.

## Preparing for the Demo

No matter what demo you're giving, preparation is key to success. Some of this can feel a bit nosey or intrusive, just remember that the goal is to give you background information which can help direct the demo.

### Know the Customer

Your sales representative will have done some work here already, but there are a few steps that you should take as a pre-sales engineer, and it only takes a few minutes to get a feel for the company:

- Do they have a tech blog? Take a look at what they've been publishing lately.
- Are they a public company? Take a look at the CEO's message in the latest annual report.
- What is the output of their marketing? Take a look at Twitter & LinkedIn to see what they are talking about.

### Know the Audience

Note that this is separate from knowing the customer, this is about getting to know the individuals you will be working with. You should have a meeting agenda with the email addresses of all attendees, hopefully at least a few days before the demo! Book half an hour in your diary before the meeting to do some research on Twitter, LinkedIn, blogs etc.

Researching the attendees is not meant to be creepy, and it is unlikely that you would (or should) use anything that you learn explicitly in the call. Understanding the background of the attendees will help you spot opportunities, focus the demo and prepare for obvious objections.

### Know the Technology

This entirely depends on the solution that you are demoing, but it is important to know the technology in place. If your solution is web-based, use a tool like [BuiltWith](https://builtwith.com/) to see what is being used already. This allows you to prepare for objections, technical complications and good synergies with other products.

You will need to do a bit more preparation if you are in a competitive position with an incumbent supplier. Make sure you know who you are competing with and your relevant strengths and weaknesses. Be prepared to [handle objections](#handling-objections) based on the competitive solution(s).

## Know your Demo

There are three software demos you are likely to give, in order of the maturity of the opportunity:

**The Qualification** - "Ooh shiny!"

**The Confirmation** - "This solves all of my problems!"

**The Proof** - "Now I've seen it with my own data, here's my credit card!"

It is important to be mindful of which stage the **customer audience** is at, and give the correct demo accordingly. Are you giving the same demo for all three stages? No worries, most of us do. Differentiating between these, even it's only the first few screens that change, can turn a low-engagement demo into an "I can spare another thirty minutes for this, please carry on" demo. You could be months into an engagement with many demos delivered, but it's likely that you need to give a qualification demo if you have to present to someone new to the company or the solution.

Whatever the demo, ensure that you follow tell-show-tell for each feature you include:

- Tell them what you're going to show them and what problem it solves
- Show them how your solution solves their problem
- Tell them what you showed them!

### The Qualification Demo

You have two goals in the qualification demo:

1. Create enough interest in the customer to progress to a proof of concept (or close the deal, if you're lucky!)
2. Qualify that your software actually meets the customer's requirements

We often forget the second point in our enthusiasm for our solutions. Both sides should be evaluating whether this is a good fit, just like in a job interview.

It is tempting to jump straight into talking about *you*: your company, your product. It is critical to understand the customer's pain before showing any software screens at all. Push back if you have been invited to a meeting by a salesperson without any qualification or background, your management and leadership should definitely support you here. It is the salesperson's job to qualify opportunities and share that information with you.

It is quite likely that at some point you will find yourself in a meeting with a new customer without any background information. This is not your fault, and can be recovered: start the meeting with questions. This is actually an opportunity to build a trusted advisor role earlier in the relationship than normal, although obviously not ideal overall. Stick to **open** questions about their business, with a leaning towards the problems that your software solves:

- "How do you solve challenge X at the moment?"
- "How successful do you feel that is?"
- "How important is X to the business?"
- "If you could change anything about X, what would it be?"

In fact, do this even if you do have a good qualification document from the salesperson. Why wouldn't you ask these questions, two heads are better than one and there's a good chance that the answers you receive may give you more information. These questions have three effects on the rest of the meeting:

- It demonstrates that you care about the customer
- It gives the customer an opportunity to discuss their challenges with a technical advisor
- It helps to build rapport and establish two-way communication

All of this helps to avoid the dreaded *radio broadcast demo*, where the customer sits in near-silence for an hour or more. This can happen remote or in-person, and is awkward for both sides!

Once you have built rapport and understood or confirmed the qualification, it's time to demo! Remember the goals for a qualification demo are two-fold: show the customer that it solves their problem, and qualify your software does actually solve their problem.

Think back to your last qualification demo where the customer had an "aha!" moment. What was the first screen of your software you showed? Now think to the "aha!" moment, what screen were you showing then?

We often have a number of linear demo flows we follow, building "bridges" from customer challenge to software solution. In fact, the cover for the (very good) pre-sales book [Demonstrating to Win!](https://www.amazon.com/Demonstrating-Win-Indispensable-Complex-Products/dp/0615477097/?tag=webnin-21) is a bridge!

Whilst this is great in a software training session or a late stage demo, it can be cumbersome in the qualification demo. Remember the screen from the "aha!" moment? What was it showing? Show that first. Use this as the conversation starter, and build a flow from here to key features based on the feedback from the audience. It can feel awkward at first, but try giving a five minute walk-through of a single screen, no clicking around! Once you have talked through this screen, confirm that the customer understands what you've shown them and ask if they have any questions. Leave the silence to hang for 20 - 30s, this allows people time to think and un-mute. It's important to get feedback at this point, if no-one asks any questions, turn it back on the audience and ask a leading question to a friendly in the audience by name. For example, you might ask "Jane, is this what you were expecting to see? What do you think would be most important to see next?"

During the demo, take note of where the audience appears most engaged or asks the most questions. This will help to build your closing summary, and should help you build your business case.

### The Confirmation Demo

The confirmation demo may be combined with the qualification demo, depending on the complexity of your software and the maturity of the opportunity.

Think about your goals for a confirmation demo. If you offer free trials or proofs of concept, the goal of this demo is to *skip the free trial*. If you can sell your software without a trial you will save time, cost and Salesforce administration! Other CRMs are available 🙂

I would expect an opportunity to be in a reasonably mature stage by the time you are preparing a confirmation demo. This means that this demo should prove to the customer that you know what they need, and that your software solves those requirements. This means that you should be investing the majority of your time on this demo, and you should have the following prepared:

- Attendees and roles for the demo
- Personal needs for each attendee
- A list of key requirements (perhaps from an RFP or procurement)
- The name of the decision maker

This is a lot of information, and a lot of work to prepare. The majority of this should be provided by your sales representative, but you can (and should) reach out to folks who will be on the demo to ask them questions about what they would like to see, what problems they hope that you can solve and what their role is in the purchasing decision.

A demo flow comes quite naturally once you have this information prepared. Don't start your demo with a demo, but a summary of this information. For example you might say:

> Thank you everyone for speaking with me over the past few days. It seems to me that the critical requirement is xxx, that the selected solution should also provide yyy. Does that sound right?

Then let the silence hang for 10 - 20 seconds. It will feel uncomfortable, but it is really important to get confirmation at this point. You will also find out who is your 'friendly' from the customer at this point, make a mental note of their first name and use it once or twice in the demo to ask for confirmation. This makes you part of the team, and helps to break down the sales / prospect barrier.

Once you have confirmation of the requirements, now we can show a single screenshot / screen of the solution. It might be worth preparing a few, in case your assumptions above are wrong, but the key is to have a single screenshot which showcases **your solution solving the critical requirement.**

Talk through this screenshot and discuss how it solves their requirement. Once you are satisfied that you have spoken to their requirement, ask them for confirmation. If they agree, ask them what else they would like to see. Once you have noted everything that the audience mentions, quickly build a demo flow based on their requirements. Then for each requirement, take the shortest possible route in your software to show how you solve the requirement. If your software requirement cannot solve a requirement, start with a very brief objection handling statement to qualify the requirement:

> "I hadn't heard that requirement before, is that something we need to cover off now or can we take it offline?"

Once you have gone through a requirement, confirm with the requestor that you have satisfied them. Asking this question can come off as combative, especially if it is phrased such as "Does that answer your question" or "Is that what you wanted to see". It is possible to take the edge off the question by adding humility:

> "I hope that answers your question", "it seems to me that should meet your requirements, would you agree".

### The Proof

This is my favorite demo. It's normally the culmination of weeks or months of work, has lots of pressure and lots of attendees! It is your opportunity to show off how great your solution is, and how well you can meet the customer's requirements.

By this time in the sales process, you should know exactly what the customer needs to see in order to confirm that your solution is the right fit for their challenges, it makes sense then that your demo shows these features and nothing else. If you only need to show 10% of your features to show the deal, only show those 10% of features! There is strength in simplicity, and value in closing a meeting early if you can.

While I suggest that you should keep the demo as short as possible, it is critical to make sure that the customer knows what they're about to see and how it solves their problem. People have bad memory, so you'll also need to recap it at the end. A good structure for this demo might look like:

- Introductions
  - Include the names and roles of everyone in the meeting. If you don't know someone, politely ask who they are and what their interest is in the solution: "Hi Joe Blogs, I don't believe we have met before? What would you like to see in today's demo?"
- Background of all of the customer's relevant challenges
  - Specific challenges
  - Solution requirements
  - Potential value of solving the challenges
- Challenge 1
  - More background of the customer's challenge ("you told me that...")
  - Specific capabilities of your solution which would solve this challenge
  - Value of solving the challenge (hours saved, $ gained, insight provided)
  - Shortest path to demo the capability
  - Confirmation - "would you agree that this solves your challenge?"
- Challenge 2, 3, 4...
- Summary
  - Confirmation of challenges - "we heard from you that..."
  - Confirmation of solutions - "we have you shown that..."
  - Confirmation of understanding - "is there anything I have missed or could make more clear?"
  - Confirmation of intent - "based on this information, it seems that our solution meets your requirements"

I've seen a lot of these demos start with a few slides about the pitching company's history, position in the market, how amazing they are etc. Why? This meeting is about your customer, not you. Leave that out, or have some slides in your pocket in case you are challenged on your ability to deliver. Otherwise, keep *everything* in the demo about your customer.

## Pitfalls

With a customer-driven demo, there are a few pitfalls to avoid. It becomes much easier to handle these once you know how to recognize them.

### The rabbit hole

![Illustration of Alice in Wonderland falling past playing cards, a clock and a mirror down the rabbit hole](https://simonhearne.com/images/great-demos/rabbit-hole.jpg)
*Alice goes down the rabbit hole to Wonderland*

The rabbit hole is when we spend a disproportionate amount of time on a small feature of the solution or customer challenge. This can come from either the demonstrator or the customer, more commonly from a confused or belligerent customer. It's also possible if you have a favorite or 'safe' feature which you are comfortable spending time in, or if you have a particular affinity for a specific feature.

The escape route is relatively simple, once you have identified that you're in the rabbit hole. If you need to leave a feature, just ask a closed leading question of the audience: "I'm conscious that we have a limited amount of time to cover a lot of requirements, is it ok if we move on and come back if we have time at the end?".

### The drunkards walk

![Illustration of a drunkard weaving an erratic dashed path back and forth around a lamp post](https://simonhearne.com/images/great-demos/random-walk.png)
**

Often a symptom of a lack of information before a demo, the drunkards walk is a seemingly random jump from feature to feature. This will end up confusing the customer and making your solution seem more complex than it is.

When giving the demo, don't be tempted away from a feature until you have given a summary. If a customer asks about something that will take you away from your current flow, take a note and let the customer know you'll get to it:

> "Great question! That is covered in another part of the solution, I've taken a note to cover that when we get to it. Please remind me if I miss it out!"

### The harbour tour

![Weighted graph diagram of 21 numbered nodes connected by 31 edges, like a travelling salesman route map](https://simonhearne.com/images/great-demos/tsp.jpg)
**

"This is what you've been training for!" you think as you prepare your demo, a checklist of the full feature set of your software ready in front of you.

It is incredibly tempting to try to show every single feature of your product in a demo. The logic is pretty solid: if you show every feature then the customer will understand how powerful the solution is! Unfortunately it doesn't work out that way, a harbour tour of your software - moving from feature to feature - will likely have the opposite effect. It is often hard to tell a story with your software during the harbour tour, making the transition from one feature to the next jarring for the customer. Demonstrating every feature can also lead to quite a dry demo, and may even give the impression that your solution is overly complex, or does more than the customer could ever need.

The harbour tour is differentiated from the runaway train and the drunkards walk as it is, in most respects, a great demo! The challenge here is knowing when to stop. Remember to focus on the minimum viable demo, what are the key features which will close the deal? Let the customer tell you if you've missed anything that is important to them, rather than throwing everything against the wall and seeing what sticks.

### The runaway train

![Motion-blurred photo of an orange freight locomotive speeding down the tracks](https://simonhearne.com/images/great-demos/runaway_train.jpg)
*The runaway train doesn't stop until it gets where it's going*

Probably the most common pitfall in technical demos, the runaway train is the well-rehearsed step by step walk-through that has been delivered hundreds of times before, which stops for nothing! This is generally delivered by veterans who have honed their demos over time, or newbies who are following a safe demo path (and likely the product training). A robotic demo routine can be both dull and fragile, customer questions can throw you off your tracks and disrupt your flow for the rest of the demo.

The solution for the runaway train pitfall is simple, let the customer drive the demo! Ask confirmation questions after each feature and get feedback as often as possible. It can be hard to break this pattern, so try practicing with a colleague: ask them to be the customer and 'drive' the demo, then follow their lead. Once you've done this a few times it will feel even more natural than your well-rehearsed demos!

### The endless objection

![Photo of TV detective Columbo with the caption \"Just one more thing....\"](https://simonhearne.com/images/great-demos/columbo.jpg)
**

Some customers are just there to make your life difficult! If you have a member of the audience who constantly throws objections (or will not let an objection go) there are two potential solutions.

If the objections are valid and following a theme, you need to [address it](#handling-objections) as soon as you can to prevent it derailing your demo. You can do this by acknowledging the objection and defer the responsibility to another time: "great question, I think it would be best for me to follow up by email with a complete answer. Would that be ok for everyone?". By using the word "everyone" in the confirmation question you put the pressure of the objection back on the person raising it, it is their decision to use everyone's time on this question. It also leaves an opening for someone else on the call to confirm the objection, in that case it really is important to handle it in the demo!

## Screen-sharing Hygiene

You are likely to be sharing your screen in either remote or in-person demos. Whilst the technology to do this is generally robust, it can open you up to some unnerving possibilities.

<figure>
<video autoplay muted loop playsinline poster="/images/great-demos/dnd.png" style="max-width:500px" preload="metadata">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/w_500,q_auto/v1612776931/videos/dnd.webm" type="video/webm">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/w_500,q_auto/v1612776931/videos/dnd.mp4" type="video/mp4">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/w_500,q_auto/v1612776931/videos/dnd.mov" type="video/mov">
</video>
<figcaption>Enable Do Not Disturb mode on MacOS from the notification center</figcaption>
</figure>

In all cases, it makes sense to mute your notifications. On MacOS this is as simple as <span class="key">⌥</span> + click on the notification center icon. Windows makes this a little more tricky - you need to head to control panel to disable notifications.

![Windows 10 Notifications & actions settings screen with the notifications toggle switched off](https://simonhearne.com/images/great-demos/windows_dnd.png)
*Disable Windows notifications in Notifications Settings*

One of the most cringe-inducing moments of a demo is the "can you see my screen?" awkward beginning. Save yourself some time and frustration by getting prepared to make this silky smooth. If the screen sharing software is not your normal software, e.g. if the customer has shared theirs, make sure you practice sharing your screen before the meeting. If you are on MacOS Catalina, you will be familiar with the screen sharing permissions issue. This new permissions setting means that new apps are not allowed to record your screen or audio until you allow them, and allowing it might require restarting the app! You will know how frustrating this is if you've had this happen to you on a call with a customer. Thankfully there are a couple of ways to avoid this nightmare.

The first trick is to avoid the native applications! Almost every video conferencing product supports joining via the web, in order to be compatible with Linux and ChromeOS. They might hide this ability, but if you look hard enough you will find it 🙂 The benefit is that it will inherit your browser's permissions, so you only need to give the permissions once.

![Meeting join screen with an arrow pointing to the 'Join from your browser' link](https://simonhearne.com/images/great-demos/join_from_browser.png)
*Look for a 'join from browser' link*

The second tip is to proactively give permission. The app you have to download might not ask for permission until you attempt to share your screen, and you might not be able to share your screen until your meeting starts! In MacOS, head to System Preferences -> Security & Privacy -> Privacy -> Screen Recording and enable the app.

<figure class="no-shadow">
<img src="/images/great-demos/accessibility_settings.png"
    alt="MacOS Security & Privacy Screen Recording settings listing apps allowed to record the screen"
    loading = "lazy"
    maxwidth = 500
/>
  <figcaption>Preemptively give apps Screen Recording &amp; Microphone access</figcaption>
</figure>

Once you are able to share your screen, let's make sure that starting your screen share is as smooth as possible. If you are presenting from a slide deck, make sure you are in presenter mode *before* you share your screen, this makes it easier to share the correct app or screen and will make the process more smooth.

There are a few methods to avoid the dreaded "can you see my screen?" intro. Most simply, agree with another person on the call (from your company) that they will let you know when they can see your screen, ideally via an internal chat tool that you can see from your phone. A slightly more involved option is to join the meeting twice, once as a presenter and once as an attendee. This way you will see exactly what the customer sees! There are obvious drawbacks here, especially if you have limited bandwidth available, but there's nothing like seeing the customer view to give you confidence about what you are presenting. I do this for very important meetings or webinars with an iPad on my desk, so I can see at a glance what the presentation looks like.

### Screen-sharing with Multi-monitor Setups

Screen-sharing becomes quite simple with multi-monitor setup. Use your secondary monitor for the demo / presentation and share the whole screen. This means that you can switch between apps without having to re-share them.

If you have to switch between screens or windows, have these staged and ready to go. On MacOS I recommend using multiple desktops - this allows you to easily switch between full screen apps using a three-finger swipe or <span class="key">control</span> + <span class="key">→</span>. On Windows, have your apps lined up in the task switcher so you can quickly move from one to the other.

### Screen-sharing with Single-monitor Setups

Sharing your screen becomes a little tricky if you only have one monitor. If you are physically plugging in to a projector or TV - you can avoid this by setting it as a second monitor, not mirrored! If you're presenting remotely you will have to make do with the one monitor you have. A good option is to only share the application you need to. In many cases, you can spend 100% of your time in a browser window. Use a web-based presentation tool (Google Slides or HTML!) and get your tabs set up in the order that you will need them. Use <span class="key">⌘</span> + <span class="key">[n]</span> to switch tabs where [n] is the tab index, i.e. <span class="key">⌘</span> + <span class="key">1</span> will switch to the left-most tab.

If you have to share your whole screen, for example if you need to switch between multiple applications, make sure your workspace is as clear as possible. In MacOS, multiple desktops are your friend here, just make sure your other applications (email, IM etc.) are minimized so you don't accidentally flash them to your customer when you confuse swiping left for swiping right!

### High Resolution Displays

If you have a newer laptop or a nice high-definition external monitor, you will likely have a horizontal resolution between 3,000 and 4,000 pixels. While this is great for productivity, it can be a nightmare for screen-sharing. If your customer is showing the display on a 720p projector, or a window on their 1080p display, your monitor will be scaled back significantly. If we take the 720p example, each projected pixel will have to squeeze in nine pixels from your 4k monitor. Ever heard customers say that it's hard to read text from your screen? This could well be why. Sharing at 4k might also consume significant resources on your machine as it tries to compress the information to share it over the internet.

There are three options to resolve this issue. No matter what route you take, it's worth confirming with your customer that they can read the content you are sharing without straining their eyes.

1. Zoom in! This is the simple option, most apps will respond to <span class="key">⌘</span> + <span class="key">+</span> or <span class="key">control</span> + <span class="key">+</span> to zoom in.
2. Share a window. Instead of sharing your full 4k screen, share an application window which is scaled to a quarter of your display.
3. Change your monitor resolution. This is something that often happens automatically when you connect to a projector, but not in remote presentations. In MacOS you can change the scaling in display settings, and in Windows you should be able to change the display resolution in Control Panel.

![screenshot of macos resolution settings](https://simonhearne.com/images/great-demos/macos_resolution.jpg)
*Adjust your effective resolution in Display preferences*

### Sharing your browser

It is quite likely that you will be sharing your browser window at some point in a demo. This can be a bit risky! There are two options to reduce how much information is shared when you share your browser. The first is to have a separate user profile for your demos. Most browsers support multiple profiles which isolate bookmarks, credentials and extensions. The challenge can be making sure you have all of the credentials set up correctly in the demo profile!

![Chrome profile switcher listing separate profiles for personal use, work accounts and a demo account](https://simonhearne.com/images/great-demos/chrome_profiles.png)
*Separate your Chrome profiles by use*

It is also a good idea to tidy up your browser before sharing your screen. This can be as simple as making sure your 'new tab' page is a blank page, and hiding your bookmarks and extension bars. You can quickly toggle the bookmark bar in Chrome on MacOS with <span class="key">⌘</span> + <span class="key">⇧</span> + <span class="key">b</span>. Your extension bar should have a grab handle so you can shrink it. Removing these distractions allows your audience to focus on what you're showing them, and prevents you from revealing anything you shouldn't.

![Chrome toolbar comparison: cluttered bookmarks and extension icons before, a clean toolbar after](https://simonhearne.com/images/great-demos/tidy_chrome.png)
*Remove distracting bookmarks and extension icons*

## Ending a Demo

How you end a demo will influence your customer's impression more than you might expect. A good ending cements the messages you want to convey in the demo and will be remembered more than the other 55 minutes. This is the logic behind tell-show-tell:

- Tell them what you're going to show them
- Show them
- Tell them what you showed them

The demo is only there to confirm the points that you have pitched at the beginning. The end, then, is to summarize those points and confirm understanding with the audience.

A weak ending puts the whole demo at risk, so make sure you are prepared to make it concise and impactful. The following four points should give a good structure to a ~2.5 minute close before the sales representative takes over:

- "At the beginning of the call we said..."
- "During the demo you have seen how we can solve your challenges..."
- "Is there anything else you were hoping to see in this demo?"
- "Would you say that this solution solves your challenges?"

Hopefully the customer responds with a yes to the last question! Don't be afraid to wait 10-20s for an answer. If the silence hangs too heavy, you can follow up with an open, leading question to give the customer an opportunity to share any objections:

> "it seems like you are not 100% sure that the solution solves your challenges, what have we not shown you today that could get you to 100%?"

This closes the demo with the customer agreeing that they need your solution, rather than you telling them they do. This is subtle but important - there is a subconscious difference between being told that the solution is good, vs. having to make the statement yourself. The worst case scenario with this ending is that the customer tells you what you have to hit first in the follow-up demo 😃

Finally, take stock of any objections and follow-up actions you have noted down. Make sure each has an owner from your side and the customer side, and follow up with this by email shortly after the call.

Having your demo structure written down in front of you, or in speaker notes, is a great way to ease in to a new demo approach. It will become second nature surprisingly quickly.

## Handling objections

Often viewed as a challenge, objections are actually great opportunities to learn more about your customer and their challenges. When I say objections here, I mean any question that challenges what you are presenting, for example "can your solution do this?" or "that looks complex to me". There could be a whole book on the subject, but the sections below cover the high-level steps required to handle objections.

### 1 - Acknowledge the objection

The first thing you should do is to look after the ego of the person who raises the objection. This is often accomplished with a simple acknowledgement, and a statement of empathy:

> "Great question!"

### 2 - Confirm understanding

This step is often skipped, but can really help to clarify the objection. It also gives the audience member an opportunity to take their objection back or even answer it for themselves!

> "It sounds like you're asking if..."

### 3a - Provide the answer

In the most simple of scenarios, the question is easy to answer. In this case it is tempting to skip steps 1 & 2 and give an instant answer, but this misses the opportunity for the customer to answer their own question or, even better, someone else from the audience to answer it for you!

When giving the answer, make sure you directly address the objection:

> "The solution provides feature X which directly resolves objection Y. It seems to me that this answers your question, would you agree?"

Asking the customer to agree serves two purposes: confirming that you have actually resolved the objection, and making the customer state that you have resolved the objection.

### 3b - Defer the answer

You will not have all the answers in all cases. It is perfectly fine to say "I don't know"!. What the customer really wants in this case is for you to have acknowledged and understood the question, and have a route to answering it in the future:

> "I don't know the answer to that question, I will take a note and get back to you after the call. Does anyone else know the answer?"

It's important to ask the audience if anyone else can answer the question, ideally someone from the customer side will! This also saves you a bit of work in follow-up.

### 3c - Admit defeat

This is the most awkward objection, and the one where your response is most important. The question to which you don't have the *right* answer. This could be missing functionality in your solution, inability to deliver in a certain time-frame or any other negative response.

There are two steps to admitting defeat. The first is to be up-front and confirm the requirement:

> "Our product does not do that. Is this a critical requirement for you?"

If they don't really care about it, then you can acknowledge it and move on. If they do care about it, you need to take an action:

> "I appreciate that you have a critical requirement for the solution to do X. I will take a note to speak to the product owner and see if it is on the road map"

Make sure that the action is noted down and be sure to follow up after the call.

## Wrapping up

In the introduction I said that spending a little time to think more about your customer and your demo can result in a more engaging, more enjoyable and more successful meeting. In this post we have discussed how important it is to invest in preparing for a demo, doing research on the company, the competition and the audience.

We then looked at the three most important demo types: the Qualification, the Confirmation and the Proof. Knowing which demo you need to give is a great way to improve your demos, and will leave your customers feeling more engaged with you and your solution.

Knowing the common pitfalls for demos makes it easier to spot them, and we introduced some simple techniques to overcome these pitfalls. This gives you the power to recover a less than perfect demo early on in the call, and turn it around into the kind of demo that leaves you and the customer feeling great.

Remote presentations require a little more thought and preparation than in-person meetings. We looked at some simple screen-sharing hygiene to make sure you don't spend the first few minutes arranging windows, or get interrupted half-way through by an inappropriate Slack message!

We know how disappointing it is when a movie has a weak ending, the same is true for demos! It is easy to get carried away and run out of time, before you know it you're at two minutes past the end of the meeting and folks are starting to drop off the call. Ensuring that you prepare a few minutes for a strong close will leave a lasting positive impression with the audience.

One of the biggest fears most people have in a demo is the "tough question". The objection handling steps we outlined will help you to buy time, win favor and leave the audience satisfied with your answer. No matter what question they ask!

We've only touched the surface of technical demos in this post, but hopefully there are some useful nuggets here for you. The following section lists some books that I have found particularly beneficial to improving my pre-sales skills, if you want to dig deeper! As a bit of a spoiler: all of these revolve around putting the customer first. Active listening, acknowledgement, mirroring and understanding are all key techniques to build trust. Without trust, it is unlikely that you will be successful.

## Further reading

There are hundreds of books on this topic. The selection below are from my library, and each adds value to technical presenting. I've roughly prioritized them by how applicable they are to technical demos, they all have something to teach which can be valuable for pre-sales engineers.

**[Mastering Technical Sales: The Sales Engineer's Handbook](https://www.amazon.com/Mastering-Technical-Sales-Professional-Development/dp/1608077446/?tag=webnin-21)** by John Care

This book is a great resource for those new to technical pre-sales. It covers much more than software demos including RFPs, needs analysis and whiteboarding. There are three chapters on demonstrations, though, starting with one of my favorite quotes from pre-sales:

> The eager rush to demonstrate your product, which we’ll refer to in this chapter as the "dash to demo," often sounds the death knell for any sales opportunity

**[Demonstrating To Win!: The Indispensable Guide for Demonstrating Complex Products](https://www.amazon.com/Demonstrating-Win-Indispensable-Complex-Products/dp/0615477097/?tag=webnin-21)** by Robert Riefstahl

Demo2Win is the golden standard for demonstrating complex software products. The book was originally published in 2000, but the second edition (2011) has been updated for remote presentations. The focus is on multi-hour demos for complex software, but most of the techniques are applicable to any software. This is where I first read about 'tell-show-tell'. Robert pitches demos as if they are movies: the intro has to capture attention and introduce the characters. The story has to follow an arc which introduces the challenge, the complication, the solution and the success. The ending has to wrap everything up and leave you speechless.

**[The Speed of Trust: The One Thing that Changes Everything](https://www.amazon.com/SPEED-TRUST-Thing-Changes-Everything/dp/1416549005/?tag=webnin-21)** by Stephen M. R. Covey

A technical pre-sales engineer should be a trusted advisor. This book discusses the importance of trust and how it is earned and lost. A good book for life in general, and quite applicable to software demonstrations.

**[SCIPAB Pocket Reference](https://www.mandel.com/landing-pages/scipab-pocket-prompt-ti)** by Mandel

This is not a book, but a quick reference to the SCIPAB methodology to start demos: **S**ituation, **C**omplication, **I**mplication, **P**osition, **A**ction and **B**enefit. This methodology is a valuable tool to help craft your ~two minute pitch before a demo.

**[Great Demo! How to Create and Execute Stunning Software Demonstrations](https://www.amazon.com/Great-Demo-Stunning-Software-Demonstrations/dp/059534559X/?tag=webnin-21)** by Peter Cohan

Great Demo! introduces the concept of showing the most powerful screen first, and has some good ideas around slides that you should prepare to top-and-tail a demo. The book is a little long for the actionable content, but the key concepts are solid.

**[Never Split the Difference: Negotiating as if Your Life Depended on It](https://www.amazon.com/Never-Split-Difference-Negotiating-Depended/dp/1847941494/?tag=webnin-21)** by Chris Voss

I think of this book like How to Win Friends and Influence People, but with more terrorists. Chris was a hostage negotiator and shares his experience and techniques to resolve negotiations successfully. One of the most valuable sections of the book is on how to find out what people really want, you can't negotiate until you know what you should be offering! This is of critical importance to delivering excellent demos and closing opportunities.

**[How to Win Friends and Influence People](https://www.amazon.com/How-win-Friends-Influence-People/dp/8189297813/?tag=webnin-21)** by Dale Carnegie

The classic book on communication strategies. While the book does not discuss software demonstrations in depth, it will help you spot your own flaws, tics and mannerisms. Once you are aware of your own issues, you can work on them to improve your communication and demo skills.

**[How to Talk so Kids Will Listen... And Listen So Kids Will Talk](https://www.amazon.com/How-Talk-Kids-Will-Listen/dp/B007YZX6LE/?tag=webnin-21)** by Adele Faber and Elaine Mazlish

This one is definitely the odd one out. I read it before having children (I can't remember why!) and I've referred back to it regularly since having kids. Working with children involves a lot of negotiation, objection handling and reading between the lines. Working with customers involves... a lot of the same!


==============================================================================

# Who Opts-in to Save-data?

> A primer on save-data, what it means for your users and what it means for you.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2020-05-29
Canonical: https://simonhearne.com/2020/save-data/
Tags: WebPerf

---
## In Summary

Save-data is a client hint which indicates that your users want you to send less content to them. It is an option on Google Chrome Mobile and it's used predominantly in Asia, Africa and South America. There is a strong relationship between the value of a user's device and their likelihood to opt-in to save-data: the more expensive the device, the less likely they are to opt-in. While the opt-in rate is highest on cheaper devices and in less economically developed countries, this is not an absolute delineation. For example: users in Canada have an opt-in rate of over 34% (compared to ~7% in the US) and users on the latest Samsung flagship have an opt-in rate of almost 18% globally. Income inequality is a factor in national opt-in rate in most countries.

Google Chrome Mobile will use this hint to provide a proxied web experience in some scenarios, but this process is quite opaque. It may also defer scripts, force `font-display: swap` and force lazy loading of iframes and images.

Website owners should not rely on the browser to make these optimizations, but instead should allow users to opt-in to a data-saving experience. Save-data is a 'set-and-forget' setting which is rarely changed; providing a UI to allow users to opt-in or out of a 'lite' mode is a great progressive enhancement for modern websites.

## Introduction

Save-data (often conflated with Data-Saver Mode / Chrome Lite Pages / [Chrome Lite mode](https://support.google.com/chrome/answer/2392284?co=GENIE.Platform%3DAndroid&hl=en)) is a client preference for reduced data consumption. The hint is currently available to JavaScript via the Network Information API and as an HTTP request header. The preference is set in browser settings, with Chrome mobile suggesting that you enable it when you go first open the application:

![screenshot of chrome recommending lite mode on mobile](https://simonhearne.com/images/save-data/litemode.jpg)
*Chrome suggests enabling Lite Mode on first open*

The save-data hint in itself does nothing, but there are a number of scenarios where service providers and website owners can react to the hint to deliver a better experience:

- Google Chrome may force interventions such as deferring external scripts and lazyloading iframes and images
- Google Chrome may proxy requests to improve performance for users who have opted in to 'Lite Pages' *and* have a poor network connection
- Website owners can deliver lighter versions of their applications, e.g. by lowering image quality, delivering a server-side rendered page or reducing the amount of third-party content
- ISPs can transform HTTP images to reduce size on the last-mile

There have already been a number of posts looking into Chrome Lite Pages, I'd recommend you take a look at Tim Kadlec's [blog post](https://timkadlec.com/remembers/2019-03-14-making-sense-of-chrome-lite-pages/](https://timkadlec.com/remembers/2019-03-14-making-sense-of-chrome-lite-pages/)) for a great introduction to the feature and the problem it is trying to solve. In this post we're going to focus on the users who choose to opt-in.

## Collecting Data

Akamai mPulse collects performance data from a range of sources in the browser, including the [Network Information API](https://developer.akamai.com/mpulse/whats-in-a-beacon/#mobile-connection-details). This API provides information about the device's current connectivity, as well as a `saveData` property. The open source [mobile plugin](https://github.com/akamai/boomerang/blob/master/plugins/mobile.js) for boomerang provides access to the API, and the data is available in [Akamai mPulse](https://www.akamai.com/uk/en/products/performance/mpulse-real-user-monitoring.jsp) as well as the open source [boomerang](https://github.com/akamai/boomerang).

The data is only available in Chromium-based browsers which support Lite pages, for now. A quick review of the data retreived in May shows that only four browsers have a reported save-data rate above 0.1%:

| Browser Name   | Save-data Rate | Relative Traffic Share |
|----------------|--------|--------------------|
| Chrome         | 0.61%  | 26%                |
| Chrome Mobile  | 28.14% | 15%                |
| Yandex Browser | 2.92%  | 0.42%                 |
| Puffin         | 35.23% | 0.003%                 |

We will be looking at Chrome Mobile hits in the following research, this represents the largest population of opted-in traffic and allows comparitive analysis of opted-in vs opted-out.

It is worth noting that this data is aggregated from mPulse beacons, which will show a natural weighting towards North America and Europe due to the customer base. The data is taken from a time period in May with a total of about two billion user experiences.

## By Country

This section reviews the save-data opt-in rate by country. Analysis by country allows us to compare the opt-in rate against a number of factors which are accessible by country, such as cost of data and average income. The opt-in rate is the percentage of pageviews which had the `saveData` property set to `true`. Scroll down to move through various dimensions of analysis.

<div class="chart-sticky-container" id="sticky-container">
  <div class="vega-chart loading chart-animate chart-sticky chart-active" id="vis3" data-spec="/data/save-data/scatter2.json">Loading chart...</div>
  <div class="stage" data-stage="0">
    <h3 id="average-cost-per-gb" class="no-link">Average Cost per GB</h3>
      <p>
      When considering why people might opt-in to save data, the first answer that came to mind was the cost of the data. If the cost of data is high, then you are more likely to try to reduce your data usage, you might assume. I pulled the latest data for the average price paid for 1GB of data, normalized to USD, and compared it with the save-data opt-in rate by country.
      </p><p>
      Looking at this first scatter plot we can see that there is no immediately obvious correlation between cost of data in a country and the opt-in rate. A positive correlation would look like the points forming a line diagonally from bottom-left to top-right.
      </p><p>
      The result do show a few interesting points. The strongest correlating factor for save-data appears to be continent. The split in the data at around 40% opt-in separates Asia, Africa and South America (with high opt-in rates) versus North America, Europe and Oceania. The exceptions to this are Japan in Asia, which creeps below 40% mark at just 34.1% and Mexico in North America which is well above 40% at 66.4%.
      </p><p>
      Another standout feature of this chart is India. It is high on the y-axis at 62% opt-in, but almost zero on the x-axis at an average cost of $0.09 per GB. I first thought this was an error but it turns out that data really is <a href="https://www.business-standard.com/article/pti-stories/india-mobile-internet-rates-remain-lowest-in-world-prasad-119120201245_1.html">that cheap</a> in India!
      </p><p>
      Whilst this chart does give us some interesting information about why people might opt-in to save data, it does not show the strong correlation that I had expected.
      </p>
  </div>

  <div class="stage" data-stage="1">
    <h3 id="average-individual-income" class="no-link">Average Individual Income</h3>
    <p>
    The separation between continents in the first chart led me to a second hypothesis, that save-data opt-in rate is correlated with the wealth of a country. I pulled the latest income per capita values available from the World Bank and plotted this against opt-in rate.
    </p><p>
    Whilst again there is a clear separation between continents, this has become more pronounced. The countries that are less likely to opt-in to save-data are clustered towards the right-hand-side of the chart, on the higher end of average personal income. The countries that are more likely to opt-in are fairly evenly spread across the x-axis, with a slight positive trend which disagrees with the hypothesis.</p><p>
    UAE, at $43k GDP per capita, is near the right-hand-side of the chart but has a 64% opt-in rate. Compared this to Canada at $46k and 34% respectively.</p><p>
    One country that exhibits strange behavior here is Ireland. At $79k GDP per capita it is one of the richest countries in the world, but the average income of $44k gives it a income / GDP ratio of 0.56, one of the lowest globally. Compare this to the Netherlands which has a similar income but lower GDP at $53k, making an income / GDP ratio of 0.83. This isn't necessarily related to save-data, but interesting nonetheless and is perhaps a symptom of the tax advantages for basing technology companies in the country.
    </p><p>
    Saudi Arabia shows that a high average individual income does not necessarily correlate with low save-data opt-in. We know the dangers of taking averages, and this may highlight the impact of wealth disparity amongst some countries.
    </p>
  </div>
  <div class="stage" data-stage="2">
    <h3 id="disposable-income" class="no-link">Income Inequality</h3>
    <p>
    This chart plots income inequality, as the percentage of total income earned by the bottom 50% of earners. A higher number represents a more even distribution of income.</p><p>
    The plot shows that the countries where save-data opt-in rates are higher, have greater income inequality - or relatively a greater population of low earners. Here we can see that at just 7.8% of income going to the bottom earners, Saudi Arabia has a significant wealth divide. This helps to explain the high opt-in rate.
    </p>
  </div>
  <div class="stage" data-stage="3">
    <h3 id="hours-worked-for-1gb" class="no-link">Hours Worked for 1GB</h3>
    <p>
    Using the data collected above, we can calculate the average number of hours required to work to earn enough to purchase 1GB of data (assuming 2,000 working hours per year). This measure should get more close to the perceived cost of mobile data. My hypothesis is that the higher the relative cost of data in a country, the greater the likelihood for opt-in to save-data. This would look like a rough line from top-left to bottom-right (note that the x-axis is reversed).
    </p><p>
    The two obvious extremes here are Malawi on the far left and Israel on the far right of the chart. Malawi has an average annual income of $312 and an average data cost of $27 per GB, meaning that the average Malawian would have to work 256 hours to earn enough for 1GB of mobile data. In Israel, the average income is $36k and the average cost for 1GB is 11¢. This makes the hours required for one GB 0.006, just 22 seconds!
    </p><p>
    The left-hand-side of the chart is populated almost wholly by African countries, where salaries are low and data is expensive. The data generally agrees with the hypothesis, but does not show a direct correlation.</p>
  </div>
  <div class="stage" data-stage="4">
    <h3 id="average-network-speed">Average Network Speed</h3>
    <p>
    So far we have looked primarily at financial motivators for saving data, which has shown a rough correlation between relative cost of data and the likelihood to opt-in to save-data. Another motivator might be network performance. The Chrome Lite Pages opt-on prompt promises up to 60% reduced data and faster page loads, if a user knows that their connection is poor they might be more likely to opt-in to Lite Mode, and thus save-data.</p><p>
    The data shows that countries with an average mobile connection speed of less than 20Mbps have a much higher save-data opt-in rate, and countries with faster speeds are less likely to have high opt-in rates. One obvious outlier is Norway, with the highest average speed at 48.2Mbps but a high opt-in rate of 37%. On the other end of the spectrum, Belarus has an average speed of 7.7Mbps but an opt-in rate of 20%.
    </p>
  </div>
  <div class="stage" data-stage="5">
    <h3 id="4g-network-availability" class="no-link">4G Network Availability</h3>
    <p>
    Much like the download rate correlation, 4G availability shows little correlation with opt-in rate. In fact a stark difference can be observed between Ireland, at 63.7% 4G penetration and 25% opt-in versus The Netherlands with at 92.8% penetration and 25.4% opt-in rate.</p><p>
    India is an outlier at 62% opt-in rate, but with one of the best 4G availability rates at 91%!
    This indicates that 4G penetration on its own is not a critical factor in likelihood to opt-in to save-data.
    </p>
  </div>
</div>

## By Device

We've filtered the results down to Chrome Mobile in this analysis, which limits us to Android devices. But Android is a diverse ecosystem: from the $50 Alcatel 1 through to the $2,500 Huawei Mate XS. Device specifications and network capability vary greatly across the spectrum of devices, meaning that simply looking at Chrome Mobile covers a diversity of capability.

The chart below shows the top devices in the dataset, from the top 25 manufacturers:

<div class="chart-sticky-container" id="sticky-container">
<div class="vega-chart loading chart-sticky" id="vis4" data-spec="/data/save-data/manufacturers.json">Loading chart...</div>

<p>The results show that users of cheaper devices are more likely to opt-in to save-data. Take a look at Samsung devices for example. The highlighted devices are the <a href="https://www.gsmarena.com/samsung_galaxy_s9-8966.php">Galaxy S9</a>, the <a href="https://www.gsmarena.com/samsung_galaxy_a50-9554.php">Galaxy A50</a> and the <a href="https://www.gsmarena.com/samsung_galaxy_j7_prime-8314.php">Galaxy J7 Prime</a>. These phones represent three reasonably similar devices from Samsung across their flagship, premium and budget product lines respectively. The <strong>J</strong> range is marketed to younger or more cost-conscious users and has relatively low specifications. The <strong>A</strong> series devices tend to inherit the components of the <strong>S</strong> series with an ~18 month delay.</p><p>The J7 was released 1.5 years before the S9, almost three years before the collection of this data. A such, this is likely to be purchased at a discount or used by those on pre-paid connections. The A50 was released in 2020, the same year of this analysis, so is likely purchased as a good value premium device or offered as a contract upgrade. The S9 was a flagship device released at ~$800 one year prior to the analysis, so is likely owned by users on a high-cost contract which includes a large amount of included data.</p>

<p>The difference in save-data opt-in is stark: just <strong>17.7%</strong> for the premium S9 vs <strong>36%</strong> for the A50 and <strong>64%</strong> for the budget J7. Could this be users trying to keep their device <em>feeling</em> fast as it ages? Or budget-conscious users preserving small data budgets?</p>

<p>Another clear difference can be observed between premium and budget manufacturers. For example: Google devices average 21% opt-in, whilst Realme devices average 67%!</p>
</div>

## By Connectivity

It is a fair assumption that folks on poorer connections are more likely to opt-in to save-data, so let's look at the data! In the same API that gives us the save-data preference, we can also get the browser's current estimated downlink speed and network latency. This data is rounded and sometimes randomized to prevent fingerprinting, but is generally a good indicator of network quality. Take a look at your current advertised data below:

<div id="netinfo">
<pre>waiting...</pre>
</div>

The `effectiveType` is derived from the `rtt` and `downlink` values, you can see the derivation in the [API Documentation](https://wicg.github.io/netinfo/#effective-connection-types). Perhaps confusingly, `4g` is effectively infinite speed - any connection faster than 0.7Mbps and 270ms RTT is given this effective type.

When we extract that data and plot it with the save-data opt-in rates we see very little correlation. The majority of experiences fall into the 4g bucket, with the `Cable/DSL` (e.g. home wifi) bucket showing a 21.7% average opt-in rate, vs. ~31.6% opt-in rate for `Dialup` and `Cellular`. There are a large number of hits in the `Not Set` connection group, these are from connections which cannot be classified through NetInfo or IP lookup.

The results imply that users who do not have a home wifi connection are more likely to use the save-data hint.

<div class="vega-chart loading" id="vis5" data-spec="/data/save-data/sd_by_ect.json">Loading chart...</div>

The differences here are much smaller than between devices, which is likely due to the fact that save-data is a set-and-forget setting. Users are unlikely to change it based on their connectivity.

## Performance

It may seem strange that I haven't mentioned performance results yet, given my interests! The reason is that it is impossible to separate correlation from causation with aggregated real user monitoring data. If a page is slow with save-data set, it is difficult to determine the cause of the slow performance. We do not know if any interventions are used by Chrome or the website.

To have a go at analyzing the data, we can separate out a single device and plot the performance of page loads with save-data set or not. The following chart is a five-number summary showing the 5th, 25th, 50th, 75th and 95th percentiles of DOM Ready time for the most common device in the dataset: the Samsung Galaxy S9. The results show that the value of th save-data flag makes zero appreciable difference to the performance, as measured by DOM Ready. This is as expected for the majority of sites which do not react to the hint, and thus deliver the same content. It is also unclear whether beacons are sent for cloud-rendered pages delivered by Google, so it is possible that these pages will be faster but not included in the data set.

<div class="vega-chart loading" id="vis6" data-spec="/data/save-data/s9_perf.json">Loading chart...</div>

## Let's not Forget Lite Pages

Lite pages are proixed through Google Servers to improve performance and reduce size over the network. Google will optimize non-secure images and force performance interventions on the users device (e.g. [lazy loading](https://web.dev/native-lazy-loading/) images and iframes). It can also redirect you to a 'cloud rendered' version of the page...

<blockquote class="twitter-tweet" data-dnt="true"><p lang="en" dir="ltr">Interesting. What&#39;s the analytics / reporting outcome if a Lite Page is served (cloud rendered)? Are the rendered versions cached? Will analytics scripts fire? Does the cloud browser have a unique UA? Can you track Lite -&gt; full sessions?<br>I&#39;ve got so many questions!</p>&mdash; Simon Hearne (@simonhearne) <a href="https://twitter.com/simonhearne/status/1145603877275873281?ref_src=twsrc%5Etfw">July 1, 2019</a></blockquote>

You can try this for yourself (if you have a device with Chrome Mobile) by enabling "Lite Mode" in your settings and overriding effective connection type to 2G in [chrome://flags](chrome://flags):

<div class="two-fig-cols">
![screenshot of lite mode settings claiming it has saved 38MB out of 310MB downloaded](https://simonhearne.com/images/save-data/lite-savings.jpg)
*Chrome Lite Mode settings screen*
![screenshot of chrome flags option for connection type set to slow 3G](https://simonhearne.com/images/save-data/ect-flag.jpg)
*ECT Flag set to Slow 2G*
</div>

The logic as to when Chrome will serve a Lite Page is not necessarily consistent, and Lite Pages will not be served for authenticated sessions. I believe analytics tags will not fire on Lite Pages although I'm happy to see data showing otherwise, or indeed any information from Google on the matter.

You can use the [Reporting API](https://developers.google.com/web/updates/2018/09/reportingapi) to see if your pages are being proxied by Chrome Lite pages. Set a `Report-To` endpoint and look out for a report with the type `intervention` and an id `LitePageServed`:

```json
{
  "type":"intervention",
  "url":"https://yoursite.com/page",
  "body":{
    "id":"LitePageServed",
    "message":"Modified page load behavior on the page because the page was expected to take a long amount of time to load. https://www.chromestatus.com/feature/5148050062311424"
  }
}
```

## Opting-out of Save-data

You may find that users complain that your page is broken in Lite Mode, or you may just not like Chrome making this intervention on your behalf. Some pages do not render quite right, as can be seen below for the BBC.

![screenshot of bbc loading with an odd layout in lite mode](https://simonhearne.com/images/save-data/bbc-lite.jpg)
*BBC Lite looking worse for wear*

Developers can opt-out of cloud-rendered Lite Pages by including the `cache-control: no-transform` response header. This should prevent any intermediary (e.g. ISP, Chrome, CDN) from modifying the response, but it is a heavy-handed approach. `no-transform` may also interfere with CDN optimizations such as HTML minification and dynamic script injection.

## Next Steps

### Getting the data

Before doing anything about save-data, you need to know how many of your current users are sending the client hint. You can collect this data in your analytics tool of choice, for example with Google Analytics:

```
ga('create', 'UA-XXXX-Y', 'auto');
ga('set', 'savedata', navigator.connection.saveData);
ga('send', 'pageview');
```

Preferably, collect this data with your performance analytics tool. You can configure this easily in the mPulse UI:

![screenshot of mPulse UI configuration of custom dimension](https://simonhearne.com/images/save-data/cdim.png)
*mPulse Custom Dimensions are configured in the UI*

### Reacting to the hint

If more than 1% of your users are sending the save-data hint, it makes sense to do something about it. This can be anything from delivering lighter images through your CDN to delivering a static version of your site.

Images are the biggest source of transferred bytes on most web pages, so are the obvious candidate for saving data.

![http archive chart showing growth of images delivered to mobile, from 75kb in 2011 to 910kb in 2020](https://simonhearne.com/images/save-data/img-weight.png)
*HTTP Archive - Image Growth. Source*

As we have seen earlier in the analysis, save-data is not limited to low-end devices. This means that you are likely sending 2x or 3x images to devices which have opted-in to save-data. A simple change to reduce size would be to only send 1x images to devices with save-data. If you have a CDN or image delivery service, it is likely possible to automate this! For example, Cloudinary can [automatically](https://cloudinary.com/blog/the_holy_grail_of_image_optimization_or_balancing_visual_quality_and_file_size#automatic_quality_default_values) reduce image quality if the save-data hint is set to true. Akamai [Image Manager](https://developer.akamai.com/akamai-image-and-video-manager) can even reduce quality based on network connectivity, with endless possibilities for custom behaviors based on the save-data client hint.

One of the biggest bottlenecks on the web is JavaScript. If your site is one of the 50% that ship more than 417kB of JS, sending a page without it will likely be a far superior experience for users with limited connectivity and under-powered devices.

![http archive chart showing growth of javascript delivered to mobile, from 50kb in 2011 to 417kb in 2020](https://simonhearne.com/images/save-data/js-weight.png)
*HTTP Archive - JavaScript Growth. Source*

Do you need to ship your whole framework for a landing page?

For even more creative ideas on this topic, I highly recommend watching Tim Vereecke's talk [Data-S(h)aver Strategies](https://www.youtube.com/watch?v=X0BbSvSbwD0) from London Web Performance in 2019:

Video: https://www.youtube.com/watch?v=X0BbSvSbwD0

Something that Tim recommends is to allow users to opt-out of the lite mode without having to disable save-data. This can be achieved with a UI toggle and a cookie which overrides the save-data preference. This option, like dark theme toggles, allows users to choose the experience that best suits their current situation. It is quite possible that a large number of users opted-in to save-data when they first configured Chrome Mobile and have not thought about it since, and that an equally large number of users ignored the intial prompt and don't know that the feature exists.

## Conclusion

We have seen that there can be many reasons for users to opt-in to save-data, and that the performance delivered to devices doesn't (yet) vary much based on the hint. A lack of awareness from both developers and web users mean that a potentially valuable web platform feature is under utilized. We currently rely on Chrome to make interventions on our behalf, potentially resulting in broken layouts, missing analytics and an unknown result with ad impressions. Could this be because Chrome is the only browser to embrace Save-Data?

Mindful observation of the save-data hint can provide a better user experience and save your visitors' data budgets. If you're not reacting to it now, think about what can be done to proactively reduce the data sent to your users.

<script defer src="/js/save-data-charts.js"></script>


==============================================================================

# Measuring Performance behind consent popups

> Opting in to cookies can load multiple megabytes of additional content. Here are tips on how to measure web performance both with and without cookie consent.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2020-05-13
Canonical: https://simonhearne.com/2020/testing-behind-consent/
Tags: WebPerf, webpagetest

---
## Introduction

Cookie consent has given users control over the data that is shared with websites and third-parties, but it has made measuring performance difficult. Cookie consent is managed outside of standard APIs, so it can be implemented any way a website wants. Within the EU (and generally for websites hosted in the EU) consent is not assumed, so a visitor must opt-in to third-party tracking etc.

This makes testing performance difficult as standard synthetic test tools such as WebPageTest will only measure the *opted-out* experience. Conversely, real user measurement (RUM) tools such as mPulse will only measure the *opted-in* experience. See the problem? This also means that large-scale datasets such as [HTTP Archive](https://httparchive.org/) and Google's [CrUX](https://developers.google.com/web/tools/chrome-user-experience-report) present the performance of the opted-out and opted-in experience, respectively.

The problem with RUM data is generally unavoidable, as performance measurement is not a functional requirement, although few websites have so far implemented a true opt-in solution for performance tracking. For reference, mPulse provides the [inline consent plugin](https://developer.akamai.com/tools/boomerang/docs/tutorial-howto-opt-out-or-opt-in.html) as a simple method to implement this correctly.

The problem of synthetic tests is thankfully more simple to solve: we simply need to have the test 'opt-in' as a user would. Once we have the ability to opt-in, there will be multiple scenarios we can explore:

- **Opted-out first view** - this is the easy one
- **Opted-out repeat view** - cached experience
- **Opted-in first view** - consent is given, page not cached
- **Opt-in after page load** - measure just the opt-in activity
- **Opted-in repeat view** - cached experience, with cookies

The results can be... eye opening. In the following sections we will walk through how to succesfully test these states, how to compare the results and present a small case study based on a corporate website.

## Opt-in Techniques

There are generally two methods to test the opt-in process in synthetic tests: setting the opt-in cookie manually or effectively clicking on the "accept" or "I agree" elements. I'll show you how to do both using WebPageTest scripts, this is one of the many surprisingly powerful features of WPT and I recommend reading up on [the documentation](https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/scripting). To learn even more about WPT, I recommend you grab a copy of the book [Using WebPageTest](https://www.amazon.com/_/dp/1491902590?tag=webnin-21)

The Script UI is shown below for reference.

![screenshot of adding a custom script to web page test](https://simonhearne.com/images/wpt-cookies/wpt1.png)
*Adding a script to WebPageTest*

### Setting an opt-in cookie

This can actually be quite tricky and requires some detective work. Consent is *normally* stored in a cookie, which means we can simulate the opt-in state by setting that cookie value manually. First, though, we need to find out what cookie to set!

Open a new private / incognito tab and crack open the developer tools on your browser of choice. Head to the section which shows cookies (`Application` in Chrome) and load the page under test.

Observe the cookies that are set on the initial page load (and screenshot them if you wish). Then accept the cookie banner and watch what changes! Unfortunately, there are likely to be a lot of new cookies, so spotting the one you care about can be difficult. Here, for example, is an enticingly named `CONSENT` cookie, but it is not the cookie we are looking for. We can tell this as the cookie domain is `google.com`, not that of the page under test:

![Right-click context menu in developer tools showing copy selector option](https://simonhearne.com/images/wpt-cookies/cookies.png)

You can also try filtering the cookies by the term `consent` to see if anything jumps out, and if all else fails? View-source!

If you view-source and search for "cookie" or "consent", you are likely to find the script responsible for the cookie banner. It might look something like this:

```html
<!-- OneTrust Cookies Consent Notice start -->
<script src="https://cdn.cookielaw.org/scripttemplates/otSDKStub.js" type="text/javascript"></script>
<script type="text/javascript">
function OptanonWrapper() {window.dataLayer.push({event:'OneTrustGroupsUpdated'});}
</script>
<!-- OneTrust Cookies Consent Notice end -->
```

This in turn points me at [https://cdn.cookielaw.org/scripttemplates/otSDKStub.js](https://cdn.cookielaw.org/scripttemplates/otSDKStub.js). Loading that script shows me right at the top of the file: `this.optanonCookieName="OptanonConsent"`. Sleuthing complete! Back in developer tools we can filter cookies to `OptanonConsent` and see if any have been set.

![Developer tools showing optnanon cookie is set](https://simonhearne.com/images/wpt-cookies/optanon.png)
**

Once you have the cookie, copy the value that has been set. It may be something simple like `true` or `1`, or it could be more obtuse and include a unique ID, timestamp and any number of other variables. We can add the cookie to our test scripts using a WebPageTest script and the `setCookie` command. Note that the domain should have `http://` as the protocol and should *not* have a trailing slash. Double check this if you see results like `(Test Error: Unhandled exception in test run: coercing to Unicode: need string or buffer, int found)`

```text
setCookie http://www.akamai.com OptanonConsent=consentId=1f6c1653-871e-468d-913c-378d05a8539e&datestamp=Tue+May+12+2020+21%3A42%3A07+GMT%2B0100+(British+Summer+Time)&version=5.15.0&interactionCount=2&isIABGlobal=false&landingPath=NotLandingPage&groups=&hosts=&legInt=
setCookie http://www.akamai.com OptanonAlertBoxClosed=2020-05-13T06:20:21.745Z
navigate %URL%
```

If the cookie value has a timestamp it may become invalid after a while, so if you are running these tests in an automated environment (e.g. CI testing or monitoring) then you may need to make this dynamic at the time of the test.

You may also notice that I've set two cookies in the script above. The first is the actual consent, allowing Google Tag Manager to inject third-parties. The second is necessary to prevent the consent banner re-appearing. Why are they separate? Who knows.

### Clicking on the Element

This is more simple, conceptually, than setting the cookie in a script. Here we emulate a user actively opting in by triggering the event in the website's code. To do this in WPT, we just need a valid selector for the accept action. Right-click on the element in your browser of choice and hit "Inspect".

![Right-click context menu in browser showing inspect element option](https://simonhearne.com/images/wpt-cookies/inspect-elem.png)

Once you have the element highlighted in the developer tools window, you can quickly observe if there is an easy selector such as an element ID (present in this example). If not, you can right-click on the element here and copy a selector. This can often end up being pretty long if the element does not have an ID, but it should at least give you a good starting point:

![Right-click context menu in developer tools showing copy selector option](https://simonhearne.com/images/wpt-cookies/copy-selector.png)

Once you have your selector, check it works by heading to the browser console and attempting to select it:

```js
document.querySelector('#onetrust-accept-btn-handler')
```

If the result is the element then you are on to a winner! We now just need to add this to our WPT script. There are a number of ways to do this, including WPT functions like `click` and `clickAndWait`. I find that this can get frustrating at times when switching between WPT and the browser console, so I tend do use `exec` or `execAndWait` which just execute the JavaScript you provide. If you need to chain multiple actions such as opening a modal then clicking a button, make sure your last action is `execAndWait` and preceding actions are `exec`. Assuming a simple interation, we will want a script that looks like:

```text
setEventName loadPage
navigate %URL%
setEventName acceptCookies
execAndWait document.QuerySelector('#onetrust-accept-btn-handler').click()
```

This will load the page at the URL you have entered in the URL field as the first step, then accept cookies and record any further activity as a separate step / page load. If you'd like to combine these you can use `combineSteps`, although note that there may be a small gap in the waterfall between the page loading and the script executing:

```text
combineSteps
navigate %URL%
execAndWait document.QuerySelector('#onetrust-accept-btn-handler').click()
```

In this case, the button will be clicked shortly after the initial page load, close to the expected user behaviour. Both test options are valid, and you should try both!

### Bonus Tips

Here are a few handy tips for working with WPT and cookie consent prompts.

#### Clearing up cookies

It is quite likely that you will need to emulate a first-time viewer multiple times in this process. The most simple method to reset state is to clear site data. In Chrome, this is under the "Applications" tab. Once cleared, hit Cmd+R / Ctrl+F5 to perform a hard reload.

<figure  class:"no-shadow">
<img src="/images/wpt-cookies/clear-data.png"
  alt="Screenshot of developer tools showing clear site data option"
  loading="lazy"
  clickable=true
  />
</figure>

#### Debugging WPT Scripts

WPT scripts do not provide debugging information, but we can debug the output. In this case we are expecting to see either a cookie being sent with the page request, or an element being clicked in the page load. Use the filmstrip view or page screenshot to check that the page looks as you expect for an opt-in user, i.e. the cookie banner is not present. If the banner is not present, you need to verify that your JavaScript actually works in your browser or check that the cookie value is being set.

To check the cookie value, click on the test waterfall to get to the Details view, then click on the first request to open the modal request viewer. Click on the request tab and check that your cookie header is being set:

![Screenshot of web page test showing cookie data in request header](https://simonhearne.com/images/wpt-cookies/wpt-cookie.png)

If not, you might need to try experimenting with syntax until it works 🙂

#### Consent Button in iframes

Some cookie consent services may load the banner in an iframe, this makes selecting content a little more tricky. First you need to select the iframe document and then select the button / confirmation element within it:

```text
execAndWait document.querySelector('iframe[name=consentFrame]').contentWindow.document.querySelector('#confirmBtn').click()
```

#### Geographic Variations

Some sites may have different policies based on the location of the visitor, for example respecting CCPA in California vs. Cookie Law in EU. I would recommend testing from at least one North American location and one European location. Countries within Europe may have slightly different laws, so testing from the Netherlands (known as relatively strict) is a good choice.

## Results

We need a test subject to use as an example for the following sections. I wanted to use a website which implements cookie consent, but is not heavily driven by third-party content (i.e. not a digital publisher). Corporate websites are great for this, so that's what we'll look at.

The headlines are:

- Opted out experiences are ~35% faster
- Opting in downloads 2.5MB of additional JavaScript
- Opted in repeat views are twice as slow as opted out

The following sections go into a bit more detail about each test. Below are links to the test results:

- [Opted-out first view](https://www.webpagetest.org/result/200512_86_c05f465592d3345f7563427f9ec0b2c3/)
- [Opted-in first view](https://www.webpagetest.org/result/200513_G2_e2b7bbe825a7d280cecce020b83d0ff4/)
- [Opt-in after page load](https://www.webpagetest.org/result/200512_B0_60886c4f172e0e4bc39db656a9217872/)

### First View Comparison

This is the most common type of comparison you are likely to perform, what does a full page load look like with and without consent? I've used the handy [wpt-compare.app](https://wpt-compare.app/) by [Matt Hobbs](https://twitter.com/therealnooshu) to make this a little easier 🙂

<figure class="scroller no-shadow">
<img src="/images/wpt-cookies/filmstrip1_250.png" alt="Filmstrip images showing cookie opt-in is slower" loading="lazy"
srcset="/images/wpt-cookies/filmstrip1_125.png 1x,/images/wpt-cookies/filmstrip1_250.png 2x, /images/wpt-cookies/filmstrip1_375.png 3x">
</figure>
  <figcaption class="scroller">Filmstrip comparison showing similar visual performance. <a href="https://www.webpagetest.org/video/compare.php?tests=200512_86_c05f465592d3345f7563427f9ec0b2c3-l%3AOpted+out-r%3A5%2C200513_G2_e2b7bbe825a7d280cecce020b83d0ff4-l%3AOpted+in-r%3A4&thumbSize=200&ival=500&end=visual&bg=fff&text=000">Source</a></figcaption>

The visual performance comparison shows relatively little difference between the initial render performance as we would expect, except for the visual addition of the cookie banner. The total load time was somewhat higher for the opted-in test, at 5s vs 4.2s for opted-out.

![Filmstrip images showing cookie opt-in is slower](https://simonhearne.com/images/wpt-cookies/timings-first-view.png)
*Timing comparison showing significant increases for opt-in on first view. Source*

Looking at more of the timers we can see some significant degradations. Notably, CPU Busy Time increases 2.5x from 4.9s to 12s, indicating that the third-party content has a significant impact on the browser main thread. This could cause a laggy experience, especially on mobile devices. The visual performance of the page is impacted as well, with the visual complete time increasing by 35% from 5.4s to 7.3s.

The request maps below show the significance of the opt-in on the diversity of domains on the webpage. The number of requests increases 2.4x from 60 to 144, with an additional 30 TCP connections established.

<div class="two-fig-cols">![Request map of opted-out experience showing few domains](https://simonhearne.com/images/wpt-cookies/reqmap-1a.png)
*Requestmap for opted-out experience. Source*![Request map of opted-out experience showing few domains](https://simonhearne.com/images/wpt-cookies/reqmap-2a.png)
*Requestmap for opted-in experience. Source*
</div>

### Repeat View Comparison

Whereas first-party content is likely to be cached for a repeat view, third-party content may be downloaded again. Third-party trackers and analytics scripts will also execute on every page view, potentially increasing the relative impact of these on cached experiences.

<figure class="scroller no-shadow">
<img src="/images/wpt-cookies/filmstrip2_250.png" alt="Filmstrip images showing cookie opt-in is slower" loading="lazy"
srcset="/images/wpt-cookies/filmstrip2_125.png 1x,/images/wpt-cookies/filmstrip2_250.png 2x, /images/wpt-cookies/filmstrip2_375.png 3x">
</figure>
<figcaption class="scroller">Filmstrip comparison showing opt-in performance is slower. <a href="https://www.webpagetest.org/video/compare.php?tests=200512_86_c05f465592d3345f7563427f9ec0b2c3-l%3AOpted+out-r%3A3-c%3A1%2C200513_G2_e2b7bbe825a7d280cecce020b83d0ff4-l%3AOpted+in-r%3A3-c%3A1&thumbSize=200&ival=200&end=visual&bg=fff&text=000">Source</a></figcaption>

The visual performance comparison above shows a close race, with the opted-out experience winning by a small margin of 200ms. Note that the opted-in experience filmstrip continues for another 2.5s. Again, the timers tell a more interesting story:

![Filmstrip images showing cookie opt-in is slower](https://simonhearne.com/images/wpt-cookies/timings-repeat-view.png)
*Timing comparison showing significant increases for opt-in on repeat view. Source*

CPU Busy Time again shows a significant jump, as do the visual metrics. These show that the relative impact of the opted-in experience is much greater on cached views. Page load increases over 160% from 1.2s to 3.2s and visual complete more than doubles from 1.9s to 4.9s!

### Opt-in after page load

The sections above have shown the impact of cookie consent on page load performance, but what about the actual action of consenting? Clicking the button immediately triggers a cascade of requests from third-parties, each hoping to collect as much data as possible. We can measure this performance impact by splitting the navigate and action steps, then we can analyse just the activity which occurs after the opt-in event.

![Filmstrip images showing cookie opt-in is slower](https://simonhearne.com/images/wpt-cookies/optin-breakdown.png)
*Content breakdown for the opt-in interaction. Source*

Almost 2.5MB of JavaScript content is loaded across 33 objects, at a cost of 680kB of bandwidth and 1.8s of download time. Knowing this, would you opt-in to cookies? What's the performance cost for your opt-in experience?

As for the UX of cookie consent, Vitaly Friedman has [written about](https://www.smashingmagazine.com/2019/04/privacy-ux-better-cookie-consent-experiences/) the varied (and extensive) issues with the current state of cookie prompts. Vitaly covers this and more in his excellent [closing keynote](https://www.youtube.com/watch?v=Jz3Fu1o356g) at performance.now() conference 2019.


==============================================================================

# When Network is Faster than Cache

> We assume that browser caches are near-instantaneous, but data shows that retrieving an object from cache can take hundreds of milliseconds!

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2020-05-07
Updated: 2020-05-11
Canonical: https://simonhearne.com/2020/network-faster-than-cache/
Tags: WebPerf

---
## Introduction

I recently discovered that Firefox introduced a feature called `RCWN` (Race Cache With Network) [in 2017](https://bugzilla.mozilla.org/show_bug.cgi?id=1392841). This feature was intended to improve web performance by racing cached requests against the network.

![Firefox network panel showing cached resources 'raced' againt the network](https://simonhearne.com/images/network-faster-than-cache/ff-raced.png)
*Fig 1. Firefox network panel showing cached resources 'raced' againt the network*

This seemed odd to me, surely using a cached response would always be faster than making the whole request again! Well it turns out that in some cases, the network is faster than the cache. There is some [discussion on mailing lists way back in 2016](https://groups.google.com/a/chromium.org/forum/#!msg/net-dev/DGVo2J4GKzc/fWNljySeAQAJ) about the performance differences between browsers. "Firefox cache is performing better than Chrome on windows. Almost 25% fewer >1 second loads" reads one forum. More recently, someone mentioned on the [Web Performance Slack](https://webperformance.herokuapp.com/) that they had seen their cache take over three seconds to retrieve an object! This led me down a bit of a rabbit hole investigating cached responses:

![Chrome network panel showing resources served from cache taking 3.4 seconds](https://simonhearne.com/images/network-faster-than-cache/chrome3scache.png)
*Fig 2. A resource served from cache taking >3,400ms (Chrome v77 on SSD Macbook Pro 2015)*

The data in the mailing groups was relatively sparse and tended to focus on a specific browser, so I wondered if I could use [mPulse](https://www.akamai.com/uk/en/products/performance/mpulse-real-user-monitoring.jsp) to build a bigger picture. mPulse collects resource timing data from every visit to a client's website using the Resource Timing API. It is possible to determine if a resource was served from cache using a simple filter when querying the data: `bool resourceFromCache = (transferredSize == 0 && encodedSize > 0)`. This filter sets a boolean of whether the resource was fetched from cache, based on the resource having a file size, but no network download. This allows us to group aggregated resource timing data by whether it was served from cache or network.

Responses to conditional GET requests (if-not-modified and if-modified-since) will be excluded from this data, as the transferred size is greater than zero due to the presence of response headers.

![Chrome console showing resource timing entry for a cached resource](https://simonhearne.com/images/network-faster-than-cache/cache_RT.png)
*Fig 3. Resource timing entry for resource in Fig 1., showing the timing points available to JavaScript*

So why would a cached response take so long to fetch? Well the answer, frustratingly, is a bit of a shrug. There have been bugs in Chromium with request prioritisation, where cached resources were delayed while the browser fetched higher priority requests over the network. There is also the obvious cost of fetching the resource from memory. It is fair to assume that the memory cache at a CDN or ISP proxy server is higher performance than the average mobile device, and the cost of an additional request over an H/2 connection on a good connection is negligible. For desktop devices, reading from disk may have a significant cost compared to a multi-megabit wired network connection. This comment from one of the firefox bugs raised on `RCWN` shows a desktop user with 1.3% successful network race. While the bug is complaining that the races are too frequent, I'm surprised that they win at all!

> On my OSX box I'm seeing us race more than we probably need to:
>
> Total network request count: 5574
> Cache won count 938
> Net won count 13
>
>That's racing almost 16% of the time, but only winning 1.3% of the time.  We should probably back off on racing a bit in this case, at least.
[Tune RCWN racing parameters (and make them pref-able)](https://bugzilla.mozilla.org/show_bug.cgi?id=1425268)

This strange phenomenon will also mean that you could see your browser making fetches for cached immutable assets! This was recently noticed by [Eric Lawrence](https://twitter.com/ericlaw) of [Fiddler](https://www.telerik.com/fiddler) fame:

<blockquote class="twitter-tweet" data-dnt="true"><p lang="en" dir="ltr">Firefox 78 periodically refetching &quot;immutable&quot; resources: <a href="https://t.co/0UtRiSSpvf">pic.twitter.com/0UtRiSSpvf</a></p>&mdash; Eric Lawrence 🎻 (@ericlaw) <a href="https://twitter.com/ericlaw/status/1257469057319817218?ref_src=twsrc%5Etfw">May 5, 2020</a></blockquote>

This all led me to dig into some data at scale and see if there are any interesting patterns we can use, and what we can do about it! I'll be looking at `cache retrieval` time, which I define as the time between request start and response end, for a resource served from cache.

The results show that cache retrieval performance is impacted most by two factors:

* Device hardware
* Total number of assets served from cache

Read on to learn about the methodology for collecting the data, or skip straight to [the results](#results).

## Data Collection

All data used in this analysis is collected using the open source library [Boomerang](https://github.com/akamai/boomerang). Boomerang is the jQuery of performance measurement in the browser, smoothing over the cracks in browser APIs to provide a consistent source of performance data back to ancient versions of IE. To take a look at the data available in your browser run the following in your developer console:

```js
window.performance.getEntriesByType['resource'](0)
```

This will expose the resource timing data for the first object in the current page (excluding the HTML document - that's in the [Navigation Timing API](https://developer.mozilla.org/en-US/docs/Web/API/Navigation_timing_API) data).

Boomerang will then compress this data using a [clever trie algorithm](https://nicj.net/compressing-resourcetiming/) and send it to a collector, for every pageview. mPulse is an enterprise solution to manage this process, collect and store the data and provide dashboards and alerts. Resource Timing data is stored for 14 days, so I have two weeks of data to analyse at any time. Along with the timing information, mPulse stores a large amount of data about the device which made the request - including manufacturer, available memory, battery state, network state and more! All of this depends on the data being available to JavaScript, so it is not guaranteed across platforms.

### Limitations

There are a number of limitations to data collected from the browser (field data).

* **Timing-Allow-Origin** - resource timing information is not available for cross-origin domains, unless the host opts in by setting a `timing-allow-origin` header. This means that we lose data for approximately 70% of third-party requests. These are rarely cacheable, and it is not a problem that we can fix, so it is ignored.
* **Resource Timing Support** - RT is [reasonably well supported](https://caniuse.com/#feat=resource-timing) at the time of writing, but we do miss Opera browser entirely.
* **Late Loaded Resources** - Boomerang collects data for resources which occur before the beacon is fired, this is typically at window loaded. As such we do not have data for resources loaded later in the page lifecycle.
* **Size of Data** - There could easily be billions of data points for a query across all resource timing data, I tended to narrow down my queries to a minimum of a 24hr time window and use basic data hygeine to exclude bad results.
* **Non-direct Comparison** - It is not trivial to compare directly a cache miss and a cache hit in large-scale RUM data, as this would require a comparison at almost the individual device level. We are using aggregates here to look at general performance rather than a direct comparison of cache vs. network.

## Results

I wanted to find out if there were any common dimensions which will lead to a slow cache retrieval time. We saw earlier that Firefox might be faster than Chrome, but what about small vs. large resources, scripts vs. images, mobile vs. desktop?

You could simply compare statistics on this data, but where's the fun in that? You will find a set of distributions in the following sections, each distribution shows the performance of cache retrieval by the given dimension. Sharp peaks to the left indicate fast and consistent retrieval times, broad and low charts indicate slow and variable retrieval times.

The mailing lists (and the complaint on Slack) showed retrieval times that took multiple seconds. In the data I see this happen extremely infrequently, so instead I focus on retrievals in the time range ≤250ms to spot differences across the dimensions. You can change the scale of both axes using one of the sliders below the charts. There is also a checkbox to show cache miss results (i.e. requests that go to the network), we can expect these to normally be quite slow so the relative height of the chart will be small.

### By Resource Size

My first hypothesis was that there might be a correlation between resource size and cache retrieval duration. It makes sense that it would take longer to retrieve an 800kB image from cache than a 10kB one. Well, it turns out that cache performance is pretty consistent across resource sizes. Once you get to 100kB - 1MB the retrieval times take a slight (<1ms) dip, but overall not a huge shift.

<div class="vega-chart loading" id="vis1" data-spec="/data/sizecachehists.json">Loading chart...</div>

### By Resource Type

My second hypothesis was that browsers might treat different resources differently, a cached CSS asset should be a higher priority than a cached image one might assume. Again, there is no real correlation here.

<div class="vega-chart loading" id="vis2" data-spec="/data/assettypehists.json">Loading chart...</div>

### By Browser

Now we're getting to the good stuff, do different browsers perform differently? My hypothesis was that we would see a difference between browsers used on mobile and desktop devices. We see that this is true, for some browsers!

Desktop Chrome has a much more consistent cache retrieval time than Chrome Mobile, with Samsung Internet showing a really broad distribution. On the other hand, Mobile Safari appears to perform better than desktop Safari - indicative of Apple's continued investment into mobile hardware.

<div class="vega-chart loading" id="vis3" data-spec="/data/browsercachehists.json">Loading chart...</div>

### By Operating System

Ok but Browsers are installed across lots of different devices, what if we broke this down to Operating Systems? Here we're starting to get some interesting data. Desktop Operating Systems perform better than mobile, and Android performs worse than iOS. This makes sense, Apple controls the hardware on which iOS runs, whereas Android (through the Android Open Source Project) could be running on anything from a $25 feature phone right up to a $1,000+ flagship.

<div class="vega-chart loading" id="vis4" data-spec="/data/osnamehists.json">Loading chart...</div>

### By Manufacturer

Let's dig down a bit further and look at the performance for actual devices. Here we see a bubble plot including the device type, operating system and reported device manufacturer. You will notice that the vast majority of bubbles are Android - this is due to a combination of two factors:

* Desktop devices generally do not advertise manufacturer in the User Agent String
* Android has, by far, the greatest manufacturer diversity of all Operating Systems

I've highlighted on the chart those devices which have an median cache retrieval time of >100ms. This is **per resource**, user experiences on these devices are likely to suck. On the other end of the spectrum, high-end manufacturers such as RED, Razer & Apple demonstrate consistently fast retrieval times, with a median <8ms.

<div class="vega-chart loading" id="vis5" data-spec="/data/cachetimings.json">Loading chart...</div>

### A Bit Deeper

I decided to make some focussed queries in order to get a more definitive picture of cache retrieval performance. The most obvious thing to do is track down a single cacheable asset and focus on that. A quick look at the data shows the most common resource is, unsurprisingly, Google Analytics: `analytics.js`.

Unfortunately the Google Analytics JS is not sent with a `timing-allow-origin` header. As such, we cannot tell if it was delivered from cache or not 😔 I modified my search to only include resources which were both cacheable and had the `timing-allow-origin` header. The top result was again from Google, this time their ad tracking script: `conversion_async.js`.

This chart shows the retrieval times for `conversion_async.js` by manufacturer. Try reducing the y-scale to see the cache misses appear on the charts, note that the long-tail of the cache hits overlaps well into the cache miss performance. This shows how often cache performance is worse than network, and how consistently fast Apple devices are.

<div class="vega-chart loading" id="vis6" data-spec="/data/manufacturerhists.json">Loading chart...</div>

## The Obvious Bottleneck

I received a comment from Uwe shortly after publishing this post which got me thinking:

<blockquote class="twitter-tweet" data-conversation="none" data-dnt="true"><p lang="en" dir="ltr">Thank you for this piece of research. <br><br>If resource size does not correlate much with hit/miss, we are back in HTTP/1 times, where the number of requests is crucial: Concatenate scripts and style sheets as much as possible to avoid requests - not only over the network.</p>&mdash; Uwe Trenkner (@utrenkner) <a href="https://twitter.com/utrenkner/status/1258726950388015105?ref_src=twsrc%5Etfw">May 8, 2020</a></blockquote>

Earlier in the post I mentioned that there was a Chromium bug where network requests might block cache requests. Taking that idea a little further: what if our optimisations on the network stack such as HTTP/2 [Request Multiplexing](https://developers.google.com/web/fundamentals/performance/http2#request_and_response_multiplexing) have degraded the performance of cache retrieval? File concatenation is generally considered an [anti-pattern in H/2](https://blog.cloudflare.com/http-2-for-web-developers/#stopconcatenatingfiles), but how do device caches handle concurrent reads?

It turns out that there is indeed a strong correlation between the number of resources retrieved from cache and the average retrieval time! This holds especially true for Android & Chrome OS devices. For example: Chrome OS average cache retrieval doubles from ~50ms with five cached resources up to ~100ms with 25 resources.

<div class="vega-chart loading" id="vis7" data-spec="/data/cachesbydevicetype.json">Loading chart...</div>

This data is from a large dataset, but still shows some variance due to the potentially small dimension set sizes such as less popular operating systems. The data shown is only for page loads that had between 1 and 250 assets retrieved from cache. Data points are only included if there were at least 100 occurrences of the combined dimensions (operating system, device type & number of cached resources).

It turns out that Chrome actively throttles requests, including those to cached resources, to reduce I/O contention. This generally improves performance, but will mean that pages with a large number of cached resources will see a slower retrieval time for each resource.

> The short-term proposal is to throttle H2+QUIC requests just as we do for HTTP/1.1 requests today. This ensures that a started request will have to wait for no more than 9 other requests, reducing the TTFB delay for the first request from 400ms to 40ms. Doing so shows an average 25% reduced time-to-first-contentful-paint (TTFCP) for a cached <https://android.com.>
<p class="align-right pull-up small"><a href="https://docs.google.com/document/d/1Aa7OKFRdtmn4IFzgHYfqeqk5lnyTiv4jkwCzMoAlrTU/edit#heading=h.zdlavartcrrw">source</a></p>

## So What

I was intrigued to dig into this data after reading about browsers racing cache vs. network and I enjoyed pulling data and plotting the charts. But what can we as humble developers actually do with this information?

### The Digital Divide Grows Wider

It turns out that this reinforces a well known but oft-ignored fact: folks on the lower end of the social scale have a worse web browsing experience. We know that cheap devices will have slower processors, less memory and lower bandwidth, and that this leads to a worse perceived page speed. Now we know that even cached experiences are slow for these users. This spreads our concept of a digital speed divide even wider. We also know a little more about browser internals!

The best thing we can do to deliver an inclusive web is to deliver the minimal viable experience, especially to low-end devices. This will benefit all users.

### Cache Will Not Save Us

There is an assumption that cached assets are retrieved instantly and at zero cost. What we have discovered here is that there is in fact a cost to retrieving assets from cache based on the number of cached assets (not file size) and the user's devices. Concatenating / bundling your assets is **probably** still a good practice, even on H/2 connections. Obviously this comes on balance with cache eviction costs and splitting pre- and post-load bundles. Make sure you test performance on real devices, and definitely start tracking [repeat-view performance](https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/quick-start-quide#TOC-Repeat-View) of changes you make to bundling and cache configuration!

If nothing else, you might remember this the next time you're stumped by some strange cache behaviour 🙂

Let me know by interacting below if you have any thoughts on cache performance, or if you have ideas on other interesting dimensions to drill in to!


==============================================================================

# Is it time for a Web Performance rebrand?

> Web Performance is a niche term in a broad market, should we widen our appeal to reach more people?

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2020-05-03
Canonical: https://simonhearne.com/2020/web-performance-rebrand/
Tags: WebPerf

---
Web performance has been a hot topic in web development and browser engineering since the first public web pages. The first book on the topic, [Web Performance Tuning: Speeding Up the Web](https://www.amazon.co.uk/Web-Performance-Tuning-Speeding/dp/059600172X/ref=as_li_ss_tl?ie=UTF8&linkCode=ll1&tag=webnin-21) was published in 1998, with Steve Souders kicking off the Web Performance Optimisation industry when he published his book [High Performance Websites](https://www.amazon.co.uk/High-Performance-Web-Sites-Essential/dp/0596529309/ref=as_li_ss_tl?dchild=1&keywords=high+performance+websites&qid=1588522271&s=books&sr=1-1&linkCode=ll1&tag=webnin-21) nine years later in 2007.

As an industry-within-an-industry we have built resources, adopted hashtags ([#WebPerf](https://twitter.com/search?q=%23webperf), [#PerfMatters](https://twitter.com/search?q=%23perfmatters)) and created meetups and conferences ([LDNWebPerf](https://ldnwebperf.org), [#PerfMatters](https://perfmattersconf.com), [Performance.now](https://perfnow.nl)). We've focused in on a subset of web development, whilst simultaneously complaining that it's hard to get business buy-in and wider appreciation of the topic. We've even [created a site](https://wpostats.com) to collect proof that web performance is important to business success! We know that web performance is important, but we consistently struggle to convince the wider world to prioritise speed as a feature.

Something that has long troubled me when discussing web performance with customers (and friends outside the industry) is that the term "web performance" is ambiguous, it means different things to different people. To someone in marketing it might mean the performance of web campaigns, measured in click-through rate and engagements. To ecommerce it might mean the business performance of the website, measured in conversions and revenue. To SEOs it could be the ranking of key pages, and to performance artists it means using the web as a medium for their art!

This ambiguity can work in our favour; it means our messaging lands with lots of different parties within a business: everyone wants to improve web performance! This often leads to confusion in the first meeting, requiring early clarification of what web performance means to us and our tools & services. I find myself describing *our* version of web performance as "measuring and optimising the speed of a web or mobile application to maximise user experience and business success".

> Web Performance is measuring and optimising the speed of a web or mobile application to maximise user experience and business success

If we are to correctly engage the broadest possible audience with our messaging, we could do better with more targeted language. Should we be using a different term when generating marketing material and engaging with external stakeholders?

<blockquote class="twitter-tweet" data-dnt="true"><p lang="en" dir="ltr">What is <a href="https://twitter.com/hashtag/webperf?src=hash&amp;ref_src=twsrc%5Etfw">#webperf</a> most commonly known as, outside of the web development industry? (E.g. Marketing, UX, Execs)</p>&mdash; Simon Hearne (@simonhearne) <a href="https://twitter.com/simonhearne/status/1256103539342000128?ref_src=twsrc%5Etfw">May 1, 2020</a></blockquote><script async src="/js/twitter_widgets.js" charset="utf-8"></script>

Site speed or page speed are often mentioned by executives and analysts. I think we can assume that Google is to thank here! Web performance stats in Google Analytics are under the `Site Speed` section, and `PageSpeed Insights` is the Google tool to [find performance issues](https://developers.google.com/speed/pagespeed/insights/). How often do we hear "We need to improve page speed" vs. "We need to improve web performance"?

Whilst certainly not scientific, a quick look on Google Trends shows that `page speed` is by far the most common search term, followed at some distance by `site speed`, with `web performance` barely registering at ~5% relative interest.

![Google trends chart showing page speed has the highest popularity, followed by site speed, with web performance only 5% relative interest](https://simonhearne.com/images/rebrand2.png)
*Google Trends for top three web performance terms. Link to original query.*

Take a look at [#SiteSpeed](https://twitter.com/search?q=%23SiteSpeed) on Twitter and you will see tweets from a range of SEO, business and Marketing folks, whereas the [#WebPerf](https://twitter.com/search?q=%23WebPerf) hashtag is mainly technical tweets from folks in the industry, tools providers or front-end developers.

Web Performance and Web Performance Optimization are still valid and descriptive terms for our industry, but we might benefit from a change to our language when working with others. The language we use could be critical to the success of making the web a faster and more accessible place.

I'll be testing this out with my messaging to clients, I'd be interested in your thoughts too. Some other ideas from the community:

* **SSO** (Site Speed Optimisation) - from [Aaron Peters](https://twitter.com/aaronpeters)
* **UXSpeed** - from [Sergey Chernyshev](https://twitter.com/sergeyche)
* **WebSpeed** - from [Šime Vidas](https://twitter.com/simevidas)


==============================================================================

# Collecting SmartThings data in InfluxDB on Raspberry Pi

> How to collect data from your Home Automation system into a local timeseries database.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2020-01-07
Canonical: https://simonhearne.com/2020/pi-smartthings-influx/
Tags: homeautomation, raspberrypi

---
Now that you've [got your Pi set up with Influx & Grafana](/2020/pi-influx-grafana/), you're [collecting some system stats](/2020/pi-speedtest-influx/), and [measuring network performance](/2020/pi-speetest-influx/), it's time to collect data from SmartThings!

[Samsung SmartThings](https://www.samsung.com/uk/smartthings/) is a multi-protocol smart home hub produced by Samsung, it supports Z-Wave, Zigbee and Bluetooth and is surprisingly configurable. One of my main frustrations with the service is the lack of visibility into the huge amounts of data that it collects, so I decided to store that data myself! This way I can query it and visualise it in Grafana, set alerts and get insight into how my home is performing.

## Step 0: Set up InfluxDB and Grafana

This post assumes that you have an InfluxDB and Grafana server running already, take a look at my post on [getting a Raspberry Pi configured](/2020/pi-influx-grafana/) if you need to get set up first.

## Step 1: Adding the SmartApp

We need to install a SmartApp to SmartThings to collect the sensor data. This is a relatively straighforward process but is daunting the first time you do it.

1. Log in to [the IDE](https://graph.api.smartthings.com/)
2. Navigate to the `My SmartApps` section
3. Click the blue `+ New SmartApp` button in the top right of the screen
4. Select the `From Code` tab
5. Copy and Paste the groovy code from [influxdb-logger](https://raw.githubusercontent.com/codersaur/SmartThings/master/smartapps/influxdb-logger/influxdb-logger.groovy)
6. Hit `Create` - you should see a green "Created SmartApp" banner
7. Click the `Publish -> For Me` button to make the app available

## Step 2: Configure the Logger

Now the SmartApp is enabled, you should see it in your SmartThings mobile application under SmartApps:

![New InfluxDB Logger SmartApp.](https://simonhearne.com/images/pi-smartthings-influx/sm_2.png)

Click on the app to open the settings:

![Configuring InfluxDB Settings.](https://simonhearne.com/images/pi-smartthings-influx/sm_3.png)

Select all devices which you would like to log:

![Selecting Devices to Log.](https://simonhearne.com/images/pi-smartthings-influx/sm_4.png)

Save the settings and make sure that the app is enabled, you'll see data in your InfluxDB immediately if everything is working correctly. You can debug by viewing the live logs of your SmartThings location - click on `Live Logging` in the menu of the SmartThings IDE:

![Viewing Live Logs in SmartThings.](https://simonhearne.com/images/pi-smartthings-influx/sm_5.png)

With any luck you'll have an InfluxDB filling up with device data!


==============================================================================

# Collecting SpeedTest results in Influx on Raspberry Pi

> How to set up regular SpeedTest reports into a local InfluxDB on a Raspberry Pi

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2020-01-07
Updated: 2020-05-01
Canonical: https://simonhearne.com/2020/pi-speedtest-influx/
Tags: homeautomation, raspberrypi

---
Now that you've [got your Pi set up with Influx & Grafana](/2020/pi-influx-grafana/) and you're [collecting some system stats](/2020/pi-metrics-influx/), it's time to measure network performance!

[SpeedTest.net](https://www.speedtest.net/) by Ookla is probably the most popular connection testing services, and they handily [provide a CLI](https://www.speedtest.net/apps/cli) to run tests programmatically. We'll create a simple python script which runs a test and sends the data to influxdb, then set it to run once every 15 minutes using Cron.

First, we need to install the `speedtest-cli` client:

```bash
sudo apt install -y python-pip
sudo pip install speedtest-cli influxdb
```

Then we can create a python script `rpi-speedtest-influx.py` in our home directory to run tests:

```python
#!/usr/bin/env python

import datetime
import speedtest
from influxdb import InfluxDBClient

# influx configuration - edit these
ifuser = "grafana"
ifpass = "<yourpassword>"
ifdb   = "home"
ifhost = "127.0.0.1"
ifport = 8086
measurement_name = "speedtest"

# take a timestamp for this measurement
time = datetime.datetime.utcnow()

# run a single-threaded speedtest using default server
s = speedtest.Speedtest()
s.get_best_server()
s.download(threads=1)
s.upload(threads=1)
res = s.results.dict()

# format the data as a single measurement for influx
body = [
    {
        "measurement": measurement_name,
        "time": time,
        "fields": {
            "download": res["download"],
            "upload": res["upload"],
            "ping": res["ping"]
        }
    }
]

# connect to influx
ifclient = InfluxDBClient(ifhost,ifport,ifuser,ifpass,ifdb)

# write the measurement
ifclient.write_points(body)
```

[View gist on GitHub](https://gist.github.com/simonhearne/eabf033bae8cee77972b9e96d2775eea#file-rpi-speedtest-influx-py)

We'll need to make it executable: `chmod +x rpi-speedtest-influx.py` and then we can run it to test it out: `./rpi-speedtest-influx.py`. You should be able to see the measurement in influx as soon as the script has exited, check by running `influx` CLI and execute the following to check your measurement (swapping database and measurement names for those you defined in your python script):

```sql
use home
select * from speedtest
```

If everything looks ok, we can add a cron job for our user (using `crontab -e`) to run this script at a regular interval, e.g. every 15 minutes: `*/15 * * * * /home/pi/rpi-speedtest-influx.py`. Any more than 15 minutes is probably unnecessary and may have an adverse effect on your perceived network quality.

You can now simply add a panel to your Grafana instance to show the download, upload and ping results over time. [Here is a sample dashboard](https://gist.github.com/simonhearne/1b2b62e818fe6e097f8f8b2cbd6f2365) you can import which includes the panel shown below:

![SpeedTest Grafana dashboard.](https://simonhearne.com/images/grafana-net-stats.png)


==============================================================================

# Reporting Raspberry Pi System Metrics to InfluxDB

> How to set up regular system stat reports from a Raspberry Pi into a local InfluxDB

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2020-01-07
Canonical: https://simonhearne.com/2020/pi-metrics-influx/
Tags: homeautomation, raspberrypi

---
Now that you've [got your Pi set up with Influx & Grafana](/2020/pi-influx-grafana/), it's time to start collecting some data! We will use a simple Python script to collect system statistics and push it to Influx - you can use this as-is or modify it to your needs.

We will need to first install the PIP package manager if it is not already on your Pi:

```bash
sudo apt install -y python-pip
```

And then use pip to install the python packages `psutil` and `influxdb`:

```bash
sudo pip install psutil influxdb
```

The python script below will use `psutil` to collect some system stats and `influx` to insert them into your influx db:

```python
#!/usr/bin/env python

import datetime
import psutil
from influxdb import InfluxDBClient

# influx configuration - edit these
ifuser = "grafana"
ifpass = "<yourpassword>"
ifdb   = "home"
ifhost = "127.0.0.1"
ifport = 8086
measurement_name = "system"

# take a timestamp for this measurement
time = datetime.datetime.utcnow()

# collect some stats from psutil
disk = psutil.disk_usage('/')
mem = psutil.virtual_memory()
load = psutil.getloadavg()

# format the data as a single measurement for influx
body = [
    {
        "measurement": measurement_name,
        "time": time,
        "fields": {
            "load_1": load[0],
            "load_5": load[1],
            "load_15": load[2],
            "disk_percent": disk.percent,
            "disk_free": disk.free,
            "disk_used": disk.used,
            "mem_percent": mem.percent,
            "mem_free": mem.free,
            "mem_used": mem.used
        }
    }
]

# connect to influx
ifclient = InfluxDBClient(ifhost,ifport,ifuser,ifpass,ifdb)

# write the measurement
ifclient.write_points(body)
```

[View gist on GitHub](https://gist.github.com/simonhearne/7a8ef5a792050b89bd708fb754be0bb4#file-rpi-stats-influx-py)

Create this script in your home directory, e.g. `/home/pi/rpi-stats-influx.py` and make the script executable with `chmod +x rpi-stats-influx.py`.

Test the script out by running it with `./rpi-stats-influx.py` and check that no errors are thrown. If all is good, we can set the script to run reqularly using `cron`

Run `crontab -e` to edit your user's crontab and add an entry `* * * * * /home/pi/rpi-stats-influx.py` to the bottom of the file, this will run the script every minute. The script is very lightweight so running every minute should be fine.

See [the docs for `psutil`](https://psutil.readthedocs.io/en/latest/) to get ideas for other stats you can collect.

You can now simply add a panel to your Grafana instance to show your system stats over time. [Here is a sample dashboard](https://gist.github.com/simonhearne/e4fd4d9f94c302a5847324a48724c6eb) you can import which includes the panel shown below:

![Pi Stats Grafana dashboard.](https://simonhearne.com/images/grafana-pi-stats.png)


==============================================================================

# Installing InfluxDB & Grafana on Raspberry Pi

> I recently treated myself to a RaspberryPi 4 Model B to use as a(nother) home server. This is a brief guide on getting InfluxDB and Grafana set up to store and graph timeseries data.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2020-01-07
Canonical: https://simonhearne.com/2020/pi-influx-grafana/
Tags: homeautomation, raspberrypi

---
## Step 0: Initial Setup

Follow these steps first if you have a brand new rPi:

1. Download the latest *lite* Raspbian image from [raspberrypi.org](https://www.raspberrypi.org/downloads/raspbian/)

2. Get a reasonable micro SD card - 32GB is the maximum supported size. I use a [SanDisk Ultra](https://amzn.to/37DqONM)

3. Burn the image to your SD card, I recommend using [balenaEtcher](https://www.balena.io/etcher/).

4. Balena will eject your SD card once it is complete, re-mount your card by physically removing and re-inserting it, then create an empty file called `ssh` in the root of the SD card - this enables SSH access which we'll need later.

5. Insert the SD card into your Pi, connect to your router via ethernet and power on.

6. Determine the auto-assigned IP address of the Pi by logging in to your router interface (see a guide on finding your router IP address [here](https://nordvpn.com/blog/find-router-ip-address/)) and navigating to LAN / DHCP settings - the pi should be recognised as `raspberrypi`. Now is a good time to assign a static IP for your Pi to make life easier in the future. You'll need to power cycle your Pi if you change from the auto assigned IP address.

7. SSH into the Pi using the IP address you have determined / assigned: `ssh pi@<yourip>`. You should probably update the password for the `pi` user now by running `passwd`.

8. (optional!) put your Pi in a case to keep it safe and cool. I found a cheap (£6 / $8) [case with heatsinks and fan](https://amzn.to/2rXT924) on Amazon.

## Step 1: Getting up to date

First off we'll make sure everything is up to date. This could take a while, especially on a new Pi:

```bash
sudo apt update
sudo apt upgrade -y
```

## Step 2: Install Influxdb

First we add Influx repositories to apt:

```bash
wget -qO- https://repos.influxdata.com/influxdb.key | sudo apt-key add -
source /etc/os-release
echo "deb https://repos.influxdata.com/debian $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/influxdb.list
```

Update apt with the new repos, & install.

```bash
sudo apt update && sudo apt install -y influxdb
```

Then start the `influxdb` service and set it to run at boot:

```bash
sudo systemctl unmask influxdb.service
sudo systemctl start influxdb
sudo systemctl enable influxdb.service
```

We should now be able to run the influx client with `influx` and create a user for later (here I use a single admin user `grafana` for simplicity):

```bash
create database home
use home

create user grafana with password '<passwordhere>' with all privileges
grant all privileges on home to grafana

show users

user admin
---- -----
grafana true
```

That's it! You can now exit the Influx client by typing `exit`.

## Step 3: Install Grafana

Again we need to add the Grafana packages to apt:

```bash
wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
echo "deb https://packages.grafana.com/oss/deb stable main" | sudo tee /etc/apt/sources.list.d/grafana.list
```

We can now update and install the binaries:

```bash
sudo apt update && sudo apt install -y grafana
```

Then simply enable the service and set to run at boot:

```bash
sudo systemctl unmask grafana-server.service
sudo systemctl start grafana-server
sudo systemctl enable grafana-server.service
```

Now we can check that grafana is up by loading it in a browser: `http://<ipaddress>:3000`. If so, you can log in with the username and password = `admin` and set a new admin password.

## Step 4: Add Influx as a Grafana data source

Now we have both Influx and Grafana running, we can stitch them together. Log in to your Grafana instance and head to "Data Sources". Select "Add new Data Source" and find InfluxDB under "Timeseries Databases".

As we are running both services on the same Pi, set the URL to localhost and use the default influx port of `8086`:

![Grafana configuration for influxdb 1](/images/grafana1.png)

We then need to add the database, user and password that we set earlier:

![Grafana configuration for influxdb 2](/images/grafana2.png)

That's all we need! Now go ahead and hit "Save & Test" to connect everything together:

![Grafana configuration for influxdb 3](/images/grafana3.png)

## Step 5: Collecting some data

Now you've got your Influx & Grafana setup running, you can [collect system stats](/2020/pi-metrics-influx), run [regular network speed tests](/2020/pi-speedtest-influx) and collect [data from your home automation system](/2020/pi-smartthings-influx)!


==============================================================================

# HTTP Headers for fast & secure static sites

> An introduction to key HTTP response headers for speed and security, with implementation guides for Netlify & CloudFlare

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2019-12-10
Canonical: https://simonhearne.com/2019/http-headers-fast-and-secure/
Tags: WebPerf, security

---
This website is powered by Netlify, it also has Content-Security-Policy and cache-control HTTP response headers to improve client security and performance. In this short article I'll describe the key response headers you should be aware of and how how I configured them on Netlify. The configuration is simple when you know how, but it can take a while to work out!

## Intro to HTTP Headers

I am discussing HTTP response headers in this article. These are a set of key-value pairs sent by a web server at the beginning of a response, separated by a colon. The headers are directives which tell HTTP clients how to treat responses (where clients can be CDNs, proxies and web browsers). No headers are required by the HTTP protocol, but `status` is a requirement for an HTTP response.

You can see response headers in your browser developer tools, just select a response and view the headers:

![Screenshot of response headers from chrome developer tools](https://simonhearne.com/images/resp_headers.png)
*Response headers shown in Chrome Developer Tools for a page on this site.*

In the following sections we will walk through some of the key use cases for response headers, when to use them and what they look like.

### Content

Content headers are critical for content that can be negotiated between client and server, such as image formats and encoding algorithms. The following response headers indicate that the response body is binary encoded using gzip, and that the content is an HTML document with a UTF8 charset:

```http
Content-Encoding: gzip
Content-Type: text/html; charset=utf-8
```

Thankfully your CMS or CDN should take care of these for you. One thing to look out for is support for gzip and brotli compression. If there is no `Content-Encoding` header on text files (HTML, CSS, JS etc.) than your static assets are being sent uncompressed. Some providers (such as [Heroku](https://devcenter.heroku.com/articles/compressing-http-messages-with-gzip)) do not compress content for you, so you'll need to implement it in your application.

### Caching

There are a wide range of response headers which impact how browsers cache content. Caching static content is important for delivering fast page loads: the fastest request is the one that is never made. Some content should never be cached, however, such as responses from a login or balance API or pages with very dynamic content. Response headers provide fine-grained control over how HTTP clients cache content. Caching headers should be present on every response, below are some examples of the Cache-Control header for a number of scenarios:

```http
Cache-Control: public, max-age=31536000, immutable # any cache can store this for a year use it without revalidating
Cache-Control: public, max-age=2628000 # any cache can store this for a month
Cache-Control: private, max-age=3600 # the browser may store this for one hour
Cache-Control: no-store # no cache may store this at all
```

Unfortunately, [Netlify defaults to](https://www.netlify.com/blog/2017/02/23/better-living-through-caching/) `Cache-Control: max-age=0, must-revalidate, public` and an `ETag` header on all responses, instructing caches not to serve cached content without revalidating against the origin that asset has not changed. This can have a negative impact on performance, especially for users on high-latency connections, as a conditional GET request will be made for all resources. It does, however, mean that changes you make to your site are reflected on the web almost immediately. You can override default Netlify behaviour by setting the response header to blank, e.g. `Etag = ""`. This might be particularly useful to prevent revalidation on assets such as webfonts, to prevent a flash of [unstyled or invisible text](https://css-tricks.com/fout-foit-foft/) while the page renders and revalidates the cached font files.

There are a bunch of headers which impact caching: `Etag`, `Age`, `Last-Modified`, `Expires` and `Pragma` are all considered by the browsers when determining cache state, although `Cache-Control` supersedes some behaviours such as `Expires`. Further reading: [HTTP Cache Headers on Key CDN](https://www.keycdn.com/blog/http-cache-headers).

### Content Security Policy

There are a number of headers which enable security features in browsers. The most exciting (for me) header is Content-Security-Policy (CSP), this gives the browser a list of approved content sources, and what they are allowed to do.

A number of recent public cases of [Magecart attacks](https://threatpost.com/magecart-ecommerce-card-skimming-bonanza/147765/), data skimming and compromised third-parties make the CSP header a must-have for any transactional site. CSP will also block any malicious browser extensions from injecting ads onto your pages which slow the experience down. There is no reason to not set up the CSP header on your static site.

CSP directives align roughly with HTML5 and JavaScript functionality, a full list of directives is included below:

<style>
  table {width: fit-content;margin: 1em auto}
  table th {text-align: left}
  table td:first-child, th:first-child {text-align: right; padding-right: 1em}
  table tr:nth-child(even) {background-color: rgba(128,128,128,0.1)}
</style>
| CSP Directive                        | HTML / JS Features             |
|--------------------------------------|--------------------------------|
| `default-src`                        | `*`                            |
| `connect-src`                        | `fetch()`, `WebSocket()`, etc. |
| `style-src`                          | `<link rel="stylesheet">`      |
| `script-src`                         | `<script>`                     |
| `form-action`                        | `<form>`                       |
| `font-src`                           | `@font-face`                   |
| `child-src`                          | `<iframe>`, `Worker()`         |
| `object-src`                         | `<object>`, `<embed>`          |
| `media-src`                          | `<video>`, `<audio>`           |
| `img-src`                            | `<img>`                        |
| `manifest-src`                       | `<link rel="manifest">`        |

CSP also allows violations reports to be sent to specified URLs, using a deprecated `report-uri` directive and its replacement, [the Reporting API](https://developers.google.com/web/updates/2018/09/reportingapi), via a `report-to` header. CSP headers should *only* be sent on document responses, i.e. the HTML file.

An example CSP directive for this site is shown below. It allows some inline scripts (such as the Google Analytics snippet), scripts from Twitter and embedded content from [noti.st](http://noti.st). Al other content will be blocked, and a CSP violation report will be sent to [report-uri.com](http://report-uri.com) for analysis.

```http
default-src 'self';
  script-src 'self' 'sha256-6/iD6t0SQvujSE2Zwae43Lq7XJSEA98rpBEWsYJd5RU=' platform.twitter.com ...;
  img-src 'self' data: simonhearne.com *.google-analytics.com *.twitter.com *.twimg.com res.cloudinary.com caniuse.bitsofco.de;
  connect-src 'self';
  style-src 'self' 'unsafe-inline' platform.twitter.com;
  child-src 'self' *.twitter.com noti.st caniuse.bitsofco.de;
  worker-src 'self';
  report-uri https://simonhearne.report-uri.com/a/d/g;
  report-to report-uri;
```

Inline scripts are generally a bad idea, but CSP allows us to define which should be allowed to excute by either providing a `nonce` - a unique identifier, or a hash of the script content. As pointed out by Amier Saleh [on twitter](https://twitter.com/AmierOSaleh/status/1206706768555855872?s=20), it is important not to use `nonce` on static sites, otherwise an attacker can simply copy the nonce string to inject their `<script>`! As such, we must use hashes for inline scripts. Helpfully, Chrome will tell you what the hash should be when it finds an inline script which violates your policy:

![Screenshot of suggested hash value from chrome developer tools](https://simonhearne.com/images/sha_chrome.png)
*Suggested hash value shown in Chrome Developer Tools for inline script.*

Further reading: [HTTP Security Headers on keycdn](https://www.keycdn.com/blog/http-security-headers) and [Google's CSP FAQs](https://csp.withgoogle.com/docs/faq.html)

### HSTS

HTTP Strict Transport Security (HSTS) is a simple header which tells a browser that your site should not be accessed via non-encrypted (i.e. `http`) connections. This prevents https downgrade attacks and ensures that your visitors' information is sent securely.

While services like Netlify and CloudFlare offer https upgrade via redirects, the `hsts` header is a belt-and-braces approach to ensure that https is always used. This will prevent potential man-in-the-middle attacks which attempt to steal cookie data by making requests to non-existent non-secure subdomains. All non-secure requests will be automatically upgraded to https *before* the requests are made, preventing any data being accidentally sent over plaintext.

If you enable an HSTS header, ensure that your HTTPS configuration is correct! Browsers which have seen the header will refuse to connect over http for the duration of the max-age directive, including if the certificate is not trusted for any reason. Because of the risk to the availability of your site, it may be worth using a short max-age directive. Below is an example which tells the browser to enforce HTTPS connections across all subdomains for 24 hours. The recommended max-age is two years. A `preload` directive may be added to the header, which hints to browsers that this domain should always be accessed over https, this can be important as HSTS headers are only valid on sites served over secure connections.

```http
Strict-Transport-Security: max-age=86400; includeSubDomains
```

Further reading: [Understanding HSTS by Troy Hunt](https://www.troyhunt.com/understanding-http-strict-transport/) and [HSTSPreload.org](https://hstspreload.org/)

### Network Error Logging

Network Error Logging (NEL) instructs browsers to send reports to a defined endpoint when it fails to load a page. NEL reports are sent in a number of situations, from DNS resolution failures to 5xx and 4xx server responses. This opens a whole new world of reporting, some of which has previously been a black-box to website owners. Note in the example below I have added a Reporting API header which defines the reporting group for the NEL header to use. NEL headers should *only* be sent on document responses, i.e. the HTML file.

```http
Report-To: {"group":"report-uri","max_age":31536000,"endpoints":[{"url":"https://simonhearne.report-uri.com/a/d/g"}],"include_subdomains":true}
NEL: {"report_to":"report-uri","max_age":31536000,"include_subdomains":true}
```

Further reading: [Network Error Logging: Deep Dive by Scott Helme](https://scotthelme.co.uk/network-error-logging-deep-dive/)

### Feature Policy

Feature Policy is like CSP, but for web platform features. This header allows you to disable access to features such as the camera, microphone and geolocation on your site. This prevents any malicious attempts to use these features by third-party scripts.

A reasonable assumption can be made that you know whether any part of your site needs access to these features, and if not you should set a feature policy which disables access:

```http
Feature-Policy : camera 'none'; geolocation 'none'; microphone 'none'
```

Policy headers can be split out, and like CSP you can limit access to the parent page or to specific domains:

```http
Feature-Policy: unsized-media 'none'
Feature-Policy: geolocation 'self' https://example.com
Feature-Policy: camera *;
```

Further reading: [Introduction to Feature Policy on Google Web Fundamentals](https://developers.google.com/web/updates/2018/06/feature-policy)

### Client Hints

Client Hints instruct the browser to send information about itself along with requests. This includes information about the current connection, device memory, screen resolution and whether the user has opted in to features such as Lite Pages. This can greatly simplify the deployment of responsive images and dynamic content based on device capability. I do not currently use this data for anything dynamic, but it is trivial to see how this can be integrated into an image delivery solution. Client hint headers should *only* be sent on document responses, i.e. the HTML file.

```http
Accept-CH: Downlink,RTT,Device-Memory,Save-Data,DPR,Width
```

Further reading: [Automating Resource Selection with Client Hints on Google Web Fundamentals](https://developers.google.com/web/updates/2015/09/automating-resource-selection-with-client-hints)

### Custom

Headers generally have to match the defined set of header keys, but there is an allowance for custom headers. All custom headers must be prefixed with `x-`, but then can be anything you like. These are often used to include debug information, such as which server served the content, what policies were applied and de facto standard headers such as `X-Forwarded-For` [used by CDNs](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For). If you have a [wordpress.com](http://wordpress.com) site, you may see these humorous headers:

```http
x-hacker: If you're reading this, you should visit automattic.com/jobs and apply to join the fun, mention this header.
x-nananana: Batcache
```

## Configuring Headers on Netlify

So now we know what headers are used for, let's configure them for a Netlify site. I'll use the `netlify.toml` [file-based configuration](https://docs.netlify.com/configure-builds/file-based-configuration/) in this example, but it could be translated easily in to the `_headers` [configuration](https://docs.netlify.com/routing/headers/). Unfortunately, there is [currently no way](https://community.netlify.com/t/setting-response-headers-only-on-documents/) to set a header on a document but not on static assets. As such, we have to set our security headers on all responses. This is not ideal, and in the H/1.1 world would be untenable due to the overhead of the header data. Luckily, H/2 [compresses header data](https://blog.cloudflare.com/hpack-the-silent-killer-feature-of-http-2/) and uses a shared dictionary which reduces the impact of this Netlify limitation. If you have Cloudflare in front of your Netlify configuration, [this worker script](https://gist.github.com/simonhearne/2715250eaccb911ebfff9e2315b2ffee) will strip unnecessary headers from static assets.

Note that header values which include double quotes must be escaped with toml multiline string markers: triple single quotes `'''`.

```toml
# netlify.toml

[[headers]]
  # Set the default header to the one we want for documents
  for = "/*"
    [headers.values]
    cache-control = "public,max-age=60"
    Referrer-Policy = "no-referrer-when-downgrade"
    X-Frame-Options = "SAMEORIGIN"
    X-Content-Type-Options = "nosniff"
    Feature-Policy = "camera 'none'; geolocation 'none'; microphone 'none'"
    Strict-Transport-Security = "max-age=2592000; includeSubDomains"
    Accept-CH = "Downlink,RTT,Device-Memory,Save-Data,DPR,Width"
    Report-To = '''{
      "group":"report-uri",
      "max_age":31536000,
      "endpoints":[
        {"url":"https://simonhearne.report-uri.com/a/d/g"}
      ],"include_subdomains":true}'''
    NEL = '''{
      "report_to":"report-uri",
      "max_age":31536000,
      "include_subdomains":true
    }'''
    Content-Security-Policy = '''
    default-src 'self';
    script-src 'self' 'sha256-i+rYvjE2MLQRpsO7Qygbp0RqJCgJE9pO2xyOIhf5LZE=' www.google-analytics.com ...;
    img-src 'self' data: simonhearne.com *.google-analytics.com ...;
    connect-src 'self';
    style-src 'self' 'unsafe-inline' platform.twitter.com;
    child-src 'self' *.twitter.com noti.st;
    worker-src 'self';
    report-uri https://simonhearne.report-uri.com/a/d/g;
    report-to report-uri;'''

[[headers]]
  # Override cache duration for assets with periods in the filename (i.e. static assets)
  for = "/*.*"
    [headers.values]
    cache-control = "public,max-age=360000"
```

## Configuring Headers on CloudFlare

If you are using CloudFlare in front of GitHub Pages for your static site, chances are that your caching is less than ideal. The default cache duration for static assets on GitHub Pages is five minutes, and pages are not cacheable. CDNs will generally be transparent, so these cache headers will not be changed.

CloudFlare configuration allows you to set a standard cache duration across all assets with a simple dropdown selection. While this makes it easy to improve cache durations, there is no granularity to cache some assets for different durations.

![Screenshot of CloudFlare UI to configure cache durations](https://simonhearne.com/images/cf_cache.jpg)
*Coarse cache control configuration in Cloudflare.*

If you want more fine-grained control over headers, you will need to move your site away from GitHub pages or use CloudFlare workers to set custom headers. This is also required to set `Report-To` and `Content-Security-Policy` headers. Scott Helme has posted a recommended worker script to add important security headers: [worker.js on GitHub](https://github.com/securityheaders/security-headers-cloudflare-worker/blob/master/worker.js).


==============================================================================

# Six Web Performance Technologies to Watch in 2020

> Don't bet on 5G and HTTP/3 to make things faster, there are six technologies that I believe will have a bigger impact on web performance in 2020.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2019-11-25
Canonical: https://simonhearne.com/2019/2020-predictions/
Tags: WebPerf

---
## Introduction

Reading the technical press you would be forgiven for thinking that 2020 is going to be a great year for web performance. Repeatedly touted are the blazing fast speeds we will achieve with 5G, the fundamental improvements that HTTP/3 will bring and how new devices will be faster than premium laptops.

I fear this outlook could make us lackadaisical about performance. 5G isn’t expected to reach even 20% of subscribers by 2025, HTTP/3 doesn’t change the speed of light nor the size of our JavaScript bundles, and ultra-fast devices will only reach the hands of the affluent few. In this article I'll look at some of the technologies which I believe could have a significant impact on web performance in 2020.

You can jump straight ahead to the sections on:

1. [JAMstack](#1---jamstack)
2. [Web Assembly](#2---wasm-web-assembly)
3. [Edge Compute](#3---edge-compute)
4. [Observability](#4---observability)
5. [Browser Platform Improvements](#5---platform-improvements)
6. [Web Monetisation](#6---web-monetisation)
7. [Honourable Mentions](#7---honourable-mentions)

> Tl;dr: there will be no quick fixes. If we want a fast web in 2020 we have to take action now.

![](https://simonhearne.com/images/2020-predictions/gsma_5g.svg)
*Predicted 5G adoption rates.*

5G will deploy slowly and will never replace 4G, that’s not the point of it. I believe we will see better traction with 5G home broadband than mobile connections. Not least because handling multiple gigabits per second of data will extract more juice from your battery than the technology of 2020 allows.

HTTP/3 will hopefully fix the [flawed implementations](https://github.com/andydavies/http2-prioritization-issues) of stream prioritisation in HTTP/2. The implementation of the protocol is fundamentally different from H/2 (and H/1.x for that matter) but it still works within the same constraints. It will not rely on TCP’s implementation of congestion control - but we do still control, so it will be handled at the application layer instead. H/2 was hailed as a breakthrough for performance over H/1, until it was widely launched. For many sites it made little difference, for some it made a positive improvement and for some it made things worse. H/3 will likely show a similar outcome, with the actual gains defined by the architecture of individual sites.

Both of these technologies, 5G and H/3, sit at the network level - how the pipes are built and how to best get information across them. But in 2019 JavaScript has the single biggest impact on user experience. Shipping a 1MB JS application bundle to a mid-range Android phone 500ms faster will do little to help the time it takes to process the code. I say mid-range Android phone because that is the median device that web users have, Apple’s market share in terms of 2019 Q1 sales in the [US is 39%, with Samsung at 28%](https://www.statista.com/statistics/620805/smartphone-sales-market-share-in-the-us-by-vendor/), while the rest is a range of Android devices which sell for between $500 and $50 dollars.

![](https://simonhearne.com/images/2020-predictions/usage.svg)
*Digital platform usage by age - young people spend 72% of their time in apps.*

We know that the average American changes their phone [every 32 months](https://www.npd.com/wps/portal/npd/us/news/press-releases/2018/the-average-upgrade-cycle-of-a-smartphone-in-the-u-s--is-32-months---according-to-npd-connected-intelligence/) and worldwide the combined market share of premium brands Apple and Samsung is [just 33%](https://www.idc.com/promo/smartphone-market-share/vendor). The majority of users world-wide are on low- to mid-range Android phones, these consumers are far more likely to use a native app vs. the mobile web with perceived performance and data usage limits being the primary factors. The web is an open and democratic place, when people are forced to use only native apps they have a limited window on the web experience and the knowledge available therein.

![](https://simonhearne.com/images/2020-predictions/gsma_newsubscribers.svg)
*Predicted population sources for half of the next billion users.*

There has been a fundamental shift over the past half-decade on the web: from thick servers and thin clients to thin servers and thick clients. The inexorable rise of frameworks such as Angular, React, Vue and their many cousins has been led by an assumption that managing state in the browser is quicker than a request to a server. This assumption, I can only assume, is made by developers who have flagship mobile devices or primarily work on desktop devices. We now find ourselves in a position where every website might bring with it a whole application framework, perhaps with hacks like server-side rendering and client-side rehydration to appease SEO crawlers. While this trend is unlikely to change course in 2020, we may see better web platform features to account for it.

There are a few exciting technologies which I think will be a positive change for web performance in 2020:

## 1 - JAMstack

The JAM (JavaScript APIs Markup) stack is an [architecture](https://jamstack.org/) for delivering fast sites, using static site generators. SSGs such as [Jekyll](https://jekyllrb.com/), [Hugo](https://gohugo.io/) and [11ty](https://www.11ty.io/) allow for a pleasant developer experience and rapid iteration, but compile to HTML - the native language of the web. These generators are currently widely used for personal blogs or small company sites due to their small footprint. Moves towards [headless CMS](https://www.cmswire.com/web-cms/13-headless-cmss-to-put-on-your-radar/) solutions, client-side [payment APIs](https://developer.mozilla.org/en-US/docs/Web/API/Payment_Request_API) and cloud deployment could make SSGs viable for anything from news websites to ecommerce sites, effectively removing server processing time as a critical factor in performance and simplifying the process of deployment.

[Netlify](https://www.netlify.com/) has emerged as the market leader in workflow for the JAMstack. Netlify manages the whole workflow from development through to CDN deployment and augments the static site with a number of dynamic features. These features, such as identity management, forms and edge functions, broaden the scope of potential for JAMstack. There are also some great developer-friendly features such as deploying branches to unique URLs for testing, so there is no more need for a staging network. While Netlify is the clear leader in this space in 2019, I expect multiple solutions to appear in 2020 and build a competitive marketplace for enterprise JAMstack deployments. This has the potential to significantly erode the Wordpress marketplace, which currently accounts for [34% of the web](https://www.whoishostingthis.com/compare/wordpress/stats/)!

**Who to follow:** [@PhilHawksworth](https://twitter.com/philhawksworth) & [@shortdiv](https://twitter.com/shortdiv).

## 2 - WASM (Web Assembly)

<figure align="center">
<p style="margin-bottom: -1.625em;" class="ciu_embed" data-feature="wasm" data-periods="future_1,current" data-accessible-colours="false">
<a href="http://caniuse.com/#feat=wasm"><picture>
<source type="image/webp" srcset="https://caniuse.bitsofco.de/image/wasm.webp">
<img src="https://caniuse.bitsofco.de/image/wasm.png" alt="Data on support for the wasm feature across the major browsers from caniuse.com">
</picture></a></p>
<figcaption>WebAssembly has shipped to all major browsers.</figcaption>
</figure>

WASM has come a long way in the past year and 2020 might see broader adoption. [WASM](https://webassembly.org/) compiles in-browser 20 times faster than JavaScript, and WASM scripts will shortly be importable as modules into JS projects. The exciting possibility here is that the most processor intensive modules of popular frameworks, such as React’s Virtual DOM, can be re-written in a performant language such as Rust and compiled into WASM. In this example a single React release could raise the tide of web performance for all React-based websites. Until then, WASM provides a method to implement compute-intensive tasks in a language that performs much better than JavaScript (for [applicable tasks](https://webassembly.org/docs/use-cases/)).

**Who to follow:** [@PatrickHamann](https://twitter.com/patrickhamann) & [@_JayPhelps](https://twitter.com/_jayphelps).

## 3 - Edge Compute

Code executing at the CDN level allows us to offload processing from both client and server into the CDN platform. An existing implementation of WASM (or at least similar) is available on Fastly’s [edge compute](https://www.fastly.com/products/edge-compute) platform. All major CDNs have now launched some form of edge compute service (see [AWS' Lambda](https://aws.amazon.com/lambda/), [CloudFlare Workers](https://www.cloudflare.com/en-gb/products/cloudflare-workers/), [Akamai EdgeWorkers](https://developer.akamai.com/akamai-edgeworkers-overview)), interestingly all are built totally differently! Compute at the edge allows us to offload logic from both client and server, moving legacy server logic closer to the user, and intensive logic away to more controlled compute resources. Great applications of this are personalisation and image manipulation, both are tough to implement on the client but work well in server environments.

## 4 - Observability

As the saying goes: "if you can't measure it, you can't manage it". 2019 has seen some significant improvements in observability: we have been moving away from purely objective measures (such as page load time) for a while now but this year we are seeing more visibility into how browsers process and render content. Metrics such as [Cumulative Layout Shift](https://web.dev/cls/), [Total Blocking Time](https://web.dev/tbt/) and [Largest Contentful Paint](https://web.dev/lcp/) aim to get closer to user experience measures. These simplify the process of measuring speed and put less dependence on developers to instrument their applications. [HTTP Archive](https://httparchive.org/) and [Chome Real User Experience report (CrUX)](https://developers.google.com/web/tools/chrome-user-experience-report) both provide a great comparative database of performance statistics, and [Lighthouse CI](https://github.com/GoogleChrome/lighthouse-ci) provides an integration point to take all of this data and add good performance testing to build processes.

Google [has proposed](https://blog.chromium.org/2019/11/moving-towards-faster-web.html) a "slow" badge in search results for websites. The stated goal is noble: "moving towards a faster web", but time and time again we have seen that the 'performance police' approach fails to affect meaningful change. Who knows what the workaround techniques will be: loading screens like 1999, screenshots rendered while the page loads in the background, UA sniffing to deliver a faster experience to googlebot.

<figure class="no-shadow">
  <img src="/images/2020-predictions/chromeslowbadge.jpg"
    alt="Chrome screenshot showing a warning that 'this page usually loads slow'"
    loading="lazy"
    clickable=true
/>
  <figcaption>Chrome's proposed 'slow' badge</figcaption>
</figure>

As a corollary to that statement: performance metrics for the slowest one-third of traffic saw improvements of up to 20% when Google [included page speed as a ranking factor](https://webmasters.googleblog.com/2019/04/user-experience-improvements-with-page.html) in mobile search results. So it seems that imposing these perceived penalties does work, perhaps because technical SEO gets more budget than web performance?

> For the slowest one-third of traffic, we saw user-centric performance metrics improve by 15% to 20% in 2018. As a comparison, no improvement was seen in 2017.

This approach of [stick vs. carrot](https://en.wikipedia.org/wiki/Carrot_and_stick) is a bit of a poke in the eye, especially considering that a huge hole in performance observability comes from Google Analytics. GA is the [most widely used](https://almanac.httparchive.org/en/2019/third-parties#providers) third-party tag on the planet, collecting data from the majority of the sites you visit around the web. GA provides 'Site Speed' reports, which should be a good thing, yet out of the box GA only supports the most basic of legacy performance measures. These are randomly sampled from 1% of pageviews with no ability to correlate performance with business outcomes. Worse still, the only aggregate statistic available is the arithmetic mean, which is known to be [nearly useless](http://highscalability.com/blog/2012/5/23/averages-web-performance-data-and-how-your-analytics-product.html) in web performance measures.

<figure class="no-shadow">
  <img src="/images/2020-predictions/ga_timers.png"
    alt="Google Analytics screenshot showing the list of timers: dom content loaded, dom interactive and page load time"
    maxwidth=400
    loading="lazy"
    clickable=true
/>
  <figcaption>Google Analytics default timers are very limited</figcaption>
</figure>

There is still a long way to go to improve observability. As an industry we could instantly improve observability for at least [thirty million](https://trends.builtwith.com/analytics/Google-Analytics) websites by simply improving the default performance data presented in Google Analytics.

It is _critical_ to measure site speed using a Real User Measurement solution, otherwise you have zero visibility of how your users perceive the performance of your application (or worse, metrics that point you in the wrong direction). For the most simple setup, go for [SpeedCurve Lux](https://speedcurve.com/features/lux/). For the most flexibility and an enterprise solution, go for [Akamai mPulse](https://www.akamai.com/uk/en/products/performance/mpulse-real-user-monitoring.jsp). If you already have an Application Performance Measurement (APM) solution they may have a plugin to measure real user performance, such as [Dynatrace UEM](https://www.dynatrace.com/support/doc/appmon/user-experience-management/) and [New Relic Browser](https://newrelic.com/products/browser-monitoring). Whatever solution you choose, make sure that you can measure what is important to your users and your product, and correlate speed with business success.

I can only hope that the web will get faster as metrics improve, RUM becomes more prevalent and performance statistics become part of executive reports. Slow sites cost money, and observability helps us to allocate funds to improve performance.

**Who to follow:** [@anniesullie](https://twitter.com/anniesullie) & [@eanakashima](https://twitter.com/eanakashima).

## 5 - Platform improvements

A lot of improvements are coming from browsers like Chrome. It has offered [Data Saver mode](https://support.google.com/chrome/answer/2392284?co=GENIE.Platform%3DAndroid&hl=en-GB) since 2013, which has very high opt-in rates for low-end devices. This mode in most cases presents a hint to web servers to [deliver a lighter page](https://calendar.perfplanet.com/2018/data-shaver-strategies/) (which could be handled at the edge to deliver a fast, light experience). Chrome also [released](https://web.dev/native-lazy-loading/) the `loading` attribute in 2019 - a hint as to the loading priority of resources. Combining these two, Chrome will by default [lazy load offscreen images](https://web.dev/native-lazy-loading/#load-in-distance-threshold) for users on poor connection or if data saver is enabled. This simple change to the web platform should have a significant impact on web performance, without the intervention of web developers. While this may feel intrusive, browsers have long enforced certain policies for reasons of privacy. Adding performance as a signal in these decisions should mean a more equitable web for all users.

<figure class="no-shadow">
<p style="margin-bottom: -1.625em;" class="ciu_embed" data-feature="loading-lazy-attr" data-periods="future_1,current" data-accessible-colours="false">
<a href="http://caniuse.com/#feat=loading-lazy-attr"><picture>
<source type="image/webp" srcset="https://caniuse.bitsofco.de/image/loading-lazy-attr.webp">
<img src="https://caniuse.bitsofco.de/image/loading-lazy-attr.png" alt="Data on support for the loading-lazy-attr feature across the major browsers from caniuse.com">
</picture></a></p>
<figcaption>Loading attribute support is limited to Chromium-based browsers.</figcaption>
</figure>

Here's hoping that these changes driven by the Chrome team are standardised and make it to other browsers. This simple change could have a drastic effect on user experiences for those on the worst network connections and those with limited or metered data connections.

**Who to follow:** [@KatieHempenius](https://twitter.com/katiehempenius) & [@AddyOsmani](https://twitter.com/addyosmani).

## 6 - Web monetisation

Much of the _slowness_ of the web comes from our attempts to monetise visitors. From advertising and tracking to personalisation and live chat, these technologies all get in the way of the user experience in order to generate revenue. The business model on the web is not as clear cut as it is on the high street or in the newsagent, and we are still struggling to reconcile the user expectation of a free and open web with the business requirement to generate revenue.

Improving monetisation through the web using built-in APIs such as the [Payment Request API](https://developer.mozilla.org/en-US/docs/Web/API/Payment_Request_API), [Web Monetization API](https://webmonetization.org/) and even [Brave Rewards](https://brave.com/brave-rewards/) should make it easier to monetize content without the need for intrusive third-party content. This change could see a drastic improvement in experience for the web users of 2020, especially on news and magazine websites.

People [spend longer on fast websites](https://wpostats.com/tags/engagement/), so this could be one of the elusive win-wins of web development.

**Who to follow:** [@Coil](https://twitter.com/Coil) & [@Interledger](https://twitter.com/Interledger).

## 7 - Honourable mentions

Deliberately missing from these predictions are [Google's AMP](https://moz.com/blog/accelerated-mobile-pages-whiteboard-friday), [Facebook Instant Articles](https://instantarticles.fb.com/), [Service Workers](https://developers.google.com/web/fundamentals/primers/service-workers), [Web Workers](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers), [Progressive Web Apps](https://www.smashingmagazine.com/2016/08/a-beginners-guide-to-progressive-web-apps/), [Web Virtual Reality](https://webvr.info/) and AI. I don't think that these technologies are critical for shifting the needle on the median person's web experience, for a range of political and technical reasons. The closest on the list is Progressive Web Apps, as they may be the key to closing the gap between the web and native apps. Samsung recently [announced support](https://medium.com/samsung-internet-dev/introducing-progressive-web-apps-to-samsung-galaxy-store-47ecd317725b) for including PWAs in their US App Store, although it does require you to email them your URL and sign some additional licenses. Google has an [odd workaround](https://medium.com/@firt/google-play-store-now-open-for-progressive-web-apps-ec6f3c6ff3cc) where you can ship your website as an app in the Play Store, but the process is far from straightforward. Apple, unsurprisingly, is steadfastly protecting its App Store revenue (which is [greater than its revenue from device sales](https://www.theverge.com/2019/3/20/18273179/apple-icloud-itunes-app-store-music-services-businesses)) by barely entertaining the idea, although Safari does [support some of the critical features](https://medium.com/@firt/whats-new-on-ios-12-2-for-progressive-web-apps-75c348f8e945). Unfortunately I don't think we will see a market shift in this respect in 2020.

There are, however, changes afoot which might mean a significant improvement to web UX in the EU. It looks like EU regulators have accepted that existing GDPR regulations have led to a web experience laden with oft-ignored (or ignorantly accepted) GDPR notices. These have become a blight on the web and do little to protect users - the average consent form takes 42 seconds to opt-out of non-essential cookies. If this consent management moves to the browser, a single native UI could be the only requirement for users to manage their consent, improving default security posture as well as the web experience. This is one of the options being considered for [ePrivacy](https://en.wikipedia.org/wiki/EPrivacy_Regulation_(European_Union)) - a proposal for better regulations in the EU. At the speed of government though, I'm not sure we will see change formalised in 2020, let alone adopted and regulated.

> The cookie provision, which has resulted in an overload of consent requests for internet users, will be streamlined. The new rule will be more user-friendly, as browser settings will provide for an easy way to accept or refuse tracking cookies and other identifiers. The proposal also clarifies that no consent is needed for non-privacy-intrusive cookies improving internet experience (like to remember shopping cart history) or cookies used by a website to count the number of visitors.

Explicit opt-out, or preferences managed in the browser, could dramatically improve the performance on many sites. Hello magazine is an example where over 11MB of content is loaded on the mobile first view, a significant amount of which is third-party. The opt-in to this is explicit: "By browsing this site you are agreeing to this", and the link to manage your preferences is inaccessible once you interact with the site.

![Hello magazine with explicit GDPR opt-in loading 471 requests](https://simonhearne.com/images/2020-predictions/hello.png)
*Hello magazine has explicit opt-in, opt-out is not possible once you scroll the page.*

If you are looking at the future of web performance, the most important thing you can do is to make your experiences fast now. The future is not bringing any silver bullets, which means it is more important than ever to measure and optimise your users' experience.

## Further Reading

- [Performance Metrics for Blazingly Fast Web Apps](https://blog.superhuman.com/performance-metrics-for-blazingly-fast-web-apps-ec12efa26bcb)
- [5G Will Definitely Make the Web Slower, Maybe](https://www.filamentgroup.com/lab/5g/)
- [Speed tooling evolutions: 2019 and beyond [video:23min]](https://www.youtube.com/watch?v=iaWLXf1FgI0)
- [GSMA Intelligence: Global Mobile Trends 2020 [pdf:3.5MB]](https://www.gsmaintelligence.com/research/?file=c5f35990dcc742733028de6361ccdf3b&download)
- [GSMA Intelligence: The Mobile Economy 2019 [pdf:3.3MB]](https://www.gsmaintelligence.com/research/?file=b9a6e6202ee1d5f787cfebb95d3639c5&download)
- [Comscore Global State of Mobile 2019 Whitepaper [registration required]](https://www.comscore.com/Insights/Presentations-and-Whitepapers/2019/Global-State-of-Mobile)
- [All of the talks from Performance.Now() conference 2019](https://perfnow.nl/speakers) :)

<script>!function(){document.addEventListener("DOMContentLoaded",function(){for(var e=document.getElementsByClassName("ciu_embed"),t=0;t<e.length;t++){var a=e[t],i=a.getAttribute("data-feature"),n=a.getAttribute("data-periods"),s=a.getAttribute("data-accessible-colours")||"false",r=a.getAttribute("data-image-base")||"none";if(i){var o="https://caniuse.bitsofco.de/embed/index.html",d='<iframe src="'+o+"?feat="+i+"&periods="+n+"&accessible-colours="+s+"&image-base="+r+'" frameborder="0" width="100%" height="400px"></iframe>';a.innerHTML=d}else a.innerHTML="A feature was not included. Go to <a href='https://caniuse.bitsofco.de/#how-to-use'>https://caniuse.bitsofco.de/#how-to-use</a> to generate an embed."}var c=window.addEventListener?"addEventListener":"attachEvent";(0,window[c])("attachEvent"==c?"onmessage":"message",function(t){var a=t.data;if("string"==typeof a&&a.indexOf("ciu_embed")>-1)for(var i=a.split(":")[1],n=a.split(":")[2],s=0;s<e.length;s++){var r=e[s];if(r.getAttribute("data-feature")===i){var o=parseInt(n)+30;r.childNodes[0].height=o+"px";break}}},!1)})}();</script>


==============================================================================

# Deep dive into third-party performance

> A deep-dive into the impact of third-party tags on web performance.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2019-11-22
Canonical: https://simonhearne.com/2019/third-party-deep/
Tags: WebPerf, video, talk
Video: https://www.youtube.com/watch?v=uXv9JFvrnwo
Slides: https://simonhearne.com/presentations/third-party-deep/#/

---
Every third-party tag is someone else's code, running on your users' devices, with your name on it. This talk is seven times that went wrong.

## About the talk

By late 2019 the median website was 37% third-party requests by count (HTTP Archive, October 2019). Analytics, advertising, optimisation, JavaScript CDNs and tag managers all arrive with a business case, and all of them execute with the same privileges as the code you wrote yourself.

This talk works through seven real incidents — outages, revenue losses, and one case of a "performance" tool making things measurably slower — and pulls a defence out of each.

## What's covered

- **A thirty-second wait.** A social sharing script blocked `document.ready` and took critical application logic down with it during a China launch. Sync is still king, so feature-flag it and defer it.
- **A 30% conversion drop.** A silent third-party update broke a checkout button. Subresource Integrity prevents unannounced changes, and was live on just 5.15% of pages.
- **The tool that cost a second.** An unload-event listener added roughly a second of latency. Synthetic tests never saw it; RUM did.
- **A 26% revenue swing.** A review provider degraded Android badly enough that removing it lifted Android revenue by 26%. Adaptive tag loading is the middle path.
- **Malicious ads and crypto miners.** Content Security Policy (6.11% adoption) and `connect-src` restrictions, plus CPU metrics to catch what network metrics miss.
- **A/B testing tools.** Render-blocking by design, with a 500ms default SLA. Proxy through your CDN, self-host, or move it to the edge.
- **A loading order.** Immediate for experimentation and tag management, fast for session and performance tracking, late for ads, reviews and live chat.

As Tim Kadlec puts it in the deck: "Everything should have a value because everything has a cost."


==============================================================================

# Native image lazy loading has landed in Chrome, maybe don't use it

> Native lazy loading for iframes and images landed in Chrome 76, but don't go ripping out your existing lazy loading strategy just yet.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2019-08-13
Canonical: https://simonhearne.com/2019/native-lazy-loading-in-chrome/
Tags: WebPerf, images

---
## Lazy Loading Images

Web browsers are complex pieces of software, but still treat images relatively na&#239;vely. See, for example, a section of this waterfall where almost 100 images are loaded immediately on first-view:

<figure align="center">
<img src="/images/waterfall_images.png"
    alt="Waterfall section showing bandwidth consumed by off-screen images delays javascript resources"
    loading="lazy"
   />
<figcaption>Waterfall section showing bandwidth consumed by off-screen images</figcaption>
</figure>

On this client's site all but four of the images downloading in the screenshot (shown in purple) are off-screen. Those images are totally invisble to the user on desktop and mobile, but compete for bandwidth with critical resources like the application JavaScript (yellow). Making matters even worse - browsers will download images in the order that they appear in the HTML document, and this client has a dozen images in their hero navigation menu which is at the very top of the HTML. Again, these are invisible to the user at page load time, but are loaded before the critical hero images.

Lazy loading is an approach to provide a hint (or trick) the browser so that visible images are prioritised higher than offscreen images. There have been multiple libraries which offer this functionality, see [lazysizes.js](https://github.com/aFarkas/lazysizes). The general premise is to remove the `src` attribute, so the browser does not download anything, then add a `src` later in the page load which will trigger an immediate download by the browser.

## Native lazy loading

As of [Chrome 76](https://www.chromestatus.com/feature/5645767347798016), released Jul 30 2019, web developers can provide a native hint to the browser that an image can be loaded lazily by adding the attribute `loading='lazy'` to the `img` element. See more details on the [web.dev](https://web.dev/native-lazy-loading) blog.

## Not ready for prime time

This release has received a lot of attention on developer twitter. Multiple tweets show developers removing their 'legacy' lazy loading approaches and replacing them with the simple `loading` attribute. While adding the `loading` attribute is a positive step for sites without any prior lazy loading strategy, **moving to it as a replacement will almost definitely degrade overall user experience**.

The browser support for `loading` is [less than 1%](https://caniuse.com/#feat=loading-lazy-attr) at the time of writing. This number will climb to about 60% globally once the Chrome 76 rollout to mobile and desktop completes. There will be a further ~1% increase in global support when MS Edge updates.

<figure align="center">
<img src="/images/lazy_caniuse.png"
    alt="CanIUse.com for native lazy loading on Aug 13 2019 shows 0.15% support globally"
    loading="lazy"
  />
<figcaption>CanIUse.com for native lazy loading on Aug 13 2019 (<a href="https://caniuse.com/#feat=loading-lazy-attr">see latest stats</a>)</figcaption>
</figure>

Safari is notably lacking from the support list. This is not a surprise, as Apple are notoriously slow to update their browser (and are opaque about roadmap), but is a serious concern for most western websites. If your Safari traffic is greater than 50%, like many of my publishing and fashion clients, moving to the `loading` attribute will result in a net-negative performance change.

## The future of lazy loading

The initial implementation of `loading` in Chrome is not complete. There are [issues with printing and exporting](https://bugs.chromium.org/p/chromium/issues/detail?id=875403&q=component%3ABlink%3ELoader%3ELazyLoad%20print&colspec=ID%20Pri%20M%20Stars%20ReleaseBlock%20Component%20Status%20Owner%20Summary%20OS%20Modified) (looks to be resolved in v77) and the options `eager` and `auto` are currently [identical in functionality](https://chromestatus.com/feature/5645767347798016). In the future, the plan is for `auto` (the default behaviour if no `loading` attribute is supplied) to be determined by the user's network conditions or data saver preference (or is it [Lite Pages](https://timkadlec.com/remembers/2019-03-14-making-sense-of-chrome-lite-pages/)?). This means that the performance of your pages should be improved for users with the worst experience, but outside of your control (unless you add the attribute and set it to `eager` or `lazy`).

If you're not currently [collecting the data saver preference](https://calendar.perfplanet.com/2018/data-shaver-strategies/) in your analytics tool of choice, this could result in some confusing metrics!

## Conclusions

Supporting lazy loading hints natively in the browser is a huge step forward for web performance. Using the `loading` attribute as the sole approach for lazy loading will be the recommendation once WebKit and Safari implement support. For now, though, it should only be used as an enhancement and not as a replacement for existing strategies.

A great worked example was [published by the BBC](https://medium.com/bbc-design-engineering/native-lazy-loading-has-arrived-c37a165d70a5), where they added the `loading` attribute to images in an internal app, presumably a very high proportion of Chrome Desktop traffic. The results were a 50% reduction in load time and 40 fewer requests per pageview. This is a great use case for native lazy loading.

In the interests of full disclosure, I use the `loading` attribute on this site (try inspecting the images in this post). I had no lazy loading strategy beforehand, so it is a progressive enhancement. Over 70% of the traffic to this site is from Chrome, so this should be a reasonable improvement. I had previously experimented with a home-brewed lazy loading technique, but the JavaScript overhead was significant enough to negate most performance benefits.


==============================================================================

# WebPageTest Private - 'no successful results'

> A potential quick fix for "The test completed but there were no successful results." in WebPageTest private instances.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2019-08-09
Canonical: https://simonhearne.com/2019/wpt-private-no-successful-results/
Tags: WebPerf, webpagetest

---
I recently configured a new [private WebPageTest instance](https://github.com/WPO-Foundation/webpagetest-docs/blob/master/user/Private%20Instances/README.md) on AWS to use SSL provided by [Let's Encrypt](https://letsencrypt.org/), using EFF's awesome [CertBot](https://certbot.eff.org/).

When running my first set of tests I noticed big delays (multiple minutes per test) and when the tests finally completed they all reported `The test completed but there were no successful results.`.

A quick look at the access log showed that the agents were getting jobs, but not submitting the results back to the server. I saw a bunch of `HTTP 301` responses to the workdone POSTs:

`POST /work/workdone.php?run=1... HTTP/1.1" 301 178 "-" "python-requests/2.22.0"`

Seeing the `301` responses (and no subsequent `200`s) made me realise that CertBot had added a default redirect from port 80 to port 443 to my nginx config (found at `/etc/nginx/sites-enabled/default` for the WebPageTest AMI).

```js
server {
    if ($host = private.wpt.host) {
        return 301 https://$host$request_uri;
    } # managed by Certbot
       listen 80;
       server_name private.wpt.host;
    return 404; # managed by Certbot
}
```

Simply commenting out this `server` block to remove the redirect and bouncing the nginx server immediately resolved the issue. CertBot does ask if it should add the redirect, so simply saying no during configuration would have solved the issue! It would be trivial to allow test agents to communicate on :80 based on the workdone path to get the best of both worlds.

If that is not the cause of your issue, it may be caused by the use of 64-bit windows clients which cannot do traffic shaping using dummynet. Simply adding `connectivity=LAN` to `locations.ini` for the test locations should resolve the issue.


==============================================================================

# Why Site Speed is Critical for Online Publishers

> Site speed is a critical factor in the success of an online publishing business, whether revenue is driven by adverts or subscriptions. This post demonstrates the relationship between speed and publishing revenue, and suggests some quick wins to improve both!

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2019-08-07
Canonical: https://simonhearne.com/2019/site-speed-for-publishers/
Tags: WebPerf

---
It has been well documented that site speed is a major factor in business success for e-commerce websites. If you can demonstrate that improving page load times by 500ms increases revenue by 5% there is a clear return on investment to make the speed improvements.

Digital publishing has not had so much documented evidence for the benefits for site speed. There are multiple factors behind this, based on the business model of the site. Subscription-based publishers have a captive audience - if you've already paid to access content, surely an article loading a second slower won't stop you reading the content? For ad-driven publishers there is a conflict between AdOps teams piling on tens or hundreds of third-party tags (each promising a slice of ad revenue) and the development team trying to ship editorial content.

> Optimizely added artificial latency to the Telegraph and saw page views plummet: by 11% for a 4 second delay and 44% for a 20 second delay.
<p class="align-right pull-up small"><a href="https://wpostats.com/2016/07/04/optimizely-pageviews.html">source</a></p>

## Subscription-based Publishers

There are two angles to look at site speed for subscription-based publishers: acquisition and retention. When acquiring new subscribers, these sites can be viewed very much like an ecommerce player - you're trying to convert users from paying nothing to purchasing a subscription.

The e-commerce model does not work when it comes to retention, however. Once a user has purchased a subscription they will have a choice of whether to renew or not. That decision will be based at least partly on their experience of the product over the previous period, there will be a calculation of the value of the content they consume versus the ongoing cost of subscription.

As the FT found out, slower pages result in fewer articles viewed. The drop in articles gets worse over time, presumably as subscribers get increasingly frustrated.

> Financial Times added a one second delay to every page view and saw a 4.9% drop in the number of articles users read over a 7 day window. A two-second delay resulted in a 4.4% drop, and a three second delay saw a 7.2% drop. After twenty-eight days the two and three second variants both resulted in further drops in engagement.
<p class="align-right pull-up small"><a href="https://wpostats.com/2016/04/13/ft-page-view.html">source</a></p>

Whilst the FT study clearly showed a direct correlation between speed and articles consumed, they did not continue to measure the effect on subscription renewal. It is fair to assume, though, that subscribers who have a poor experience (and thus consume less content) are less likely to renew their subscription. You may be missing out on critical business data if you're not tracking subscriber views, performance and attrition.

> GQ cut load time by 80% and saw an 80% increase in traffic. Median time spent on the site also increased by 32%.
<p class="align-right pull-up small"><a href="https://digiday.com/media/gq-com-cut-page-load-time-80-percent/">source</a></p>

The interactive chart below shows the relationship between load time and the likelihood that a user will bounce or continue to a different page (retention) for a range of publishing websites.

<div class="vega-chart loading" id="vis1" data-spec="/data/retention.json">Loading chart...</div>

## Ad-driven Publishers

The standard e-commerce model again does not work for publishers who make the majority of their revenue from advertising. Success is measured by ads viewed and interactions. There are a number of ways that performance impacts these key metrics:

* Slow pages lead to higher bounce rates
* Slow pages lead to lower interaction rates
* Slow pages lead to [poorer organic SEO rankings](https://webmasters.googleblog.com/2010/04/using-site-speed-in-web-search-ranking.html)
* Slow pages lead to [higher ad costs](https://www.synapsesem.com/slow-site-load-times-crippling-digital-marketing-programs/)

At the most basic level, we know that faster pages lead to an increase in page views. This increase comes from a number of factors: reducing bounce rate (is this page ever going to load?), increasing session length (I've only got a minute to catch up on the news) and favourable rankings from search engines which include speed as a ranking factor. Assuming that there is a near-linear relationship between page views and ad revenue, there is an obvious benefit to improving site speed.

> Tokopedia reduced render time from 14s to 2s for 3G connections and saw a 19% increase in visitors, 35% increase in total sessions, 7% increase in new users, 17% increase in active users and 16% increase in sessions per user.
<p class="align-right pull-up small"><a href="https://wpostats.com/2018/05/30/tokopedia-new-users.html">source</a></p>

The interactive chart below shows the relationship between first-page load time and session length (pages viewed per session) for a range of publishing websites. Session length can drop by over 60% when comparing the fastest page loads with the slowest!

<div class="vega-chart loading" id="vis2" data-spec="/data/session_length.json">Loading chart...</div>

### Sponsored content, takeovers and advertorials

The publishing industry is looking to monetise their visits in novel ways as ad & content blockers become more prevalent. Full page takeovers fill otherwise unused whitespace on the page with sponsored promotional material. The commercial arrangement is similar to sponsored content and is typically based on the duration of the takeover or a goal number of pageviews. Increasing pageviews means either higher value for a duration-based takeover, or a shorter duration for a pageview-based takeover. Either way, this could mean more revenue for the publisher. On a secondary note, takeover content _should_ load after the main content of the page. Making the page faster will mean that the promotional content will be loaded sooner and thus seen by more visitors.

## Improving Speed

Site speed should be measured using metrics which are closest to user experience. Time to interactive & first meaningful paint are generally the best metrics to use for publishing sites where third-party content may significantly skew the load event. Some key methods to improve time to interactive and paint timing metrics are to prioritise critical content:

1. [Defer all JavaScript](https://medium.com/the-telegraph-engineering/improving-third-party-web-performance-at-the-telegraph-a0a1000be5#4123)
1. [Keep HTML documents < 14kB to maximise the first round-trip](https://hpbn.co/building-blocks-of-tcp/#slow-start)
1. [Use the `loading='lazy'` attribute (cautiously) to defer non-critical iframes and images](https://www.searchenginejournal.com/lazy-loading-attribute/302155/)
1. [Use `preload` for critical fonts](https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/webfont-optimization#preload_your_webfont_resources)
1. [Use `font-display: swap` to improve render times](https://developers.google.com/web/updates/2016/02/font-display)
1. [Keep track of third-party tags](https://simonhearne.com/2015/find-third-party-assets/)
1. [Remove unnecessary and unused third-party tags](https://medium.com/the-telegraph-engineering/improving-third-party-web-performance-at-the-telegraph-a0a1000be5)

---

If you would like advice on measuring, improving and advocating for site speed, please [get in touch](mailto:simon@hearne.me)!


==============================================================================

# Lead Developer Conference (London) - Day 1!

> Lead Developer is a conference for senior and lead software engineers. These are my raw notes from day one.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2018-06-27
Canonical: https://simonhearne.com/2018/leaddev-day-1/
Tags: WebPerf, conference

---
## Alice Goldfuss - The Container Operator’s Manual

Containers are generally presented in theory, Alice tells us how they work in real-life. Alice is an SRE at GitHub and has run Docker in production for 3.5 years, so knows a bit about containers in the wild!

1) Containers have strengths.

- Stateless applications / processes

2) Containers have weaknesses.

- Stateful applications / databases
Don't containerise databases - use cloud services

3) Containers need friends.

- Multiple tools, vendors and products required to manage, deploy, network, monitor etc.

4) Containers need headcount

- You need a whole team of engineers (6-8 people ideally) to build a container platform:
- operations
- deployments
- tooling
- monitoring
- kernel
- network
- infosec
- internal adoption
- project manager

> Do you want containers, or a blog post?!

## Dan Persa - First Steps as a Lead (⚡️ talk)

@danpersa

Dan is an engineering lead at Zalando, based in Berlin.

> Dealing with people problems is much more interesting than dealing with development problems

The responsibility of tech leads varies by company, from a technical lead role to a line management role. It is important to understand the role at your company and whether it fits your career goals.

Dan had three months of intense mentoring from an experienced tech lead at Zalando as part of a transition process from engineer to lead.

Dan's tips on how to learn fast:

 1) get a mentor
 2) _apply_ what you read in books
 3) reflect on your past - how you were managed

Trust instead of control:

 1) earn the respect of the team through pair-programming
 2) regular 1:1s
 3) transparency around performance evaluations

The lead's role is to help the team discover solutions to challenges, not to solve them yourself.

Not taking a decision is a decision in itself.

Give feedback as soon as possible, with a balance of positive and constructive feedback. Always ask for feedback in return.

## Alexandra Hill - The art of giving and receiving code reviews gracefully (⚡️ talk)

Code reviews are a key factor in improving:

- functionality
- readability
- meaintainability
- scalability

But, they can be a big cause of conflict.

Low impact issues can be automated away.

Dual concerns model for conflict resolution.

Solutions to improve code reviews:

1) Pair program
2) Discsuss tasks prior to implementation
3) Don't silo codebases
4) Ensure that everyone reviews and everyone is reviewed

Reviewers can raise by a grade or two.

Use 'we' instead of 'you'. Ask questions instead of making statements. Don't make demands of people or use challenging language.

Don't forget to give positive feedback.

## Pia Nilsson - Growing teams to continuously deliver

@pia_nilsson

Increasing flow in the back-end continuous delivery squad:

1) WIP limits to increase focus, but also increased stress as engineers focussed on finishing tasks, even when delays were out of their control.
2) TDD & DDD
3) Pairing & Mob programming

"Yes, and" - never say "No" or "but".

Non-violent communication method.

Objectives and key results (OKR).

Despicable design. Define the worst possible experience.

## Menno van Slooten - How I learned to stop worrying and love meetings

@mennovanslooten

> Becoming a lead: the subtle art of not making a difference.

1) Let go of programming - chasing your old job prevents you from seeing the value in what you _are_ doing
2) If you attend a meeting your developers don't have to, so they can focus on what they love to do
3) Learn to love interruptions - they are opportunities to help people

How do you say no??

## Jenny Duckett - Building sustainable teams to handle uncertainty

@jenny_duckett

Have a single clear goal for your team, reiterate it constantly. Use it to add context to tasks.

[Mobbing](https://gdstechnology.blog.gov.uk/2016/09/08/our-top-12-mob-programming-tips-and-thoughts/)

Use every task as an opportunity for the developer to grow & learn.

> Make yourself dispensable

Don't insulate your team too much, make it clear where their work fits in the wider picture.


==============================================================================

# Delta V Conference - Day 2!

> Delta V Conference is a new two-day, single-track web performance conference in London, hosted by me, Perry Dyball and Jo Franchetti. Here's my raw notes from the fantastic content from our lovely speakers on day two.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2018-05-11
Canonical: https://simonhearne.com/2018/deltav-day2/
Tags: WebPerf, conference

---
## Tammy Everts - Hunting the Unicorn

Tammy started researching user experience on the web in a laboratory setting - measuring individuals as they used websites. Now she works with SpeedCurve trying to connect the dots between real people and the stream of data being collected from Real User Monitoring.

> How do we scale empathy?

As we collect millions or billions of user experience data points, how do we remind ourselves that there are real people on the other end of the data?

The average web user believes that they waste **two days a year** waiting for web pages to load.

We know that slow sites suck, but how do we define **slow** and how do we define **suck**? We are perception brokers. But perceptions are fluid and, well, perceptual. How can we measure something so dynamic?

Our traditional web performance metrics like start render, page load, speed index do not help us to measure perception!

The best UX metric needs to correlate to what users actually see, it needs to be easy to use and work out-of-the box in browsers, it should recognise that not all pixels and page elements are equal and should be customisable based on our knowledge of our sites.

**First meaningful paint (FMP)** - times the paint after which the biggest above-the-fold layout change has happened or when web fonts are loaded, whichever comes later. Available in Chrome.

**Time to first interactive (TTFI)** - when the page is first expected to be usable and will respond quickly to user interactions. Available in WebPageTest and mPulse.

**Hero rendering times** - the time at which the largest image, largest background image and top-level heading render.

To help work out the best metrics, SpeedCurve created a [test page](http://lab.speedcurve.com/rendering/picker.php) where you can see what metric correlates best with your perception of page load. The results unsurprisingly show that there is no metric that works for all sites, nor one that works for all people.

**Custom metrics** are the only way to measure what you know is important to your page, for your users. E.g. time-to-first-scroll, time to product image. Unfortunately these aren't necessary accessible or easy to use.

Whatever you measure:

1. Know your most meaningful metrics
2. Track them with synthetic and RUM
3. Don't focus on averages and medians
4. Set goals / budgets
5. Make yourself accountable with alerts and reports
6. Mind regressions

## Jeremy Keith - What is a progressive web app?

There is a lot of confusion and misinformation about what progressive web apps (PWAs) are. They do not require a single-page application JavaScript framework!

Most of what makes a PWA is just what makes a got website - secure, fast and responsive. You can measure that a site has https, service worker and a web app manifest - that's what makes a **measurable** PWA.

> Add to home screen is a bit of a distraction for PWAs, don't worry too much about it

Service Workers are the key part of a PWA, and they enable varying levels of offline functionality:

1. An "offline" error page - e.g.
2. A full offline experience - e.g.
3. A user-initiated page download - e.g. [Una Kravet's](https://una.im) website

> Should your site be progressive web app? Yes, it should

Jeremy has [published a book](https://abookapart.com/products/going-offline) about using Service Workers to create great offline experiences.

## Mark Leach and Andy Davies - Fast Fashion

[Slides](https://www.slideshare.net/AndyDavies/fast-fashion-how-missguided-revolutionised-their-approach-to-site-performance-deltav-conference-may-2018)

Mark works at Missguided.com - a fast fashion brand aimed at trend-conscious women aged 18 - 29.

> Our job is to empower young women to feel strong and confident

Mark's customers are all on social media, and are keen to complain when the website is slow.

When working with NCC Group Web Performance (now Eggplant) Mark was told that there could be an additional £1.2M through performance improvements.

When analysing the website, it was discovered that the median Android page load time was **over 14 (fourteen) seconds**.

Adding ```<link rel="preload">``` for web fonts and fixing some issues with Optimizely brought the Android page load time down by two seconds.

Removing BazaarVoice improved load time by a further **3 (three) seconds** for Android devices. This improvement led to an increase in weekly revenue of **56%** 😮. When baselined against all other traffic with BazaarVoice still enabled, the net gain in revenue was **26%**.

When presenting to the board, initially they were pessimistic about the potential, but then saw the real data. Then they wanted more!

In October 2017, Missguided set a goal of a three second load time. Andy used [source map explorer](https://github.com/danvk/source-map-explorer) to visualise the JavaScript and CSS bundles. The JavaScript bundle was reduced by 42%!

Missguided then hired someone to manage A/B testing and experiments to improve speed, as Optimizely could not be removed. This involved removing the bundled jQuery from the Optimizely bundle, removing old experiments and those for non-production environments.

Overall, median performance on Android was improved from 14 seconds to just over 5 seconds.

## Lauren Younger - Managing Performance like a Product

Performance and product management have a lot in common, they are both about improving conversion and revenue, retaining users, manage the user experience and understanding people.

Four basic principles for performance management:

1. Understand
Market, competition, stakeholders, business KPIs.
**Qualitative** and quantitative data - using e.g. surveys and satisfaction scores. What does a 49% bounce rate mean? Were 49% of customers unsatisfied?

2. Define
Create a specific business case with SMART goals then get stakeholder buy-in.
Think beyond standard performance metrics (bounce rate, load time, session length), what about operations resource cost, global reach.

3. Prioritise
Prioritisation is hard, we're not very good at it and we tend to rely on opinion. Use data to create scores and ranking. It's easy to focus on the slowest pages, but you should use business information such as conversion impact to determine priority.

4. Communicate
We can't use the same communication methods when working with all of the stakeholders of your website. Marketing, operations and engineering all have different requirements and communication styles.

Measure the results. What went well? What could have been improved in the process?

**Share your success** and **thank the team**!

## Dan Shappir - Improving Performance for 100 Million Websites

Wix made performance a top priority feature in 2017. Performance has to be a feature in order to get resources, buy-in and a specification.

Wix runs >1,000 concurrent experiments and pushes to production around 200 times per day.

In order to protect performance, performance is measured during development, in pre-release and in production.

The biggest benefits seen by Wix (measured by change in first contentful paint):

**Server-Side Rendering** - 70% improvement for mobile devices and 40% improvement for desktop. However the time between render and interactive became longer due to the additional delay in downloading the JavaScript bundle.

**Caching** - before going crazy with service workers, ensure that your assets are actually cacheable at the client!

**Smart Bundling** - dyanmically create app bundles depending on the features required on the site in question

**HTTP/2** - 10 - 20% speed improvement by switching to HTTP/2.

**Resource Hints** - Using preconnect, preload and prefetch gave Wix a 10% speed improvement.

**Progressive Rendering** - by only rendering above-the-fold content in the first pass, Wix saw a 5 - 10% improvement.

**JavaScript Optimisation** - Wix saw a 10% improvement by profiling and optimising their common JavaScript bundle.

Wix are in the process of transitioning from a JavaScript-driven layout to CSS Grid!

## Robin Marx - QUIC in theory and practice

Robin introduces the three main problems with HTTP/1.1:

1. 4xRTT (round-trip-time) connection setup
2. TCP head-of-line blocking
3. HTTP head-of-line blocking

We can fix a few of these issues with TLS 1.3, fast-start and resumption. Leading to a single RTT connection establishment.

HTTP/2 removes the TCP head-of-line blocking, but requires a single connection which is susceptible to packet loss. Robin's research has shown that HTTP/2 is up to five times slower than HTTP/1.1 in high packet-loss environments.

We can't just upgrade TCP - it is too pervasive and there are too many pieces of technology which are too slow to upgrade (e.g. proxies). The solution to this is to use a different transport-layer protocol: UDP.

QUIC is built on top of UDP with custom packet-loss recovery logic. Solving TCP head-of-line blocking. Beyond that, it encompasses all of our network learning over the past three decades. By design, QUIC mitigates against all known DDoS attacks, prevents middlebox meddling through full transport-layer encryption. It also uses unique connection IDs to prevent connections closing down when IP addresses change, and allows communication over multiple networks (e.g. 4G and WiFi).

QUIC further improves on HTTP / TCP with custom congestion control.

Google have launched their own version of this protocol: **gQUIC**. A standardised version, **iQUIC** is expected to land at the end of 2018.

Google Search speed improvements: 8% on desktop, 3.6% on mobile. 16% improvement in 99th percentile.

20% less rebuffering on YouTube India, 14% faster on 4G, 20% slower than HTTP/1.1 with packet loss (HTTP/2 is 200% slower).

But, some research shows 75% lower throughput and QUIC costs 2x the CPU time on servers.

For QUIC connections on mobile devices, 58% of the time the network is faster than the CPU!

## Harry Roberts - It's my (third) party and I'll cry if I want to!

[Slides](https://speakerdeck.com/csswizardry/its-my-third-party-and-ill-cry-if-i-want-to).

We use third-parties because they provide convenience, but this increases our exposure to risk.

Third-party content is inevitably added to provide value to the business, not the web users.

1. Understand

**Security** - insecure content, man-in-the-middle attacks, compromised third-parties.

**Delays** - network, third-party infrastructure, third-party runtime.

> Use WebPageTest's **block** and **SPoF** features to determine the impact of each third-party.

**Failures** - what happens when JavaScript fails?
Use the **block request URL** feature in Chrome DevTools.

**Runtime cost** - defer expensive third-party JavaScript until after critical content.

**Who owns what?** - use [requestmap.webperf.tools](https://requestmap.webperf.tools/) to plot the third-party assets on your page. Harry has created a [Google Sheet](https://csswz.it/2rm29JY) to help visualise the CSV output from request map!

In summary: use as few third-parties as possible, for as short a period of time as possible.

## Dora Milataru - Where are the women?

Dora leads the engineering team at FT.com responsible for the changes required to be compliant with the General Data Protection Regulation (GDPR).

Globally, the Financial Times employs more women than men. Board representation is greater than 30% women, with a target of 50%.
This is a **diversity quota**, which can be dangerous. What happens when the quota is reached? Is that diversity done?

> Quotas may be the only way of achieving a world where quotas are obsolete

Diversity is intersectional, discussing diversity along one dimension (gender, race, sexuality, religion, culture...)

If you search for why diversity is important in tech, you will find hundreds of articles which show that a diverse workforce improves business results. This helps get stakeholder buy-in, but does not help public opinion.

> Caring about diversity has to mean caring about people.

Virtue signalling - showing off how **good** you are. Self-congratulation does not help society.

Social networks show that diversity alone doesn't help, especially in decision making processes. Inclusion is the glue that can fix this.

**We** gravitate to people like **us**.

> Diversity is being invited to the party, inclusion is being glad that you're there

The recent Stack Overflow developer survey shows that men don't rank diversity as important when selecting a job, while women and non-binary people rank diversity and working environment as more important than salary.

An internal [Google study](https://www.forbes.com/sites/quora/2013/06/28/is-there-a-link-between-job-interview-performance-and-job-performance/#3df2990a458b) showed that there is zero correlation between a candidate's interview score and their future job performance.

Reach out to under-represented groups. Be honest. Celebrate success. You're not a saviour. Try to understand.

> Tolerating difference is not the same as celebrating diversity

## Léonie Watson - There's more to performance than meets the eye

Screen readers

Accessibility APIs are provided by Operating Systems, these are exposed to assistive technologies (ATs) such as screen readers. As developers we don't have access to these APIs.

The Accessibility Tree provides a sister hierarchy to the browser's document object model (DOM). This contains element roles (e.g. <code>button</code>) and properties (e.g. <code>disabled</code>) as well as any text content (e.g. <code>Checkout Now</code>).

Microsoft Internet Explorer allows screen readers to inject themselves into the application process and communicates directly with the browser, while in MacOS the screen reader communicates with the browser across OS APIs.

The Microsoft Internet Explorer model provides a fast time to interaction, but is prone to instability. If the browser crashes it can take the screen reader down with it. Chrome and Firefox provide greater stability by proxying the accessibility API calls, Chrome also now caches the accessibility tree. This can lead to a slower initial TTI but fast subsequent page hits. This delay in TTI can be multiple seconds.

Firefox uses intelligent caching to improve TTI performance. Microsoft Edge, however, does not allow screen readers to inject into the process, depending on communication via the OS APIs. This makes communication between screen reader and Edge can be up to 50x slower than Internet Explorer.

Safari on MacOS is in-process, providing fast TTI.

## Tobias Baldauf - 25% faster hotel search. Web performance? Trivago.

1. Research
2. Pitch
3. Build
4. Verify
5. Merge

When launching a large marketing campaign in Brazil, Trivago discovered that Brazil performance was one second slower than the rest of the world. By analysing the performance by proxying into Brazil, they discovered that their CDN was not sending Brazil traffic to their Brazilian edge servers, but up to North America!

At Trivago, web performance is equal to user value, especially for emerging markets.

Feature teams are created from multiple disciplines around projects and given the freedom to build the feature.

A team of twelve people run a release controller tool: a real-user performance testing framework to test the impact of new features with canary traffic.

> The behaviour of your existing user base will not change instantly, they have trained behaviour and expectations. It took three months to see the positive impact of performance improvement.

Part of the CDN rollout produced a delay on the hotel results display, which led to an increase in conversions! By keeping the UI clean while users interacted with the calendar widget and filters, users were more likely to continue their user journey.

Trivago has a performance team - they are not responsible for performance optimisation. The performance team owns the performance tooling and provides data to feature teams, enabling the developers to improve the performance of their own features.

Automated performance and accessibility testing is performed across at least five browser testing tools for every build, 30 - 50 times a day! Issues are added as build badges to the git branches.

> Web performance is a feature owned by everyone. Specialists consult and evangelise. Fast feedback loops help to iterate quickly and long-term data helps discover trends and define direction.

> Beware of what to measure: business metrics may change unexpectedly when improving web performance. Choose the right tool for the job.

> Web performance is equal to but not superior to the main features of a website sand can be temporarily traded in for learnings.

## Henri Helvetica - The shape of the web

The first browser, Mosaic, recently celebrated its 25th birthday.

> this software is going to change everything

Mosaic achieved 65M new users in 18 months, now we have over 4 billion people online.

We do everything on the web, and it's reshaping everything we do.

> the web is the most hostile software engineering environment imaginable

The next users on the web are coming from emerging markets with poor connectivity and low-end devices. Many users in these markets turn off their data connection to save money.

We need to treat **offline** not as an error case, but as a normal use case. This is the shape of the web. Service Workers and Progressive Web Apps are the greatest updates to browsers in a decade.


==============================================================================

# Delta V Conference - Day 1!

> Delta V Conference is a new two-day, single-track web performance conference in London, hosted by me, Perry Dyball and Jo Franchetti. Here's my raw notes from the fantastic content from our lovely speakers on day one.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2018-05-10
Canonical: https://simonhearne.com/2018/deltav-day1/
Tags: WebPerf, conference

---
## Tim Kadlec - Redefining Web Performance

There is a lot of under-appreciated creativity in technical work.
Definitions change - dictionaries are out of date as soon as they are printed - like JS frameworks.

**Network Constraint** - 71% of global connections are 2G or 3G, we live with a rose-tinted view of the world.

**Device Constraint** - new adoption is on low-end Android devices. 90th percentile JS payload is 1MB (compressed) / 7MB (uncompressed).

Performance cannot be fixed top-down (AMP / FIA / Apple News).
It took the New York Times 10 years to fix their performance problem.

**Speculative execution** - a performance optimisation without considerations for the implications. (Spectre / Meltdown).

**Access & accessibility** - he keystroke-level model for user performance time with interactive systems.
Performance is about the level of effort, human memory limits, task completion.

> Web Performance: how quickly users can complete their task.

## Sarah Dapul-Weberman - The first 10 months of a dedicated web performance team at Pinterest

**Discovering the value of performance**
Migration of user pages to React -> 20% performance gain. Led to 10 - 20% engagement increase.
Non-auth pages 30% faster -> 15% more signups, 10% increase in SEO traffic.

**No ownership of performance** -> started dedicated performance teams for web, Android and iOS.

**Pinner Wait Time (PWT)** as the key performance metric - a composite of the time to render the key image and the comments / metadata.

Get rid of vanity metrics - prove correlation with business goals or drop them.
Performance is a **lever** to change the business metrics.

**Impact Assessments** - artificially delay key metrics and measure the impact on user behaviour / business metrics.

**PR campaign** to publicise the performance team on all-hands calls.
Each critical page is tested multiple hundreds of times on every commit - the 90th percentile of PWT is taken and plotted, threshold-based alerts.

**"Perf Detective"** performs a binary search with performance testing (like git bisect) to find the exact commit which caused the regression.

**Changes are prototyped first** to provide estimated performance wins - before development starts! Once the projects are decided on they are tested in an A/B framework which has performance and engagement metrics built in!
A/B test results are segmented by geography, user behaviour profiles etc.

The team beat the original goal of 14% improvement by an additional 20%!
Vision: five pillars:

 1. Scalability (team) - we can’t be the only people working on performance. Enable product teams to manage performance.
 2. Ownership - each team owns their surface performance. Decentralised monitoring.
 3. Knowledge base - create a central place to document what performance looks like at Pinterest
 4. Tooling - create / purchase the tools which developers want to use
 5. Strategy - performance team will own the performance strategy - where to focus, tool management. Distribute the work.

## Rhys Evans - Speeding up FT.com, without slowing down

[Slides](https://speakerdeck.com/player/845b3dbdbfab441b86a059966bb68d69) | [@wheresrhys](https://twitter.com/wheresrhys)

The old FT.com site (Falcon) was slow and high-risk. One release a month!

> If we ever need to start from scratch again, we've gone badly wrong

For the new site, the team insisted on control of all web content, including third-parties.

Deploy local -> production in 10 minutes using CircleCI and Heroku. Feature flags enable roll-out of features to live traffic.

~100 microservices create the FT.com back-end stack.
Componentisation at the front-end using [Origami](http://origami.ft.com).

> These practices free us to release quality software quickly, with confidence

Microservices add extra latency to user requests due to dependency chains. Ft.com uses multiple caching layers to resolve that.

Preflight enables intelligent caching at the edge. But this itself is another performance bottleneck with multiple microservices! **Every service** is measured, medians and percentiles are used and the number of timeouts are counted and plotted too.

**Code impatiently** - HTTP agent keep alive to create connection pools. Pessimistic timeouts - fail fast and provide error details.

Fast releases reduced asset reuse - with no inlined CSS or shared JS bundles!

The singular focus on performance made releases harder, alienated the development team.

> Performance is not a core front-end skill. It's weird, complex and **optional**

Performance optimisations should be discussed with the whole team, especially junior developers. Optimisations should be released early and often, tested and balanced with developer convenience.

> You don't have to be perfect to get results!

[Further reading!](https://medium.com/ft-product-technology/making-a-request-to-the-financial-times-b2119a2f422d)

## Seren Davies & Bruce Lawson - This is for everyone

### Seren

Accessibility issues are much broader than we consider, with two broad topics: **permanent** and **temporary / situational**.

**Cognitive Accessibility** e.g. dyslexia, ADHD and epilepsy
Animations should add value and not distract.

**Drunk**
Use high contrast colours and large text. City mapper increases the size of the "get me home" button as the evening progresses!

**Touch or movement** - e.g. broken arm, holding a baby.

Link areas should be larger than the text - provide large tap targets.

**Sight** - beyond screen-readers, including colour-blindness.
Text on images is really tricky - ensure that contrast ratio is preserved for all characters.

**Deaf / hard of hearing** - always add connections

### Bruce

4 Billion people are not on the web - including 2 billion from India and China, and 51 million in the USA!

We need to sell into growing markets. These markets are overwhelming getting online with low-powered Android devices, with limited RAM on 2G networks. The infrastructure supporting mobile networks takes decades, while new devices take months.

A standard-sized Android app will take 30 minutes to download on a 2G connection, except it's unlikely that the download will succeed during that time.

Progressive Web Apps are the solution. Flipkart lite increased returning visitors by 40% when they released their PWA.

Images still account for >50% of page weight so that's a good place to start!

This matters not to the people loading your site from Silicon Valley, it's for the people who pay per MB.

Worldreader.org

> If you want to liberate a country, give it the internet

The web is for everybody.

## Inian Parameshwaran - Chrome Real User Experience Report

Measuring speed - Synthetic & RUM
HTTP Archive is the archive for synthetic tests, CrUX is the archive for RUM data.

Rather than absolute values, CrUX stores histograms. Inian defines **Site Experience Benchmark (SEB)** as the fraction of users whose time to first contentful paint is under one second.

SEB reduces by 36% on average between desktop and mobile.

On average, SEB is 225% better on non-framework sites compared with those using a single-page app frameworks. This grows to 486% when isolating to mobile traffic.

Using RUM data gives a much wider perspective on geography and device diversity than synthetic testing, so you can use it to help inform technology choices. E.g. React server-side rendering improves SEB by 47%, 69% on mobile.

google.com is 40% faster than google.com.cn for visitors in China. 🤨

Lack of information on the number of data points in CrUX makes it hard to do cross-site analysis.

## Laura Sheridan & Nick Webb - Web performance at N Brown, it's a business affair

N Brown are a fashion brand group that focuses on under-served groups - those that are larger and / or older.

First online store in 1999 as a small IT project. Now 75% online sales, 76% of traffic is smartphone. 4 million sessions per week!

After getting inspired at Velocity Berlin 2011, N Brown launched a project to remove duplicate scripts, set a weight budget for the homepage and purchased a StrangeLoop on-premise FEO solution.

Making the business understand performance - using RUM data to correlate page load time with bounce rate. Then created department-specific metrics, e.g. total image weight for the design team.

Laura held roadshows to share performance data for each of the 14 brand websites, with tips on how to improve. They also developed a web performance action plan - including scheduled work on specific operating systems, device types and page types.

To ensure performance does not regress, a weekly report is produced for each brand, device and page group combination. This is used to encourage internal competition and provide insight into regressions.

RUM data allows N Brown to modify their development plans between brands, based on the user profiles of the brand customers and their web environment.

Tag mapping for identification of what's on the site, as well as analysis for GDPR compliance. This led to a tag governance process.

> The right visualisation can do our job for us

48/52 split between iOS and Android traffic, but sales are much lower on Android. To help fix the issue, rules were added to deliver the **mobile** version of the site to Android tablets with display sizes under ten inches.

As N Brown re-platform, there are weekly team lead meetings to discuss performance and a UX Director to ensure the performance of the new sites.

## Uve - Making a Progressive Web Application

This was a live coding demo - [here's a relevant blog post!](https://medium.com/samsung-internet-dev/a-beginners-guide-to-making-progressive-web-apps-beb56224948e)

## Ian Feather - Frontend Resilience

**Resilience is** function in a hostile environment.

We don't ship compiled products - there are many components we have to ship over the network to create a functional product.

We tend to develop in the ideal environment for a great user experience, but we can't guarantee that. We need to guarantee the most basic level of acceptable UX.

### How systems can fail

**HTTPS** is a must-have to protect you from malicious interference. Third-party scripts are a nightmare.

> Pick your third-parties and trust them well.

13 million JavaScript requests fail per month (1%) - that's a greater proportion of page views than IE11. 9% of visitors use content blockers, and 4% of users don't load web fonts - 40 million page views per month with no fonts.

> Know what's broken before Twitter does

### How we can design for failure

Have a working user experience in the first request. I.e. know what the critical part of your user experience is and ensure that it can be usable with the first HTML response.

Tell your developers when things break, but don't tell your users unless you have to. If you have to tell your users - give them an informative message and reassurance.

### How we can mitigate failures

**Lock your dependencies** - use sub-resource integrity (SRI) for all third-party assets. But this then requires a well defined update process between you and your third-parties.

**Build in redundancy** - have two of everything, and use a proxy where possible to create an active / active asset delivery system.

Buzzfeed has **Plan Z** - a static copy of buzzfeed.com hosted both on AWS S3 and Google Cloud Platform and updated every few minutes. If everything is on fire, the DNS is updated to point at a static backup!

Serve stale content - use cache control headers to enable the CDN to continue to deliver assets when origin is unavailable.

### How we can learn from our failures

Run blameless postmortems religiously to ensure that knowledge is gained from every failure. Consider how incidents were handled as a team and how it can be prevented from happening in the future.

Run fire drills and chaos testing to give the team practice on responding to incidents.

## Ada Rose Cannon - New web technology

This one is best viewed the way Ada presented it, don't forget to [view-source](https://ada.is/grid-slides/new-web/).

## Denys Mishunov - debugger; for developers

### Perfectionism

There are two types of perfectionism: healthy and unhealthy.

Steve Jobs was not satisfied with any of the 2,000 shades of beige given as options for the first Apple computer. This is perfectionism paralysis.

Perfectionists tend to pick small details to work on, where there is a guarantee that they can achieve the perfect result.

Unnecessary tasks (the cherry on top). Near the end of the project you might think of a small feature that would bring the project a little closer to perfect.

We cannot force perfectionists to stop being perfectionists, but we can help to turn negative productivity into positive.

### Imposter Syndrome

2 out of 5 successful people suffer from imposter syndrome, and 70% of the general population have experienced it at some point in the career.

You think that your success is due to luck, others might discover that you are not as intelligent as they think you are or that everyone is better than you.

True frauds rarely experience imposter syndrome.

> The trouble with the world is that the stupid area cocksure and the intelligent are full of doubt

Feeling like an imposter is a symptom of gaining expertise.

Measure yourself against you, not others. Communicate your fears and feelings.

### Long Hours

One of the most common death-bed regrets is **working too much**.

There are two types of long hours: temporary and permanent. Temporary long hours are associated with hard-working people, while permanent long hours indicates workaholics.

Stress and burnout are eventually harmful to productivity, even if there is a temporary increase at the beginning.

Every year [Guolaosi](https://en.m.wiktionary.org/wiki/guolaosi) costs China 1,600 people per day due to overwork.


==============================================================================

# Third-Party Content - the weak link?

> The Resource Timing API allows site owners to measure how third-parties perform for every single pageview. Combining this data with business intelligence provides new insight to the impact of third-party providers on business performance.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2018-01-14
Canonical: https://simonhearne.com/2018/third-party-weak-links/
Tags: WebPerf, video, talk
Video: https://www.youtube.com/watch?v=9ol9MNGqVOM
Slides: https://docs.google.com/presentation/d/1pUo2SrWPvJ5vwYIbpebqFqnhAwGZ2pVryKP2kzqA_Uw/edit?usp=sharing

---



==============================================================================

# Twitter Lite isn't as good as the app, and that's okay

> Twitter's Progressive Web App is finally out of testing, and it's not as good as the native app. Missing push notifications, wonky images, small text boxes, lack of sharing intents and confusing pull-to-update behaviour all highlight the challenge of building a PWA. The fact that Twitter is on-board with PWAs is a great step towards better web experiences for all.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2017-04-07
Canonical: https://simonhearne.com/2017/twitter-pwa/
Tags: WebPerf, PWA

---
## The promise of PWAs

Progressive Web Apps bring a native-like application experience to websites. This is achieved through a combination of specifications like the web application manifest, service worker, web push API and (soon) webAPKs.

The advantages of using a website over a native application are numerous for consumers: faster install, more rapid updates and a significant reduction in the storage required. There are also advantages for application publishers: less friction to install (it's just a URL!), instant updates (no app store update process) and they are easier to develop for cross-platform audiences.

Normal websites can be upgraded to add great features such as a good offline experience and a more immersive browser view. This means that PWAs aren't just an alternative to native applications, it makes sense to roll these features out on websites anyway.

<br>

Have you ever started to install an application, then stop because it requested permission to record audio, spam all of your contacts, install a trojan horse or turn your phone pink? The web has a well developed permissions model for things like access to the camera, vibration, location and sensors. A PWA inherits all of this development in web permissions.

> I want to point out here that, although I am picking at the flaws in Twitter Lite, I have great respect for the team that created it. Twitter releasing a progressive web app is such a positive sign for the web. I'm also aware that I am not the key audience of Twitter Lite: I have a phone that cost an eye-watering amount to buy and consistent access to 4G coverage. The issues I highlight below are minor compared to the increased reach provided by Twitter Lite: to users who cannot access the latest devices, who may only have 2G network coverage and might rely solely on their phone for communication. Twitter Lite is a positive step for the web! Browsers just need to catch up with what we are now expecting of them.

## Twitter Lite

![](https://simonhearne.com/images/twitter-pwa/twitter-app-size.png)
*104MB of storage for the Twitter application*

Twitter's native application has over 500M downloads on the Google Play Store, and presumably a similar number on the App Store. Twitter has been experimenting with their mobile web experience over the past six months, adding PWA features to groups of users and measuring the response. On April 6 Twitter announced 'Twitter Lite', a fully-fledged progressive web app. The [marketing](https://blog.twitter.com/2017/introducing-twitter-lite) angle around the release is that it will improve the experience for users with low bandwidth mobile connections, especially those in Asia Pacific, Latin America and Africa. I don't fit that demographic, I am extremely fortunate to have a smartphone that costs an eye watering amount to buy and I have 4G internet most of the time. But I am a massive PWA fan, and couldn't continue to call myself that without switching to the Twitter PWA!
I must point out that the PWA *is* a massive step for Twitter users worldwide and for the industry. But there is also a lot of room for improvement both with the PWA and browser implementations.

<blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">Introducing Twitter Lite on mobile web! 📱<br><br>Loads quickly, takes up less space, and is data-friendly. Learn more: <a href="https://t.co/Zd825WOdQz">https://t.co/Zd825WOdQz</a> <a href="https://t.co/l1n0cYJuPc">pic.twitter.com/l1n0cYJuPc</a></p>&mdash; Twitter (@Twitter) <a href="https://twitter.com/Twitter/status/849866660882206721">April 6, 2017</a></blockquote>

I jumped on the bandwagon, why would I want a massive native application, with frequent ~70MB updates, when the mobile website will give me the same experience?!

<blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">Goodbye native app, hello fast web experience! <a href="https://t.co/xfQ3tY0hYW">pic.twitter.com/xfQ3tY0hYW</a></p>&mdash; Simon Hearne (@simonhearne) <a href="https://twitter.com/Twitter/status/849970819258343424">April 6, 2017</a></blockquote>

> I'm sad to say that I have now reverted to the native application, after less than 24 hours of using the PWA.

## Browser dependence

Before I discuss why I've gone back to the native application, it's worth discussing the inherent limitations of PWAs. The core PWA specifications of [service worker](http://caniuse.com/#feat=serviceworkers), [web app manifest](http://caniuse.com/#feat=web-app-manifest) and [web push](http://caniuse.com/#feat=push-api) are currently only supported on mobile devices running Android.

![](https://simonhearne.com/images/twitter-pwa/caniuse-pwas.png)
*PWA Support*

This also means that the performance of the PWA is dependent on browser performance. Native applications can be optimised at a code-level to give the best performance, while PWAs have to rely on the browser to interpret the HTML, CSS and JavaScript provided. This may mean that PWAs perform more poorly than native applications.

## Things that just don't work

There are five key issues I have with the twitter PWA which have led me back to the native application, for now.

### 1) Text boxes are tiny, and broken

I use an increased font-size in my browser configuration, this is because so many websites use tiny font sizes and disable zooming on mobile. It's easier for me to just have bigger text everywhere. Unfortunately this breaks the way Twitter's input boxes grow with text, making it really awkward to edit or review a tweet.

<figure align="center">
<video autoplay muted loop controls="true" playsinline style="width:80%; max-width:480px;" preload="metadata">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/w_480,q_auto/v1612776931/videos/text.webm" type="video/webm">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/w_480,q_auto/v1612776931/videos/text.mp4" type="video/mp4">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/w_480,q_auto/v1612776931/videos/text.mov" type="video/mov">
  <source src="/images/text.webm" type="video/webm">
  <source src="/images/text.mp4" type="video/mp4">
</video>
<figcaption>Text boxes are too small and do not overflow correctly</figcaption>
</figure>

### 2) Push notifications are less reliable

I noticed a few times during my trial of the PWA that I would open Twitter to find unread notifications, but I had received no push notifications. On re-installing the native application I often get more notifications from the application than the PWA. The Web Push API depends on a [third-party server](https://developers.google.com/web/fundamentals/engage-and-retain/push-notifications/sending-messages) to send notifications, and it's generally fire-and-forget with no ability to re-send failed notifications.
Also frustrating is the lack of grouping of notifications, a popular tweet can quickly fill up your notifications!

![](https://simonhearne.com/images/twitter-pwa/twitter-pwa-notifications.png)
*Fewer notifications from the PWA*

A small secondary issue is that newline characters are not escaped in the notifications. I'm not sure if this behaviour changes between devices, if so this would be really tough to test. It makes the PWA feel a bit immature.

![](https://simonhearne.com/images/twitter-pwa/twitter-newlines.png)
*New lines are not escaped in notifications.*

### 3) Portrait images aren't re-oriented

When attempting to add a portrait picture, which the native application uploads with the correct orientation, the PWA has it sideways. This wouldn't be such an issue if the image could then be rotated in the PWA, but it cannot. As such this is a killer bug for me.

Another slight irritation is shown in the video below. There is no indication to the user that the image is being uploaded! This is on wifi and there is an eight second delay with no user feedback, while the whole browser is locked up. Considering that the PWA is designed for users with low bandwidth mobile connections, this delay could be significantly longer.

<figure align="center">
<video autoplay muted loop controls="true" playsinline style="width:80%; max-width:480px;" preload="metadata">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/w_500,q_auto/v1612776931/videos/image.webm" type="video/webm">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/w_500,q_auto/v1612776931/videos/image.mp4" type="video/mp4">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/w_500,q_auto/v1612776931/videos/image.mov" type="video/mov">
</video>
<figcaption>Images aren't correctly rotated, and there's no way to fix it</figcaption>
</figure>

### 4) Pull-to-...

The native Twitter application has trained me to pull-to-update on the feed. Pulling down in other screens does nothing. Unfortunately, due to the dependence on the browser, this behaviour is not consistent in the PWA. It will reload the whole page if you pull on most pages. Implementing pull-to-update across the whole PWA might make this more consistent.

<figure align="center">
<video autoplay muted loop controls="true" playsinline style="width:80%; max-width:480px;" preload="metadata">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/w_500,q_auto/v1612776931/videos/ptr.webm" type="video/webm">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/w_500,q_auto/v1612776931/videos/ptr.mp4" type="video/mp4">
    <source src="https://res.cloudinary.com/simonhearne/video/upload/w_500,q_auto/v1612776931/videos/ptr.mov" type="video/mov">
</video>
<figcaption>Pull-to-update behaviour is inconstistent.</figcaption>
</figure>

### 5) Sharing

One of the key features of the web is being able to share content. The native Twitter application adds sharing intents to menus, so that you can share content via tweets or direct messages from any other application (or browser) on your device. This is missing in the PWA which makes day-to-day use significantly more tricky - copy to clipboard, back to homescreen, open PWA, click the tweet button, tap-and-hold, paste text.

![](https://simonhearne.com/images/twitter-pwa/twitter-share-fail.png)
*Lack of integrated share options is frustrating.*

## Minor annoyances

1. Can't have multiple accounts logged in simultaneously
2. Saved searches are gone
3. No option to add to list
4. No swipe actions
5. No tweet analytics

As the PWA is [built on the Twitter API](https://blog.twitter.com/2017/how-we-built-twitter-lite) its functionality is limited to what is currently available to all other Twitter clients.

## Conclusion

So I've moved back to the native application. Of course that doesn't mean I can't use the PWA, as I can just open up [mobile.twitter.com](https://mobile.twitter.com/) and get the full PWA experience. The Twitter PWA is trying to do a lot, while browsers are still getting to grips with implementing the specifications. Having Twitter invested in the PWA ecosystem should really help to push the development along.

Committing to PWAs will continue to be a tough sell until Safari on iOS supports service worker, web push and web app manifests.


==============================================================================

# Your Analytics Lies to You

> You can't rely on your web analytics for performance data and it's understating bounce rate and traffic numbers from old devices.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2017-03-29
Canonical: https://simonhearne.com/2017/analytics-lie/
Tags: WebPerf, analytics

---
## The site speed fallacy

In a recent webinar, [Andy Davies](https://twitter.com/AndyDavies) and I asked a poll of the attendees: where do you get your site speed data? Bear in mind that the majority of the attendees on the webinar were from Ecommerce or Digital Management. The results were just about what we were expecting, over half of attendees use web analytics to get their speed data:

![](https://simonhearne.com/images/analytics-hero.png)
*Poll showing that 58% use web analytics to get site speed data.*

> Miss the webinar? You can [watch the recording here](https://attendee.gotowebinar.com/recording/224707572878182658)

A further 29% of attendees use synthetic monitoring as their source of site speed data. While synthetic monitoring is great for measuring operational performance and tracking changes over time, it gives little indication of the real user experience. Only 2% of attendees stated that they use real user monitoring.

Using analytics for site speed is logical, if you have a website you will have analytics. Access to products like Google Analytics is widespread throughout an organisation, and it provides reporting on site speed data. Google Analytics (GA) is actually the best we've seen for reporting site speed. You can view multiple timing metrics and even segment by device type or location. Adobe Analytics has no default site speed data, you have to manually add it to the analytics code, once it's being collected there is very little you can do with it.

![](https://simonhearne.com/images/ga-speed-sampling.png)
*GA quietly samples site speed data to 1%, independent of traffic sampling.*

GA's site speed data is flawed. The data is sampled, limited by default to [no more than 1% of pageviews](https://developers.google.com/analytics/devguides/collection/analyticsjs/field-reference#siteSpeedSampleRate) even for Analytics 360 paid accounts. This sample rate can be changed to 100%, although that will dramatically skew your data towards traffic in the early hours of the morning, [especially if your site attracts over 1M pageviews per day](https://developers.google.com/analytics/devguides/collection/analyticsjs/user-timings#sampling_considerations). Even if GA collected 100% of site speed data, it cannot be correlated with any other metric. The speed data is aggregated and averaged by time and not linked with sessions. As such it is impossible to determine the user experience of a whole session, or to correlate speed with goal completion.

<figure align="center">
<code>ga('create', 'UA-XXXX-Y', {'siteSpeedSampleRate': 100});</code>
<figcaption>Set a 100% sample rate for GA site speed. p.s. <a href='https://developers.google.com/analytics/devguides/collection/analyticsjs/user-timings#sampling_considerations'>do not do this</a>!</figcaption>
</figure>

Most of my job as a performance consultant is convincing companies that site speed is critical to their business, GA does not make that easy!

## Phantom bounces

Anyway, so you've changed your sampling rate and got 100% of your pageviews sending site speed data. The average page load time is seven seconds, that seems pretty slow to you. Bounce rate is just over 50%, that seems pretty bad too.
It's scary to think about it, but your bounce rate is likely much higher than 50%. Bounces are sessions which have only one pageview, it doesn't count sessions which have zero pageviews... Bear with me.

A recent [study by Google](https://www.doubleclickbygoogle.com/articles/mobile-speed-matters/) compared DoubleClick and Google Analytics data to find that 53% of mobile ad clicks never result in a pageview, when the page has a load time of three seconds or more. Your analytics shows you that your mobile page load time is eight seconds, how many users who click on a link to your site *never* result in a pageview?

![](https://simonhearne.com/images/ga_cancelled.png)
*GA beacons cancelled when user navigates away during a slow page load.*

These non-pageviews are *phantom bounces*. They don't appear anywhere in your analytics, but they could be a significant proportion of your traffic. What's worse, your marketing budget is being spent on traffic which may never reach your landing pages.

## Device representation

Most of my clients base their device testing strategy on their analytics. I vividly remember a story I was told by an employee at a large UK retailer: when first joining the mobile team as a contract resource she questioned why there was no testing on the first generation iPad. She was told that their analytics showed no traffic from such an old device (three years after it was launched) so there was no need to test on it. The sceptical contractor got out her very-much-still-working iPad 1 and loaded the homepage. Safari crashed. No analytics event was recorded.

This is a textbook case of the [tail wagging the dog](https://en.wiktionary.org/wiki/tail_wagging_the_dog).

It is common for my clients to focus their attention on flagship devices from Apple and Samsung, "because all of our customers have them". Unfortunately your analytics is coming up short again. For one, GA will group all iPhone models into a homogenous 'iPhone' device; it does the same with iPads. How can you develop a solid device lab when you don't know which of the eleven iPhone models generate the most traffic?

Just because Apple and Samsung make up for 80% of your smartphone traffic, that doesn't mean that you can ignore the other 20%. As we've seen above, that 20% could be under-represented due to older devices failing to load your site *at all*.

## Solutions

Right, that's enough of problems. How do we go about getting the right information?

### Analytics

There's little that you can do to improve your analytics. With GA you can increase the site speed sample rate, but the data still doesn't have much value. It's the same with Adobe Analytics. Unfortunately I think it's necessary to separate performance and analytics products, even though they have a significant overlap. There's a niche waiting to be exploited! NCC's RUM solution has been designed to help build a business case for site speed improvements. I highly recommend a [three week free trial](https://www.nccgroup.trust/rum-webinar/).

### Solving phantom bounces

Phantom bounces are an interesting challenge. From the DoubleClick study we know that there is a disparity between the clicks being logged and the pages being loaded. The flow goes something like this, assuming a click from Google's SERP:

 1. User clicks a link (actually loads a Google link)
 2. Google logs the click, redirects user to your site
 3. User's browser makes a request for your site
 4. Your site responds with some HTML
 5. HTML, CSS, JavaScript are processed
 6. Page loads
 7. Analytics beacon fires

 Ideally, we would get data from the first step - how many people make the intention to load the page. This information is extremely hard to get, although an advertising provider should report on the number of clicks. If you have a good URL strategy then you might be able to do your own DoubleClick-style study, correlating ad clicks with pageviews.

 It's also possible to correlate page requests with pageviews. Web server or CDN access logs can be compared against reported page impressions to determine the analytics loss. Further correlation can be run against the useragent of the requests to identify devices which are more underreported in analytics than others. I have seen loss rates around 20% before.

### Better device representation

The homogenous 'iPhone' and 'iPad' issue can be solved pretty easily using [WURFL.js](https://web.wurfl.io/#wurfl-js). Simply instrument the JavaScript snippet on your site, integrate with GA and voila: iPhone and iPad models in your analytics.

Unless your website has a very specific audience, base your device testing on your target markets not your analytics. [Amazon's best sellers list](https://www.amazon.co.uk/Best-Sellers-Electronics-Mobile-Phones-Smartphones/zgbs/electronics/5362060031/ref=zg_bs_nav_ce_3_1340509031) is usually a great place to start, bolt on the last 2.5 years of Apple and Samsung flagships, plus the Chinese upstarts OnePlus and Xiaomi, and you'll get reasonable coverage.

The top 12 devices should give you around 20% coverage of the whole smartphone market. You'll need 60 devices to get to 50%! Testing on the latest flagships should be a given, testing on the budget devices that make up around [half of the market](http://opensignal.com/reports/2015/08/android-fragmentation/) is the next step to take.

Get in touch via [email](mailto:simon@hearne.me) or [twitter](https://twitter.com/simonhearne) if you need help getting better performance data, optimising your analytics configuration or setting up a device lab.


==============================================================================

# How to Optimise CSS Image Sprites

> Optimising sprites might be more important than you think, and easier than you expect!

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2017-01-20
Canonical: https://simonhearne.com/2017/optimising-css-image-sprites/
Tags: WebPerf, images

---
CSS image sprites can provide a performance benefit for most sites.
A sprite combines multiple small images into one file and CSS slices out the relevant sections of the image when they are required.
This reduces the number of requests required to render a page, potentially meaning that the page renders faster over HTTP/1.1.

The humble sprite can become critical to web performance. It is common to see the main website logo buried in the middle of image slider controls and social media icons.
Unfortunately, like all images, sprites can become bloated and slow.

Here are some top tips to improve your sprite's performance:

## Remove your logo image

Serve your logo image as a separate file, referenced in an ```<img>``` element.
This will be downloaded earlier than the sprite file (which is discovered late by the browser).

![](https://simonhearne.com/images/css-sprites/amazon-sprite.png)
*Amazon used to put its logo in a sprite. Why?!*

A secondary benefit is that the logo asset can be cached for a very long period,
unaffected by minor changes to the other icons in the sprite.

## Remove icons that are available in UTF-8

It's common to see simple elements such as left and right arrows included in sprites.
Oftentimes there is a UTF-8 symbol which provides a suitable match. <a href="https://www.utf8icons.com/">UTF-8 Icons</a> provides a good reference.

![](https://simonhearne.com/images/css-sprites/utf8-icons.png)
*Some of the characters available for free in UTF-8*

## Use CSS filters

Want to change the colour of an icon when someone hovers over it? Don't add another image to the sprite, use CSS filters to do it automatically.
A ```:hover``` rule like ```filter:hue-rotate(90deg);``` or ```filter:invert(100%);``` might just do the job!

## Separate icons with very different colours

PNG compression works best for a limited colour palette. If your sprite has images with complex colours, consider moving them to a separate image.
A good test of this is to save your image as a PNG8 and see if any of the colours are lost in the process.

## Save as a PNG8

Most image editors will save PNG files that have transparency as PNG32 by default.
This allows for 24 bits of colour data (true colour = >16.7 million colours) as well as 256 levels of transparency.

PNG8 allows for 255 colours plus on/off transparency. This should be plenty for most sprites and significantly reduces file size.

<a href="https://tinypng.com/">TinyPNG.com</a> is a good place to start optimising your sprite.

## Remove whitespace

Whitespace is totally unnecessary in a sprite, as the CSS is defined at a pixel level.
While extra whitespace will not have a significant impact on filesize, thanks to PNG compression, the memory footprint on the device will be increased.
In a world of £50 Android phones from Amazon, we have to be extremely conservative with memory usage.

![](https://simonhearne.com/images/css-sprites/sprite-whitespace.png)
*That's a lot of wasted memory*

## Keep it under 14kB

On the web, a packet is around 1,500B of data (the [maximum transmission unit](https://en.m.wikipedia.org/wiki/Maximum_transmission_unit)). In general you should send up to 10 packets in the first round trip of a TCP connection (the [TCP initial congestion window](https://en.m.wikipedia.org/wiki/TCP_congestion_control#Congestion_window)).

Assuming you expect the image sprite to load early in the page load, it might be the first object in one of the six available TCP connections (in HTTP/1.1), and thus subject to the initial congestion window. Keeping it under 14kB means that the image can download in one network round-trip (assuming header data is under 1kB), reducing the effect of latency and packet loss.

## Make it square

This may be a micro-optimisation, but square images are more efficient than rectangles on memory-constrained devices.


==============================================================================

# Three WebPerf Takeaways from Velocity Europe

> Velocity Amsterdam has just come to a close. Here are the three key things that I am taking away from the two days of sessions.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2016-11-08
Canonical: https://simonhearne.com/2016/three-takeaways-from-velocity/
Tags: WebPerf, conference

---
## 1) Performance KPIs are broken

This was one of the themes that ran like an undercurrent through the whole conference, starting with [Steve Souders](https://twitter.com/souders) at WebPerfDays talking about the death of window.onload for performance measurement. It's been known for a while that the onload event is flawed for measuring performance, but it has been universal across browsers so was the lowest friction measure. The problem with measuring onload is that the number can be [gamed](http://www.leanblog.org/tag/gaming-the-numbers/). Better numbers can be achieved by moving content around, while the measure can also be influenced by third-party content. The onload does not necessarily bare any relation to user experience, especially with JavaScript-driven websites.

There are a great many alternatives to the onload event for measuring performance. [Speed Index](https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/metrics/speed-index) measures the average time at which parts of the page are displayed in synthetic tests. [User timings](http://www.webperformance.io/custom-user-timings) can measure specific events such as time-to-first-interaction for real user monitoring. None of these are a silver bullet: Speed Index is impacted by cookie banners and modals, and is dependent on test speed and viewport size. User timings have to be carefully thought out and defined for specific page types, making them labour intensive.

> recommendations and statistics are objective measures of performance, but user perception is not mathematics

[Denys Mishunov](https://twitter.com/mishunov) also spoke at WebPerfDays, he says that "recommendations and statistics are objective measures of performance, but user perception is not mathematics". Denys had some great examples of objectively fast experiences that had a poor user experience, such as a coin counting machine that users didn't trust because it [counted coins too quickly](http://www.90percentofeverything.com/2010/12/16/adding-delays-to-increase-perceived-value-does-it-work/). He extended this concept to the idea of passive- vs active-waiting: we can trick the user into thinking an interaction is just the right speed by manipulating the experience. [Optimistic UIs](https://www.smashingmagazine.com/2016/11/true-lies-of-optimistic-user-interfaces/) are an example of using this in the wild. This concept makes measuring simple timing points of pages almost totally irrelevant, instead we need to measure when the user thought that the key event occurred, the delay until it actually occurred and the failure rate of the optimistic UI interaction.

One of the questions asked by the audience after [Kevin Bowman](https://twitter.com/__kb)'s great talk about [surviving big events](http://conferences.oreilly.com/velocity/devops-web-performance-eu/public/schedule/detail/53557) was 'what is the uptime SLA for Sky Betting'. Kevin's response was that they didn't have an official SLA, everyone just tries their best and they trust others to do the same. From my experience, having an SLA leads to gaming the numbers. This is especially true when there are incentives (i.e. a financial bonus) linked to the reported figure.

> Targets always result in gaming

Gaming the numbers certainly is not restricted to web performance. The UK National Health System has been highlighted as an organisation where KPIs have led to poor standards of care, leading to [unnecessary patient deaths](http://www.nationalhealthexecutive.com/Health-Care-News/nhs-performance-management-putting-standards-of-care-at-risk), even though the care givers give great concern to the welfare of their patients. KPIs are easy for senior stakeholders to set and measure - on paper they are the ideal way to get visibility into performance. Unfortunately static goals are open to abuse through manipulating the numbers in any number of ways.

To improve performance we should enable teams to be successful, to measure their own performance and set their own definitions of success.

## 2) Team success = business success

This was a positive theme that ran through a lot of the talks at velocity. We must enable our teams to do their jobs, let them make their own decisions and remove micro-management.
A great example of this was given by Kevin from Sky Betting. In the run up to the biggest event in SkyBet's calendar, they knew it was going to be tough. Kevin even said "we knew we were going to run out of compute resource, we just didn't know where". Team dynamics are critical in order to survive an event where just minutes of downtime can cost £100,000s of customer bets. SkyBet has cross-team 'squads', each of which has a responsibility on the production system. These squads are self-managing, so when something goes wrong they just get on and fix it. There were some close calls during the big event, with the platform almost coming to a stand-still. Kevin believes that the trust and independence of the squads allowed the issues to be resolved as quickly as possible, without large-scale war rooms which would have wasted valuable time.

> Successful teams are trusted and manage themselves.

[Sarah Wells](https://twitter.com/sarahjwells) from the FT emphasized the importance of enabling teams to clearly state their key goal(s). Without clear, agreed upon direction, teams will never realise their maximum potential. As part of this goal, teams at the FT are empowered to choose whatever tools and processes they think will be the best fit for their problems. In order to maintain supportability these teams are then responsible for supporting their chosen technologies 24x7, if they're not already covered by first-line support.

[Arnoud Vermeer](https://twitter.com/funzoneq) from Leaseweb described the journey from [waterfall to agile developement](http://conferences.oreilly.com/velocity/devops-web-performance-eu/public/schedule/detail/52562). One of the key concepts that Arnoud attributes to the success of their agile methodology is autonomous, empowered teams. Each team is responsible for one product. That responsibility extends from planning through development to in-live support. This enabled Leaseweb to produce predictable development, with priorities based on business value. The development roadmap is transparent to the whole company, with all employees involved in strategic direction.

To keep Leaseweb on track, there is a shared dashboard which shows the results of an anonymous post-sprint retrospective survey. This gives an early indication of when team dynamics begin to fail.

![](https://simonhearne.com/images/team_dashboard.png)
*Leaseweb's Retro Dashboard (source)*

[Astrid Atkinson](https://twitter.com/shinynew_oz) listed seven factors which lead to successful teams in her keynote, which by proxy lead to business success:

- Diversity is strength - only with a diverse set of personalities can you truly develop strong ideas
- The best teams are inclusive - all ideas should be treated with respect and openness
- Seek understanding, not skills - when hiring, look for candidates who will be a good personal fit
- Give power away - empower teams to make the right (or wrong) choices
- Create safety - there should be no fear of speaking out, even to the CEO of the company
- Mission matters - the company, team and individual must all have a clear common mission

## 3) We are not using great Web features

There are a number of great Web features that are available to us today, yet few websites are utilising them. One example is the challenge of under-utilised bandwidth: currently, there is a delay while the browser downloads and parses the HTML document, discovers a ```<link rel='stylesheet'>``` directive and requests the CSS asset(s) required to render the page.

This process takes time. This delay is unnecessary as the server already knows what assets are critical to render the page!

Enter ```<link rel='preload'>```.

Preload is an HTML directive or HTTP header which hints to the browser that the linked asset should be pre-emptively loaded, as soon as possible. As an HTML directive it doesn't offer much of a performance benefit in the described scenario. As an HTTP header however, it can offer a performance gain: the browser is made aware of a critical asset before the HTML is even downloaded.

HTTP/2 takes this paradigm one step further. [HTTP/2 Server Push](https://www.igvita.com/2013/06/12/innovating-with-http-2.0-server-push/) allows the server to actively push the asset to the browser. Cloudflare's CDN currently [implements this automatically](https://blog.cloudflare.com/announcing-support-for-http-2-server-push-2/) when it sees the preload header.

[Colin Bendell](https://twitter.com/colinbendell) of Akamai spoke about the [promise of Push](http://conferences.oreilly.com/velocity/devops-web-performance-eu/public/schedule/detail/53584). Although there is still some work to do on mainstream support (test your browser using Colin's [canipush.com](https://canipush.com)), we can deploy Push now. Pushing CSS shows performance gains, while pushing anything else might actually make things worse. ```<link rel='preload'>``` may still be the best option for WebFonts, for example.

![](https://simonhearne.com/images/push.png)
*Potential win area for Push*

Another technology mentioned a lot, again, was Progressive Web Apps. The magical combination of [web app manifests](https://developers.google.com/web/updates/2014/11/Support-for-installable-web-apps-with-webapp-manifest-in-chrome-38-for-Android), [service worker](https://jakearchibald.com/2014/using-serviceworker-today/) and [web push notifications](https://developers.google.com/web/fundamentals/engage-and-retain/push-notifications/) which give us a near-native web app experience on Android devices. The fundamental technologies for PWAs have [been around since mid-2015](https://infrequently.org/2015/06/progressive-apps-escaping-tabs-without-losing-our-soul/), and yet adoption is extremely low. Only 0.5% of mobile pages tested by HTTP Archive include a ```manifest``` object.

[Jason Grigsby](https://twitter.com/grigs) gave an [interesting keynote](http://conferences.oreilly.com/velocity/devops-web-performance-eu/public/schedule/detail/54697) about PWAs. He explained that every step towards a PWA [makes sense on its own](https://cloudfour.com/thinks/progressive-web-apps-simply-make-sense/). As such it's shocking that web app manifests, the first step towards PWAs, have such a low penetration.

From speaking with my clients, it seems that there is a hesitance to 'jump in to bed' with PWAs. There are questions around the lack of support on iPhones as well as the feared erosion of native application market share. However, as Jason said, each step makes sense on its own. I recommend that my clients start work towards PWAs with a web app manifest and start to investigate the potential of service worker.

## BONUS - Empathise with your audience

One of the key differentiating factors for the presentations I attended was whether the speaker empathised with their audience or not. By that I mean there were some speakers who had clearly thought about the Velocity audience and showed empathy for their pains and challenges at the very beginning of their talks. There were other speakers who appeared to be speaking for their own benefit, and had not thought about why the audience might want to hear what they have to say.

I'll do my best to keep that in mind when speaking, I hope that others will do the same.


==============================================================================

# Too hot to handle - optimizing for low powered devices

> We know what web performance is—getting pages to render fast for as many customers as possible. Traditionally, the focus has been on the network, improving the time to deliver critical content, but the combination of modern web applications, better network connectivity, and HTTP/2 is reducing the impact of the network on site speed. Our next great challenge is how well our applications perform on our customer’s CPU. As our dependency on the CPU has inevitably increased, our devices have gotten smaller, and processors have struggled to keep up with Moore’s Law. Meanwhile, our ability to profile and analyze the CPU footprint of our web applications has barely improved beyond the timeline in Chrome’s Developer Tools.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2016-10-14
Canonical: https://simonhearne.com/2016/too-hot-to-handle/
Tags: WebPerf, video, talk
Video: https://www.youtube.com/watch?v=__KEbUEFHpE
Slides: https://speakerdeck.com/simonhearne/too-hot-to-handle-optimizing-for-low-powered-devices

---
The phone in a developer's pocket is not the phone in a user's pocket. Everything in this talk follows from that gap.

## About the talk

Responsive design solved layout. It did not solve the fact that a mid-range Android handset has a fraction of the CPU budget of the flagship it was designed on, and that CPU, rather than bandwidth, is what most modern pages are actually constrained by.

This talk is about that constraint: how wide real device diversity runs, why the conveniences that make development pleasant make execution expensive, and what to do when a large share of your users are on hardware you have never held.

## What's covered

- **How wide the gap is.** Android held 53% of the US market against the iPhone's 43%, and 19% of US adults were smartphone-dependent — their phone was their only route online.
- **What it costs.** 53% of mobile visits were abandoned past three seconds (DoubleClick, September 2016). Mobile accounted for 16.9% of digital revenue that year.
- **The framework tax.** Sites built with SPA frameworks measured 43% slower by Speed Index, and took 500ms longer to start rendering on mobile.
- **The users you cannot see.** Roughly 15% of a 5,400-person survey were browsing without working JavaScript.
- **Six rules.** Responsive design is not enough; question the framework; test on low-end hardware; measure CPU, not just network; build a device lab; split the codebase when you must.
- **What actually helps.** Pre-rendering single-page apps, progressive enhancement around a core experience, and testing with JavaScript disabled in WebPageTest.

Presented at Velocity New York 2016.


==============================================================================

# Beyond the Waterfall - Meaningful Web Performance Visualizations

> With all the different measurements we can take of web performance, the data we can report on becomes huge. Waterfall charts just don’t cut it any more – they are a technical tool to diagnose technical issues. What we need now are business-focused visualizations that quickly represent the impact of performance on user experience.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2016-10-14
Canonical: https://simonhearne.com/2016/meaningful-visualisations/
Tags: WebPerf, video, talk
Video: https://www.youtube.com/watch?v=l1RpGiP3FpM
Slides: https://noti.st/simonhearne/tODEWH/beyond-the-waterfall-meaningful-web-performance-visualisations

---
A waterfall chart is a tool for people who already know what they are looking at. This talk is about everyone else.

## About the talk

Waterfall charts are diagnostic instruments. They reward expertise, and they assume the reader already knows what a request is, why blocking matters and what a long green bar means. That makes them excellent for engineers and close to useless for the people who decide whether performance work gets funded.

This talk argues for visualisations built for that second audience: images that carry their meaning at a glance, because we read pattern and shape faster than we read text. It walks through the visual vocabulary available, then applies it to real performance data.

## What's covered

- **Infographics versus visualisations.** The hand-made and the computer-generated, and why the distinction matters once your data changes daily.
- **The visual vocabulary.** Size for importance, position for relationship, colour for category, plus shape and animation — the raw materials of any chart worth putting in front of a stakeholder.
- **Six ways to show a site's performance.** Waterfalls, filmstrips, heatmaps, request maps, dashboards and the RUM wall, and which audience each one actually serves.
- **A worked third-party problem.** Using word clouds and request maps to make the third-party load at a large UK publisher legible to people who don't read waterfalls.
- **The tools.** WebPageTest, the heatmap, request map and RUM wall tools on `webperf.tools`, and Graphite for trends.

Further reading suggested in the deck: *Information Is Beautiful* by David McCandless, and *Storytelling with Data* by Cole Nussbaumer Knaflic.


==============================================================================

# Velocity NY 2016 - Wrap-up

> Velocity New York seems like a distant memory now, especially with Velocity Europe around the corner. With that said, I have copious notes from the two day conference which I have summarised here. There were a lot of nuggets of wisdom which I’ll try to organise into a logical order.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2016-10-11
Canonical: https://simonhearne.com/2016/velocity-ny-2016-wrap-up/
Tags: WebPerf, conference

---
(p.s., the [video of my talk](https://www.youtube.com/watch?v=__KEbUEFHpE) on performance for low powered devices is up!)

## Real User Monitoring is getting exciting, but has gaps

Everyone is using RUM. The vendors are maturing rapidly to be able to offer either:

* Single view of customer - integration with Application Performance Monitoring.
* Insight - analysis of user behaviour and business events to create actionable insight.

An example of a vendor going down the second route is SOASTA’s mPulse; there was a lot of discussion around [machine learning](http://conferences.oreilly.com/velocity/devops-web-performance-ny/public/schedule/detail/51082). The key conclusions from [Tammy](https://twitter.com/tameverts) and [Pat](https://twitter.com/patmeenan)'s [talk](http://conferences.oreilly.com/velocity/devops-web-performance-ny/public/schedule/detail/51082) were that:

* the best predictor of bounce rate is DOM Ready time
* the number of ```<script>```s and DOM elements are inversely correlated with conversion rate

![Line chart: conversion probability peaks near 400 DOM nodes then declines steadily as node count rises](https://simonhearne.com/images/velocity-2016/domnode_conversion.png)
*Conversion likelihood by number of DOM nodes.*

SOASTA also introduced [measuring continuity](http://conferences.oreilly.com/velocity/devops-web-performance-ny/public/schedule/detail/50522), using cool new script snippets to measure user behaviour. For example: measuring dead clicks, frame rate, time between intent and action. It’s all exciting stuff but I have concerns about the impact on user experience that these measurements will have (see [observer effect & bias](http://www.dnb.com/perspectives/data-management-and-analytics/recognizing-observer-effect-issues-in-data-science.html)!). Their experiments are available in a public [GitHub repo](https://github.com/soasta/measuring-continuity).

There is clearly a lot of potential in using RUM to make predictions and measure outcomes. What we’re still missing is tight integration with experiment technology (e.g. Maxymiser or Google Experiments). Without that, we can’t use RUM to properly manage development decisions. [Stuart McMillan](https://twitter.com/mcmillanstu) from Schuh recently mentioned that they use Google Analytics for performance data during experiments, probably because the integration works well. Unfortunately, the site speed data from Analytics is pretty poor… In the example below site speed metrics were only collected for under 1% of pageviews!

![Google Analytics site speed report showing only 64,603 of 7,961,930 pageviews sampled](https://simonhearne.com/images/velocity-2016/sitespeed_sample.png)
*Screenshot from Google Analytics low sample rate for site speed metrics.*

## Synthetic Monitoring vendors are disappearing from Velocity

There was virtually no presence from synthetic monitoring vendors at Velocity NY, the exception being [Mehdi](https://twitter.com/mdaoudi) from [Catchpoint](http://www.catchpoint.com/). This is a trend that I’ve noticed over the past three years, probably driven by the commoditised synthetic monitoring being bundled into APM solutions (e.g. New Relic’s [Synthetics](https://newrelic.com/synthetics)).

I think this says a lot about the advancement of RUM and APM. Synthetic gives the heartbeat monitoring and alerting for operational awareness, but if you know that your infrastructure and applications are ok, and that customers are having a good experience, then you don’t need synthetic monitoring to tell you a page is available.

## WebPageTest is bigger than ever

Almost every talk referenced [WebPageTest](http://www.webpagetest.org/) or [HTTPArchive](http://httparchive.org/) (which uses WebPageTest under the hood). It has certainly become the defacto web performance testing tool, with companies forming off the back of it such as [SpeedCurve](https://speedcurve.com/). What seems to be missing is contribution back in to the project. Looking at the contributors page on GitHub shows that [Pat](https://twitter.com/patmeenan) is responsible for almost all development on the project.

Perhaps the industry ([including myself](https://simonhearne.com/2015/find-third-party-assets/)) is taking this for granted.

![GitHub contributors chart: pmeenan has 3,097 commits, dwarfing the next three contributors combined](https://simonhearne.com/images/velocity-2016/webpagetest_contributors.PNG)
*WebPageTest contibutors page on GitHub showing Pat's huge contribution.*

## Big companies are talking about site speed

Ancestry and GoDaddy both spoke at Velocity. [Jed Wood](https://twitter.com/silentrant)’s talk about [creating a performance culture](http://conferences.oreilly.com/velocity/devops-web-performance-ny/public/schedule/detail/51033) at Ancestry was insightful, describing the journey from quick wins (gzip, images etc.) to a full understanding of performance. To do this, Ancestry track business metrics such as conversions, alongside user-centric performance metrics such as time to an ancestor’s name rendering. The focus on both business metrics and user-centric metrics means that Jed can demonstrate a correlation to the business to help drive further work on site speed.

Improving the Ancestry.com sign up page from 2.7 seconds to 1.7 generated a 7% increase in conversions. Interestingly, further work to get to 1.3 seconds made no further improvement to conversion. The chart below is taken from SpeedCurve and shows the team's progress over time to reduce Speed Index. What I really like about this is the use of annotations to mark where changes and releases occurred, so changes in performance can be traced back to a specific build of the website.

![SpeedCurve dashboard showing Speed Index dropping 4.6s (55%) over 90 days, annotated with release markers](https://simonhearne.com/images/velocity-2016/ancestry_speedindex.PNG)
*Screenshot of Ancestry.com's SpeedCurve Speed Index dashboard.*

One of the ways Ancestry maintains performance is by adding an artificial delay to third-party scripts executing. I wonder if [requestIdleCallback](https://developers.google.com/web/updates/2015/08/using-requestidlecallback) could be used for this?

[Jim Pierson](https://twitter.com/perfmangodaddy) from GoDaddy took a different approach. To prove that performance was important, he and an engineer [secretly improved the performance](http://conferences.oreilly.com/velocity/devops-web-performance-ny/public/schedule/detail/50588) of the GoDaddy homepage in India, while there was no change in marketing activity. The result of improving load time by 50% was an additional $35,000 of revenue per day. Now if that doesn’t sell the value of performance I’m not sure what will. Those performance tweaks are now rolled out across all of GoDaddy.

Jim used a maturity model to describe his journey in web performance, with anomaly detection, regression analysis and communication being at the top. I think the most important point that Jim made was the need for solid understanding of performance impact across the business. There's nothing quite like $35,000 to do that, I suppose.

![Pyramid diagram of four performance maturity levels: Novice, Goal oriented, Controlled, Best](https://simonhearne.com/images/velocity-2016/godaddy_maturity.PNG)
*Jim Pierson's web performance maturity pyramid.*

## Single Page Apps are slow

I was really happy that someone else said this out loud. [Boris](https://twitter.com/livshitz98) and [Manuel](https://twitter.com/MD_A13) from Akamai talked about [making SPAs faster](http://conferences.oreilly.com/velocity/devops-web-performance-ny/public/schedule/detail/51232) through selecting the right SPA framework, using JS bundlers, server-side rendering and tricking the user with a skeleton page. All of these are hacks around the fundamental problem with client-side applications. As such, I’m not a fan!

I also spoke a lot about SPAs being slow [in my presentation](http://conferences.oreilly.com/velocity/devops-web-performance-ny/public/schedule/detail/51254). In my study, a SPA will generally be 43% slower than a traditional web page. Of course this difference is magnified on mobile devices.

## AMP is not the killer feature

The [Accelerated Mobile Pages](https://www.ampproject.org/) project is over a year old now. There were two talks about AMP which took very different approaches. [Malte Ubl](https://twitter.com/cramforce) (creator of AMP) [gave a talk](http://conferences.oreilly.com/velocity/devops-web-performance-ny/public/schedule/detail/50798) about the current state of AMP and what’s coming in the future, while [Nic](https://twitter.com/nicj) and [Nigel](https://twitter.com/querymetrics) of SOASTA used analytics data gathered by mPulse to [describe what consumers were doing with AMP pages](http://conferences.oreilly.com/velocity/devops-web-performance-ny/public/schedule/detail/51319).

![AMP promotional slide claiming +73% increase in visits from Google mobile search for Slate](https://simonhearne.com/images/velocity-2016/amp.png)
*Misleading statistic from the AMP Year in Review.*

One of the interesting points brought up by Malte was that as AMP pages are almost always (>99%) served by the AMP CDN, there is a lot of potential for static performance optimisation. For example, the average image on the AMP CDN can be further compressed to reduce the file size by 40%. Rolling out these optimisations at the CDN level will have a great bang-for-buck across all AMP pages.

Analytics gathered by SOASTA paint a rather gloomy picture for publishers using AMP. While AMP pages are almost six times faster than the regular page, they take users out of the publishers’ domain. The probability of a reader of an AMP article going to the *article publishers’ own site* in the next 30 days is only 3%. So it seems there is a significant trade-off to be had: in order to have super-fast articles that are *promoted by Google in search results*, you have to sacrifice engagement and brand awareness.

![Slide comparing bounce rate: 58% for regular pages versus 82% for AMP pages](https://simonhearne.com/images/velocity-2016/amp_bounce.PNG)
*Scary engagement statistic for publishers.*

This all feels very walled-garden, especially as AMP pages are just optimised web pages, which anyone can make. I like the fact that it promotes fast content as better content, but I don’t think it’s in the publisher’s best interest. [Tim Kadlec](https://twitter.com/tkadlec) has proposed an open alternative to AMP, the [Content Performance Policy](https://timkadlec.com/2016/02/a-standardized-alternative-to-amp/), which is definitely worth read.

## There are lots of underutilised performance and security features on the web

[Sonia](https://twitter.com/soniaburney) and [Sabrina](https://twitter.com/sabrina_burney) Burney of Akamai promised a talk on the [cross-overs in web security and web performance](http://conferences.oreilly.com/velocity/devops-web-performance-ny/public/schedule/detail/51203). The talk was fast-paced and covered a lot of ground including how features we use already (iframes, pre-load etc.) have additional security options that few people use. This was the most practical session of the conference for me, and I need to list all of these out to make sense of it:

### iFrames

Iframes are great for performance, kinda.
Over 60% of sites use iframes, yet virtually none of them use the [sandbox attribute](https://www.html5rocks.com/en/tutorials/security/sandboxed-iframes/). Sandbox allows you to define what access the iframe has to the parent page and whether it can execute scripts. It will also deny pointer lock, form submissions and a whole host of other scary stuff.

![Code sample showing an iframe with a bare sandbox attribute versus one with sandbox=\"allow-scripts\"](https://simonhearne.com/images/velocity-2016/iframe_sandbox.PNG)
*Using sandbox with iframes.*

### Prefetch / Preload `As`

Prefetch allows web developers to provide hints to browsers about assets which should be optimistically downloaded. This is great for objects which are critical to render, such as CSS and WebFonts. Prefetch can be used in <link> tags or in HTTP headers, the advantage of a header being that it can be used by the browser before the HTML document has been downloaded and parsed.

Preload builds on prefetch by forcing the browser to download the asset, whereas prefetch hints can be ignored.

Both of these have an additional optional attribute: `as`. This allows us to define the type of asset to be loaded, e.g. 'image', 'script' or any one of the [standard fetch types](https://fetch.spec.whatwg.org/#concept-request-destination). Adding the type of asset allows the browser to send the correct Accept header, as well as ensuring that any content security policy can be applied correctly to the preloaded asset.

![Code sample showing rel=\"prefetch\" and rel=\"preload\" link tags with an as attribute for scripts and styles](https://simonhearne.com/images/velocity-2016/preload_prefetch-9863b9.png)
*Using prefetch and preload with `as` attribute.*

### Content Security Policy

Speaking of content security policy... This is the header directive which tells a browser what permissions each domain used on a site has. For example, fonts.example.com should not be able to execute scritps, and static.example.com might only be for serving images. Using a [content security policy](https://developers.google.com/web/fundamentals/security/csp/) ensures that only the correct domains have access to the browser, which is obviously a security win. The performance benefit comes mainly from the audit and analysis required in order to write a CSP - what domains are being used on our site, for what, and why?!

```http
Content-Security-Policy:
    default-src 'self' ;
    img-src 'self' https://*.google.com ;
    script-src 'self' http://www.google-analytics.com 'unsafe-inline'
    http://*gstatic.com ;
    style-src 'self' https://fonts.googleapis.com ;
    font-src 'self' https://themes.googleusercontent.com ;
    frame-src 'self' http://3rdparty.com
    sandbox 'allow-scripts';
```

### Service Worker

Now we come to service worker, the magic JavaScript proxy thread. This is the concept most fundamental to 'offline-first': a developer-controlled proxy which can intercept network requests, build an internal cache and do all sorts of other magic.

For performance, service worker is critical for good performance in poor network conditions, if you haven't seen [Jake Archibald](https://twitter.com/jaffathecake) talk about service worker yet, check out his [video from Google IO](https://www.youtube.com/watch?v=cmGr0RszHc8). No, seriously, go watch that video.

Now on to security. As service worker has access to network requests, it can be used to enforce rules in a similar way to content security policy. You can even maintain your own black- and white-lists of blocked assets or domains, and let the service worker manage how these are enforced in the browser. Cool, eh?

I think the next logical step is for service workers to apply time thresholds to third-party downloads. That tracker tag taking over a second to load? Kill the request, give a safe response to the browser, and log the event to keep track of how often it happens. The possibilities are almost endless!

![Diagram of a service worker intercepting network requests and blocking one based on a list of thresholds](https://simonhearne.com/images/velocity-2016/sw_whitelist.png)
*Using service worker as a client reputation enforcer.*

### Sub-Resource Integrity

Sabrina and Sonia only briefly mentioned SRI but I think it's worth covering in a little more detail here. SRI allows you to take a hash of an asset (e.g. jQuery v1.9.1, minified) and add that to your ```<script>``` tag. If the downlaoded asset does not have the same hash, then the browser will refuse to execute it. This is really handy for third-party content that has the potential to effect user experience and/or security, or whose provenance is not entirely clear (JavaScript CDNs, anyone?)

```html
<script src="https://example.com/example-framework.js"
        integrity="sha384-oqVuAfXRKap7fdgcCY5uykM6+R9GqQ8K/uxy9rx7HNQlGYl1kPzQho1wx4JwY8wC"
        crossorigin="anonymous"></script>
```

## Progressive web apps aren't all that. Yet.

There was a lot of conversation around [progressive web applications](https://developers.google.com/web/progressive-web-apps/), but the lack of good browser support for some of the key features means that large organisations are holding off implementation: Web Push is only [available on Firefox and Chrome](http://caniuse.com/#feat=push-api), service worker is only [partially supported on Firefox, Chrome and Opera](http://caniuse.com/#feat=serviceworkers).

I'm also wary of websites going fully PWA - which may not provide the desktop with a good experience. Definitely a 'watch this space'.

In summary, an awesome conference with huge amounts to mull over.


==============================================================================

# Blocking requests in webpagetest? Don't use Chrome.

> We know that third-party content can harm the user experience on a website. Measuring that impact can be tricky.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2016-07-11
Updated: 2020-04-16
Canonical: https://simonhearne.com/2016/blocking-requests-in-webpagetest-dont-use-chrome/
Tags: WebPerf, webpagetest

---
<blockquote class="info"><p>Note: this issue has since been resolved by the Chrome team, please proceed with caution!</p></blockquote>

TL;DR: Blocking third-party requests in webpagetest shows you the impact they're having on a page, but Chrome is inefficient at blocking the requests. So use another browser such as Internet Explorer to do third-party impact testing.

In order to demonstrate the performance impact that third-party content can have, I like to remove the offending requests to show a before & after view of a web page. Nothing says 'your third-parties are harming user experience' quite like a video of a page loading twice as fast without them.

To do this, I run a test in webpagetest and get the list of domains as JSON. I parse this to filter out first- & second-party domains then re-run the test with all of the third-party domains blocked by webpagetest. Often, though, just blocking tag container(s) will do the job.

Let's take an example: [cyclingweekly.co.uk](http://www.cyclingweekly.co.uk/). Not for any reason other than I like bikes. When you load the homepage, approximately 150 objects are loaded from 60 domains:

![Request map showing cyclingweekly.co.uk connected to around 60 third-party domains for ads, tags and tracking](https://simonhearne.com/images/blocking-requests/cyclingweekly_requestmap.png)
*Requestmap of cyclingweekly.co.uk.*

Now to test the impact of this large number of third-parties lets block them and re-run the test. Here's the result:

![Filmstrip comparing page render with third-parties blocked (top) versus unblocked (bottom) in Chrome](https://simonhearne.com/images/blocking-requests/cyclingweekly_chrome_blocked.jpg)
*Filmstrip comparison without & with third-party content.*

Kind of what I expected, the third-parties mean the page takes longer to finish. But I had secretly hoped that they were also affecting render performance - that the blocked version would render quicker.

To dig in a bit deeper I looked at the waterfall chart of the blocked version, and I noticed two things: the requests seemed to be sequential rather than parallel, and the CPU was pegged at 100% for most of the time. Curious.

![WebPageTest waterfall showing requests loading one after another instead of in parallel, with one taking 1448ms](https://simonhearne.com/images/blocking-requests/cyclingweekly_waterfall.png)
*Waterfall chart showing sequential requests when third-parties are blocked*

Whenever the CPU is pegged in a Chrome webpagetest result, here's a pro-tip: enable timeline capture.

![WebPageTest advanced settings with the Chrome 'Capture Dev Tools Timeline' checkbox highlighted](https://simonhearne.com/images/blocking-requests/cyclingweekly_devtoolscapture.png)
*Enabling timeline capture in webpagetest*

This gives you a json file that you can drag into your local Chrome developer tools window:

![WebPageTest results page with the 'Timeline (view)' link highlighted next to the waterfall and screenshot](https://simonhearne.com/images/blocking-requests/cyclingweekly_timelinelink.png)
*Download timeline json.*

Loading this up in Chrome Canary lets you analyse the network waterfall in the timeline view, where I saw significant gaps in the processing timeline, and a number of grey requests:

![Chrome DevTools timeline highlighting gaps in processing where blocked requests still appear in the network list](https://simonhearne.com/images/blocking-requests/cyclingweekly_timeline_gaps.png)
*Blocked requests appearing and gaps in timeline.*

The grey request was one which should have been blocked by webpagetest. You can't drill into any more details from this view, although it looks like the blocked requests still consume time yet don't have a response.

In [issue 584](https://github.com/WPO-Foundation/webpagetest/issues/584) on the webpagetest GitHub repo Pat Meenan states:
> blocking (or modifying request headers) in Chrome is exceedingly expensive

So, trying again in Firefox and Internet Explorer we get the following results:

![Side-by-side waterfalls for Chrome, Firefox and Internet Explorer, each with third-party domains blocked](https://simonhearne.com/images/blocking-requests/cyclingweekly_allwaterfalls.png)
*Waterfall charts for three browsers while blocking third-party domains.*

And for visual performance, IE gives the best result:

![Filmstrips for Firefox, IE and Chrome with third-parties blocked: IE reaches 100% render by 2.0s, Chrome not until 3.0s](https://simonhearne.com/images/blocking-requests/cyclingweekly_allfilmstrips.png)
*Waterfall charts for three browsers while blocking third-party domain.*

So the final result, to show the impact of third-party content on the user experience comes out below:

![Filmstrip comparing cyclingweekly.co.uk rendering much sooner with third-party assets blocked (top) than unblocked (bottom)](https://simonhearne.com/images/blocking-requests/cyclingweekly_ieblocked2.png)
*Filmstrip image showing with cyclingweekly.co.uk loading without & with third-party assets.*


==============================================================================

# How fast is fast enough?

> Webinar with Andy Davies on measuring site speed and using RUM data to work out when performance starts to change customer behaviour.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2016-06-29
Canonical: https://simonhearne.com/2016/how-fast-is-fast-enough/
Tags: WebPerf, talk
Slides: https://speakerdeck.com/simonhearne/webinar-how-fast-is-fast-enough

---
Do you know when your website's speed starts to impact your customers' behaviour?

The speed of your site affects how your visitors behave: how many pages they view, how much they spend or even whether they convert.

Join me and one of the world's leading web performance experts, Andy Davies, as we discuss what you should consider when measuring your site's speed and how to determine when speed is a problem.

You'll leave this webinar with a good understanding of why speed matters, what good performance looks like and how it impacts your business.

We're really lucky to have some concrete RUM data and experience to work out what's important. The webinar has been and gone, but you can see the slides below.

<br/>

<script async class="speakerdeck-embed" data-id="ebcabd2299de462dbd6dbc47c6b8c7a3" data-ratio="1.77777777777778" src="//speakerdeck.com/assets/embed.js"></script>


==============================================================================

# Getting Reliable Visual Performance Metrics

> Tips to get more reliable & consistent visual metrics from WebPageTest

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2016-06-08
Canonical: https://simonhearne.com/2016/getting-reliable-visual-metrics/
Tags: WebPerf, webpagetest

---
Visual metrics such as Speed Index, render start and visual complete are the best proxies we have for user experience. Unfortunately two of these are totally thrown by rotating carousels, popups, cookie banners and adverts.

To give an example, below is the homepage of Transport for London loading at Cable speed from London. It's clear that the main content of the page is visible by 6.5 seconds. The metrics we get from WebPageTest don't tally though, with visual complete coming in at almost 12 seconds.

![Filmstrip section showing TfL appear visually complete at 6.5 seconds.](https://simonhearne.com/images/tfl_6-5.JPG)

The reason that the WebPageTest metrics disagree with what we can see is that a few seconds after the 'first' visual complete, an advert appears which pushes content down the page. Notice that in the image above the visual completeness is reported at 90%. The image below shows what happens between 9 and 12 seconds.

![Filmstrip section showing TfL visually complete at 12 seconds.](https://simonhearne.com/images/tfl_9-12.JPG)

This effect can be caused by a number of issues:

- Cookie banners which appear after the page loads
- Adverts which load late
- Carousels which auto-advance
- Auto-play videos
- Subscription popups

Unfortunately this is normally outside of my control as a consultant. It is an especially pertinent issue when I'm comparing multiple websites or pages, as some can appear artificially worse than others.

While I believe that anything which creates this effect is bad for user experience (e.g. [rotating carousels](http://shouldiuseacarousel.com/)), it's not my place to make recommendations on their use. So, we need a way to get reliable visual metrics given that a page under test can have one or more of these features.

When testing a page, WebPageTest has a default activity threshold of 2 seconds. That is: after all page activity stops, the test will continue for a further 2 seconds. If there is network activity within this time the timeout will reset. If the screen changes within this time, the final frame (after the timeout) will be used as the reference point for visually complete and Speed Index.

Thus, a potential way to improve visual metrics is to reduce this timeout. To do so in WebPageTest you can use a script, e.g.:

```text
setActivityTimeout    100
navigate    http://tfl.gov.uk
```

The result is a page which finishes before the advert pushes content down, as shown below.

![Filmstrips of TfL tested with and without a 100ms activity timeout.](https://simonhearne.com/images/timeout.jpg)

![Visual progression over time.](https://simonhearne.com/images/visual_progression.PNG)

So if you want more reliable visual performance metrics, reduce the activity timeout threshold to something reasonable - like 100ms.

Unfortunately this can only be done through scripted sessions, or by changing the [default in WebPageTest private](https://github.com/WPO-Foundation/webpagetest/blob/master/agent/browser/ie/pagetest/PagetestBase.h) as shown below.

```js
/* PagetestBase.h */
#define ACTIVITY_TIMEOUT 2000
#define REQUEST_ACTIVITY_TIMEOUT 2000
#define FORCE_ACTIVITY_TIMEOUT 2000
#define DOC_TIMEOUT 2000
```


==============================================================================

# Render Conf 2016

> Render Conference - previously known as jQuery UK - was held in Oxford last week. It's a front-end conference for front-end people, which makes me a bit of an outsider. Performance is not the focus of Render Conference. Render is an opportunity for me to hone up my front-end skills and get an idea of what front-end developers are up against and the latest technologies in play.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2016-01-21
Canonical: https://simonhearne.com/2016/render-conf-1/
Tags: WebPerf, conference

---
### [Bruce Lawson](https://twitter.com/@brucel) - web.next

#### [Description](http://2016.render-conf.com/talks.php#webnext) | [Slides](https://speakerdeck.com/brucel/web-dot-next-render) | [Video](https://vimeo.com/album/3953264/video/165995132)

I had assumed that Bruce's talk would be identical to the [last few I've seen](/2015/velocity-europe-2015-report-1/#ensuring-a-high-performing-web-for-the-next-billion-people---bruce-lawson-opera-asa). Bruce's previous talks on web.next have focused on where customers are coming from and what devices they are using. This talk was much more focused around the web technologies that will *allow* the next 4 billion people to use the web.

Progressive Web Apps (PWAs) [will replace native mobile apps](http://loxima.com/blog/the-death-of-apps/) and this is a good thing: a relatively light-weight 20MB app might take over 30 minutes to install on 2G - by which time the network will probably have flaked out, Bruce notes.

PWAs increase accessibility to mobile users with limited devices and limited connectivity.

In telling his story, Bruce highlighted how browser manufacturers have had to collaborate to create better web technologies - and thus better user experiences. Mozilla, Google, Opera and Microsoft are trying to improve their relationship with web developers to produce scalable and useful functionality. AppCache is the counter-example Bruce uses to demonstrate what happens when this relationship is not built correctly.

Another great talk by Bruce, inspiring a room of front-end engineers to get involved with the Web Platform Incubator Group ([https://www.w3.org/blog/2015/07/wicg/](WICG)).

----------

### [Val Head](https://twitter.com/@vlh) - Designing Meaningful Animation

#### [Description](http://2016.render-conf.com/talks.php#designing-meaningful-animation) | [Video](https://vimeo.com/album/3953264/video/165995133)

Val believes that animation is often mis-used and mis-understood on the web. Throughout the talk Val uses a mocked-up registration confirmation form to demonstrate how animation can be used to reinforce a brand message.

Val references Disney's [12 basic principles of animation](https://en.wikipedia.org/wiki/12_basic_principles_of_animation) and highlights the key factors to focus on when animating on the web:

 * Staging
 * Follow Through
 * Timing
 * Secondary Action

Using a live example and the Chrome Developer Tools [animation controls](http://valhead.com/2015/01/06/quick-tip-chrome-animation-controls/) Val shows that animations can convey meaning and emotion when these four factors are considered properly.

----------

### [Alicia Sedlock](https://twitter.com/@aliciability) - The landscape of front-end testing

#### [Description](http://2016.render-conf.com/talks.php#the-landscape-of-front-end-testing) | [Slides](https://speakerdeck.com/aliciasedlock/the-landscape-of-front-end-testing) | [Video](https://vimeo.com/album/3953264/video/165995134)

Alicia knows that testing is hard, and that front-end developers aren't traditionally great at testing. By introducing testing framework language Alicia sets the scene for unit testing, introducing core concepts for front-end developers. The focus is on automation, making testing as simple as possible to drive adoption and engagement.

The testing concept then gets extended to visual regression testing, where Alicia lists the following automation tools to make this simple:

 * [PhantomCSS](https://github.com/Huddle/PhantomCSS)
 * [BackstopJS](https://github.com/garris/BackstopJS)
 * [Wraith](https://github.com/BBC-News/wraith)
 * [WebDriver.io](http://webdriver.io/)
 * [Percy.io](https://percy.io/)

Of course no talk on testing would be complete without a nod to accessibility. Alicia lists two tools to help with automating accessibility testing:

 * [a11y](https://github.com/addyosmani/a11y) / [grunt-a11y](https://github.com/lucalanca/grunt-a11y)
 * [pa11y](http://pa11y.org/)

And the final chapter in automated testing is performance (yay!). Alicia mentions a number of tools for automated testing:

 * [grunt-perfbudget](https://github.com/tkadlec/grunt-perfbudget)
 * [gulp-size](https://github.com/sindresorhus/gulp-size)
 * [perf.js](https://gist.github.com/tkadlec/7e352b74b1961a3e36d7)

I would add to the list a notable exception:

 * [SiteSpeed.io](https://www.sitespeed.io/)

----------

### [Harry Roberts](https://twitter.com/@csswizardry) - CSS for software engineers for CSS developers

#### [Description](http://2016.render-conf.com/talks.php#css-for-software-engineers-for-css-developers) | [Slides](https://speakerdeck.com/csswizardry/css-for-software-engineers-for-css-developers) | [Video](https://vimeo.com/album/3953264/video/166790749)

Harry starts his talk with a look back over his family history, highlighting that we have been use modern programming languages since 1959, the year his parents were born. CSS has only been around since 1996, but we have decades more software engineering experience which we can apply to CSS to make our projects more robust, scalable and predictable. I've noted down the core principles Harry advocates below:

 * Don't Repeat Yourself (DRY)
   * Your source should be DRY but the output can have repetition.
   * Use a preprocessor to store data in variables
   * Some repetition is better than a bad abstraction
 * Single Responsibility Principle
   * Use a class to denote a single responsibility, e.g. *class="button button-large button-positive"*
 * Separation of Concerns
   * Do not use CSS classes for JavaScript and vice versa
   * Do not bind CSS rules on to accessibility concerns, e.g. *role="navigation"*
   * Do not write CSS with JavaScript
 * Immutability
   * Do not allow any CSS rule to change, e.g. with a media query
   * Use responsive suffixes, as Harry defines in [BEMIT](http://csswizardry.com/2015/08/bemit-taking-the-bem-naming-convention-a-step-further/#responsive-suffixes)

I will never look at CSS the same. While a lot of this seems obvious when written down, how many times have you broken these simple rules?!

----------

### [Sara Soueidan](http://twitter.com/@SaraSoueidan) - SVG in motion

#### [Description](http://2016.render-conf.com/talks.php#svg-in-motion) | [Slides (PDF)](https://sarasoueidan.com/slides/SVG-In-Motion.pdf) | [Video](https://vimeo.com/album/3953264/video/166790778)

Sara is well-known as the SVG guru, in this talk she guides us through the perils and pitfalls of embedding SVG in our applications. It turns out that the way you embed an SVG image directly impacts what you can do with it. The table below summarises how you can animate SVG based on the way it is embedded, with the most flexible being an &lt;svg> element. This has the drawback of increasing your HTML size and reducing cache-ability, however.

| Embedding Technique | CSS Animations | JS Animations |
|:-:|:-:|:-:|
| &lt;img> | Inside &lt;svg> | N/A |
| url(); | Inside &lt;svg> | N/A |
| &lt;picture> | Inside &lt;svg> | N/A |
| &lt;iframe> | Inside &lt;svg> | Anywhere |
| &lt;object> | Inside &lt;svg> | Anywhere |
| &lt;svg> | Anywhere | Anywhere |

Sara then details the various methods of animating SVG, with the following table summarising her recommendations:

| Animation | Technique |
|---|--:|
| Transforms | JavaScript (or CSS) |
| Path Morphing | JavaScript |
| Line Drawing | JavaScript (or CSS) |
| Colour & Simple Animations | CSS (or JavaScript) |

The rest of Sara's talk goes in to detail on specific animations, path drawing and some cross-browser issues with transform-origin. Not to mention the fact that CSS transforms do not work in either Internet Explorer nor Microsoft Edge.

The part of Sara's talk I found most interesting was around manipulating the SVG viewbox property. By changing the value of the viewBox, you change the area of the canvas that is visible inside the SVG viewport. This enables you to zoom in to specific areas or objects. This can also enable you to manage SVG sprites and even step-animation in SVG.

----------

### [Robin Christopherson](http://twitter.com/@USA2DAY) - Technology – the power and the promise

#### [Description](http://2016.render-conf.com/talks.php#technology-the-power-and-the-promise)

Robin used a selection of videos and personal stories to remind us of how transformative the web is for people with disabilities. Even with the increased focus that developers put on accessibility, many web sites are totally inaccessible to blind people like Robin. New technologies such as automated intelligence, voice recognition and driver-less cars are all going to help people with disabilities, if only the designers and developers bear that in mind.

----------

### [Frederik Vanhoutte](http://twitter.com/@wblut) - Rendering the Obvious

#### [Description](http://2016.render-conf.com/talks.php#rendering-the-obvious)

Frederik started his talk with a reference to web performance - render the obvious. He mentions that we should show customers what they've got as soon as they've got it. From there the talk rapidly morphed into a psychedelic exploration of teaching and knowledge, using 'how a rainbow works' as a metaphor of a knowledge abstraction.

Children don't need to know exactly how a rainbow works, so a simple demonstration of a prism refracting light and creating the colours of the rainbow suffices. This is an abstraction away from how rainbows actually work, involving reflection and refraction in spheres of water. You can't hope to know everything, so accepting abstractions of knowledge in some areas is fine, but you must understand that everything you know is an abstraction...


==============================================================================

# Prioritising Site Speed Recommendations

> Sometimes prioritising recommendations is more important than making them.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2016-01-18
Canonical: https://simonhearne.com/2016/prioritising-sitespeed-recommendations/
Tags: WebPerf

---
One of my favourite tasks as a consultant is to provide reports to clients which make recommendations for improving site speed. Often these reports are taken very seriously, with senior managers using them to help drive the development roadmap for the next few months.

The reason I enjoy these reports so much is that you can often see a direct positive change off the back of them. Like most IT professionals I miss the intrinsic pleasure of working hard to create something with my hands, to have the [sense of completion](http://changingminds.org/explanations/needs/completion.htm) at the end of a project.

Seeing changes based on my recommendations, speed indexes improving and conversion rate increasing, is one of the few 'tactile' pieces of feedback I have on my work.

> The vast majority of the improvements are best practises and relatively easy to observe

Perhaps the most surprising part of delivering these reports is that quite a few of the recommendations I make are already in the client's backlog. Luckily there is invariably at least one 'aha' moment when something truly unknown is discovered, but the vast majority of the improvements are best practises and relatively easy to observe.

The last few of these types of engagements have led me to a realisation; for most of my clients these reports are used as a formal document to *prioritise* site speed improvement work, not just to discover it.

> There's little value in telling a client about the best practices they've already implemented.

My document structure has evolved due to this. There's little value in telling a client about the best practices they've already implemented, nor is there any point in documenting the fact that they could shave another 10kB off their image weight.

So in my reports I now suggest that site speed optimisations can be ranked by two critical factors:

 * what impact will this change have on the users
 * how hard will the change be to implement

If you can put a rough estimate for these two factors against each recommendation then you can rapidly determine the highest priorities for development. The [MoSCoW](https://en.m.wikipedia.org/wiki/MoSCoW_method) Method gives a good methodology around describing priorities and I often reduce this to a three star system: ★ is could do up to ★★★ for must do. For the ease of implementation ranking: ★ means difficult and ★★★ is simple to change.

I've borrowed from a recent report to demonstrate this theory in practise, below is a small section taken from the results table:

|Recommendation|Impact on Users|Ease of Implementation|
|-------------:|:-------------:|:--------------------:|
|Enable gzip on CSS|★★★|★★★|
|Reduce upstream cookie size|☆★★|☆★★|
|Replace icon fonts with SVG|☆★★|☆☆★|
|Review third-party content|☆☆★|☆☆★|
|Remove redundant JavaScript|☆★★|☆☆★|
|Remove redirects|☆☆★|☆★★|

The sum of the stars give a good idea of priority. Take the uncompressed CSS issue for example: enabling gzip compression should be trivial on any modern set-up, and as CSS is critical to render a page it should be delivered as soon as possible.

> I place the highest priority issues in the executive summary, with the full table in the conclusion.

This gives enough up front to give an idea of the issues that have been found, with the addition of the full list for internal prioritisation.
The rest of the report describes the methodology for making the recommendations, as well as how each MoSCoW recommendation is made.

For the structure of the recommendations, I usually break it down into four of these main focus areas:

 * Reduce Requests
 * Reduce Transmitted Bytes
 * Improve Render Performance
 * Review Third-Party Content
 * Improve Back-end Performance

In each one I detail what has been discovered, why it is important and how it can be fixed. Perhaps there's another post in that.


==============================================================================

# Velocity Europe 2015 - The Good Ones!

> This post is a review and summary of the talks at Velocity Europe that I attended. I've summarised each talk and given links to the slides, videos and ratings.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2015-11-01
Canonical: https://simonhearne.com/2015/velocity-europe-2015-report-2/
Tags: WebPerf, conference

---
> [Another post](/2015/velocity-europe-2015-report-1/) lists the talks that got five stars from me!

### Continuous Performance - *Stijn Polfliet (CoScale)*

**[Description](http://velocityconf.com/devops-web-performance-eu-2015/public/schedule/detail/45495) | [Video (5m)](https://www.youtube.com/watch?v=D4tVpzj6BvE) | [Slides (ppt)](http://cdn.oreillystatic.com/en/assets/1/event/134/Continuous%20performance%20Presentation.pptx) | My Rating: ★★★★☆ | Average Rating: ★★★★☆**

Stijn's keynote piece was only short (as all sponsored keynotes should be) but he used the time well to highlight CoScale's killer feature.

It appears that CoScale may have solved the biggest problem we have in Web Performance - integrating site speed and business performance data. Everyone in our industry knows that site speed is key to business performance, but we often struggle to spread that message beyond the IT department. By integrating business metrics and performance data we can make better technical decisions and tell a better story about the impact site speed has on our business.

It's worth stating here that [Google Analytics](https://www.google.co.uk/analytics/), [Blue Triangle](http://www.bluetriangletech.com/) and [SOASTA's mPulse](http://www.soasta.com/performance-monitoring/) also collect and report on this data. It remains to be seen who will win over this currently narrow segment of the market.

Stijn lost a star from me as the talk was not as engaging as it could have been and felt like a sales pitch.

----------

### Stranger danger - *Guy Podjarny & Assaf Hefetz (Snyk)*

**[Description](http://velocityconf.com/devops-web-performance-eu-2015/public/schedule/detail/46147) | [Video (14m)](http://www.youtube.com/watch?v=iXA14OFXxZA) | My Rating: ★★★★☆ | Average Rating: ★★★★☆**

Guy and Assaf got 15 minutes to demonstrate [Snyk.io](https://snyk.io/) in their (sponsored?) keynote.

Snyk.io (So Now You Know) solves a problem that I didn't know we had - but that I now realise is a huge issue. This problem is that we all use third-party code when we develop applications, and through both direct and indirect dependencies we are continually introducing a huge threat landscape to our applications. Who knows how often vulnerabilities are introduced into open source code, which then get pushed into other code through dependency updates? Snyk.io do.

> The typical Node.js app has over 200 dependencies and 59% of reported vulnerabilities in Maven packages remain unfixed. The mean-time-to-repair (MTTR) for a Maven package vulnerability is *390 days*. This presents a serious threat to projects using these dependencies and at present it is very hard to get any visibility of that threat.

Snyk.io is run from the command line (which calls their remote API service). It can analyse all dependencies in a project and automatically identify *known* vulnerabilities. The killer feature, though, is the ability to automatically patch the vulnerabilities and update your configuration to maintain the patched version in future updates. Very cool.

Snyk.io currently only works with Node.js projects but they will be expanding to other languages shortly. This is definitely a security service to watch.

Guy and Assam lost a star from me as the demo could have been done in less time and the talk felt a little like a hard sell on Snyk.io, even though it sells itself!

----------

### A PaaS for government - *Anna Shipman (Government Digital Service)*

**[Description](http://velocityconf.com/devops-web-performance-eu-2015/public/schedule/detail/46814) | [Blog Post](https://gdstechnology.blog.gov.uk/2015/10/27/looking-at-open-source-paas-technologies/) | [Video (15m)](https://www.youtube.com/watch?v=OLOaq-Xf5zU) | My Rating: ★★★☆☆ | Average Rating: ★★★★☆**

Anna took us through the thought process of the Government Digital Service as they introduce a Platform as a Service (PaaS) for government websites.

Anna did a great job in introducing what GDS do and how important it is that people can quickly access their services. A few examples were given such as the ability to book a prison visitation and voter registration.

The government has a problem, though. For each new digital service that goes live (over 800 of them are on the cards!) the project team procures the infrastructure, development environment, database, logging, monitoring etc. This makes for a lot of repetition and a slow time-to-market for new services.

Anna's team are hoping to make the release of a new digital service quick and simple by introducing a Government-wide PaaS. She takes us through the thought process of starting a PaaS at government scale - the challenges that are faced when choosing the correct platform as well as service provider. We were shown a number of candidates and a run-down of pro's and con's to each of them and were promised updates on the [GDS Blog](https://gdstechnology.blog.gov.uk/) as the PaaS is rolled out. I'll certainly be following that closely.

Anna lost a couple of stars from me as the talk was dry and could have focussed more on the human element - who would be helped by this and how much would it affect them? What are the barriers to adoption within the Government?

----------

### Probabilistically sampling a stream of events with a sketch - *Baron Schwartz (VividCortex)*

**[Description](http://velocityconf.com/devops-web-performance-eu-2015/public/schedule/detail/43888) | [Slides (pdf)](http://cdn.oreillystatic.com/en/assets/1/event/134/Probabilistically%20sampling%20a%20stream%20of%20events%20with%20a%20sketch%20Presentation%201.pdf) | My Rating: ★★★☆☆ | Average Rating: ★★★★☆**

In this talk Baron took us on a technical tour of the event sampling algorithm behind [VividCortex](https://www.vividcortex.com/). VividCortex is a cloud based intelligence platform for database system performance and health, it works with all of the big open source database players.

The challenge that Baron described is that the agents installed on customer hardware generate huge volumes of data; sending all of this data back to VividCortex would be infeasible due to both network constraints and the potential resource impact on the customers' host servers.

To overcome this, VividCortex takes a sampling approach to events. While time-series data is sampled once a second, full event traces are sampled on a less frequent basis (about one per hour per category). The challenge here comes in choosing which events to sample. Baron was open about their choices failing a few times and described a number of approaches that were attempted before detailing their chosen methodology.

I felt that the talk became too technical near the end in detailing exactly how the statistical sketch was created, with not enough high-level description of how it can be applied and, more importantly for the audience, where else this kind of sampling could be used. Less technicality in the talk but with a reference to a technical blog post to back it up may have worked better.

Baron lost a couple of stars from me for this reason, as well as the strong feeling of being sold to. It is unfortunate as the general concept is very interesting to me and I would have loved for this to be a five-star talk.

----------

### Fake it until you make it - *Jean-Pierre Vincent (BrainCracking)*

**[Description](http://velocityconf.com/devops-web-performance-eu-2015/public/schedule/detail/44543) | [Slides (ppt)](https://docs.google.com/uc?id=0ByrWN83fytRxQzc3b01wQmJOaEE&export=download) | My Rating: ★★★★☆ | Average Rating: ★★★★☆**

In this talk JP uses stories from his clients to show how interface design can overcome some of our inherent performance challenges.

JP's first example was of work he did on a container shipping company. Their challenge was that the web application to book and manage multi-million dollar shipments took sixty seconds to load for Chinese customers. Understandably these customers raised complaints about performance. After doing some work to halve the page load time complaints were still coming in. After all, it was still a thirty second page load!

As a developer with an eye for design, JP found a design solution - he added a spinner to indicate loading on the application page. Complaints dropped a little. He then found another design solution - a five second animation of a seagull flying across the screen. This significantly reduced complaints.

A number of other examples were given of how we, as web developers and designers, should consider performance by design. This is something that [Lara Hogan](http://larahogan.me/design/) has been preaching for quite some time. A new idea to me was using the well-known research showing that we must give feedback within 200ms and show progress within 1000ms. JP showed us how mobile applications in particular are following these rules well. To keep the user engaged they use transition animations between actions, hiding the delay in downloading or preparing content while giving the user a feeling of continuity.

I think JP's conclusions could have been stronger - for example giving your user feedback within 200ms and designing transition pieces to keep the user involved. Overall, though, the talk was very strong and introduced or reinforced a large number of key performance concepts for me.

----------

### Bigger, faster, and more engaging while on a budget - *Nathan Bower (Zillow)*

**[Description](http://velocityconf.com/devops-web-performance-eu-2015/public/schedule/detail/46156) | My Rating: ★★★★☆ | Average Rating: ★★★☆☆**

Nathan introduced [Zillow](http://www.zillow.com/) as one of the biggest property websites in the United States. Zillow have faced a number of challenges around web performance, Nathan detailed one challenge in particular as a case study - when the mobile homepage size increased by 12MB.

By using [SpeedCurve](https://speedcurve.com/) and performance budgets Nathan was alerted to this immediately, he even managed to get web developers on the case while he was commuting to work.

It turned out that an A/B testing feature that was not being used was pulling down a large amount of content which was never even rendered.

Nathan's conclusions were that performance budgets are critical, but they must be on the right metrics. His main suggestion was to use the [User Timing API](http://www.stevesouders.com/blog/2014/08/21/resource-timing-practical-tips/) to set custom metrics and track those as well as [SpeedIndex](https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/metrics/speed-index) and the more traditional metrics. He also concluded that you can download a whole load of useless data on mobile devices and customers don't care - I'm not sure that I would repeat that one too much!

----------

### Finding bad apples early: Minimizing performance impact - *Arun Kejariwal (Machine Zone)*

**[Description](http://velocityconf.com/devops-web-performance-eu-2015/public/schedule/detail/43716) | My Rating: ★★★☆☆ | Average Rating: ★★★☆☆**

Arun used to work at Twitter where he helped to develop an open source [anomaly detection library](https://github.com/twitter/AnomalyDetection). In this talk he presented how he uses anomaly detection to find bad server nodes in large clusters at his new job with [Machine Zone](https://www.machinezone.com/) (they make the hugely popular mobile game [Game Of War](http://www.gameofwarapp.com/)).

I was unfortunately a little lost by the technicality of the presentation and would have loved more visuals to explain what Arun was talking about - he is obviously an extremely clever chap.

The general idea, I believe, was that he uses clustering on time-series data such as CPU and RAM utilisation across all nodes to identify node behaviour. Once these known-good patterns are learned, anomalies can be detected by comparing the timeseries data of each node against all node clusters. This is performed using the [Euclidean distance measure](https://en.wikipedia.org/wiki/Euclidean_distance) between the data matrices of each cluster-median against the individual node.

The outcome from this work is that the system can now automatically identify nodes which are behaving abnormally in the context of their peers. This allowed Arun to successfully identify a heterogenous server environment where two different types of CPU architecture had been mistakenly installed in nodes in the same cluster.

Arun lost a couple of stars from me as I found it hard to follow the presentation and there were too few conclusions presented. What else can this type of anomaly detection be used for? And will it ever be open source? (The tool discussed in this presentation is different to the open source library mentioned at the beginning).

----------

### Designing for Security Outcomes - *Eleanor Saitta (Dymaxion.org)*

**[Description](http://velocityconf.com/devops-web-performance-eu-2015/public/schedule/detail/46786) | [Video (18m)](https://www.youtube.com/watch?v=zqnQ0Mvi-as) | [Slides (pdf)](http://cdn.oreillystatic.com/en/assets/1/event/134/Designing%20for%20security%20outcomes%20Presentation.pdf) | My Rating: ★★★☆☆ | Average Rating: ★★★☆☆**

Eleanor introduced her keynote by telling us we've got security all wrong, that we focus on technology and not people. She then tells us how we can get better at designing our teams and applications with security in mind.

> Security design is the process of
understanding user culture, goals,
and workflows, organizational
technical capabilities, and adversary
capabilities and dispositions and
synthesizing a satisficing solution.

Eleanor suggests that we should all be building thorough threat models (rather than threat lists) at the very early stages of projects. This will allow us to capture potential security concerns early, as we will have a better understanding of the potential adversaries we face.

I'm afraid I got lost at a few points in Eleanor's keynote, the slides were occasionally distracting and some un-introduced security terms threw me off. That's why she lost a couple of stars from me.

----------

### A day in the life - An immersive data experience - *David Boloker (IBM Corporation)*

**[Description](http://velocityconf.com/devops-web-performance-eu-2015/public/schedule/detail/47746) | [Video (10m)](https://www.youtube.com/watch?v=6vMR6p-ff9U) | My Rating: ★★★☆☆ | Average Rating: ★★★★☆**

David had a hard job in his sponsored keynote. He started by introducing the limitations we currently have with our Human-Computer Interaction (HCI) and that we need a more natural way to communicate with machines.

To demonstrate this David quickly switched into a live demonstration of a piece of conversational software. The demo was unfortunately contrived and awkward; the speech recognition did not work well in the keynote setting, the feedback from the software was slow and machine-like and the quality of data was not much better than you would get from Siri.

----------

### Thinking about thinking (about people) - *Courtney W. Nash (O'Reilly)*

**[Description](http://velocityconf.com/devops-web-performance-eu-2015/public/schedule/detail/48025) | [Video (10m)](https://www.youtube.com/watch?v=tkJtr6xJp4g) |  My Rating: ★★★★☆ | Average Rating: ★★★★☆**

Courtney's background is in cognitive science and that really comes across in this keynote.

I think Courtney's general point is that humans are not inherently sensible or unbiased. We are weird creatures that do not follow the rules. An example was given of a 'shared space' created in a town which mixed pedestrians, cyclists and drivers with no traffic lights or stop signs. This forced people to be more considerate of those around them and in-turn increased safety.

This 'shared space' was eventually reverted, though. Even though empirical safety had increased significantly, the residents still *felt* that the junction was less safe and campaigned against it.

Courtney then related this to the web performance industry by citing an [article](https://www.uie.com/articles/download_time/) published in User Interface Engineering in 2001. The article states that if a user achieves what they set out to do on a website, they perceive it as 'fast'. This perception of speed, however, has no correlation to the actual speed of the site! Courtney's conclusion from this is that we need to measure and monitor better metrics, [time to first tweet](http://blog.alexmaccaw.com/time-to-first-tweet) was referenced.

Courtney then took a turn towards operations for a conclusion that everyone should share responsibility for the uptime of a site or application, where uptime is measured as the ability of a customer to complete their action.

----------

### Measuring the performance of single page web applications - *Philip Tellis & Nic Jansma (SOASTA)*

**[Description](http://velocityconf.com/devops-web-performance-eu-2015/public/schedule/detail/44019) |  My Rating: ★★★★☆ | Average Rating: ★★★★☆**

Philip and Nic work on SOASTA's Real User Measurement (RUM) product mPulse. In this talk they gave a run-down on the work they are doing with [boomerang](https://github.com/lognormal/boomerang/) to improve the data captured on single-page applications (SPAs).

SPAs give us a fundamental challenge when it comes to monitoring - the old paradigm of clicking a link causing a page navigation which can be tracked is no more. When SOASTA enabled SPA measurement for one client the amount of datapoints collected increased four-fold!

The work Philip and Nic are doing is around developing [plugins](https://github.com/lognormal/boomerang/tree/soasta-2015/plugins) for boomerang which instrument the network activity created by SPAs.

The conclusion is that browsers need to catch up with vendors - it should not be so difficult to monitor and measure real users.

----------

### Open components as microservices in the front-end world - *Matteo Figus (OpenTable)*

**[Description](http://velocityconf.com/devops-web-performance-eu-2015/public/schedule/detail/44289) |  My Rating: ★★★★☆ | Average Rating: ★★★☆☆**

In this talk Matteo introduced his open source project [open components](https://github.com/opentable/oc). Matteo describes how difficult it is to develop a website in the complex environment at OpenTable, with many engineers based out of multiple countries all working on the same site.

To solve the challenges faced with front-end development at OpenTable, they decided to break the development down to micro-sites. By utilising common components and removing interdependencies the developers can now rapidly develop these micro-sites and push to live indepently of each other.

Open components provides a registry and commandline service to create front-end components.

----------

### A real-life account of moving 100% to a public cloud - *Julien Simon (AWS) & Antoine Guy (Viadeo)*

**[Description](http://velocityconf.com/devops-web-performance-eu-2015/public/schedule/detail/44042) |  My Rating: ★★★☆☆ | Average Rating: ★★★★☆**

Julien and Antoine were part of the team that migrated [Viadeo](http://gb.viadeo.com/en/) to a public cloud stack. In this talk they describe the technical process they went through to get the site successfully migrated.

The main outcome of the migration was the move to [CloudFormation](https://aws.amazon.com/cloudformation/) to automate Infrastructure as Code.


==============================================================================

# Velocity Europe 2015 - My Allstars

> This post is a review and summary of the talks at Velocity Europe that got a 5 star review from me. I've summarised each talk and given links to the slides, videos and ratings.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2015-11-01
Canonical: https://simonhearne.com/2015/velocity-europe-2015-report-1/
Tags: WebPerf, conference

---
> [Another post](/2015/velocity-europe-2015-report-2/) lists the talks that didn't quite get five stars from me!

### The Physical Web is a Speed Issue - *Scott Jenson (Google)*

**[Description](http://velocityconf.com/devops-web-performance-eu-2015/public/schedule/detail/45231) | [Video (18m)](https://www.youtube.com/watch?v=7H_E_ZbFAn0) | My Rating: ★★★★★ | Average Rating: ★★★★☆**

Scott's keynote was probably the highlight of this Velocity for me. He discusses how the web will change over the next few years with the advent of the Physical Web and the Internet Of Things.

Scott used a number of examples to show how this will affect our daily lives, with a parking meter being the clearest demonstration.

Imagine a parking meter which has a light-weight web server and low-energy bluetooth. The bluetooth (or other low-energy, low-range wireless technology) continuously publishes a unique URL for the web server. You park your car and pull out your phone, in your notification tray this URL has silently appeared. You click on the link and immediately load a payment page for the parking meter, one tap and you've got an hour's worth of parking and the meter screen immediately updates.

This sounds pretty futuristic, but almost all of the technology we need is already in place. The one part that's missing is the silent notifications - broadcasting a link that nearby devices can pick up but without irritating every passer-by (like iBeacon). Scott's team at Google is working on this very challenge, there is an open source repository with a functioning Chrome extensios on iPhone (but not Android!) on [GitHub](https://github.com/google/physical-web). This technology is based on Google's open Bluetooth low-energy protocol [Eddystone](https://github.com/google/eddystone).

The title of Scott's talk took me a while to decipher: "The Physical Web is a Speed Issue". By this I think he means that the Physical Web will make our day-to-day lives more 'speedy'. This will be achieved by automating link sharing, thus making tedious tasks such as paying for parking or interacting with every-day items much easier and faster. In order for this to work, though, we need *fast mobile web applications*.

We cannot and will not install an application for every light bulb, parking meter or door lock which makes up the Physical Web. Technologies such as [service worker](https://github.com/slightlyoff/ServiceWorker/) and [quic](https://www.chromium.org/quic) are going to be essential to make the Physical Web fast and, more important, accessible.

> The Physical Web is like a really awesome QR Code

This talk really hit home for me. As someone who runs two QR Code services ([QR Explore](https://qrexplore.com) and [ThisQR](http://thisqr.com)) I am more than familiar with the drawbacks of QR Codes in western society - no one wants to be seen taking a picture of a poster! This means that QR Codes work best in a closed environment such as a factory (which is why they were invented) or a science center. The potential for having QR Code-like functionality (link discovery) without the awkward scanning issue is an absolute game-changer.

----------

### Discovering Operations Expertise - *John Allspaw (Etsy)*

**[Description](http://velocityconf.com/devops-web-performance-eu-2015/public/schedule/detail/47779) | [Video (20m)](https://www.youtube.com/watch?v=OJ21jwJThq4) | [Thesis (pdf)](http://bit.ly/AllspawThesis) | My Rating: ★★★★★ | Average Rating: ★★★★☆**

John's keynote was a very brief summary of his work towards a thesis in Human Factors Engineering.

We often focus on analysing failures to find out what has gone wrong. John's point was that we need to focus on success as well in order to find out why something has not gone wrong. No matter how much we automate, humans will always be the lynchpin of success or failure in deployment; if we analyse what comes together to make successful human behaviour then we can improve success rate at the individual level.

John suggests that in order to increase efficiency, especially in times of crisis, we also need to find out what engineers do in the event of failure - what are the rules of thumb that are followed? Once we know that, we can analyse and improve our processes.

This talk was a refreshing change from the technical focus of the conference, instead proposing that as humans are the critical factors in all that we do, so we must focus there first.

----------

### Ensuring a high performing web for the next billion people - *Bruce Lawson (Opera ASA)*

**[Description](http://velocityconf.com/devops-web-performance-eu-2015/public/schedule/detail/44920) | [Video (20m)](https://www.youtube.com/watch?v=f6As5HEkG5E) | My Rating: ★★★★★ | Average Rating: ★★★★★**

The highest rated talk of Velocity. I saw Bruce give this talk at Velocity Santa Clara earlier this year and it has got even better.

Few of us consider proxy browsers when we develop websites, but Opera Mini alone serves over 350 Million customers. By developing Javascript-rich websites or by using icon fonts we may be excluding a huge portion of our future market.

----------

### One year later: How Marks & Spencer revolutionized PerfOps after Velocity 2014 - *Andrew Neilson (M&S)*

**[Description](http://velocityconf.com/devops-web-performance-eu-2015/public/schedule/detail/44100) |  My Rating: ★★★★★ | Average Rating: ★★★★☆**

Andy runs PerfOps and marksandspencer.com - a large UK retailer that does 20% of its business online. In this talk he described how Velocity Europe 2014 inspired him to refactor performance operations at M&S.

Andy gave a practical run-down of the tools and metrics that M&S use to track performance with a number of examples to illustrate his points.

A key turning point for the business was when SpeedCurve showed a huge increase in third-party calls on the M&S homepage. This led Andy to investigate what had happened using my [requestmap](http://requestmap.webperf.tools/) and discovered that a third-party service had started calling more third-party domains. This discovery led to a piece of analysis showing that third-parties were responsible for 50% of the load time on the homepage.


==============================================================================

# Measuring Webpage Jank

> Webpage jank can harm the user experience, here's an easy way to measure it on your pages

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2015-09-11
Updated: 2026-03-02
Canonical: https://simonhearne.com/2015/jank-meter/
Tags: WebPerf, jank

---
Here's the bookmarklet:
<a style="cursor:alias;" class="btn btn-block" href='javascript:(function(e)%7Bfunction t()%7Bfor(var e=document.getElementsByClassName("fpser-lay"),t=e.length-1,n=t;n>=0;n--)try%7Bdocument.body.removeChild(e%5Bn%5D)%7Dcatch(r)%7Bcontinue%7Dvar r=document.getElementById("fpser");r&&r.parentNode.removeChild(r)%7Dfunction n()%7Bif(t(),e.scrollTo(0,0),count=0,sh=Math.max(document.documentElement.clientHeight,e.innerHeight%7C%7C0),h=Math.max(document.body.offsetHeight,document.body.scrollHeight)-sh,chunk=100,h-100<0)throw alert("I need to scroll, please reduce your browser height or choose a longer page!"),"Too short";fA=%5B%5D,e.requestAnimationFrame(r),ll=!1%7Dfunction r()%7Breturn ll?(e.scrollBy(0,chunk),tl=performance.now(),fA.push(parseInt(1e3/(tl-ll))),ll=tl,count+=chunk,void(count>=h?(e.scrollTo(0,0),l(fA)):e.requestAnimationFrame(r))):(ll=performance.now(),void e.requestAnimationFrame(r))%7Dfunction o(e)%7Bvar t=350,n=60,r=JSON.stringify(%7Btype:"bar",data:%7Blabels:e.map(function(_,i)%7Breturn i%7D),datasets:%5B%7Bdata:e,backgroundColor:e.map(function(v)%7Breturn v<50?"rgba(255,55,0,0.7)":v<59?"rgba(255,255,0,0.7)":"rgba(55,255,55,0.7)"%7D),borderWidth:0%7D%5D%7D,options:%7Blegend:%7Bdisplay:false%7D,scales:%7BxAxes:%5B%7Bdisplay:false%7D%5D,yAxes:%5B%7Bticks:%7BbeginAtZero:true%7D%7D%5D%7D,plugins:%7Bdatalabels:%7Bdisplay:false%7D%7D%7D%7D);return"https://quickchart.io/chart?w="+t+"&h="+n+"&bkg=transparent&c="+encodeURIComponent(r)%7Dfunction l(e)%7Bvar t=e.slice(0);e.sort(function(e,t)%7Breturn e-t%7D);var n=Math.floor(e.length/2);if(e.length%252)var r=e%5Bn%5D;else var r=(e%5Bn-1%5D+e%5Bn%5D)/2;for(var a=e%5B0%5D,c=e%5Be.length-1%5D,i=0,s=0;s<e.length;s++)i+=e%5Bs%5D;var p=parseInt(i/e.length),d=o(t),u=document.createElement("div");u.id="fpser",u.style.cssText+=";font-family:sans-serif;font-weight:bold;width:400px;background:rgba(200,200,200,0.9);position:fixed;top:10px;right:10px;z-index:10002;padding:5px;box-shadow: 0px 0px 5px 2px rgba(0,0,0,0.75);text-align:center";var m=document.createElement("p");m.innerHTML="FPS: median = "+r+", mean = "+p+", min = "+a+", max = "+c,p2=document.createElement("p"),50>=r?(p2.style.color="red",p2.innerHTML="This page is JANKY (Framerate is below 50 FPS)"):r>=59?(p2.style.color="green",p2.innerHTML="This page is SMOOTH (Framerate around 60 FPS)"):(p2.style.color="yellow",p2.innerHTML="This page is ALMOST JANKY (Framerate above 50 FPS)"),p3=document.createElement("p"),p3.innerHTML="<a href=%5C"https://simonhearne.com/2015/jank-meter%5C">What is this?</a> %7C <a href=%5C"%23%5C" onclick=%5C"window.FPS.cl();%5C">Hide this</a>.";var g=document.createElement("img");g.src=d,g.style="width:100%",u.appendChild(m),u.appendChild(g),u.appendChild(p2),u.appendChild(p3),document.body.appendChild(u);for(var f=0,g=sh;g<=h+sh;g+=100)%7Bvar v=document.createElement("div");v.className="fpser-lay";var b="";if(b=parseInt(t%5Bf%5D)<50?"255,55,0":parseInt(t%5Bf%5D)<59?"255,255,0":parseInt(t%5Bf%5D)>0?"55,255,55":"55,55,55",v.style.cssText+=";border-bottom:1px solid black;line-height:"+chunk+"px;text-align:center;font-weight:bold;z-index:10001;position:absolute;width:80px;height:"+chunk+"px;left:0;top:"+g+"px;background:rgba("+b+",0.5);",parseInt(t%5Bf%5D)>0)%7Bvar w=document.createElement("p");w.style.cssText+=";margin:0px",w.innerHTML=t%5Bf%5D+" FPS",v.appendChild(w)%7Ddocument.body.appendChild(v),f++%7Dvar v=document.createElement("div");v.className="fpser-lay",v.style.cssText+=";z-index:10001;position:absolute;width:80px;height:"+sh+"px;left:0;top:0;background:rgba(55,55,55,0.5);",document.body.appendChild(v)%7Dn(),e.FPS=%7B%7D,e.FPS.cl=t%7D)(window);'>Jank Meter</a>
 Drag it to your bookmarks bar or just add as a bookmark.

## What is jank?
Our eyes can't perceive visual changes at around 60 frames per second (FPS). As such, for smooth visual experiences most modern screens have a refresh rate of 60FPS. In order for our webpages to appear smooth to a user while interacting, all frames must render within 16ms (1000ms / 60 FPS).

## What causes jank?
Anything that is expensive to process and causes the browser to repaint can cause jank. Common causes include large background images that move on scroll (parallax effect) and Javascript functions bound to scroll events. Paul Lewis ([@aerotwist](https://twitter.com/aerotwist)) wrote a [good post](http://calendar.perfplanet.com/2013/the-runtime-performance-checklist/) on some of the causes of jank back in 2013.

Here's an example of the jank meter on a web design agency's homepage. They use a jQuery plugin to create a parallax effect on large (but barely visible) background images.

![Jank Meter showing poor scroll performance](https://simonhearne.com/images/janky_site2.png)
</figure>

Conversely, here's jankfree.org with a near-perfect scrolling experience:

![Jank Meter showing good scroll performance](https://simonhearne.com/images/jankfree_site2.png)
</figure>

## How do you detect jank?
You can detect jank using the Timeline tab in Chrome Developer Tools. Simply hit the record button and scroll, then stop the recording and look for slow frames:

![](https://simonhearne.com/images/devtools.png)
*Dev Tools showing Jank*

You can also enable the framerate overlay in Chrome to show the live render performance:

![](https://simonhearne.com/images/framerate.png)
*Dev Tools showing FPS meter*

This can all be a bit time consuming though, recently I've found myself spending a lot of time in the timeline view just to detect jank.
The framerate meter also makes it difficult to identify exactly where in the page the jank is coming from.

I figured that there must be an easier way to do this. [requestAnimationFrame](http://www.paulirish.com/2011/requestanimationframe-for-smart-animating/) has been around for a few years now, it lets you make timer-based animations more efficient by synchronising reflow and repaint events.
This also gives us a clue as to how fast the browser can synchronise these events: ideally requestAnimationFrame should call your function every 16ms to get that silky-smooth 60FPS you want. If, however, it calls your function after 50ms you know that there has been something hogging the browser's time when it should have been repainting.
I created a small Javascript bookmarklet around this concept, scrolling down a page and measuring the effective framerate.

The bookmarklet produces a small overlay on the top-right of the screen showing the framerates achieved while scrolling the page. It also adds a bar to the left of the page showing what framerate was achieved for that scroll movement.

## Jank meter limitations
* Style is inherited, so it may look funny
* It depends on being able to scroll, so the page has to be longer than 2 x viewport height
* It relies on performance.now() and requestAnimationFrame() so it won't work in all browsers (latest Firefox and Chrome are fine)
* The chart image may not load on secure sites

There are probably a few more limitations, give it a go and let me know if you have any issues.
The code is in [a gist](https://gist.github.com/simonhearne/ef145e2732f2082771d3) so feel free to fork.

## How to fix jank?

Paul Lewis has a great [live demo](https://www.youtube.com/watch?v=QU1JAW5LRKU) on youtube of debugging and fixing jank in realtime.

There are a number of resources available, [jankfree.org](https://jankfree.org) is a great place to start.


==============================================================================

# Be Mindful With Modernizr

> Modernizr might be making your site slow, especially on mobiles

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2015-08-10
Canonical: https://simonhearne.com/2015/be-mindful-with-modernizr/
Tags: WebPerf, third-party

---
According to [Modernizr.com](http://www.modernizr.com/), Modernizr is a JavaScript library that detects HTML5 and CSS3 features in the user's browser.
This framework tests for features in the user's browser then adds classes to your page declaring which HTML5 and CSS3 features are (and are not) supported in that browser.

Recently, a colleague was analysing a client's site and noticed that performance on mobile was about one second slower than expected, with a gap in the WebPageTest waterfall with no network traffic.
We did some digging on desktop and could not replicate the one second of missing network time, although there was a clue hidden in the high CPU usage on the browser main thread, visible in the WebPageTest waterfall:

![High CPU usage and a gap in the waterfall at 6.5-7.5s](https://simonhearne.com/images/waterfall_gap.png)

Something on the page was consuming the browser thread and blocking critical content including web fonts - meaning no text was rendered until this process had completed.
I decided to find more sites using Modernizr to see if the issue could be replicated - it could.

Take [www.CreativeBloq.com](http://www.creativebloq.com/) as an example. A Javascript bundle including Modernizr is loaded as object #6 on the homepage.
When loading it in a browser you can quickly see the browser thread being consumed by the Modernizr script in the Chrome Developer Tools Timeline:

![Over 150ms spent in modernizr.js - on a Core i7 processor](https://simonhearne.com/images/modernizr_desktop.png)

Note that the bulk of the time spent on this script is in one function - s.csstransforms3d. Following the stack you can see that this function causes a style recalculation.
A quick bit of digging into the Modernizr source identifies the cause. The script injects an element and then uses functions offsetLeft and offsetHeight to test whether a CSS 3D transform has worked on the injected element:

```js
tests['csstransforms3d'] = function() {
  var ret = !!testPropsAll('perspective');
    if ( ret && 'webkitPerspective' in docElement.style ) {
      injectElementWithStyles('@media (transform-3d),(-webkit-transform-3d){\#modernizr{left:9px;position:absolute;height:3px;}}',
    function( node, rule ) {
      ret = node.offsetLeft === 9 && node.offsetHeight === 3;
    });
  }
  return ret;
};
```

This process makes sense - these offest function calls are a predictable way to test for the size of a DOM element on screen.
In order to calculate the size of the injected element these functions trigger a [reflow](https://developers.google.com/speed/articles/reflow?hl=en), in order for the browser to tell you the true size of an element it must ensure that all DOM manipulations (such as 3D transforms) have been executed.
Causing a reflow is generally considered bad for web performance, Stoyan Stefanov wrote a [great post](http://www.phpied.com/rendering-repaint-reflowrelayout-restyle/) on this way back in 2009.

Now, in order for Modernizr to do its thing properly it must load early. Here's what the [Modernizr documentation](http://modernizr.com/docs/) says:

> The reason we recommend placing Modernizr in the head is two-fold: the HTML5 Shiv (that enables HTML5 elements in IE) must execute before the `<body>`, and if you're using any of the CSS classes that Modernizr adds, you'll want to prevent a FOUC.

Ideally it should load before the other Javascript and CSS on the page to ensure that the correct features and styles are applied.
That means that the reflow behaviour is called twice - once for offsetLeft and once for offsetHeight - very early in the page load.
This is _bad_ for render performance, and it is _really bad_ on mobile where reflows (and the CSS 3D transform itself) are expensive in CPU time.

So, what do we do about it? First off, check if your Modernizr build runs the CSS 3D transforms check. If it does, make sure it is necessary and if not, remove it from your build using the Modernizr [production download tool](http://modernizr.com/download/)

Of course we need to make sure that we profile all of the Javascript on our sites, first-party and third-party.
We need to do this for every release, and every time a third-party changes.


==============================================================================

# The Future of Web Performance - Part 1

> Web performance is critically important, how will upcoming technologies change the way we manage it?

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2015-08-07
Canonical: https://simonhearne.com/2015/future-of-web-performance-1/
Tags: WebPerf, future

---
Web performance is [critical](/2015/web-performance-optimisation-basics/) to a successful online business.
Keeping on top of the latest technologies, techniques and practices allow us to stay ahead of the curve and get the edge over our competition.
This competitive edge comes through improved user experience and reduced operational costs (and the pride in achieving a [sub-1,000 Speed Index!](http://www.sitepoint.com/speed-index-measuring-page-load-time-different-way/))

New technologies are emerging which will change the face of web performance forever.
These technologies range from the underlying transport mechanism for our transmitted bytes to new javascript frameworks and an ever-growing range of browsing devices.
This complex and rapidly changing environment makes it harder than ever to plan and manage web performance.
In this short series I've attempted to summarise the key upcoming issues and describe how each one may impact your day-to-day life as a stakeholder in an online business.
I've also estimated when you need to really start thinking about it:

1. Now..
  *	[No Flash!](#noflash)
  *	[HTTPS Everywhere](#https)
  *	[Responsive Images](#responsive)
  *	[Mobile-First](#mobile)
  *	[Micro-Services and Containers](#micro)
2. Very Soon...
  * HTTP/2
  * _More_ Javascript?
  * Third-Party Functionality
  *	Emerging Markets
  *	Offline-First
3. Soon...
  *	ECMAScript 6
  *	Low-Powered Devices
  *	Social-First
  *	Rich Content
  *	Web 3.0 / Semantic Web

## <a name='noflash'></a>No Flash!

Flash has been around forever (well [10 years](https://en.wikipedia.org/wiki/Adobe_Flash#Macromedia) is forever in web-years)
It allowed web designers to create rich experiences and port them directly to the web in a format compatible with most operating systems and browsers.

_This. Was. Great._

So you want to embed a game on a page? Use Flash. So you want to have an interactive museum exhibit? Flash.
The uses were nearly endless, from poorly designed website introduction animations right through to online music albums.

The reason Flash was so universal is that you had the core code installed on your machine, the website element was just to tell the code where to download the Flash file from and how to render it.
This opens the client computer up to [quite](http://www.theguardian.com/technology/2015/jul/08/warning-adobe-flash-vulnerability-hacking-team-leak) a [few](http://blog.trendmicro.com/trendlabs-security-intelligence/analyzing-cve-2015-0311-flash-zero-day-vulnerability/) [security](http://www.computerweekly.com/news/4500248673/Adobe-patches-Flash-Player-vulnerability-CVE-2015-3113) risks.
Files from the web run directly in native code installed on your machine is generally a bad thing.

Flash is actually no longer available on mobile devices and has not been developed [since 2012](http://www.techhive.com/article/258574/adobe_says_no_flash_player_for_android_41_plans_to_withdraw_app_on_aug_15.html), making it pretty useless in a mobile-first world.
HTML5 offers us much more secure and flexible ways of rendering dynamic content, such as audio and video, on web pages.
CSS and Javascript offer the power to make even more interactive web experiences.
All of these together mean that Flash is dead.

Losing Flash is a positive change. Flash was [horrible for web performance](https://www.apple.com/hotnews/thoughts-on-flash/) and the modern replacements are more than up to the challenge.
The HTML5 and interactive client-side applications that replace Flash have their own performance challenges, however... (A story for another blog post or ten).

## <a name='https'></a>HTTPS Everywhere

The Electronic Frontier Foundation created a browser plugin called [HTTPS Everywhere](https://www.eff.org/Https-everywhere) in 2011 and it has been in active development [since 2012](https://github.com/EFForg/https-everywhere/graphs/contributors)
The plugin will always try to redirect to a secure version of a site and its third-party resources. The intention of the plugin is to increase the security of your connections and to keep your data [private across the network](http://mashable.com/2011/05/31/https-web-security/ "Web Security: Why You Should Always Use HTTPS")
Google Chrome is planning to mark non-HTTPS pages as [affirmatively non-secure in the future](https://www.chromium.org/Home/chromium-security/marking-http-as-non-secure)
This means that, as a site owner, you should expect all traffic to be served securely over HTTPS (oh, and you must [use TLS not SSL](https://tools.ietf.org/html/rfc7568))
This paradigm shift has significant performance implications as secure connections take time to establish.
To optimise secure connections ensure that HTTP keep-alive is enabled (the default for HTTP/1.1), that you have [session ticketing](http://chimera.labs.oreilly.com/books/1230000000545/ch04.html#_session_tickets) enabled and that your negotiating server is optimised for rapid negotiation.
Ilya Grigorik runs the site [isTLSfastyet.com](https://istlsfastyet.com/) which has much more information on the topic (hint: the answer is yes, TLS is fast when done right!)

## <a name='responsive'></a>Responsive Images

Delivering content to multiple device-types *well* is _hard_.

Delivering content to mutliple device-types *fast* is _really hard_.

We all know that [desktop is dying](http://qz.com/393553/the-desktop-is-dying-and-mobile-is-winning-in-news-like-everything-else/), giving way to mobile and tablet consumers ([especially while commuting](http://internetretailing.net/2015/07/consumers-to-spend-9-3bn-shopping-on-mobile-while-they-commute-study-finds/ "Consumers to spend £9.3bn shopping on mobile while they commute"))
As images make up [over 50%](http://httparchive.org/interesting.php#bytesperpage) of the average site, and bandwidth is limited with mobile connectivity, delivering a fast responsive site to all users means sending the smallest images possible whilst maintaining visual quality.

This is by no means an easy task, otherwise companies would not be [selling services](https://www.resrc.it/) specifically to make it easier.
Fortunately for us, new HTML specifications are [being drawn up](http://responsiveimages.org/ Responsive Image Working Group) and [implemented in browsers](http://caniuse.com/#feat=srcset "CanIUse.com / SrcSet") which will help in our quest.
These new specifications allow us to define images with breakpoints, just [like our CSS](https://css-tricks.com/css-media-queries/), so that smaller devices get the correct size images and retina displays get their high resolution versions.
See the [srcset definition](http://www.w3.org/html/wg/drafts/html/master/semantics.html#attr-img-srcset) for a good place to start, and a useful explanation on [CSS-tricks](https://css-tricks.com/responsive-images-youre-just-changing-resolutions-use-srcset/)

Unfortunately for us, this creates more work in creating the multiple image versions on top of the additional markup required in our HTML.

Hey, I didn't say we were getting a free ride.

Some build and automation tools will make this process easier for you, however. See the [Grunt Responsive Images](https://github.com/andismith/grunt-responsive-images) plugin (thanks to [Andi Smith](https://twitter.com/andismith "Andi Smith on Twitter") as an example of responsive image automation.

If this all seems a bit much, your CDN or content delivery provider may be able to do it for you with little work on your end. Whatever happens, we need to keep marketing happy by delivering the best quality images, keep development happy by automating everything and keep IT happy by minimising cost of delivery.

This is not a small task!

## <a name='mobile'></a>Mobile-First

I've mentioned earlier in this post that consumers are moving towards mobile devices.
This shift in consumer behaviour is also causing a shift in the way we build our websites.

Mobile-first means developing your site to work on mobile devices, then scaling the design up to larger screens.
This is as opposed to traditional responsive design which tends to wrangle desktop sites into smaller form-factors.

Being mobile-first means opening up your potential audience to the next _billion_ online consumers.
This point was made extremely well by [Bruce Lawson](https://twitter.com/brucel "Bruce Lawson on Twitter") in [his keynote](https://www.youtube.com/watch?v=BHO70H9tvqo) at Velocity Santa Clara 2015.
Potential consumers in under-privileged areas *will* access your web site or application using a mobile device. Period.

Beyond the obvious advantages in being mobile-friendly, all types of online business will have customers following links from social media on their mobile phones.
Unfortunately, these types of clicks generally use the webview built in to the social application rather than a 'proper' web browser.
This means that there are even more platform / device / browser combinations to worry about and further considerations on web performance (which cache will I use? What resources will I get?)

Designing for mobile-first means layouts that start small and grow with the screen, minimal network interactions and use of key HTML5 features such as [Open Graph](http://ogp.me/)

Designing for _fast_ mobile-first means:

*   Heavy use of caching
*   Optimal use of network time
*   Removing _all_ unnecessary third-party assets
*   Removing (or at least profiling) Javascript - no event handlers on touch or scroll please!
*   Fitting first-render into one round-trip - that means critical CSS is inline, with cacheable CSS added with Javascript

## <a name='micro'></a>Microservices and Containers

Containers and Microservices are real buzz-words at the moment.
If you are unfamiliar with the terms have quick read of [this Medium article](https://medium.com/aws-activate-startup-blog/using-containers-to-build-a-microservices-architecture-6e1b8bacb7d1) on the topic.

Breaking a monolithic application into atomic components - each with their own function and service definition - is a very logical move.
Microservices have many benefits:

*   Rapid development cycles - rapidly iterate services on fixed service definitions
*   Easy replacement of functionality - changing payment provider? Replace the payment service
*   Simple testing during development - stubbing of services means that each can be tested for performance and functionality in isolation
*   Logical scalability - scale each component as necessary rather than the whole application

There's no escaping microservices. The concept offers many benefits and will apparently make web application development much more simple.

There are various performance and security issues that we need to be aware of though.
I won't try to enumerate all of the potential issues that microservices create - because [Gareth Rushgrove](https://twitter.com/garethr "Gareth Rushgrove on Twitter") did it better than I could ever do at the most recent [London Web Performance Meetup](http://www.meetup.com/London-Web-Performance-Group/)
Please find his fantastic slides [here](https://speakerdeck.com/garethr/containers-and-microservices-make-performance-worse)

Wow, that's quite a list of things we need to care about, and I haven't even started to consider the way we measure and monitor site performance.
Using [WebPageTest](https://webpagetest.org/), [sitespeed.io](http://www.sitespeed.io/), [SpeedCurve](https://speedcurve.com/) or similar to test front-end performance before any release is _critical_.
We also need to ensure that we include mobile and responsive breakpoint testing.


==============================================================================

# Questions to Ask Your Third-Parties

> We need rules to keep third-party Javascript in check.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2015-07-03
Canonical: https://simonhearne.com/2015/questions-to-ask-your-third-parties/
Tags: WebPerf, third-party

---
We've discussed how to [find the third-party assets](http://simonhearne.com/2015/find-third-party-assets/) on your site and started to look at how to [mitigate the risk](http://simonhearne.com/2015/manage-3p-risk-csp/) that they pose.
But what should you do when you add new third-parties to the site?

There are some key questions you need to ask of the third-parties to ensure that they will not impact the user experience, performance or security of your customers.
In the best-case scenario you have the opportunity to ask these questions before signing a contract with the third-party in question.
In the more common case, someone else has already created a relationship and has just requested that the tag be added to your site.
Either way, having a list of questions to ask will give you an idea of how mature the third-party is and how much work will be required to ensure that your customers continue to have a great experience.

[JSManners.com](http://jsmanners.com/) by [@triblondon](https://twitter.com/triblondon) was designed to address exactly this issue, giving a list of questions that you can ask a third-party to give them a score.
I'm a fan of scores, they're simple and easy to report on. For assessing third-parties, though, there are more qualitative assessments to make.
For example you may not be willing to use a third-party which includes jQuery outside of a protected scope (as the risk of breaking your site is too high) but you are willing to accept 1kB of cookie data on your host domain.
So I've borrowed the list of questions from JSManners and re-organised them into categories, I've also removed some and added others as well as adding some context to each point so that you can understand the reasoning for the question.

* **Critical Questions**

	*   Are common 3rd party libraries (e.g. jQuery, Underscore, Backbone) included outside of a protected scope (ie is there any chance they may conflict with other instances of the same library that may be included on the page? _- If common libraries conflict, it may break critical functionality of your site. Third-parties are on different release cycles to you so even if everything works now, it may not in the future._
	*   Do component script(s) block render in any way? _- If a third-party asset blocks the render of your page(s), it is a single-point-of-failure (SPOF). Thus if your customers cannot download their script for any reason, it will take [at least 20 seconds](http://www.stevesouders.com/blog/2014/11/14/request-timeout/) to render your page(s). You want to  know about this and mitigate the risk (by asking all of the availability questions below!)_
	*   Do component script(s) use document.write? _- if so, you cannot load them asynchronously, every DOM element below the inserted script is [blocked from rendering](http://www.stevesouders.com/blog/2012/04/10/dont-docwrite-scripts/) while it downloads and it [blocks other dynamic scripts](http://www.stevesouders.com/blog/2012/04/10/dont-docwrite-scripts/)._

	<br/>
* **Good Behaviour**

	*   How many objects/variables do component script(s) require in / add to the global scope?
	*   Do component script(s) produce any console output in normal operation? _- they shouldn't; if they do, why and what content is output?_
	*   Do component script(s) cause any JavaScript errors in normal operation? _- they shouldn't; if they do, why and what content is output?_
	*   Can component script(s) be loaded asynchronously? _- ideally scripts should be loaded asynchronously to prevent potential impact on your [users' experience](https://css-tricks.com/thinking-async/)._
	*   How many (infinitely and unconditionally recurring) timer events does the script fire per minute? _- timer events such as setInterval() can cause the main browser thread to lock up in some circumstances and can cause performance issues if the functions access the DOM or perform network actions._
	*   If the script tag is removed from the page (e.g. because it appears to be causing problems), what is the UI impact (on subsequent page loads)? _- if you have a method of removing badly-behaving scripts, you should ensure that the customer experience is not negatively impacted. This may be relevant for a third-party that adds content to the page such as adverts or personalisation._
	*   Which browsers is the component script tested in? _- if the third-party can't guarantee that the script will behave well in all of the browsers you support then you can't guarantee your user experience._
	*   When does the script interact with the DOM? _- if the script interacts with the DOM on regular, critical or unpredictable events there may be a significant performance impact._
		*   On parse
		*   On the DOMReady event
		*   On the load event
		*   On recurring timer events
		*   On user interaction events
		*   On network related events
		*   When invoked by a function call on JS API

	<br/>
* **Local Storage**

	*   Does the script set any persistent cookies on the host domain? _- this may be bad, I've seen a customer site which sends 200kB of upstream cookie data on each page load because of third-party cookies set on the host domain._
	*   Does the component depend on, or modify, any existing cookies set by the host page? _- it shouldn't, these are your cookies._
	*   What is the maximum total size of data, in bytes, stored in cookies by the script? _- depending on the domain, a large amount of upstream cookies can cause performance issues. Remember that cookies cannot be compressed on the network (at least with HTTP/1.1)._
	*   Does the component persist data in the browser using a mechanism other than cookies? _- you should be aware if the third-party relies on localstorage or equivalent and ensure that there are no security or performance risks associated with it._

	<br/>
* **DOM Interaction**

	*   If the component adds event listeners to DOM elements, does it remove them when they're no longer required?

	<br/>
* **Content Delivery**

	*   Is the script (including all non-dynamic resources) cached on a CDN with global reach? _- using a CDN ensures that the resources are delivered to your customers as quickly as possible and shows that the third-party understands the importance of performance._
	*   Where are there servers that can accept write operations performed by the script? _- if the browser needs to send data back to the third-party, having locations close to your customers ensures best possible performance and that data won't be lost due to poor connections._
	*   What is the total data size in kilobytes transferred on page load? _- large amounts of Javascript can cause performance issues, ensuring that the third-party only delivers necessary content._
	*   What is the minimum browser-cache TTL (in seconds) of files downloaded when the component script loads (ie. Cache-control max-age)? _- having good cache control on third-party assets reduces the amount of network time for repeat visits, however some third-parties will disable all caching as the scripts are dynamic._

	<br/>
* **Analytics**

	*   Do the component script and all associated resources provide ResourceTiming [opt-in header](http://www.w3.org/TR/resource-timing/#cross-origin-resources) (ie. [Timing-Allow-Origin](https://nccgroup.webex.com/mw0401lsp13/mywebex/default.do?service=1&siteurl=nccgroup&nomenu=true&main_url=%2Fmc0901lsp13%2Fe.do%3Fsiteurl%3Dnccgroup%26AT%3DMI%26EventID%3D349336237%26UID%3D504506122%26Host%3DQUhTSwAAAAJ8Y2XjDvwHvaWR52DTDaT-zR08KLPczSY835Z46AfLT52IErW9zlPt5lkk-nSo-xT6NY8Q5oCfFw2U0-39b8pJ0%26FrameSet%3D2%26MTID%3Dm4dad1c514e54409238c2142cd843ae2d): *)? _- if you are using a real user monitoring (RUM) solution, this header allows you to measure and record detailed timings of your third-party resources. There may be security implications here though._

	<br/>
* **Availability**

	*   Is the script or any web service it depends on served from a single origin (ie. is there just one origin data center)? _- this could be a single-point-of-failure for the third-party._
	*   In the event of a complete failure of the primary origin (eg unmitigated data centre power failure), what is the mean time to recovery (MTTR), in seconds? _- this checks that the third-party are aware of the consequences of the origin becoming available._
	*   How many full outages has the service experienced in the 12 months prior to today? (where full outage means a user with empty cache anywhere in the world being unable to use the service) _- this gives you an idea of how reliable the service will be. Based on the impact of the service being unavailable this could be a deal-breaker._
	*   Is there a public status page describing the current operational state of the script's backing services? _- if the third-party is open about uptime and performance this serves as a good indicator of their maturity._

	<br/>
* **Transport**

	*   What is the minimum number of HTTP requests that the script will make when invoked? _- if this number is high it suggests an inefficient design, creating more network traffic than is ideal._
	*   Is it possible to make ALL the script's network activity go over HTTPS? _- your customers trust you with their information, third-parties become potential leaks of this information and HTTPS helps to reduce the risk._
	*   Is HTTP 2.0 supported for all requests? _- HTTP 2.0 is the newest HTTP protocol which enables more efficient communications. In most browsers it is only enabled over HTTPS._
	*   Can the script's initial payload be bundled with the host page's own scripts? _- this reduces the network overhead and delay of the initial request. It can also mitigate the risk of the third-party being unavailable and keeps the core code under your control, meaning that you manage the new third-party code releases to your site._

	<br/>
* **Security**

	*   Will the component be involved with collecting user identifiable data (including any one of name, email address, credit card details, or phone number)? _- any of this can be risky and may be against the law in some of your customers' countries or open you up to PCI compliance issues._
	*   For European customers, does the script's backing service either a) not store any personal data, b) store personal data only within the EU, or c) store personal data outside the EU in compliance with the US Safe Harbour data protection standard? _- you should ensure that your third-parties are not putting your customers' data at risk, pursuant to your (and your customers') local and international laws._
	*   Does the script respect the Do-Not-Track HTTP header, either actively or by not conducting any tracking at all? _- Do-Not-Track is a [proposed HTTP header](https://en.wikipedia.org/wiki/Do_Not_Track) which suggests that customers can opt-out of cross-site behaviour tracking. If your third-party respects the DNT header then they have respect for your customers' privacy wishes._


==============================================================================

# Manage Third-party Risk Using a Content Security Policy

> A feature to prevent cross-site-scripting may make it easier to police third-party javascript.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2015-07-03
Canonical: https://simonhearne.com/2015/manage-3p-risk-csp/
Tags: WebPerf, third-party

---
Andy Davies and I have [spoken](https://www.youtube.com/watch?v=9OtUOgx0cZg "What Are Third-party Components Doing to Your Site?") at a number of conferences about the risks that third-party components can have on your website.
In these talks we try to illustrate the fact that third-party components can have all sorts of negative consequences on your site's availability and performance.
At the end of our talks we are always asked the same thing: 'what can we do about it?'.

Once you've got your third-parties under control, set up a tag management solution and identified the business value you will still have some third-parties on your site.
What can we do to mitigate the risk that those critical third-parties pose?
There are a number of methods to minimise the risk, such as loading the third-party scripts with an [async or defer tag](https://www.youtube.com/watch?v=I5uhZcJ30SA "Guy Podjarny talks about Third-party Performance") (after ensuring that the third-party script [doesn't use document.write()](http://www.w3.org/TR/html401/interact/scripts.html#adef-defer "W3C Defer Specification")!)
The problem is that these third-party scripts can still make calls to fourth- and fifth- and sixth- ... parties, which can multiply the risk posed to your site.
We know this through using tools such as [request map](http://blog.simonhearne.com/portfolio/request-map/ "Request Map") which highlights how requests to these third-parties are made.

![Third-party calls can become complex and hard to manage](https://simonhearne.com/images/requestmap_very.png)

We don't currently have a good way to prevent these nth-party calls. Or at least not that people are using in the wild.
Content-Security-Policy is a proposed HTTP Header which prevents cross-site scripting (XSS) attacks. It does this by defining whitelists of domains which can load scripts, styles images etc. from.
This prevents resources being requested from malicious domains _at the client-side_.
CSP is [pretty widely supported](http://caniuse.com/#feat=contentsecuritypolicy "Can I use... CSP") amongst the major browsers and as it is just an HTTP header it can be set by any server-side language, any web server and any CDN.
Using your CDN would allow you to define geographic-specific third-party script whitelists which would be great to prevent calls to social networks from inside China, as an example.
I've built a quick example to demonstrate the concept [on my site](http://simonhearne.co.uk/sandbox/csp/ "Console message when unexpected third-party asset is blocked"):

![Console message when unexpected third-party asset is blocked](https://simonhearne.com/images/CSP.png)

CSP will beacon back JSON every time a resource is blocked, it also has a [report-only feature](http://www.w3.org/TR/CSP/#content-security-policy-report-only-header-field "W3C CSP Report-only Mode"), where the request still happens at the client but it would allow you to report on unknown third-parties for further analysis.
A potential secondary benefit is that you can define the protocol in the CSP - thus enforcing HTTPS on the requests and better respecting your customers' privacy.

I can see a future where tag managers will integrate with CDNs (through [great APIs](https://docs.fastly.com/api/ "Fastly API Docs")) and will automatically create CSPs based on the tags that are loaded for each page and geography.
Wouldn't that be nice! Until then, though, it will take some manual intervention to get CSPs up-and-running.

Would you use this to restrict the third-parties on your site or is CSP too extreme for your needs? Please let me know if you try it out in the wild!


==============================================================================

# Using a Web Performance Heatmap to Assess Page Performance

> I've made a Heatmap to visualise when each element loads on a page.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2015-05-29
Canonical: https://simonhearne.com/2015/performance-heatmap/
Tags: WebPerf

---
<a href="http://heatmap.webperf.tools" class="btn btn-block btn-primary">Go to the tool</a>

I've recently had a few clients ask if there is a way to test when certain parts of their pages become visually complete. Imagine a media company that relies on advertising revenue who wants to measure when the above-the-fold adverts are rendered, relative to the actual content of the page.

We have a few ways to attempt this, for example [user timings](http://www.html5rocks.com/en/tutorials/webperformance/usertiming/ "HTML5Rocks User Timings Tutorial") can add a performance timing marker based on an element loading (note that this can't tell when the loaded element / image is actually rendered). To really tell when an element is rendered we either need to dig in to [paint traces](https://www.chromium.org/developers/how-tos/trace-event-profiling-tool/trace-event-reading "Chromium about:tracing page") or use a filmstrip view from e.g. [webpagetest](http://webpagetest.org "WebPageTest.org").

The only realistic solution to tell when an element on a page _actually_ renders in a controlled environment is to use a filmstrip view and scroll through until the relevent part of the page is visible. This is obviously a labour-intensive task and tricky to automate.

Thinking about better solutions to this challenge led me to imagine a heatmap of a webpage - showing in red-amber-green how late each pixel was to load. This would make it simple to tell, at a glance, what parts of the page are slow or fast to render.

To test this theory out I started running filmstrips from WebPageTest through PHP to compare pixels to the final frame and determine when each pixel achieved its final state. Doing this I discovered that by default WebPageTest filmstrip images are scaled and heavily compressed. While to remove the scaling you have to re-build the WebPageTest agent, you can disable image compression [using a hidden form field](http://jrvis.com/blog/wpt-screenshots/). Now that I had uncompressed images I could write a function that iterates through the frames and notes at what load time each pixel completes. This matrix can then be used to generate a heatmap layer based on the performance budget (e.g. if a pixel acheives final state at 8 seconds and the budget is 6 seconds, colour the pixel red).

I ended up building a wrapper on top of WebPageTest to automate this process and generate a heatmap for you: [heatmap.webperf.tools](http://heatmap.webperf.tools/ "Web Performance Heatmap").

![Pinterest Heatmap highlighting that the top images are late to load](https://simonhearne.com/images/heatmap_pinterest.png)

So go ahead - have a play. Note that it is a proof-of-concept toy not a complete tool so please let me know what you think.

_You may notice some strange results when you test your website. Many websites have rotating carousels or cookie / advert banners which push all of the page content down, this causes the heatmap to reflect potentially poor results even though content has been rendered earlier. My opinion on this is that shifting the page down for a cookie / advert banner is bad for user experience. [So is using a rotating carousel](http://shouldiuseacarousel.com/ "Should I Use a Carousel?")._


==============================================================================

# How to Find the Third-Parties on Your Site

> Third-party means not you or me, but them.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2015-03-19
Canonical: https://simonhearne.com/2015/find-third-party-assets/
Tags: third-party, WebPerf

---
<a href="http://requestmap.webperf.tools" class="btn btn-block btn-primary">Go to the tool</a>

My customers often ask how a third-party asset got on their site. A director of digital once **_told_** me that Facebook was not on their site, not at all.

I ran the homepage through WebPageTest and sure enough, there were a bunch of calls to various subdomains of facebook.com. Thankfully WebPageTest stores initiator, referer and redirect headers; so with a little work you can find out where these third-party calls come from. The director was correct, there were no calls to Facebook on their site. It was a third-party creating _fourth-party_ calls to Facebook! This can have serious ramifications if the Facebook (or _any other_ third-party call) affects customer experience. Radware explain this well in an [article from 2011](http://www.webperformancetoday.com/2011/10/13/how-vulnerable-is-your-site-to-third-party-failure/)

Trawling through the initiators and referers was a headache so I wrote a tool to do it: [requestmap.webperf.tools](http://requestmap.webperf.tools/ "Request Map")

The request map runs a test in Chrome on public WebPageTest and uses the initiator, referer and redirect data to work out where every request comes from. Here's the result when you run Google.co.uk through the tool:![ninja_map](/images/requestmap_google.png)

From the picture I can see immediately that the bulk of the content on the page comes from www.google.co.uk (size of blob is proportional to percentage of total bytes).
There is a redirect from google.co.uk -> www.google.co.uk and a number of requests from other Google-owned domains.

It all looks pretty simple on Google.co.uk, there's basically nothing on it. But when it comes to an ecommerce site or a site generating advertising revenue, third-parties are everywhere.
Whether they are for analytics, advertising, tracking, attribution... third-parties make people _money_.
As such, they tend to be added to sites without much consideration for management of the relationship, the technology and (dare I say it) third-party SLAs.

Take the answers.com homepage as an example:

[![answers_map](/images/requestmap_img.png)](http://requestmap.webperf.tools/render/150319_RK_58a22e3fe46f36365ab817e2b03f3606)

Here you can see that the root domain (www.answers.com) makes up a small proportion of the total page weight. This site uses domain sharding across a number of CDN domains (in yellow) to maximise performance for high-bandwidth customers. One CDN subdomain stands out from the others though: rxf3.answcdn.com is the big yellow blob on the right. Almost 500kB of content comes from this domain but, more interestingly, it initiates calls to Google and Facebook which then spawn requests to another six domains!

Another observation is based on the length of the edges on the map (the lines which join the blobs), these are proportional to the mean response time of that domain. Blobs that are far away from the target site on the map are far away from the customer's experience. A single asset from www.dsply.com (far left) took over 1.1s to load less than one kilobyte of content! It also took almost a second to load two kilobytes from snip.answers.com. Are these assets critical to the user experience? If so, there's a massive performance hit here.

There's a lot of work to do on analysing third-party components but visualising who is on your site (and how they got there) is a good start!


==============================================================================

# Web Performance Optimisation Basics

> Website performance is critical to user experience. We need rules to make it easier.

Author: Simon Hearne (https://simonhearne.com/about/)
Published: 2015-03-11
Canonical: https://simonhearne.com/2015/web-performance-optimisation-basics/
Tags: WebPerf

---
Our web performance industry was unquestionably seeded by Steve Souders' book [High Performance Websites](http://shop.oreilly.com/product/9780596529307.do). In his book, steve identifies [14 rules](http://stevesouders.com/hpws/rules.php) to improve web perfomance.

As part of my job at NCC Group I get to produce optimisation reports for our customers (called healthchecks). In these reports we tend to focus on five key areas, one of these is front-end optimisation. This is where Steve's rules come in. The rules were first published in 2007, over 8 years ago! In the fast-paced world of technology this is a huge amount of time, although admittedly the technology that defines the web has not changed too much since then. Web browsers and business attitudes to web performance have changed, however. Companies are focusing more effort and resource on ensuring a good online experience for their customers. Gradually, web performance is being considered as one of the key factors in the customer experience.

What we need now is a concise, logical list of technical recommendations to improve website performance.

The list needs to be logical in terms of implementation and importance, but must also have business context in terms of complexity of implementation. Generally, improving front-end performance reduces operational cost: fewer bytes over a CDN is less $ spent, fewer connections to a firewall is less utilisation. There is also an associated cost - implementing combined CSS involves process changes, new technologies etc.

The challenge is finding a generic enough list of recommendations which will apply across website industries, customer types, mobile v. desktop etc. I have tried to create a list which covers these elements but only caters for browser activity before the onLoad event (according to [wikipedia](http://en.wikipedia.org/wiki/DOM_events) the onLoad event "fires when the [user agent](http://en.wikipedia.org/wiki/User_agent "User agent") finishes loading all content within a document, including window, frames, objects and images"). Content requested after onLoad _should not_ impact customer experience - this is the realm for analytics, tracking, advertising etc. Here's my current working draft of pre-onLoad recommendations:

*   Make fewer requests
    *   Combine CSS
    *   Combine JS
    *   Sprite small images
    *   Cache everything (no ETags?)
*   Transmit fewer bytes
    *   Optimise Images
    *   Compress everything
    *   Send only what you need (responsive images?)
    *   Cache everything
*   Reduce request latency
    *   Optimise the application
    *   Use asynchronous requests
    *   Put content close to the consumer
    *   Optimise headers
    *   Use the latest protocols
    *   Set an intelligent initial congestion window
*   Prioritise critical content
    *   Inline critical CSS
    *   Inline critical images
    *   Inline critical JS
    *   Outline non-critical content
    *   Load images asynchronously
*   Remove single points of failure
    *   Use async & defer
    *   Manage third-parties (tag management?)
    *   Re-consider <link src> tags

The specific implementation of each recommendation is unimportant, but the general concepts and prioritisation are key to getting a website to perform at its best.

Feedback graciously welcomed!

Si.