Delta V Conference - Day 2!
Tammy Everts - Hunting the Unicorn
Tammy started researching user experience on the web in a laboratory setting - measuring individuals as they used websites. Now she works with SpeedCurve trying to connect the dots between real people and the stream of data being collected from Real User Monitoring.
How do we scale empathy?
As we collect millions or billions of user experience data points, how do we remind ourselves that there are real people on the other end of the data?
The average web user believes that they waste two days a year waiting for web pages to load.
We know that slow sites suck, but how do we define slow and how do we define suck? We are perception brokers. But perceptions are fluid and, well, perceptual. How can we measure something so dynamic?
Our traditional web performance metrics like start render, page load, speed index do not help us to measure perception!
The best UX metric needs to correlate to what users actually see, it needs to be easy to use and work out-of-the box in browsers, it should recognise that not all pixels and page elements are equal and should be customisable based on our knowledge of our sites.
First meaningful paint (FMP) - times the paint after which the biggest above-the-fold layout change has happened or when web fonts are loaded, whichever comes later. Available in Chrome.
Time to first interactive (TTFI) - when the page is first expected to be usable and will respond quickly to user interactions. Available in WebPageTest and mPulse.
Hero rendering times - the time at which the largest image, largest background image and top-level heading render.
To help work out the best metrics, SpeedCurve created a test page where you can see what metric correlates best with your perception of page load. The results unsurprisingly show that there is no metric that works for all sites, nor one that works for all people.
Custom metrics are the only way to measure what you know is important to your page, for your users. E.g. time-to-first-scroll, time to product image. Unfortunately these aren't necessary accessible or easy to use.
Whatever you measure:
- Know your most meaningful metrics
- Track them with synthetic and RUM
- Don't focus on averages and medians
- Set goals / budgets
- Make yourself accountable with alerts and reports
- Mind regressions
Jeremy Keith - What is a progressive web app?
There is a lot of confusion and misinformation about what progressive web apps (PWAs) are. They do not require a single-page application JavaScript framework!
Most of what makes a PWA is just what makes a got website - secure, fast and responsive. You can measure that a site has https, service worker and a web app manifest - that's what makes a measurable PWA.
Add to home screen is a bit of a distraction for PWAs, don't worry too much about it
Service Workers are the key part of a PWA, and they enable varying levels of offline functionality:
- An "offline" error page - e.g.
- A full offline experience - e.g.
- A user-initiated page download - e.g. Una Kravet's website
Should your site be progressive web app? Yes, it should
Jeremy has published a book about using Service Workers to create great offline experiences.
Mark Leach and Andy Davies - Fast Fashion
Mark works at Missguided.com - a fast fashion brand aimed at trend-conscious women aged 18 - 29.
Our job is to empower young women to feel strong and confident
Mark's customers are all on social media, and are keen to complain when the website is slow.
When working with NCC Group Web Performance (now Eggplant) Mark was told that there could be an additional £1.2M through performance improvements.
When analysing the website, it was discovered that the median Android page load time was over 14 (fourteen) seconds.
Adding <link rel="preload"> for web fonts and fixing some issues with Optimizely brought the Android page load time down by two seconds.
Removing BazaarVoice improved load time by a further 3 (three) seconds for Android devices. This improvement led to an increase in weekly revenue of 56% 😮. When baselined against all other traffic with BazaarVoice still enabled, the net gain in revenue was 26%.
When presenting to the board, initially they were pessimistic about the potential, but then saw the real data. Then they wanted more!
In October 2017, Missguided set a goal of a three second load time. Andy used source map explorer to visualise the JavaScript and CSS bundles. The JavaScript bundle was reduced by 42%!
Missguided then hired someone to manage A/B testing and experiments to improve speed, as Optimizely could not be removed. This involved removing the bundled jQuery from the Optimizely bundle, removing old experiments and those for non-production environments.
Overall, median performance on Android was improved from 14 seconds to just over 5 seconds.
Lauren Younger - Managing Performance like a Product
Performance and product management have a lot in common, they are both about improving conversion and revenue, retaining users, manage the user experience and understanding people.
Four basic principles for performance management:
-
Understand Market, competition, stakeholders, business KPIs. Qualitative and quantitative data - using e.g. surveys and satisfaction scores. What does a 49% bounce rate mean? Were 49% of customers unsatisfied?
-
Define Create a specific business case with SMART goals then get stakeholder buy-in. Think beyond standard performance metrics (bounce rate, load time, session length), what about operations resource cost, global reach.
-
Prioritise Prioritisation is hard, we're not very good at it and we tend to rely on opinion. Use data to create scores and ranking. It's easy to focus on the slowest pages, but you should use business information such as conversion impact to determine priority.
-
Communicate We can't use the same communication methods when working with all of the stakeholders of your website. Marketing, operations and engineering all have different requirements and communication styles.
Measure the results. What went well? What could have been improved in the process?
Share your success and thank the team!
Dan Shappir - Improving Performance for 100 Million Websites
Wix made performance a top priority feature in 2017. Performance has to be a feature in order to get resources, buy-in and a specification.
Wix runs >1,000 concurrent experiments and pushes to production around 200 times per day.
In order to protect performance, performance is measured during development, in pre-release and in production.
The biggest benefits seen by Wix (measured by change in first contentful paint):
Server-Side Rendering - 70% improvement for mobile devices and 40% improvement for desktop. However the time between render and interactive became longer due to the additional delay in downloading the JavaScript bundle.
Caching - before going crazy with service workers, ensure that your assets are actually cacheable at the client!
Smart Bundling - dyanmically create app bundles depending on the features required on the site in question
HTTP/2 - 10 - 20% speed improvement by switching to HTTP/2.
Resource Hints - Using preconnect, preload and prefetch gave Wix a 10% speed improvement.
Progressive Rendering - by only rendering above-the-fold content in the first pass, Wix saw a 5 - 10% improvement.
JavaScript Optimisation - Wix saw a 10% improvement by profiling and optimising their common JavaScript bundle.
Wix are in the process of transitioning from a JavaScript-driven layout to CSS Grid!
Robin Marx - QUIC in theory and practice
Robin introduces the three main problems with HTTP/1.1:
- 4xRTT (round-trip-time) connection setup
- TCP head-of-line blocking
- HTTP head-of-line blocking
We can fix a few of these issues with TLS 1.3, fast-start and resumption. Leading to a single RTT connection establishment.
HTTP/2 removes the TCP head-of-line blocking, but requires a single connection which is susceptible to packet loss. Robin's research has shown that HTTP/2 is up to five times slower than HTTP/1.1 in high packet-loss environments.
We can't just upgrade TCP - it is too pervasive and there are too many pieces of technology which are too slow to upgrade (e.g. proxies). The solution to this is to use a different transport-layer protocol: UDP.
QUIC is built on top of UDP with custom packet-loss recovery logic. Solving TCP head-of-line blocking. Beyond that, it encompasses all of our network learning over the past three decades. By design, QUIC mitigates against all known DDoS attacks, prevents middlebox meddling through full transport-layer encryption. It also uses unique connection IDs to prevent connections closing down when IP addresses change, and allows communication over multiple networks (e.g. 4G and WiFi).
QUIC further improves on HTTP / TCP with custom congestion control.
Google have launched their own version of this protocol: gQUIC. A standardised version, iQUIC is expected to land at the end of 2018.
Google Search speed improvements: 8% on desktop, 3.6% on mobile. 16% improvement in 99th percentile.
20% less rebuffering on YouTube India, 14% faster on 4G, 20% slower than HTTP/1.1 with packet loss (HTTP/2 is 200% slower).
But, some research shows 75% lower throughput and QUIC costs 2x the CPU time on servers.
For QUIC connections on mobile devices, 58% of the time the network is faster than the CPU!
Harry Roberts - It's my (third) party and I'll cry if I want to!
We use third-parties because they provide convenience, but this increases our exposure to risk.
Third-party content is inevitably added to provide value to the business, not the web users.
- Understand
Security - insecure content, man-in-the-middle attacks, compromised third-parties.
Delays - network, third-party infrastructure, third-party runtime.
Use WebPageTest's block and SPoF features to determine the impact of each third-party.
Failures - what happens when JavaScript fails? Use the block request URL feature in Chrome DevTools.
Runtime cost - defer expensive third-party JavaScript until after critical content.
Who owns what? - use requestmap.webperf.tools to plot the third-party assets on your page. Harry has created a Google Sheet to help visualise the CSV output from request map!
In summary: use as few third-parties as possible, for as short a period of time as possible.
Dora Milataru - Where are the women?
Dora leads the engineering team at FT.com responsible for the changes required to be compliant with the General Data Protection Regulation (GDPR).
Globally, the Financial Times employs more women than men. Board representation is greater than 30% women, with a target of 50%. This is a diversity quota, which can be dangerous. What happens when the quota is reached? Is that diversity done?
Quotas may be the only way of achieving a world where quotas are obsolete
Diversity is intersectional, discussing diversity along one dimension (gender, race, sexuality, religion, culture...)
If you search for why diversity is important in tech, you will find hundreds of articles which show that a diverse workforce improves business results. This helps get stakeholder buy-in, but does not help public opinion.
Caring about diversity has to mean caring about people.
Virtue signalling - showing off how good you are. Self-congratulation does not help society.
Social networks show that diversity alone doesn't help, especially in decision making processes. Inclusion is the glue that can fix this.
We gravitate to people like us.
Diversity is being invited to the party, inclusion is being glad that you're there
The recent Stack Overflow developer survey shows that men don't rank diversity as important when selecting a job, while women and non-binary people rank diversity and working environment as more important than salary.
An internal Google study showed that there is zero correlation between a candidate's interview score and their future job performance.
Reach out to under-represented groups. Be honest. Celebrate success. You're not a saviour. Try to understand.
Tolerating difference is not the same as celebrating diversity
Léonie Watson - There's more to performance than meets the eye
Screen readers
Accessibility APIs are provided by Operating Systems, these are exposed to assistive technologies (ATs) such as screen readers. As developers we don't have access to these APIs.
The Accessibility Tree provides a sister hierarchy to the browser's document object model (DOM). This contains element roles (e.g. button) and properties (e.g. disabled) as well as any text content (e.g. Checkout Now).
Microsoft Internet Explorer allows screen readers to inject themselves into the application process and communicates directly with the browser, while in MacOS the screen reader communicates with the browser across OS APIs.
The Microsoft Internet Explorer model provides a fast time to interaction, but is prone to instability. If the browser crashes it can take the screen reader down with it. Chrome and Firefox provide greater stability by proxying the accessibility API calls, Chrome also now caches the accessibility tree. This can lead to a slower initial TTI but fast subsequent page hits. This delay in TTI can be multiple seconds.
Firefox uses intelligent caching to improve TTI performance. Microsoft Edge, however, does not allow screen readers to inject into the process, depending on communication via the OS APIs. This makes communication between screen reader and Edge can be up to 50x slower than Internet Explorer.
Safari on MacOS is in-process, providing fast TTI.
Tobias Baldauf - 25% faster hotel search. Web performance? Trivago.
- Research
- Pitch
- Build
- Verify
- Merge
When launching a large marketing campaign in Brazil, Trivago discovered that Brazil performance was one second slower than the rest of the world. By analysing the performance by proxying into Brazil, they discovered that their CDN was not sending Brazil traffic to their Brazilian edge servers, but up to North America!
At Trivago, web performance is equal to user value, especially for emerging markets.
Feature teams are created from multiple disciplines around projects and given the freedom to build the feature.
A team of twelve people run a release controller tool: a real-user performance testing framework to test the impact of new features with canary traffic.
The behaviour of your existing user base will not change instantly, they have trained behaviour and expectations. It took three months to see the positive impact of performance improvement.
Part of the CDN rollout produced a delay on the hotel results display, which led to an increase in conversions! By keeping the UI clean while users interacted with the calendar widget and filters, users were more likely to continue their user journey.
Trivago has a performance team - they are not responsible for performance optimisation. The performance team owns the performance tooling and provides data to feature teams, enabling the developers to improve the performance of their own features.
Automated performance and accessibility testing is performed across at least five browser testing tools for every build, 30 - 50 times a day! Issues are added as build badges to the git branches.
Web performance is a feature owned by everyone. Specialists consult and evangelise. Fast feedback loops help to iterate quickly and long-term data helps discover trends and define direction.
Beware of what to measure: business metrics may change unexpectedly when improving web performance. Choose the right tool for the job.
Web performance is equal to but not superior to the main features of a website sand can be temporarily traded in for learnings.
Henri Helvetica - The shape of the web
The first browser, Mosaic, recently celebrated its 25th birthday.
this software is going to change everything
Mosaic achieved 65M new users in 18 months, now we have over 4 billion people online.
We do everything on the web, and it's reshaping everything we do.
the web is the most hostile software engineering environment imaginable
The next users on the web are coming from emerging markets with poor connectivity and low-end devices. Many users in these markets turn off their data connection to save money.
We need to treat offline not as an error case, but as a normal use case. This is the shape of the web. Service Workers and Progressive Web Apps are the greatest updates to browsers in a decade.