Questions to Ask Your Third-Parties

We've discussed how to find the third-party assets on your site and started to look at how to mitigate the risk that they pose. But what should you do when you add new third-parties to the site?

There are some key questions you need to ask of the third-parties to ensure that they will not impact the user experience, performance or security of your customers. In the best-case scenario you have the opportunity to ask these questions before signing a contract with the third-party in question. In the more common case, someone else has already created a relationship and has just requested that the tag be added to your site. Either way, having a list of questions to ask will give you an idea of how mature the third-party is and how much work will be required to ensure that your customers continue to have a great experience.

JSManners.com by @triblondon was designed to address exactly this issue, giving a list of questions that you can ask a third-party to give them a score. I'm a fan of scores, they're simple and easy to report on. For assessing third-parties, though, there are more qualitative assessments to make. For example you may not be willing to use a third-party which includes jQuery outside of a protected scope (as the risk of breaking your site is too high) but you are willing to accept 1kB of cookie data on your host domain. So I've borrowed the list of questions from JSManners and re-organised them into categories, I've also removed some and added others as well as adding some context to each point so that you can understand the reasoning for the question.

• Critical Questions

  • Are common 3rd party libraries (e.g. jQuery, Underscore, Backbone) included outside of a protected scope (ie is there any chance they may conflict with other instances of the same library that may be included on the page? - If common libraries conflict, it may break critical functionality of your site. Third-parties are on different release cycles to you so even if everything works now, it may not in the future.
  • Do component script(s) block render in any way? - If a third-party asset blocks the render of your page(s), it is a single-point-of-failure (SPOF). Thus if your customers cannot download their script for any reason, it will take at least 20 seconds to render your page(s). You want to  know about this and mitigate the risk (by asking all of the availability questions below!)
  • Do component script(s) use document.write? - if so, you cannot load them asynchronously, every DOM element below the inserted script is blocked from rendering while it downloads and it blocks other dynamic scripts.
  
  

• Good Behaviour

  • How many objects/variables do component script(s) require in / add to the global scope?
  • Do component script(s) produce any console output in normal operation? - they shouldn't; if they do, why and what content is output?
  • Do component script(s) cause any JavaScript errors in normal operation? - they shouldn't; if they do, why and what content is output?
  • Can component script(s) be loaded asynchronously? - ideally scripts should be loaded asynchronously to prevent potential impact on your users' experience.
  • How many (infinitely and unconditionally recurring) timer events does the script fire per minute? - timer events such as setInterval() can cause the main browser thread to lock up in some circumstances and can cause performance issues if the functions access the DOM or perform network actions.
  • If the script tag is removed from the page (e.g. because it appears to be causing problems), what is the UI impact (on subsequent page loads)? - if you have a method of removing badly-behaving scripts, you should ensure that the customer experience is not negatively impacted. This may be relevant for a third-party that adds content to the page such as adverts or personalisation.
  • Which browsers is the component script tested in? - if the third-party can't guarantee that the script will behave well in all of the browsers you support then you can't guarantee your user experience.
  • When does the script interact with the DOM? - if the script interacts with the DOM on regular, critical or unpredictable events there may be a significant performance impact.
    • On parse
    • On the DOMReady event
    • On the load event
    • On recurring timer events
    • On user interaction events
    • On network related events
    • When invoked by a function call on JS API
  
  

• Local Storage

  • Does the script set any persistent cookies on the host domain? - this may be bad, I've seen a customer site which sends 200kB of upstream cookie data on each page load because of third-party cookies set on the host domain.
  • Does the component depend on, or modify, any existing cookies set by the host page? - it shouldn't, these are your cookies.
  • What is the maximum total size of data, in bytes, stored in cookies by the script? - depending on the domain, a large amount of upstream cookies can cause performance issues. Remember that cookies cannot be compressed on the network (at least with HTTP/1.1).
  • Does the component persist data in the browser using a mechanism other than cookies? - you should be aware if the third-party relies on localstorage or equivalent and ensure that there are no security or performance risks associated with it.
  
  

• DOM Interaction

  • If the component adds event listeners to DOM elements, does it remove them when they're no longer required?
  
  

• Content Delivery

  • Is the script (including all non-dynamic resources) cached on a CDN with global reach? - using a CDN ensures that the resources are delivered to your customers as quickly as possible and shows that the third-party understands the importance of performance.
  • Where are there servers that can accept write operations performed by the script? - if the browser needs to send data back to the third-party, having locations close to your customers ensures best possible performance and that data won't be lost due to poor connections.
  • What is the total data size in kilobytes transferred on page load? - large amounts of Javascript can cause performance issues, ensuring that the third-party only delivers necessary content.
  • What is the minimum browser-cache TTL (in seconds) of files downloaded when the component script loads (ie. Cache-control max-age)? - having good cache control on third-party assets reduces the amount of network time for repeat visits, however some third-parties will disable all caching as the scripts are dynamic.
  
  

• Analytics

  • Do the component script and all associated resources provide ResourceTiming opt-in header (ie. Timing-Allow-Origin: *)? - if you are using a real user monitoring (RUM) solution, this header allows you to measure and record detailed timings of your third-party resources. There may be security implications here though.
  
  

• Availability

  • Is the script or any web service it depends on served from a single origin (ie. is there just one origin data center)? - this could be a single-point-of-failure for the third-party.
  • In the event of a complete failure of the primary origin (eg unmitigated data centre power failure), what is the mean time to recovery (MTTR), in seconds? - this checks that the third-party are aware of the consequences of the origin becoming available.
  • How many full outages has the service experienced in the 12 months prior to today? (where full outage means a user with empty cache anywhere in the world being unable to use the service) - this gives you an idea of how reliable the service will be. Based on the impact of the service being unavailable this could be a deal-breaker.
  • Is there a public status page describing the current operational state of the script's backing services? - if the third-party is open about uptime and performance this serves as a good indicator of their maturity.
  
  

• Transport

  • What is the minimum number of HTTP requests that the script will make when invoked? - if this number is high it suggests an inefficient design, creating more network traffic than is ideal.
  • Is it possible to make ALL the script's network activity go over HTTPS? - your customers trust you with their information, third-parties become potential leaks of this information and HTTPS helps to reduce the risk.
  • Is HTTP 2.0 supported for all requests? - HTTP 2.0 is the newest HTTP protocol which enables more efficient communications. In most browsers it is only enabled over HTTPS.
  • Can the script's initial payload be bundled with the host page's own scripts? - this reduces the network overhead and delay of the initial request. It can also mitigate the risk of the third-party being unavailable and keeps the core code under your control, meaning that you manage the new third-party code releases to your site.
  
  

• Security

  • Will the component be involved with collecting user identifiable data (including any one of name, email address, credit card details, or phone number)? - any of this can be risky and may be against the law in some of your customers' countries or open you up to PCI compliance issues.
  • For European customers, does the script's backing service either a) not store any personal data, b) store personal data only within the EU, or c) store personal data outside the EU in compliance with the US Safe Harbour data protection standard? - you should ensure that your third-parties are not putting your customers' data at risk, pursuant to your (and your customers') local and international laws.
  • Does the script respect the Do-Not-Track HTTP header, either actively or by not conducting any tracking at all? - Do-Not-Track is a proposed HTTP header which suggests that customers can opt-out of cross-site behaviour tracking. If your third-party respects the DNT header then they have respect for your customers' privacy wishes.