Upgrading to HTTPS: common issues and solutions

We like to keep our clients at Daylight in the know on trends in web development. In March, Alex wrote about changes coming to Google Chrome making the user more aware of insecure sites through labels. This post offers a follow-up deeper dive into the world of HTTPS. I will cover the trend of more and more sites moving to HTTPS and why they are choosing to migrate for reasons of confidentiality, integrity, and authenticity. I will go over the basics of how HTTPS works and the issues of cost, complexity, and compatibility, as well as volunteer a few simple solutions by using services like Let’s Encrypt along with Certbot, and ensuring security through the use of Content Security Policies and HTTP Strict Transport Security. I hope you find it informative and useful as you consider your site’s migration to the HTTPS protocol.

The Trends

• The average daily volume of encrypted internet traffic now has surpassed the average daily volume of unencrypted traffic (Wired 1/30/17).
• Over 30% of the world’s top 1,000,000 sites are using HTTPS (Scott Helme 8/17).
• Major browsers are demonstrating their commitment to a more secure web through the use of labels and warnings to educate users on a site’s safety.
• Google began factoring page load speed into its ranking algorithm.  HTTP/2 makes for a faster web but can only be served through a HTTPS connection which actually makes encrypted sites faster than unencrypted ones.

Why Migrate to HTTPS

Moving your site to HTTPS makes sense for one simple reason: Man-in-the-Middle attacks.

A Man-in-the-Middle (MitM) attack occurs when someone intercepts your internet traffic. These attacks can happen at a variety of insertion points and are often malicious. One of the more common places for MitM attacks to occur is on public wifi locations between the router and your browser (think coffee shops, hotel lobbies, airports, Airbnb). A wifi router isn’t the only insertion point though. Web traffic can also be intercepted between the router and an Internet Service Provider (ISP) or between an ISP and a web server. There are a lot of ways internet traffic can be intercepted and that is one of the major reasons why it is crucial the internet move towards a more secure protocol. 

Communication served over HTTP is not encrypted which means that if someone is able to intercept it through a MitM attack, they are able to see all the communication between your browser and the site you are visiting and can even change the information that is going to your browser. 

The move toward HTTPS is about doing more to ensure Confidentiality, Integrity, and Authenticity in your web experience.

Confidentiality

When you use the web to access your bank accounts, email, and social media, you expect privacy and confidentiality. You assume your communication with the page or app is understandable to just your browser and the server providing the service you are using especially on sites where you enter a lot of personal details and that to an outside observer it would just look like gobbledygook. A web page served over HTTP does not provide this. Anything served over HTTP is subject to the possibility of other people viewing it if your communication suffers a MitM attack. A good example of this occurred back in 2010 when Facebook wasn’t served over HTTPS and people were intercepting other’s Facebook traffic while they were at Starbucks on the wifi. A developer made an extension for Firefox called Firesheep which allowed anyone, not just the technically literate, to hijack a user’s session cookie and access their Facebook or Twitter accounts as if they were logged in as the user. 

Integrity

Integrity of a website or web application means the interactions of a user on a site do only what they expect them to do and what they have approved of in the terms of service and cookie policy. When you are on a website and interact with the content on that web page you expect the application or site to have integrity. You expect when you click on an edit button it edits something you’ve posted on the page or if you click submit you expect it to submit the data you just entered into that company’s database and only that. If a page is attacked with a MitM and your connection is compromised, a site communicating using HTTP can lose its integrity.  The submit button may still post your information to the company database for your order but also may be installing malware into your system or adding malicious cookies to track information about the sites you visit and your personal details. 

Authenticity

Authenticity is different than confidentiality. Authenticity means your browser is in fact communicating with the website you think it is and not a spoof. When you fill out a form on a website and click the submit button you expect it to submit the information you just entered into the browser to the company whose page you are on. If your communication is compromised hackers can intercept that pushing of information and route it to other places. Here’s how that could work. You are on a bank website and say their login page is served over HTTP before routing you to a HTTPS site once you are logged in. If someone hijacks your communication they could prompt you to re-enter personal details about yourself on another page that looks very similar to the bank you think you are visiting and be able to then go and access your account on their own.

How HTTPS Works

“By HTTPS we mean HTTP over SSL, and by SSL we mean TLS” – Etsy

When a browser (the client) visits a site (the server) and the site is served over HTTPS the first thing that happens is a TLS handshake. TLS stands for Transport Security Layer which improved upon Secure Socket Layer (SSL) that was developed in the late nineties. Both are cryptographic protocols and when you hear about SSL most of the time people are actually referring to TLS. Anyway, back to the handshake. The first thing that happens is the client and server talk to each other and figure out how they are going to communicate securely (Highest Version of TLS, Cipher Suites, etc). The Server responds to the initial client request for information with its preferences and the public key which the client (browser) can check against it’s list of Certificate Authorities to see if the server really is who it claims to be. Once this is found the client can perform a key exchange with the server encrypted with the server’s public key and they can start communicating with encrypted traffic.

The Issues: Cost, Complexity, Compatibility, and Confidence

The dangers of MitM attacks make you think everything should be served as HTTPS. The risks are too great. But why do companies hesitate in adopting the standard?

Cost
A perceived limitation for businesses looking to move to HTTPS is the cost of acquiring CA certificates and having to renew them. This can become quite expensive for large sites that require many certificates and that is why early on it was mostly companies in the financial sector who needed the security and could afford the cost. As the web has moved more and more to being served over HTTPS the cost of acquiring certificates has gone down substantially and even free options for acquiring and renewing certificates like Let’s Encrypt have sprung up.

Complexity
Another perceived limitation closely related to cost has been complexity. Enabling HTTPS on your site involved figuring out how to obtain the certificate, making sure it was in the appropriate format for being configured to your server, converting it into that format if necessary, and actually configuring your server, etc. And all this had to be done with each certificate renewal. As you can imagine this can take time and time equals money.

Compatibility
There is also the issue of compatibility. When you move a site to HTTPS that was originally written for HTTP, oftentimes the way you reference resources such as images, CDN’s, scripts, embedded videos, and advertisements will flag a mixed content warning because the content is still being referenced as HTTP and the browser knows that is insecure. 

Confidence
Finally, there is the issue of confidence. Will your efforts actually prevent man in the middle attacks? Are there still weaknesses to your site that allow someone with intent the opportunity to spoof, snoop, or inject malware?

Solutions:

Cost and Complexity: Let’s Encrypt and Certbot

So you’ve heard of the issue of cost and complexity when moving a site to HTTPS. Here are a few services to make it less painful.

The first question you need to ask yourself is, “Do you have shell access to the hosting environment you need the cert on?”  If so, Let’s Encrypt and Certbot are for you.

Let’s Encrypt is a free automated, and open certificate authority run for the public’s benefit. What Let’s Encrypt offers is a way to acquire digital certificates for free. It is sponsored by most of the major tech companies and run by the Internet Security Research Group.  Let’s Encrypt has issued over 100 million digital certificates and is fast on its way to passing Comodo and being the top provider of digital certificates. 

Free is great. Automation is even better. 

Certbot is an Automated Certificate Management Environment (ACME) client provided by EFF to run on the target environment and communicate with Let’s Encrypt to validate the domain registration process. It automates the process of acquiring, installing, and renewing the certificates which is very important because while Let’s Encrypt certificates are free they are only valid for three months. By making the renewal process automatic, the duration of the certs isn’t as relevant.

But what if you don’t have shell access?

There are a number of hosting services that offer support for Let’s Encrypt including WordPress, Squarespace, Wix, Dreamhost, etc. To see a full list of hosts click here. With these providers it is easy to add Let’s Encrypt certificates within the hosting environment managers.

Compatibility: Changing Resource Paths and Content Security Policy

A layered approach is recommended to fix the hang up of mixed content like images or videos that are referenced in the HTML as HTTP when we are trying to serve the site securely.

First, go through all of your URLs and change their reference to relative paths if they are internal links or HTTPS paths if they are linking to outside resources like a youtube video or Amazon S3 bucket. This is the safest thing to do but also can take the most time to double check, especially on a large site. The common wisdom used to be moving cross-origin resources (resources not on your domain) to protocol relative paths like ://www.youtube.com as this was a way to ensure against attempting to pull content in through HTTP on a HTTPS site, but there are a few reasons this is no longer what is most recommended. The first is speed. If you haven’t managed to lock down all your redirects and someone accesses your site using HTTP in an old browser that doesn’t allow for Content Security Policies they will use the protocol relative to the path they are on, which in this case is HTTP. Since HTTP/2 (the faster protocol which allows for concurrency) only can be used in HTTPS, the speed of your site load for that user will decrease. If you are curious at just how much of a difference HTTP/2 can make check out httpvshttps. The other reason is security. At every point we can load a resource through HTTPS we should as it reinforces the principles of confidentiality, integrity, and authenticity of users’ web experience. Mixed content warnings are a one way street only being announced on HTTPS sites and not the other way around. 

Next, add a Content Security Policy as a meta tag in your headers like this:

<meta http-equiv=”Content-Security-Policy” content=”upgrade-insecure-requests”>

The browser will be instructed to go get resources securely if you still have insecure (http) links to outside resources on your site. What this practically means is you will not lose your green padlock for links that have not been updated and you will not have areas on your site where there is no content as long as there as those resources can be accessed using https. The reason you still need to do step one and can’t completely rely on the Content Security Policy is that Content Security Policies are only supported in the newer browsers and anyone viewing your site on an older or non-compliant browser would have a broken padlock and lots of security warnings.

Confidence: HTTP Strict Transport Security (HSTS) / Pre-load HSTS

When migrating a site from HTTP to HTTPS the first thing you’ll do is set a permanent redirect of your web traffic from http://www.yourcompany.com to https://www.yourcompany.com. You do this because all web browsers currently default to the insecure protocol if someone just enters www.yourcompany.com into their browser. Unfortunately, your site is still at risk of a MitM attack. When you set up a permanent redirect on your site (Status Code 301) the response header contains the location of where the redirect is pointing to. Someone could intercept the redirect and reroute a visitor to a spoofed site. The way to protect against this is HTTP Strict Transport Security (HSTS). HSTS does a few things for your site. The way it works is once your browser sees the HSTS response header it will not make an insecure request to that domain for the length of age of the Strict-Transport-Security. There still is the weakness of the very first use however. The very first time your browser goes to the domain it will be insecure. The way to protect against this is by registering your domain to preload HSTS at Hstspreload.org. Once accepted, this will add your website to the HSTS pre-load list which is hard-coded into the major browsers. When your site’s HSTS preloading is built into the browser no one will be able to load the site insecurely. Make sure you are committed to staying delivering your site via HTTPS as this is a not a small step to take.

Conclusion

Moving your site to HTTPS requires a bit of work. It’s not as easy as clicking a button and it does involve some thought as developers work on your site, but it’s not anything to shy away from. Certificates are getting less and less painful to acquire, install, and automatically renew.  Ensuring compatibility with outside content can easily be accomplished with a knowledgeable team of developers using Content Security Policies and ensuring correct url paths. With everyday users primarily accessing the internet through their mobile devices and laptops using a wide range of wifi routers it is imperative we do more and more to move the web to HTTPS by default. 

At Daylight, we’re here to help. We have a team of strategists and developers ready to assist you with any questions you have about implementing HTTPS on your site. Let us know how we can assist you in making your website part of the trend of a more secure user experience.