SAVE 25% when you purchase our Theme Bundle and Make Plus together. View bundle deal.

Why we don’t use a CDN: A story about SPDY and SSL

By Zack Tollman on February 5, 2014

Fast scooter blurred

Last week we moved to a new SSL everywhere setup for this website. We were really excited to implement SSL across the board, but nervous about the impact on site performance. Therefore, we made it priority to focus on performance during the transition. Using a CDN (content delivery network) for the new site was a forgone conclusion, as we assumed it would help us speed things up. But, after testing with a few different CDNs, we uncovered some surprising results.

Before the switch

Before we talk about where the site stands in terms of performance, let’s take a look at how well the site did prior to this switch.

Pre-launch Waterfall.

There are a few things that are important to notice about this figure. First, the time to first byte (TTFB) is 205ms. This TTFB value is the DNS lookup time (84ms; in blue-green) + the initial TCP handshake (95ms; in orange) + the time it takes to receive the first HTML header response (26ms; in bright green). A TTFB time of 205ms is quite respectable and means that we are serving WordPress really quickly and our network connection is sufficient. Second, the whole site loads in 1600ms (1.6s; the blue line). We have some additional scripts that fire afterward, but those do not effect the painting of the page. Third, notice the diagonal trend of the waterfall. It’s a nice visualization of assets loading in serial. Modern browsers can make a number of connections at the same time to a single URL, but each of these require a separate TCP connection. This leads to the diagonal waterfall. Finally, notice the 6 unique patches of orange through item 16. These orange strips represent the time that it takes for the browser to connect to the server. As you can see, this happens quite a bit.

The old site will be our baseline for measuring performance on the new site. Unfortunately, we cannot make 1:1 comparisons between the sites because the two servers are hosted in different locations (New Jersey vs. Texas). In hindsight, I really wish we had these two servers in the same physical location as it would lead to much grander conclusions. Even so, I think the results I am about to present are still quite interesting.

Why SSL has performance issues

SSL has inherent performance issues. At the most basic level, SSL requires additional round-trip transfers between the server and the browser. To give you an example of the expected performance impact, in the figure above, the TCP handshake took 95ms. This is essentially the round-trip time (RTT) between the test location (Los Angeles) and the server (New Jersey). SSL negotiations require two extra round-trips between the browser and the server. As such, if we simply placed a well configured SSL implementation on our old servers, we would expect 95ms x 2 (190ms) for the SSL negotiation, thereby increasing the TTFB by 190ms. Additionally, if your server isn’t configured correctly for it, you will likely incur a lot of additional overhead as well. Ilya Grigorik documents how you are likely incurring 5 extra round-trips if you are using vanilla Nginx. Clearly, when adding SSL into the mix, careful consideration of the performance impact is important.

Getting Nginx ready for SSL

Heeding Ilya’s warnings, we jumped into manually compiling Nginx with the mainline version of the package. We used Nginx 1.5.9 (to avoid the “large certificate” bug), compiled against OpenSSL 1.0.1e (to enable NPN), and enabled Perfect Forward Secrecy (which adds even more security) all in an effort to incur no more than 2 extra round-trips in our effort to bring SSL to our site. We added the following directives to our server block for the site:

ssl_protocols             TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers               ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS;
ssl_buffer_size           8k;

The first line declares the TLS versions that we support. This allows the site to support a wide array of secure connections with browsers. The only browser that is not supported is IE6 and that was acceptable for us. IE6 only support SSL 3.0 or lower, which is susceptible to a renegotiation vulnerability. The next line allows us to choose the preferred ciphers used for the TLS negotiation. It allows us to prefer more secure ciphers over less secure ones. We decided to implement cipher suite recommendations from Hynek Schlawack, which are validated by Qualys SSL Labs. Finally, we reduced the size of the SSL buffer (the default is 16kb) in order to avoid a buffer overflow scenario that would cause additional round-trips to be needed. In the end, this implementation registered as an A on SSL Labs, an excellent tool for examining your SSL implementation.

With this server overhaul, we also made sure to implement SPDY with Nginx. SPDY comes standard in Nginx as of 1.3.15. Earlier versions of Nginx required manual compiling to support SPDY. SPDY has become the starting point for HTTP 2.0. SPDY is a transport layer that sits on top of SSL and provides HTTP multiplexing, prioritization, and server push. Our hope was that if we suffered any performance degradation due to SSL, we could make up the time from SPDY’s reported performance improvements. Fortunately, right before we launched, the SPDY 3.1 patch, sponsored by Automattic and MaxCDN, was released and we could roll into our Nginx implementation.

If you are interested in getting a similar Nginx setup, I recommend taking a look at our compile instructions for Nginx. These instructions also compile Nginx with PageSpeed and Nginx Cache Purge support.

A CDN for more speed

We have been contemplating implementing a CDN on the site for a while. We decided to make it happen with this update. Again, we thought that adding a speed improvement via a CDN could help mitigate the SSL performance issues. We had some really interesting experiences exploring a CDN provider, which we’ll write about later. Initially, we chose MaxCDN, which was really quick to set up and implement. We used Mark Jaquith’s CDN drop-in with a few customizations for https compatibility.

Performance with the CDN

With everything in place, I eagerly went to test the site.

From LA with CDN.

Did you know that Surf Office, Postmatic, Yeah Dave, and over 700,000 small businesses run their websites with Make, our free WordPress page builder. Discover the Make page builder now.

Just in case you forgot, this server is not located in the same location as our original server, so we have to be careful when comparing numbers between the tests; however, we can still draw some interesting conclusions.

The first thing to notice here is that there is indeed an extra color in our race to TTFB. The purple band in item #1 represents the SSL negotiation. We expected that this would be here. The initial connection, which represents our round-trip time, weighed in at 57ms and the SSL negotiation took 103ms. This suggests that we successfully avoided the extra round-trips that Ilya warned about and were somewhere in the range of 1-2 extra round-trips, which is quite acceptable. If all we did was implement SSL, we would have lost only 1-2 round-trips worth of performance, which is the expected result. This result alone made me think that we accomplished our performance goals

Unfortunately, there are 21 other lines to inspect in this figure and they did not look good. On the second line, SSL negotiation time is bothersome. This line represents the download of our site’s CSS file from MaxCDN’s servers. This request involves a DNS lookup (76ms; blue-green), a TCP connection (110ms; orange), an SSL negotiation (429ms; purple) and the content download (243ms; bright green). Notice that the SSL negotiation was roughly 4 times the length of the initial TCP connection. Clearly, the SSL negotiation is not well tuned. We spent a lot of time tuning our SSL connection for speed and now we are suffering from slow SSL negotiation on the CDN. This was not the result I was hoping for when we implemented the CDN.

Another thing to note about the CDN performance (items 2-13, 15-16) is that this negotiation is repeated 3 times. Additionally, we incur extra DNS lookups, which slows our asset loads. Fortunately, the last time, I believe that the SSL negotiation uses SSL resumption and it is much quicker. Finally, notice the diagonal waterfall of the elements. We are seeing the same serial download pattern that we saw in our pre-launch site tests.

With CDN farther from the server

One of the primary benefits of a CDN is placing assets physically closer to site visitors, which reduces round-trip time. As we are beginning to see, round-trip time is an important consideration. If you cannot reduce the number of round-trips, perhaps you can reduce the time they take. A CDN can help with this. As such, I decided to perform a test from Australia to see how our site performed, primarily because our Designer, Scott, lives down there and I wanted to see how things worked out for him.

From Australia with CDN.

Again, we are seeing great SSL negotiation times with our site. The round-trip time is in the 200ms range, which is similar to the round-trip time to our servers. What I was hoping for is that even though the initial time to connect to our site was poor, the CDN would help pick up some of the slack and serve the assets really quickly. This did not appear to happen in this case and was hindered by the slow SSL negotiation and serial connections.

Without the CDN

With the disappointing CDN results, we decided to take a look at how things performed without it.

From LA with no CDN.

These results blew my mind. We are seeing the great SSL negotiation times. Then, for items 2-15 (our assets), there is no additional SSL negotiation or connection times. SPDY’s HTTP multiplexing is allowing all of these request to occur over the same TCP connection. We do not need to make additional connections or SSL negotiations. All of the items on the page are loaded in just over 900ms, which is outstanding (yes, that’s faster than the old site, but the server is also closer).

Also notice, that there is less diagonal flow to this waterfall. This shape is due to how SPDY allows multiple HTTP requests over the same TCP connection. It is important to note that while using the CDN, we received no benefit from SPDY. By removing the CDN, we start to see the benefit of reusing a TCP connection and SSL negotiation.

Clearly, SPDY is providing a huge benefit for us here. While we lose the 70-90ms in SSL negotiation, we save so much time in not creating more connections due to HTTP multiplexing.

Without a CDN in Australia

The previous results were awesome and beyond our wildest dreams. Now, we needed to make a decision about the CDN. Yes, the CDN performance was not great, but perhaps it does provide a measurable improvement for Scott down in Australia.

From Australia with no CDN.

I honestly did not expect this result. I thought the CDN would be much better than SPDY from a single location. All of our assets loaded via the CDN in just under 5 seconds. It only took ~2.7s to get those same assets to our friends down under with SPDY. The performance with no CDN blew the CDN performance out of the water. It is just no comparison. In our case, it really seems that the advantages of SPDY greatly outweigh that of a CDN when it comes to speed.

It should be noted that we did try other CDNs. Unfortunately, we found similar results. We found CDNs with better SSL negotiation times and got the total asset load time down to 4.5s from Australia, but the multiplexing just worked better for us than placing assets in a closer physical location.

Caveats

In writing this article, I wanted to highlight the changes to our site with the new infrastructure that we implemented, while not overstating any of the conclusions. There is nothing scientific about these tests. In particular:

  • Our pre and post server locations were different. Not all tests were on the same hardware, though it was comparable.
  • All tests were conducted in Chrome, which is the most performance advance browser available. Roughly 50% of our visitors use Chrome, so it certainly makes the decision easier for us.
  • We did not look at a number of locations around the world. We focus on the US and Australia.
  • We only looked at the home page, which is quite important; however, other pages serve more images and assets.

Last thoughts

While the conclusion for us was to remove the CDN, it is important to note that this decision may or may not be the right one for you. For our site, we saw a huge benefit of HTTP multiplexing. Assets were delivered really fast, we avoided extraneous connections, and we could benefit from our speedy SSL negotiations. Speed is only one benefit of a CDN. In our case, speed was the only thing that we wanted from it and we did not get it.

In hindsight, it was almost silly for us to implement SPDY and a CDN. By sharding assets to another domain, we really lose all of the benefits of SPDY. It definitely was not clear to me how SPDY would benefit us, but after take a closer look, SPDY paid off in a major way. It was hard to find a CDN provider that used SPDY. It will pretty amazing when you can combine SPDY and CDN benefits, which is theoretically possible with a CDN provider that provides SPDY and allows SSL on CNAMEs.

As a developer or sysadmin, it is important to measure things that can be measured and make decisions based on these measurements. If we would not have tested the CDN performance, we would still be using it thinking that we were seeing a performance boost, not to mention, paying for a service that was not giving us the intended benefit. We are excited that this investment in Nginx and SPDY has paid off for us!

Enjoy this post? Read more like it in From the workshop.

49 Comments

  1. Justin Dorfman

    As much as I would love to discredit this article, it brings up great points. One project our engineers are working on is TLS optimization. We have received great feedback from Ilya Grigorik and Andy Davies on what we should be doing to optimize our SSL stack.

    As far as SPDY support, we (MaxCDN) just co-sponsored SPDY/3.1 development (with Automattic) for Nginx (http://nginx.com/news/nginx-announces-support-spdy31/) and plan on deploying it to each edge by the end of Q1.

    We are sorry to see you go as a client, but appreciate your efforts on making the web faster (not to mention our service). No hard feelings ;)

  2. Zack Tollman

    Hi Justin!

    Thanks for the comment! There are lots of ways to discredit our findings (it was far from scientific) and I hope that it was clear that we are not advocating this approach for everyone; rather, we found a lot of benefit with SPDY alone and it seemed to be the best decision for us.

    I will update the article to mention your support for SPDY. I was aware of that, but did not mention it.

    My dream is definitely a CDN with good SSL negotiation times, as well as SPDY support. When you guys have that implemented, we will be taking another really close look.

    What I think was shocking for us is the paradigm shift from trying to shard assets to different domains to keeping everything on a single domain. Basically, by keeping everything on the same domain, we get so much benefit from SPDY.

    For what it is worth, MaxCDN was really great to use. It was a fast and easy set up process for an amazing price.

  3. Ilya Grigorik

    Zack, great writeup! A couple of quick notes…

    1) You may also want to enable OSCP stapling (which nginx supports) to avoid extra latency for OSCP checks.
    2) Nginx SPDY implementation disables header compression by default, which is unfortunate, and I hope that this will be addressed in one of the upcoming releases. I recommend setting spdy_headers_comp to “6”.

    The extra TLS handshake latency with MaxCDN is definitely something worth investigating and pinging them about. They’re a responsive team, and I’m sure they’d appreciate the feedback from one of their customers.

    Finally, the most important bit… Assuming the handshake latency concerns are addressed, your best results will actually be by using CDN *and* SPDY. It’s a common misconception that CDNs are only useful for static content – that’s not the case. Routing dynamic (non-cacheable) requests through a nearby CDN edge node reduces your RTT to negotiate the TCP+TLS connection and will deliver big savings — see [1] for more. The best case scenario is: terminate TLS at an edge node close to the user, which is also SPDY-enabled, and then use the same origin / connection to fetch static resources (aka, avoid sharding).

    [1] http://chimera.labs.oreilly.com/books/1230000000545/ch04.html#TLS_EARLY_TERMINATION

  4. Zack Tollman

    Thanks for the awesome input, Ilya!

    Regarding 1 & 2, I really appreciate the pointer. I’ll definitely look into implementing those changes.

    I’ll be sure to get in touch with MaxCDN. In fact, they left a comment before yours (which you didn’t see prior to your comment due to a caching issue on our end).

    Regarding your last suggestion, I totally agree. Our CDN integration was the last part of this puzzle. We focused a lot on the SSL piece and getting that right. There is a lot of tweaking involved to get SSL both secure and performant. Given that we have a decent sized international audience, that will be an important thing to look into next.

    Thanks again for all of your work on making the web faster! Your resources were a huge help while working on this transition.

  5. Mark Gavalda (@MarkGavalda)

    This is a great article with so much information! To be honest I haven’t really looked at the details of the scripts you’re including from other domains, but (and I know that’s not the perfect solution) if you mirrored those scripts locally (and thus took advantage of SPDY) it would mean that viewers from Australia would see your site in less than 3 seconds… That’s awesome! And less than 1 sec in the US! :-)
    Anyway, thanks for this detailed article!

  6. Zack Tollman

    Hi Mark!

    You are absolutely right! I did not mention the other 3rd party scripts in the waterfall. You’ll notice that they are performing terribly. For most of them, the SSL negotiation times are not so good. Fortunately, all of these scripts are analytics related, none of which cause layout or paint events. This essentially means that the page is fully loaded by the time that the assets from our domain are loaded. The other scripts just add some event listeners in the background.

    Great observations!

  7. Kai Armstrong

    This is a really awesome post with some great stuff in it. Thanks for the super detailed write up, it’s starting to make my head spin with ideas.

    I am curious about the wp-stack-cdn.php drop in. What modifications did you make for SSL support? Something you’d be willing to share/contribute back?

  8. Zack Tollman

    Hi Kai!

    Thanks for the compliments! I’m glad you enjoyed this write-up.

    Regarding wp-stack-cdn.php, it is quite easy to update it for HTTPS support. In the script, all or your site URLs are rewritten to a new URL of your choice. This rewrite is hardcoded to rewrite the URL to an “http” address, not “https”. As such, it is easy to change this rewrite to support https. I contributed to an issue on this topic with a suggested patch.

  9. Kai Armstrong

    @Zack, That’s awesome… that happens to be why I was curious… hadn’t seen your comments in the github issue.

  10. John Blackbourn

    Great article Zack. What tool are you using to generate those graphs? I usually use Charles Proxy but its timeline doesn’t go into that level of detail.

  11. Zack Tollman

    Thanks John! I use WebPageTest. It gives a lot of detail and much more than what I have shown here. Additionally, you can use the tcpdump that it prepares and get even more details about the connections by uploading it to CloudShark.

  12. Phil Pirozhkov

    Hi

    You’re doing most of the things wrong.
    1. Google analytics script in , should be in the bottom of
    2. Use ‘s anync property
    3. Minify your HTML
    4. [“identify”,”track”,”trackLink”,”trackForm”,”trackClick”,”trackSubmit”,”pageview”,”ab”,”alias”,”ready”,”group”] … t.src=(“https:”===document.location.protocol?”https://”:”http://”)+”d2dq2ahtl5zl1z.cloudfront.net/analytics.js/v1/”+e+”/analytics.min.js”
    11 separate JS files?
    5. “DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES” and PFS? How?
    6. TLSv1 TLSv1.1? RSA+3DES?
    7. SPDY 3.1 is natively supported in ngnix 1.5.10

    There are plenty of things to improve in your setup.

  13. Zack Tollman

    Hi Phil!

    I really appreciate your feedback and agree with your assessment that there are plenty of things to improve.

    Items 1-4 are on our radar, but were not a priority in this recent set of changes. We definitely can make some gains here.

    I would love to hear more about items 5 and 6. The SSL Labs tool is verifying that we have PFS set up correctly. If we are doing this incorrectly, I would love to hear more about it.

    Regarding item 7, yep! It was released about a week after we did our relaunch and a day before publishing this post. We will be upgrading soon.

  14. Phil Pirozhkov

    First of all thanks for the investigation an writeup, looking forward to seeing how it goes on when you make it perfect.

    5. I’m far from being an expert in this area, but from those that i remember, DHE (Ephemeral Diffie-Hellman) is required to keep the exchanged key temporary and undiscoverable in the future. However I doubt that this is your primary concern taking into account the requirement to support not only edge browsers.
    6. 3DES is considered insecure, GCM is only available in TLS1.2. So most of the tunnels being made will use potentially insecure ciphers. However, same as for 5 applies here (browser support).

  15. Zack Tollman

    Hi Phil!

    Thanks for the follow-up!

    5. I took a closer look at our setup given your criticisms. Using DHE and ECDHE and according to SSL Labs, we cover all modern browsers, with the only one missing being IE8 on XP. This is something I need to look into as I previously missed that it was not covered by FS. Overall, it seems that we are doing a good job with FS.

    6. Can you point to a reference about 3DES being insecure? I am having a tough time finding good information on this. I would like to read about it. Also, FWIW, no browsers are even using 3DES to negotiate the connection. That said, I’m sure there is a way to force that, but for the casual visitor, she should still get the benefit of FS.

    I am looking to continue to improve the ciphers used. Any insights are much appreciated!

  16. Zack Tollman

    Hi Nick!

    Thanks for the comment! That’s awesome that you guys have such a great looking CDN service with optimized SSL/SPDY support.

  17. Jacob Share

    This has been a terrific post coupled with equally good comments. With your help & insights, mainly, I enabled SPDY on my nginx-served site and optimized SSL to an A+ rating from SSL Labs while having lower pageload times than what I had before over http but with a CDN.

    Using WebPageTest, I confirmed what was said above i.e. that until MaxCDN enables SPDY on their end, my site is better off not using it all.

    However, here’s my question – does anyone know how to configure Nginx so that when it sees a browser that is not SPDY-compatible, at that point it would serve content from the CDN?

  18. Zack Tollman

    Hi Jacob!

    I’m glad that you found the post helpful!

    I have not done this myself, but, supposedly, the PageSpeed module for Nginx and Apache allows you to changed on-the-fly rewrites based on whether the browser is SPDY enabled or not. Given that the PageSpeed module can do things like concatenate and minify files for you, you can set up some smart optimizations for SPDY and non-SPDY browsers. For instance, you can serve non-concatenated files for SPDY browsers and concatenated files for non-SPDY browsers.

    I hope to get PageSpeed set up on this site at some point. If I’m successful, I’ll be sure to let you know how it goes.

  19. Jacob Share

    Hey Zack
    I spent a full day testing ngx_pagespeed last week. I was pretty excited too, there were lots of options that I wanted to take advantage of – progressive jpegs, dns prefetch, trimmed urls, server-side lazy loaded images, and more. But again according to WebPageTest, ngx_pagespeed usage added (on the average) anywhere from half a second (for my homepage) to a full second (for articles) in page load times. After spending extra time to be sure that the results were conclusive, the only explanation I could come up with – which doesn’t mean it’s the right one – was that ngx_pagespeed usage increased nginx memory usage (over double in my case, according to Munin) to the point where my server needed to start using a hard drive-based swap file. I’m on AWS, so I wouldn’t expect this to have such an impact but otherwise, all the pagespeed filters I used were ‘low risk’ ( = unlikely to increase load times, according to the Pagespeed docs) so I’m not sure what else it could be. Disappointing to say the least, and I’ve since deactivated ngx_pagespeed.

    I’m definitely curious to hear how you fare with it in a completely different hosting environment.

    Do you happen to remember where you saw the conditional handling directives in the pagespeed docs?

  20. Jacob Share

    Upon closer examination of Munin, it does NOT appear that ngx_pagespeed significantly increased nginx memory usage, which I’m happy to report. But now I really have no clue why it actually increased page load times in my case.

  21. Zack Tollman

    Hi Jacob!

    It looks like I looked at that documentation too quickly. You are correct that the SPDY conditional is Apache only :(

    Again, I have yet to venture into PageSpeed beyond just reading about it, but I wonder if you have caching set up correctly. If the TTFB is higher when PageSpeed is enabled, this could be the case.

  22. Jacob Share

    Checked my testing notes and saw the TTFBs didn’t change significantly with or without PageSpeed enabled.

    Since I still couldn’t explain what happened the first time, I decided to test some more because as good as WebPageTest is, testing over the Web in small samples is far from scientific and environmental conditions really can make a difference.

    This time around, with the same set of filters enabled as above while being a little more rigorous with my change management: PageSpeed consistently decreased First View pageload times significantly, while often increasing Repeat View pageload times albeit much less significantly. Since Repeat View times were already much lower than the First View times, and considering that every visitor has a First View but not a Repeat one, this is a tradeoff I’ll happily take, so I’m leaving PageSpeed enabled.

  23. Zack Tollman

    Thanks for sharing your experiences Jacob! It sounds like you’ve set up a nicely tuned system.

  24. Jacob Share

    Happy to give back :)

    One last thing, and this may be important for some people: Even at 10x more traffic, Munin’s CPU usage graph is much more stable than what I’m seeing since I enabled PageSpeed, and I’m using a (virtual) dual-processor c1.medium AWS EC2 instance. This leads me to think that some of the PageSpeed filters, perhaps low risk from a pageload point of view, are CPU-intensive (perhaps the ones that convert image formats on the fly?). Unfortunately, the PageSpeed docs don’t seem to mention how CPU-intensive the individual filters are. I’ll be watching this over the next few days and if this continues, I’m going to have to figure out which ones are causing the most ruckus and then deactivate at least some of them.

  25. Jacob Share

    After spending a few days keeping an eye on Munin graphs and enabling/disabling various filters, there’s no question that some filters are much more CPU-intensive than others and this information is sorely lacking in the docs. I didn’t rigorously test each filter, or even each of the filters that I wanted enabled, but it didn’t take much to show that the many of the image-related filters need extra juice. My solution was to offload most of the image optimizations I wanted to the EWWW Image Optimizer plugin.

    Although I haven’t tested this yet, someone on Reddit mentioned that there’s a $spdy variable you can check for in the nginx config and decide on that whether to serve from a CDN. Here’s more info if anyone would like to try.

  26. Zack Tollman

    Hi Jacob!

    That is really interesting. Given all that PageSpeed is doing, it is not a huge surprise that there is a CPU cost. I imagine that in larger setups, you’d ideally have a few servers dedicated to this alone. Are you caching the output of the PageSpeed rewritten pages?

    I think you should write up a blog post with your findings. It would be super helpful for others considering doing such work. It might also stir up some experts on the topic.

  27. Jacob Share

    I agree with you- in a bigger setup, you would definitely want to have dedicated resources for rendering images and priming cache in various ways.

    I’m using W3TC for caching, and I guess I assumed that it was taking into account PageSpeed’s rewrites but I haven’t checked, to be frank. Is that what you’re referring to?

    I thought about blogging what I’ve done but I’ve spent so much time on the tuning, more than I estimated, that a few more hours to compile it and write it all up doesn’t appeal to me just yet. It would also be way off-topic on my site, but maybe in a few weeks when I’m comfortable saying that I’m done tweaking :)

    Some useful resources I’ve found in the meantime:

    1) SPDY Best Practices, from Google

    2) ngx-pagespeed-discuss Google group. Not overly active, but definitely looks to be a good place to ask a question.

  28. KJ

    Great write up thank you. I gave your nginx compile script a shot but couldn’t make it work.

  29. Zack Tollman

    @KJ – I see you posted a more specific message on the Gist, so I will respond over there.

  30. David Edwards

    Thank you for this fantastic write up and discussion. Being a small regional Genesis developer based in Australia I have great interest with what’s in the discussion.

    I currently use Namecheap, CloudFlare and MaxCDN for my performant clients but have been investigating Synthesis for SPDY and Genesis hosting, but was curious if I’d have to chuck my investment in MaxCDN due to SPDY support. Searching lead me here.

    Seems the picture is clear now, but soon Max will be updating.

    Realistically I’d be doing this for my own site first which hardly receives any traffic but deciding that I want to develop on Genesis pretty much exclusively going forward I think it best to test Synthesis, SPDY, etc. I do have a pretty heavy RamNode VPS but realistically I don’t have the time nor security knowledge to trust client installs to it.

    Again thanks for the discussion.

  31. Zack Tollman

    Hi David!

    Thanks for stopping by!

    but was curious if I’d have to chuck my investment in MaxCDN due to SPDY support. Searching lead me here.

    I’m not sure that this is the right conclusion for everyone. For us, it makes sense as we consider performance and cost. Ideally, as Ilya suggested earlier, moving our SSL termination to the edge would be the ultimate performance improvement, but to do so would be very expensive, and significantly increase complexity and maintainability. For our business, this is unlikely to have any measurable impact, so it makes sense for us to just use SSL and SPDY from our own servers.

    Good luck with Synthesis! I hope it goes well for you!

  32. Vidyut

    I can verify this indepedently. I just moved my blogs to https+spdy and was horrified by the slow speed. One accidental reading without the CDN opened my eyes to the obvious. Went around searching for more information and found your blog. Clearly, spdy beats cdn – unless perhaps it was a really huge site with many heavy assets and a CDN with spdy too or something. I don’t know, just trying to imagine when a CDN would be useful even if you have spdy – only case I can think of is if download times would still be too great or something. Not sure. No particular reason to want to imagine this, just that biggies like Facebook, Gmail and Twitter seem to use CDNs anyway.

  33. Zack Tollman

    Hi Vidyut!

    I’m glad you found this post informative.

    Keep in mind the comment from Ilya above. The ideal setup would be to have your SSL termination on the edge network, which would allow reduced latency due to locating the servers closer to the clients. This would then allow the initial request to be served via the same domain as the other assets via a SPDY connection. This setup would tie together geolocated servers and SPDY, which ultimately reduces latency in a number of ways. Unfortunately, such a setup is very costly at this point, but I imagine it is something we might see more of in the future.

  34. Vidyut

    Hi Zack,

    I did read the comment by Ilya. It is too much for a small blogger, IMO (which would be me), though I guess once the speed bug bites, the fun is in the chase :D

    Another thing I wanted to ask is that your article mentions server push. The spdy module page says that server push is not yet supported. What did I miss? I’d like to use that if there is a way.

    Vidyut

  35. Zack Tollman

    Hi Vidyut!

    You are correct. SPDY 3.1 support server push, but the Nginx implementation does not yet support it. I’m hoping a future update to Nginx will enable server push as it sounds like something fun to experiment with.

  36. Alexis

    Zack great read! Thanks. Its given me some food for thought. I was wanting to implement SPDY & CDN, mainly for an exercise. But didn’t consider loosing SPDY by using a CDN (duh).

    My 2 cents but slightly off topic. Im thinking to export all image assets twice as jpg|png & webp. Considering your main user base is Chrome (I think thats the case for most people these days) they could take advantage of the compression webp graphics offer. I’m running a very image heavy site so its a higher priority for me, and I’m already dynamically loading, so simpler to feature detect.

    spdy + webp = ftw

  37. Zack Tollman

    Hi Alexis!

    I’m glad that you enjoyed the article! I think the underlying point is that there is no one right answer when implementing performance improvements. At a minimum, you must look at the impact your changes have on the site.

    WebP can be a huge win. The biggest downside is the build process, especially when you run a CMS. How do you create the WebP images? Fortunately, Google has created the PageSpeed module that can do this on the fly! It’s worth checking out if you want to support WebP.

  38. Alexis

    Hi Zack,
    Thanks for the PageSpeed module tip. I had herd of it but hadn’t looked into it.

    I have been storing images in Mongo then resizing and extracting thumbs into a web directory. It sounds like a strange set up, but using graphicsmagic I can extract all the Exif meta and then run mongo queries on it. I find it easier than maintaining a file pointer in the DB.

    PageSpeed may save me having to extract the image twice, I wonder if it’ll be faster exporting them first though? If I ever get round to trialling it out I’ll let you know :)

  39. Rai

    I was thinking to move my site to a CDN… without even to measure this, now that I read this article I will test my site before and after as you did and I will see what happend.!

    Thanks a lot.

  40. I.Adam

    Hi, I suggest you to use DNS less mode of PaceCDN.com. It is something interesting to see how they eliminated DNS resolution time at client side. Also it provides a solution to handle all DNS caching issues of typical CDNs.

  41. budyk_ir

    im stil confused while others suggesting to use CDN. but your post gave me a closer thought what to decide, thanks

  42. Zack Tollman

    @budyk_ir – A CDN generally has significant performance advantages. For our number and location of visitors, taking advantage of HTTP multiplexing with SPDY seemed to be the better choice for us for now. This may change in the future, but is right for now. Please keep in mind that this article is just an account of our decision, it is not a recommendation that you shouldn’t use a CDN. We are encouraging you to take a look at your individual concerns and think about how a CDN may affect your site.

  43. Robert Payne

    Did you trial using other CDN’s besides MaxCDN? There’s a lot of providers out there with very different implementations.

    Fastly is an up and coming CDN that has amazing TTFB even over SSL. Here in New Zealand the SSL termination with the CDN is done in 20-30ms.

  44. Chris

    Hi, Zack. This is a great article and is very important in the area of both security and speed. I ended up doing some research myself and was amazed at the results. Notice the difference between HTTP and HTTPs, which uses SPDY: http://www.httpvshttps.com

  45. Vignesh S

    Great writeup.

    Would like to know how you have routed the users to http for non-spdy enabled browsers?.

Comments are closed.