Random thoughts

Web companies, big and small, face the same challenges. Our pages must be fast, our infrastructure must scale up (and down) efficiently, and our sites and services must be reliable … without burning out the team.
Velocity Europe conference Website

Three years after its inception in California O’Reilly’s Velocity Web Performance and Operations Conference finally made it to Europe. Some 500 people, web developers, architects, system administrators, hackers, designers, artists, got together at Velocity Europe in Berlin on November 8 and 9 to learn about the latest developments in web performance optimization and managing web infrastructure, exchange ideas and meet vendors in the exhibition hall.

Velocity Europe was well organized and run. There were power strips everywhere and a dedicated wireless network for the participants, although the latter barely handled the load when everyone was hogging for bandwidth. Seeing bytes trickling in slowly at a performance conference was not without irony. Some things never change: Getting connected sometimes requires patience and endurance. Back in the days I was volunteering at the W3C conferences preparation involved running cables and configuring the “Internet access room”, only then contention for network resources meant waiting for an available computer.

As expected for a techie conference, about the only people wearing jackets and ties were the AV operators, food was plentiful and good, and the sponsors handed out T-shirts, caps, and other give-aways. Plenary sessions were recorded and streamed live, and #velocityconf on Twitter also has a good collection of facts and memorable quotes for those who couldn’t attend in person.

Steve Souders and John Allspaw led through two busy days packed with plenary sessions, lighting talks and two parallel tracks on Web performance and Web operations. While bits and bytes certainly mattered to the speakers and the audience, the focus was clearly on improving the Web experience for users and the business aspects of fast and well-managed Web sites.

The conference started with a controversial talk about building a career in Web operations by Theo Schlossnagle, and I couldn’t agree more with many of his observations, from suggesting discipline and patience (and recommending martial arts to develop those virtues), learning from mistakes, developing with operations in mind to seeing security not as a feature but a mentality, a state of mind. Along the same lines, Jon Jenkins later talked about the importance of dev ops velocity, why it’s important to iterate fast, deploy fast, and learn from mistakes quickly, mentioning the OODA loop. Some of the Amazon.com deployment stats are just mind-boggling: 11.6 seconds mean time between deployments, and over 1,000 deployments in a single hour to thousands of hosts.

Joshua Bixby addressed the relationship between faster mobile sites and business KPIs. Details of the tests conducted and the short-term and long-term effects on visitor behaviour are also available in his recent blog post about a controlled performance degradation experiment conducted by Strangeloop. Another interesting observation was the strong preference of customers for the full Web sites over mobile versions and native apps: One retailer in the U. S. found that of the online revenue growth for that company was driven by the full site. 35% of the visitors on their mobile site clicked through to the full site immediately, 24% left on page 1, another 40% left after page 1, and only 1% bought something.

Performance also matters at Betfair, one of the world’s largest betting providers. Doing cool stuff is important too, but according to Tim Morrow’s performance pyramid of needs that’s not where you start:

1. It works.
2. It’s fast.
3. It’s useful. (I personally have a slight preference for useful over fast.)
4. It’s cool.

Jeffrey Veen of Hotwired, Adaptive Path, TypeKit fame kicked off the second day with an inspiring talk on designing for disaster, working through crises and doing the impossible. I liked the fancy status boards on the walls, and the “CODE YELLOW” mode, the openness and the clear roles when something bad happens. And something bad will happen, as John Allspaw pointed out: “You will reach the point of compensation exhausted, systems, networks, staff, and budgets.” A helpful technique for planning changes is to write down the assumptions, expectated outcomes and potential failures individually, and then consolide results as a group and look for discrepancies. If things still go wrong, Michael Brunton-Spall and Lisa van Gelder suggested to stay calm, isolate failing components, and reduce functionality to the core. Having mechanisms in place to easily switch on and off optional features is helpful, down to “page pressing” to produce static copies of the most frequently requested content to handle peak loads.

Several talks covered scripting performance and optimization techniques. Javascript is already getting really fast, as David Mandelin pointed out, running everything from physics engines to an H.264 decoder at 30 fps, as long as we avoid sparse arrays and the slow eval statements and with blocks. Using proven libraries is generally a good idea and results in less code and good cross-browser compatibility, but Aaron Peters made the point that using jQuery (or your favorite JavaScript library) for everything may not be best solution, and accessing the DOM directly when it’s simple and straightforward can be a better choice. Besides that, don’t load scripts if the page doesn’t need them – not that anyone would ever do that, right? – and then do waterfall chart analysis, time and again. Mathias Bynens added various techniques for reducing the number of accesses to the DOM, function calls and lookups with ready-to-use code snippets for common tasks.

For better mobile UI performance, Estelle Weyl suggested inlining CSS and JS on the first page, using data: URLs and extracting and saving resources in LocalStorage. Power Saving Mode (PSM) for Wi-fi and Radio Resource Control (RRC) for cellular are intended to increase battery life but have the potential to degrade perceived application performance as subsequent requests will have to wait for the network reconnection. Jon Jenkins explained the split browser architecture of Amazon Silk, which can use proxy servers on Amazon EC2 for compression, caching and predictive loading to overcome some of these performance hogs.

IBM’s Patrick Mueller showed WEINRE (WEb INspector REmote) for mobile testing, a component of the PhoneGap project.

Google has been a strong advocate for a faster Web experience and long offered tools for measuring and improving performance. The Apache module mod_pagespeed will do much of the heavy lifting to optimize web performance, from inlining small CSS files to compressing images and moving metadata to headers. Andrew Oates also revealed Google’s latest enhancements to Page Speed Online, and gave away the secret parameter to access the new Critical Path Explorer component. Day 2 ended with an awesome talk by Bradley Heilbrun about what it takes to run the platform that serves “funny cat videos and dogs on skateboards”. Bradley had been the first ops guy at YouTube, which once started with five Apache boxes hosted at Rackspace. They have a few more boxes now.

With lots of useful information, real world experiences and ideas we can apply to our Websites, three books signed by the authors and conference chairs, High Performance Web Sites and Even Faster Web Sites, and Web Operations: Keeping the Data On Time, stickers, caps and cars for the kids, Velocity Europe worked great for me. The next Velocity will be held in Santa Clara, California in June next year, and hopefully there will be another Velocity Europe again.

Related links

• Velocity Europe conference 2011
• Videos and presentations
• #velocityconf on Twitter

Photo credit: O´Reilly

Labels: events, javascript, metrics, networking, technology, webdevelopment

If you are like me and upgrade software only when there are critical security fixes or you badly need a few feature, you may have tried sticking to an older version of Yahoo! Messenger. I made the mistake of upgrading, and was almost cut off voice service for a few days. Fortunately, Yahoo! has a fix for the problem, which only seems to affect some users.

The new video calling capability in Yahoo! Messenger 10 didn't really draw my attention. Nevertheless I eventually gave in and allowed the automatic upgrade, if only to get rid of the nagging upgrade notice. At first everything seemed fine: The settings were copied over, and the user interface looked reasonably familiar. However, soon after, voice calls started failing with an obscure error message “Your computer is not connected to the network”. Restarting Yahoo! Messenger sometimes helped, but clearly this wasn't working as reliably as the previous version. “Works sometimes” wasn't good enough for me.

Yahoo! support was exceptionally helpful, within minutes the helpdesk agent had identified that I was running Yahoo! Messenger version 10.0.0.1270-us, which was the latest and greatest at the time but a few issues with voice telephony. He recommended uninstalling the current messenger and manually installing a slightly back-level version of Yahoo! Messenger 10, and disallow further automatic upgrades. The installation worked smoothly, and voice support in Yahoo! Messenger has been working flawlessly ever since.

Thank you, Yahoo! support.

Links:

• Yahoo! Messenger 10.0.0.1258-us for Windows installation file
• Yahoo! Messenger home

Labels: networking, technology, windows

Redirects are often discussed only in the context of search engine optimization (SEO). Here is a good example how redirects affect users as well, and why it is important to choose your redirects wisely.

The Central Intelligence Agency (CIA) in 2006 began serving its Website encrypted in an effort to improve security and privacy of the communication.

This is a clear case for a 301 redirect from the unencrypted URL http://www.cia.gov/page to the equivalent encrypted URL https://www.cia.gov/page. Instead, except for the homepage and very few other pages, all requests get redirected to a splash page informing visitors about the site changes:



Not only is this a bad idea for search since all those links out there on various sites now transfer link weight to a splash page which is marked as non-indexable. It is also an inconvenience to users who need to navigate to the specific content or go back to the previous page and try again with an edited link.

Even the old URL for the World Factbook, arguably one of the most popular resources on the site, no longer goes to the desired World Factbook homepage directly.

The CIA press release states: “We believe the inconveniences of implementing SSL for the entire website will be offset by increased visitor confidence that they are, in fact, connected to the CIA website and that their visits are secure and confidential.”

The effort to increased security and privacy is commendable, and encrypting all communication with the agency certainly isn't a bad idea. Doing so without the inconveniences would be even better though, and perfectly feasible, too.

Labels: networking, seo, technology

More often than not you will see someone recommend switching all redirects on your site to 301s because they pass “link equity” in Google. There is the simple and neat solution for search engine optimization, and it could be plain wrong. At the risk of diverging from the consensus in your favorite discussion forum or offending your SEO consultant, read on and decide for yourself.

Redirects instruct browsers and search engine crawlers that content is available under a different URL. They often go unnoticed as we access Websites. Along with the new location of the content, the server also sends a response code indicating the type of redirect. From an SEO perspective, you generally care about two response codes, 301 Moved permanently and 302 Found:

• 301 Moved permanently indicates that the resource has been assigned a different URL permanently, and the original URL should no longer be used. What this means for Search engines is that should index the new URL only. Google also transfers full link equity with a 301 redirect, which is the very reason why you will often see the advice to use 301 redirects.
• 302 Found indicates that the originally requested URL is still valid, and should continue to be used. Search engines vary in how they treat 302 redirects and which URL they show in search result pages, but generally will continue to crawl the original URL as recommended in the HTTP/1.1 specification: “The requested resource resides temporarily under a different URI. Since the redirection might be altered on occasion, the client SHOULD continue to use the Request-URI for future requests.”

Redirects are often used allow the publication of shorter or prettier “vanity” URLs. Shorter URLs are easier to scan and look much more enticing in the search engine results pages, and generally perform better according to MarketingSherpa's 2007 search marketing study and anecdotal evidence.

Choosing the right redirect response code

So which redirect response code should you use? Matt Cutts' description how Google treats on-domain and off-domain 302 redirects covers the basic principles and the heuristics that Google used at the time, which to a large extent still apply.

Let's summarize the effects of the two most common redirect status codes again:

• 301 redirects transfer link equity to the new URL.
• 301 redirects remove the original URL from the search index.
• 302 redirects often keep the original URL in the index.

The transfer of link equity is desirable in almost all cases. So the deciding factor really is whether you prefer the original URL in the index, and if that outweighs the disadvantages of potentially losing some link equity (more on that later).

The 301 redirect response is appropriate for the following scenarios:

• Content has moved to a different location permanently, for example to a different server name or a different directory structure of the same server. This may be triggered by the rebranding of content where you want all references to the original content to disappear.
• A Website is accessible under multiple host names, such as example.com and www.example.com, or typo catchers like eggsample.com and example.org, but only one name should be indexed.
• A temporary campaign URL is published in direct mail or print advertising, but the landing page has a different permanent URL that will remain accessible beyond the lifetime of the campaign.
• The requested URL does not match the canonical URL for the resource. Often extraneous session and tracking parameters can be stripped, or path information gets added to help with search rankings, for example http://www.amazon.com/Software-Development-Principles-Patterns-Practices/dp/0135974445

The 302 redirect response is the best choice for the following scenarios:

• The original URL is shorter, prettier, more meaningful, etc. and therefore should show on the search engine results page.
• Temporary session or tracking information gets added to the canonical URL. Those URL parameters should not be indexed since they will not apply to other visitors.
• Multiple load balanced servers deliver the content. Indexing an individual server would defeat the purpose of using load balancing. (There are better ways to load balance than having multiple server names, though.)

The “canonical” meta tag

How can you keep the short URL in the index and still transfer link equity? In summer 2008, we started observing that Google somehow “merged” related URLs and treated them as a single entity, showing identical page rank and identical number of inbound links to all of the URLs.

When Google introduced the “canonical” meta tag in February 2009, this suddenly made sense. Once multiple URLs are known to refer to the same page, or a slight variation of a page, the index only needs to keep one instance of the page.

The canonical meta tag helps the webmaster of a site who give search engines a hint about the preferred URL. The canonical meta tag also helps search engines since mapping multiple crawled URLs to the same page and indexing it only once just became easier.

Whether link equity fully transfers between multiple URLs mapped to the same page remains to be seen. At least within the same domain, this unification process may allow keeping the vanity URL in the index with a 302 redirect response while still transferring link equity.

PS. For an excellent and detailed description how redirects work, how to configure your Web server and what each status code does, see Sebstian's pamphlet The anatomy of a server sided redirect: 301, 302 and 307 illuminated SEO wise.

Labels: networking, seo, webdevelopment

For years rsync from the cygwin distribution and PuTTY's plink program would not work together and fail with

Unable to read from standard input: The handle is invalid.
rsync: read error: Connection reset by peer (104)

I first posted this problem to comp.security.ssh in 2005 but never received a solution, until now.

Frank Behrens published cygnative, a small wrapper program that resolves an incompatibility between cygwin and Win32 native programs. With cygnative, invoking rsync with plink as the remote shell works like a charm:

rsync -ave "cygnative plink -batch" source target

Thank you, Frank!

Labels: networking, windows