WordPress Bug Friday: Wasting your bandwidth

I intended to post this on Friday. As they say when receiving a crappy Christmas gift, It’s the thought that counts. (They lie.) I should probably give myself a break and just change the official name to WordPress Bug Near Friday. Well, so be it. Make it so. Engage. Energize. Giddyup!

Usually I’m bemoaning the existence of HTTP 304 (Not Modified) responses, but this time the problem was the non-existence of them. (Can there be such a thing as non-existence? Where would you find it? And can you deduct it from your taxes?) I noticed when looking through my web site logs that feed requests from NetNewsWire always received an HTTP 200 (OK) response from WordPress, never 304, which means that NetNewsWire downloaded the entire content of a feed on every request. Since my web site gets more hits from NetNewsWire than from any other browser, that’s quite a lot of bandwidth used. (Relatively speaking, that is. In the grand scheme of things, my page ranking is right below the site for Grasshopper Enthusiasts of Eastern Ontario.)

Brent Simmons, the creator of NetNewsWire, was kind enough to talk to me about the problem, despite the fact that my app, Vienna, has undoubtedly taken away some sales from him. (In fact, I was all set to purchase NetNewsWire myself until I discovered Vienna.) I’m not worried about Brent, though: I heard that NewsGator paid him something like a trillion dollars for NetNewsWire, give or take. Plus he gets as many Café Lattes as he likes. Anyway, he explained that WordPress does not handle entity tags correctly.

In addition to Last-Modified headers, WordPress sends out ETag headers, which are basically gobbledygook strings that identify web content. Some web browsers, such as Vienna, only send conditional If-Modified-Since requests based on Last-Modified dates, but a browser can also store ETags and send them back on subsequent visits to the site as part of conditional If-None-Match requests. If the ETags don’t match the current content on the site, then there is new content that needs to be downloaded. NetNewsWire sends both kinds of conditional request. Unfortunately, WordPress does not parse its own ETags correctly on receiving If-None-Match requests — there seems to be a problem with quoting — so a match is never found, and it always sends a 200 response, along with the full feed content.

Brent passed along a suggestion to remove or comment out the following line in the file wp-includes/classes.php:

@header("ETag: $wp_etag");

After that, WordPress no longer sends out ETags, so it relies totally on Last-Modified dates. I’ve been testing this modification for a week, and NetNewsWire now receives both 200 and 304 responses, as appropriate. Moreover, my bandwidth has been cut by more than half. Thanks, Brent! We should form a tag team wrestling duo to pin down the WordPress developers and make them fix their feed bugs. My wrestling name will be the Penultimate Warrior.

7 Responses to “WordPress Bug Friday: Wasting your bandwidth”

  1. I also ran into the 304 dysfunction, and fixed it in conjunction with adding support for WP-Cache, so I could reduce my processor usage at the same time:


  2. Heh, the comment from charles wasn’t there when I submitted. Honest!

  3. Jeff says:

    His comment got marked as spam by Akismet, so I didn’t see it until after I approved yours. That is pretty funny.

  4. Peter Hosey says:

    Thanks to both of you! I’ve just applied this to my own blog. I’ll see how well it works.

    As I mentioned on my own blog (spam prevention has been a topic there lately), I don’t use Akismet — when the Adium Trac was using it, we had a *lot* of users blocked from creating or commenting on tickets by the Trac Akismet plug-in. That, combined with their lack of detailed explanation of the operation of the plug-in, makes it one of my last resorts (along with Bad Behavior).

  5. Anonymous says:

    Unfortunately, syndicators like Bloglines require both the Etag and Last-modified headers in order for it to them to respect a 304.

  6. [...] no doubt, by “Episode V: All Hope Dashed”). I’m very happy to report that the ETag parsing bug has been fixed in WordPress 2.0.7. Thus, if you’re running WordPress 2.0.7 on your site, you [...]