Double Space Project

The crawler

Meet spacehound.

Most exhibits on this site were caught by spacehound, the project's own crawler. It reads pages that lots of people read (government sites, popular Wikipedia articles, the help centers and docs of big companies and widely used software) and sniffs out double spaces that a visitor can actually see.

How it hunts

  • It looks with a real browser. HTML folds a run of spaces into one, so the page source is full of doubles nobody ever sees. spacehound renders each page and measures every gap, character by character. Only gaps that really show up on screen count.
  • Only inside a sentence. A gap has to sit between real words, like the three dots menu. Menus, tables, code, aligned columns and the old typewriter habit after a full stop are left alone.
  • No false alarms from icons. An icon that fails to load can leave two spaces around an empty hole. spacehound checks the page's source and loads it a second time with every image in place before it believes a gap is real.
  • High-profile pages only. It keeps to well-known, much-visited sites, and skips social media, forums, shops, logins and anything written by random users.
  • A human has the last word. Every find is reviewed by a person before it becomes an exhibit. The publication is named, never the person who typed it.

How polite it is

  • It obeys robots.txt, including Crawl-delay.
  • One page at a time per site, and at most one request per second.
  • It never logs in, never fills in forms and never uses your site search.
  • If a site answers 429 Too Many Requests, or refuses it twice in a row, it stops visiting that site for good. It never retries or disguises itself.

User agent

spacehound always says who it is. Its requests carry this user agent:

Mozilla/5.0 (compatible; spacehound/2.0; +https://doublespaceproject.com/spacehound)

Blocking it

Rather not be visited? Add this to your robots.txt and spacehound will stay away:

User-agent: spacehound
Disallow: /

Found your own site among the exhibits? The quickest fix is the double space itself. We're also happy to take an exhibit down: just contact at doublespaceproject dot com.