How Shall I Measure Thee? Let Me Count the Ways.
By Lincoln D. Stein
Measuring things is a basic human need. To know this you need only grow up in a family of four brothers, as I did. We would squabble at the breakfast table over who got more cereal than the others, argue about whose collection of pebbles was larger, or heavier, or more varied, compete over who could run faster or jump farther. The key to sibling harmony was accurate, impartial measurement.
By applying a measurement to something, you understand it. Understanding it is the first step to controlling it. Take the Web. We know it's out there, and we have a sense that it's "big." Just how big, exactly? How many sites, how many Webmasters, how many terabytes of storage? How many people are visiting your site? How many are leaving feeling satisfied? If all of the Web were packed into a single disk array, just how much physical space would it occupy?
In this column I consider various ways of measuring -- and mismeasuring -- the World Wide Web.
How Big Is the Web?
Almost from the beginning, people have been interested in measuring the size of the Web. How many Web sites are there? How many pages? How many bytes of information? How much traffic goes over it in a day?
The earliest attempt to measure the Web that I know of dates back to 1994, when Matthew Gray, one of MIT's Webmasters, created a World Wide Web Wanderer robot to search out all the Web's nooks and crannies. By November 1994, the Wanderer had found all of 870 Web sites.
