Traffic Surveillance
By Randal L. Schwartz
Navigational elements of your site—how a user moves from one place to another—are integral parts of the user experience. Ask yourself several questions about your site's effectiveness and whether the adjustments you make will positively effect your site's visitors. Can you tell what users want from a Web page or cluster of Web pages? Are people drawn to your most important pages? When hits come from search engines or external links, on which pages do these visitors typically end up?
I'm not a usability expert by any means. I'm a Perl hacker. But the first step in answering most of these questions is to gather data. Much of this data is tracked by a referer log that, for each request, also records the browser's previous requested page.
Apache doesn't include referer logging right out of the box, but you can access it with one or two directives, then sit back and wait a month while statistics pour in. This month I decided to keep tabs on my site's traffic flow in an attempt to ultimately improve its user experience. I left Apache's default logging alone, and logged many other interesting items directly to a DBI log using mod_perl. (See my April 2000 column, "Web Access Logs with DBI.")
By default, logs also don't track outbound links. I would normally receive no indication when a browser followed an outside link I'd provided on my site. However, thanks to the code I wrote for "Programming with Perl" in May 1998, I've rewritten every outbound link prefixed with /cgi/go/, as in the following line:
More info at <a href="/cgi/go/http://www.p
