Forums & Blog

A SmarterTools-sponsored community.
Welcome to Forums & Blog Sign in | Join | Help
in Search

Prefilter exclude spider and other bot data

Last post 02-17-2007 11:51 AM by ST-GWerner. 2 replies.
Page 1 of 1 (3 items)
Sort Posts: Previous Next
  • 02-08-2007 10:03 AM

    Prefilter exclude spider and other bot data

    I know this has been posted back in Mar of 2006 on this very forum.  But I want to post to make sure it still remains fresh in SmarterTools todo list.

     

    We need a way to prefilter any data coming in based on the existence of a string within the user agent string.  This would allow us to filter out spiders and bot traffic (which skews the reports that SmarterStats generates).

  • 02-14-2007 10:47 AM In reply to

    Re: Prefilter exclude spider and other bot data

    Definitely needed.  All of the charts and such should be human-only traffic (except the bot/spider report).  I like how AwStats has a row at the bottom near the total for "Spiders" which gives the total spider traffic for the report.  That also is a reality check since it gives me an idea of human traffic volume versus bot volume.

    This also would imply SS would need to download a "bots" file on a consistent basis to stay current with the new crawlers out there.

  • 02-17-2007 11:51 AM In reply to

    Re: Prefilter exclude spider and other bot data

    Grady Werner
    SmarterTools Inc.
Page 1 of 1 (3 items)