Spinn3r.com offers a service that indexes all of the blogs on the internet
Title
spinn3r - About
Description
Excerpted from the website:
- In order to build Tailrank, we needed to create a spam-resistant and high performance spider (also know as a crawler) to index the blogosphere. This involved spam prevention, language categorization, complex trust ranking, realtime ping support, etc. Needless to say, building a high quality spider is a difficult task.
Languages
English
Additional Information
Spinn3r/Tailrank operates a crawler which does not follow the robots.txt protocols. It can be identified in logs from the following User-Agent:
- Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1; aggregator:Tailrank (Spinn3r 2.1); http://spinn3r.com/robot) Gecko/20021130
Related Domains
External Links
- Alexa: Spinn3r.com
- WHOIS for Spinn3r.com
