Alice in the SEO wonderland.: Parasite hosting - Or why social networking sites need to review user generated content.

To directly see the list of highly ranked spammy users pages within reddit
From wikipedia: Parasite hosting is the process of hosting a site on some one else's server without their consent, generally for the purpose of search engine benefit.

One of the most competitive keywords which everyone wants to rank for is buy viagra online

On that and many other similar searches, you would start noticing that user pages of social bookmarking sites like reddit, get very prominent rankings.
These sites have a very high TrustRank and amazing authority scores. So even off topic pages are very easily ranked high for pages created on these domains.
I was playing with the sitexplorer python library I wrote. The user pages on the reddit domain outrank even the subreddit pages.
Using a little python I found that of the top 400 pages on the reddit.com domain, 102 are user pages of the form reddit.com/user/{username}/. All of these are pages promoting prescription pills like Viagra or Cialis.
(The python program [1]
List of the user pages in the top 400 pages on reddit)

How can the social bookmarking sites combat this?
1. Use a robots.txt. For reddit.com this can be as simple as using


User-Agent: *
Disallow: /user

2. Have kill words which are not allowed in the user names. This allows the pages to be indexed with facing parasite hosting.


import re
all_pages = []
for i in range(4):
    start = i*100 + 1
    results = get_page_data('YahooDemo', u'http://reddit.com', start = start, results = 100)
    pages = [el['Url'] for el in results]
    all_pages.extend(pages)

pat = '/user/([a-zA-Z0-9_]*)'
rep = re.compile(pat)
users = [rep.search(el) for el in all_pages]
users_ = [el.groups()[0] for el in users if el is not None]