fake googlebots gobbling my band width
This is a discussion on fake googlebots gobbling my band width within the Server / Technical Administration Tips and Queries forums, part of the Webmaster Help category; or add "rel="nofollow" to the links....
| |||||||
| Register | Forum Rules | FAQ | Donate | Members List | Calendar | Search | Today's Posts | Mark Forums Read |
11-01-2008, 09:31 PM
| |||||
| |||||
 or add "rel="nofollow" to the links.
__________________ :. Web Design & Development Web Design Ireland :. Search Engines Optimization Search Engines Optimization :. Directory Submission Directory Submission :. News & Press Release Ireland GiveItSocks.com :. Used Cars Ireland, Car Parts & Car Audio Cars For Sale, Car Parts & Accessories :. I Have 2 Find It Directory SEF Directory         |
TOTW/F/M Award(s): 1
|
11-01-2008, 11:10 PM
| |||||
| |||||
| louie nofollow doesn't mean it won't be indexed. you need "no-index" for that.
__________________ my sites : irish poker / irish jobs / seo faq / advertise jobs free / green card / skiing / geansai gorm |
|
11-01-2008, 11:43 PM
| |||||
| |||||
| i think this is the post redcardinal is talking about. Official Google Webmaster Central Blog: How to verify Googlebot reverse dns lookup.
__________________ My Blog |
|
12-01-2008, 12:12 AM
| |||||
| |||||
| Paul if it doesn't follow then it can be indexed can it? nofollow - Wikipedia, the free encyclopedia Last edited by louie; 12-01-2008 at 12:19 AM. |
|
12-01-2008, 02:03 AM
| |||||
| |||||
| If you just deny the bot from the directory it won't even try to look at the links ...
__________________ Hosting & Domains|Plesk Vps Hosting|Geek / Cool Stuff|Films.ie|Cool Sites|Monetisation Tips|Movie Chat Energise your forum! Click here for info |
|
12-01-2008, 05:45 AM
| |||||
| |||||
| Quote:
__________________ Search Engine Optimisation - Red Cardinal Internet Marketing Internet Consultant Ireland | Search Engine Optimisation Services Catering Company Dublin - My sister's handmade canape company! |
|
12-01-2008, 06:25 AM
| ||||
| ||||
| Yeap if Google finds a link somewhere else to it then it'll get indexed Here's 2 good links Preventing content from appearing in Google search results Block or remove pages using meta tags |
|
15-01-2008, 01:44 PM
| |||||
| |||||
| Block google from spidering a Calander Follow on from my original post Would it be possible for me to block part of a url in robots.txt Google has indexed 12 identical pages generated by my calendar with the same content "No events found for this date." Heres is a full url google is spidering /gallery.php?picture_id=9&user_eventsDate=2008-01-13 this is what I want to remove &user_eventsDate= this is the calendar link. I would like to remove the blasted Calendar but it was part of the specs for the site. thanks again. |
|
15-01-2008, 02:12 PM
| |||||
| |||||
| I wrote an article some time ago about dealing with duplicated content here: SEO Dealing with Duplicate Content | Website Design Ireland, Website Development |
|
15-01-2008, 03:24 PM
| |||||
| |||||
| Disallow: /*&user_eventsDate=/ I have picked this up from the google web masters forum Disallow: /*&user_eventsDate=/ &user_eventsDate=2008-01-13 is the useless segment of the url which google is generating from the calendar So It is spidering pages which don't exist like /news_desk_detail.php?news_id=20&user_eventsDate=20 08-01-13 which when typed in to the browser shows the content for /news_desk_detail.php?news_id=20 the Problem with this is it is also spidering /news_desk_detail.php?news_id=20 &user_eventsDate=2008-01-13 /news_desk_detail.php?news_id=20 &user_eventsDate=2008-01-14 /news_desk_detail.php?news_id=20 &user_eventsDate=2008-01-15 and so on up to four links deep for each dynamic link it follows , which could expand very quickly to thousands of useless pages. Will the above disallow work ? |
Linear Mode
