Irish SEO,  Marketing & Webmaster Discussion

 

Badly Behaved Spiders / Bots / Automated Scripts

This is a discussion on Badly Behaved Spiders / Bots / Automated Scripts within the Server / Technical Administration Tips and Queries forums, part of the Webmaster Help category; Over the last few weeks a couple of servers running forums and other dynamic content have been getting slammed by ...


Go Back   Irish SEO, Marketing & Webmaster Discussion > Webmaster Help > Server / Technical Administration Tips and Queries

Register Forum Rules FAQDonate Calendar Search Today's Posts Mark Forums Read



Reply

 

LinkBack Thread Tools Display Modes
  #1 (permalink)  
Old 22-08-2006, 11:37 PM
blacknight's Avatar
Web Slave
Recent Blog: links for 2010-03-19
 
Join Date: Jan 2006
Location: Ireland
Posts: 7,876
Thanks: 8
Thanked 8 Times in 7 Posts
blacknight is a splendid one to beholdblacknight is a splendid one to beholdblacknight is a splendid one to beholdblacknight is a splendid one to beholdblacknight is a splendid one to beholdblacknight is a splendid one to beholdblacknight is a splendid one to beholdblacknight is a splendid one to behold
Default Badly Behaved Spiders / Bots / Automated Scripts

Over the last few weeks a couple of servers running forums and other dynamic content have been getting slammed by some form of spider or bot. It's really annoying!

Has anyone else seen anything similar?
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Spurl this Post!Reddit! Wong this Post!
Reply With Quote
  #2 (permalink)  
Old 23-08-2006, 09:01 AM
grandad's Avatar
Senior Senile Member
Recent Blog: Einstein and Saturday
 
Join Date: Feb 2006
Location: Up the Mountains
Posts: 413
Nominated 1 Time in 1 Post
TOTW/F/M Award(s): 0
Thanks: 0
Thanked 0 Times in 0 Posts
grandad has a spectacular aura about
Send a message via ICQ to grandad
Default

As a matter of interest, did this start around the 10th August?

I run a couple of small forums on Enniskerry - Gateway to the Garden of Ireland and Jack and Jill Childrens Foundation using SMF [I switched from phpBB because of hacking attempts].

Since the 10th I have been plagued with attempted postings for porn sites, dubious medical sites and the like. Most of the postings are coming from Russia [boom.ru, bk.ru, list.ru etc etc] and Egypt [gawab.com]. Every morning there is a new clutch of non-authenticated 'new members' which have to be cleared out and a new series of bans inserted.
__________________
Head Rambles
The Other Fellow
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Spurl this Post!Reddit! Wong this Post!
Reply With Quote
  #3 (permalink)  
Old 23-08-2006, 09:35 AM
blacknight's Avatar
Web Slave
Recent Blog: links for 2010-03-19
 
Join Date: Jan 2006
Location: Ireland
Posts: 7,876
Thanks: 8
Thanked 8 Times in 7 Posts
blacknight is a splendid one to beholdblacknight is a splendid one to beholdblacknight is a splendid one to beholdblacknight is a splendid one to beholdblacknight is a splendid one to beholdblacknight is a splendid one to beholdblacknight is a splendid one to beholdblacknight is a splendid one to behold
Default

They're not actually posting anything.. I keep seeing them in logs.. On my own blog I'm using a script to block bad requests, so they just create a lot of log traffic, but they have caused slightly more serious issues on a couple of other sites by opening up so many requests to the forum at once that the default MySQL settings are no longer enough ...
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Spurl this Post!Reddit! Wong this Post!
Reply With Quote
  #4 (permalink)  
Old 23-08-2006, 09:53 AM
louie's Avatar
Senior Member
 
Join Date: Jan 2006
Location: Dublin, Ireland
Posts: 2,328
Nominated 7 Times in 5 Posts
Nominated TOTW/F/M Award(s): 2
Thanks: 0
Thanked 1 Time in 1 Post
louie will become famous soon enoughlouie will become famous soon enoughlouie will become famous soon enoughlouie will become famous soon enoughlouie will become famous soon enoughlouie will become famous soon enoughlouie will become famous soon enoughlouie will become famous soon enoughlouie will become famous soon enoughlouie will become famous soon enoughlouie will become famous soon enough
Send a message via Yahoo to louie Send a message via Skype™ to louie
Default

Didn't one of my sites crashed few weeks ago twice cause something like this?
__________________
:. Web Design & Development Web Design Ireland
:. Search Engines Optimization Search Engines Optimization
:. Car Parts & Accessories Car Parts
:. Cars Ireland Cars Ireland
:. I Have 2 Find It Directory SEF Directory
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Spurl this Post!Reddit! Wong this Post!
Reply With Quote
  #5 (permalink)  
Old 23-08-2006, 09:59 AM
RedCardinal's Avatar
Richard Hearne
 
Join Date: Feb 2006
Location: Dublin/Phuket
Posts: 985
Nominated 0 Times in 0 Posts
TOTW/F/M Award(s): 0
Thanks: 0
Thanked 0 Times in 0 Posts
RedCardinal is a splendid one to beholdRedCardinal is a splendid one to beholdRedCardinal is a splendid one to beholdRedCardinal is a splendid one to beholdRedCardinal is a splendid one to beholdRedCardinal is a splendid one to beholdRedCardinal is a splendid one to behold
Default

I wonder would it be possible to mod the Bad Behaviour script for use on forums to stop bad requests?
__________________
RedCardinal.ie Search Engine Optimisation & Online Marketing
I offer Web Strategy Consulting and SEO Services.
My sister's handmade canape company - Catering Company Dublin
On Twitter I'm RedCardinal
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Spurl this Post!Reddit! Wong this Post!
Reply With Quote
  #6 (permalink)  
Old 23-08-2006, 10:09 AM
grandad's Avatar
Senior Senile Member
Recent Blog: Einstein and Saturday
 
Join Date: Feb 2006
Location: Up the Mountains
Posts: 413
Nominated 1 Time in 1 Post
TOTW/F/M Award(s): 0
Thanks: 0
Thanked 0 Times in 0 Posts
grandad has a spectacular aura about
Send a message via ICQ to grandad
Default

But would I be right in saying that this all started around the 10th?

Life was peaceful up to then. Now it's a pain in the *ss. Thay rarely do any damage as I have so many filters in place - it's just the extra time involved.

Personally I would happily castrate the lot of them with a rusty [blunt] pair of garden shears
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Spurl this Post!Reddit! Wong this Post!
Reply With Quote
  #7 (permalink)  
Old 23-08-2006, 10:12 AM
blacknight's Avatar
Web Slave
Recent Blog: links for 2010-03-19
 
Join Date: Jan 2006
Location: Ireland
Posts: 7,876
Thanks: 8
Thanked 8 Times in 7 Posts
blacknight is a splendid one to beholdblacknight is a splendid one to beholdblacknight is a splendid one to beholdblacknight is a splendid one to beholdblacknight is a splendid one to beholdblacknight is a splendid one to beholdblacknight is a splendid one to beholdblacknight is a splendid one to behold
Default

Quote:
Originally Posted by rgh View Post
I wonder would it be possible to mod the Bad Behaviour script for use on forums to stop bad requests?
I think I'll be contacting a few guys about that

Even if you do use it though you still have to process the requests. Tying it in with iptables would be a better solution
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Spurl this Post!Reddit! Wong this Post!
Reply With Quote
  #8 (permalink)  
Old 23-08-2006, 11:39 AM
frakilk's Avatar
Trapped under Gnomes
Recent Blog: Meet Max
 
Join Date: Aug 2006
Location: Malmö, Sweden (but a Galwegian)
Posts: 62
Nominated 0 Times in 0 Posts
TOTW/F/M Award(s): 0
Thanks: 0
Thanked 0 Times in 0 Posts
frakilk will become famous soon enough
Default

I use phpbb and I have been having a ton of list.ru registrations. I installed a little script that rejects any user trying to supply a website address during registration. Human visitors don't have the option to enter this when signing up. So if it is trying to be entered it is definitely a bot. I have not seen as many spam registrations since.
__________________
http://www.gardenstew.com/
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Spurl this Post!Reddit! Wong this Post!
Reply With Quote
  #9 (permalink)  
Old 23-08-2006, 11:57 AM
louie's Avatar
Senior Member
 
Join Date: Jan 2006
Location: Dublin, Ireland
Posts: 2,328
Nominated 7 Times in 5 Posts
Nominated TOTW/F/M Award(s): 2
Thanks: 0
Thanked 1 Time in 1 Post
louie will become famous soon enoughlouie will become famous soon enoughlouie will become famous soon enoughlouie will become famous soon enoughlouie will become famous soon enoughlouie will become famous soon enoughlouie will become famous soon enoughlouie will become famous soon enoughlouie will become famous soon enoughlouie will become famous soon enoughlouie will become famous soon enough
Send a message via Yahoo to louie Send a message via Skype™ to louie
Default

how do you filtered websites addresses from the textarea submitted?

Are you using regex?
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Spurl this Post!Reddit! Wong this Post!
Reply With Quote
  #10 (permalink)  
Old 23-08-2006, 12:46 PM
frakilk's Avatar
Trapped under Gnomes
Recent Blog: Meet Max
 
Join Date: Aug 2006
Location: Malmö, Sweden (but a Galwegian)
Posts: 62
Nominated 0 Times in 0 Posts
TOTW/F/M Award(s): 0
Thanks: 0
Thanked 0 Times in 0 Posts
frakilk will become famous soon enough
Default

Nothing as complex as that louie, the mod I used is here phpBB.com :: View topic - [2.0.19] Easy BotStopper Simple but effective.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Spurl this Post!Reddit! Wong this Post!
Reply With Quote
Reply

Tags
automated, badly, behaved, bots, scripts, spiders

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On



Sponsored links

Pepperjam Network
Paid On Results www.zanox.com Get Chitika Premium


All times are GMT +1. The time now is 12:38 PM.


Powered by: vBulletin Version 3.8.4, Copyright ©2000 - 2010, Jelsoft Enterprises Limited.
Hosted in Ireland by Blacknight - Test your ISP |Irish Hosting Directory| Armchair.ie|Logo by Eden Web Design|Avatars by Afterglow |Latest Blog Entries | VPS HostingAd Management by RedTyger

Search Engine Friendly URLs by vBSEO 3.3.2

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64