Irish SEO,  Marketing & Webmaster Discussion

 

Badly Behaved Spiders / Bots / Automated Scripts

This is a discussion on Badly Behaved Spiders / Bots / Automated Scripts within the Server / Technical Administration Tips and Queries forums, part of the Webmaster Help category; Over the last few weeks a couple of servers running forums and other dynamic content have been getting slammed by ...


Go Back   Irish SEO, Marketing & Webmaster Discussion > Webmaster Help > Server / Technical Administration Tips and Queries

Register Forum Rules FAQDonate Members List Calendar Search Today's Posts Mark Forums Read


Notices

Reply

 

LinkBack Thread Tools Display Modes
  #1 (permalink)  
Old 22-08-2006, 11:37 PM
blacknight's Avatar
Web Slave
 
Join Date: Jan 2006
Location: Ireland
Posts: 6,021
blacknight is just really niceblacknight is just really niceblacknight is just really niceblacknight is just really nice
Send a message via ICQ to blacknight Send a message via AIM to blacknight Send a message via MSN to blacknight
Default Badly Behaved Spiders / Bots / Automated Scripts

Over the last few weeks a couple of servers running forums and other dynamic content have been getting slammed by some form of spider or bot. It's really annoying!

Has anyone else seen anything similar?
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Spurl this Post!Reddit! Wong this Post!
Reply With Quote
  #2 (permalink)  
Old 23-08-2006, 09:01 AM
richardo's Avatar
Senior Senile Member
 
Join Date: Feb 2006
Location: Co Wicklow
Posts: 395
Nominated 1 Time in 1 Post
TOTW/F/M Award(s): 0
richardo has a spectacular aura about
Send a message via ICQ to richardo
Default

As a matter of interest, did this start around the 10th August?

I run a couple of small forums on Enniskerry - Gateway to the Garden of Ireland and Jack and Jill Childrens Foundation using SMF [I switched from phpBB because of hacking attempts].

Since the 10th I have been plagued with attempted postings for porn sites, dubious medical sites and the like. Most of the postings are coming from Russia [boom.ru, bk.ru, list.ru etc etc] and Egypt [gawab.com]. Every morning there is a new clutch of non-authenticated 'new members' which have to be cleared out and a new series of bans inserted.
__________________
curratech.ie | enniskerry.ie | Grandad's Blog
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Spurl this Post!Reddit! Wong this Post!
Reply With Quote
  #3 (permalink)  
Old 23-08-2006, 09:35 AM
blacknight's Avatar
Web Slave
 
Join Date: Jan 2006
Location: Ireland
Posts: 6,021
blacknight is just really niceblacknight is just really niceblacknight is just really niceblacknight is just really nice
Send a message via ICQ to blacknight Send a message via AIM to blacknight Send a message via MSN to blacknight
Default

They're not actually posting anything.. I keep seeing them in logs.. On my own blog I'm using a script to block bad requests, so they just create a lot of log traffic, but they have caused slightly more serious issues on a couple of other sites by opening up so many requests to the forum at once that the default MySQL settings are no longer enough ...
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Spurl this Post!Reddit! Wong this Post!
Reply With Quote
  #4 (permalink)  
Old 23-08-2006, 09:53 AM
louie's Avatar
Senior Member
 
Join Date: Jan 2006
Location: Dublin, Ireland
Posts: 1,939
Nominated 0 Times in 0 Posts
TOTW/F/M Award(s): 0
louie has much to be proud oflouie has much to be proud oflouie has much to be proud oflouie has much to be proud oflouie has much to be proud oflouie has much to be proud oflouie has much to be proud oflouie has much to be proud of
Send a message via Yahoo to louie Send a message via Skype™ to louie
Default

Didn't one of my sites crashed few weeks ago twice cause something like this?
__________________
:. Web Design & Development Web Design Ireland
:. Search Engines Optimization Search Engines Optimization
:. Directory Submission Directory Submission
:. News & Press Release Ireland GiveItSocks.com
:. Used Cars Ireland, Car Parts & Car Audio Cars For Sale, Car Parts & Accessories
:. I Have 2 Find It Directory SEF Directory
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Spurl this Post!Reddit! Wong this Post!
Reply With Quote
  #5 (permalink)  
Old 23-08-2006, 09:59 AM
RedCardinal's Avatar
Richard Hearne
 
Join Date: Feb 2006
Posts: 938
Nominated 0 Times in 0 Posts
TOTW/F/M Award(s): 0
RedCardinal is a splendid one to beholdRedCardinal is a splendid one to beholdRedCardinal is a splendid one to beholdRedCardinal is a splendid one to beholdRedCardinal is a splendid one to beholdRedCardinal is a splendid one to beholdRedCardinal is a splendid one to behold
Default

I wonder would it be possible to mod the Bad Behaviour script for use on forums to stop bad requests?
__________________
Search Engine Optimisation - Red Cardinal Internet Marketing
Internet Consultant Ireland | Search Engine Optimisation Services
Catering Company Dublin - My sister's handmade canape company!
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Spurl this Post!Reddit! Wong this Post!
Reply With Quote
  #6 (permalink)  
Old 23-08-2006, 10:09 AM
richardo's Avatar
Senior Senile Member
 
Join Date: Feb 2006
Location: Co Wicklow
Posts: 395
Nominated 1 Time in 1 Post
TOTW/F/M Award(s): 0
richardo has a spectacular aura about
Send a message via ICQ to richardo
Default

But would I be right in saying that this all started around the 10th?

Life was peaceful up to then. Now it's a pain in the *ss. Thay rarely do any damage as I have so many filters in place - it's just the extra time involved.

Personally I would happily castrate the lot of them with a rusty [blunt] pair of garden shears
__________________
curratech.ie | enniskerry.ie | Grandad's Blog
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Spurl this Post!Reddit! Wong this Post!
Reply With Quote
  #7 (permalink)  
Old 23-08-2006, 10:12 AM
blacknight's Avatar
Web Slave
 
Join Date: Jan 2006
Location: Ireland
Posts: 6,021
blacknight is just really niceblacknight is just really niceblacknight is just really niceblacknight is just really nice
Send a message via ICQ to blacknight Send a message via AIM to blacknight Send a message via MSN to blacknight
Default

Quote:
Originally Posted by rgh View Post
I wonder would it be possible to mod the Bad Behaviour script for use on forums to stop bad requests?
I think I'll be contacting a few guys about that

Even if you do use it though you still have to process the requests. Tying it in with iptables would be a better solution
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Spurl this Post!Reddit! Wong this Post!
Reply With Quote
  #8 (permalink)  
Old 23-08-2006, 11:39 AM
frakilk's Avatar
Trapped under Gnomes
 
Join Date: Aug 2006
Location: Malmö, Sweden (but a Galwegian)
Posts: 62
Nominated 0 Times in 0 Posts
TOTW/F/M Award(s): 0
frakilk will become famous soon enough
Default

I use phpbb and I have been having a ton of list.ru registrations. I installed a little script that rejects any user trying to supply a website address during registration. Human visitors don't have the option to enter this when signing up. So if it is trying to be entered it is definitely a bot. I have not seen as many spam registrations since.
__________________
http://www.gardenstew.com/
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Spurl this Post!Reddit! Wong this Post!
Reply With Quote
  #9 (permalink)  
Old 23-08-2006, 11:57 AM
louie's Avatar
Senior Member
 
Join Date: Jan 2006
Location: Dublin, Ireland
Posts: 1,939
Nominated 0 Times in 0 Posts
TOTW/F/M Award(s): 0
louie has much to be proud oflouie has much to be proud oflouie has much to be proud oflouie has much to be proud oflouie has much to be proud oflouie has much to be proud oflouie has much to be proud oflouie has much to be proud of
Send a message via Yahoo to louie Send a message via Skype™ to louie
Default

how do you filtered websites addresses from the textarea submitted?

Are you using regex?
__________________
:. Web Design & Development Web Design Ireland
:. Search Engines Optimization Search Engines Optimization
:. Directory Submission Directory Submission
:. News & Press Release Ireland GiveItSocks.com
:. Used Cars Ireland, Car Parts & Car Audio Cars For Sale, Car Parts & Accessories
:. I Have 2 Find It Directory SEF Directory
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Spurl this Post!Reddit! Wong this Post!
Reply With Quote
  #10 (permalink)  
Old 23-08-2006, 12:46 PM
frakilk's Avatar
Trapped under Gnomes
 
Join Date: Aug 2006
Location: Malmö, Sweden (but a Galwegian)
Posts: 62
Nominated 0 Times in 0 Posts
TOTW/F/M Award(s): 0
frakilk will become famous soon enough
Default

Nothing as complex as that louie, the mod I used is here phpBB.com :: View topic - [2.0.19] Easy BotStopper Simple but effective.
__________________
http://www.gardenstew.com/
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Spurl this Post!Reddit! Wong this Post!
Reply With Quote
Reply

Tags
automated, badly, behaved, bots, scripts, spiders

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump


All times are GMT +1. The time now is 12:32 AM.


Powered by: vBulletin Version 3.7.3, Copyright ©2000 - 2008, Jelsoft Enterprises Limited.
Hosted in Ireland by Blacknight - Test your ISP |Irish Hosting Directory| Armchair.ie|Logo by Eden Web Design|Avatars by Afterglow |Latest Blog Entries | VPS HostingAd Management by RedTyger

Search Engine Friendly URLs by vBSEO 3.2.0