Badly Behaved Spiders / Bots / Automated Scripts

blacknight · #1 (**permalink**) 22-08-2006, 11:37 PM

Over the last few weeks a couple of servers running forums and other dynamic content have been getting slammed by some form of spider or bot. It's really annoying!

Has anyone else seen anything similar?

richardo · #2 (**permalink**) 23-08-2006, 09:01 AM

As a matter of interest, did this start around the 10th August?

I run a couple of small forums on Enniskerry - Gateway to the Garden of Ireland and Jack and Jill Childrens Foundation using SMF [I switched from phpBB because of hacking attempts].

Since the 10th I have been plagued with attempted postings for porn sites, dubious medical sites and the like. Most of the postings are coming from Russia [boom.ru, bk.ru, list.ru etc etc] and Egypt [gawab.com]. Every morning there is a new clutch of non-authenticated 'new members' which have to be cleared out and a new series of bans inserted.

blacknight · #3 (**permalink**) 23-08-2006, 09:35 AM

They're not actually posting anything.. I keep seeing them in logs.. On my own blog I'm using a script to block bad requests, so they just create a lot of log traffic, but they have caused slightly more serious issues on a couple of other sites by opening up so many requests to the forum at once that the default MySQL settings are no longer enough ...

louie · #4 (**permalink**) 23-08-2006, 09:53 AM

Didn't one of my sites crashed few weeks ago twice cause something like this?

RedCardinal · #5 (**permalink**) 23-08-2006, 09:59 AM

I wonder would it be possible to mod the Bad Behaviour script for use on forums to stop bad requests?

richardo · #6 (**permalink**) 23-08-2006, 10:09 AM

But would I be right in saying that this all started around the 10th?

Life was peaceful up to then. Now it's a pain in the *ss. Thay rarely do any damage as I have so many filters in place - it's just the extra time involved.

Personally I would happily castrate the lot of them with a rusty [blunt] pair of garden shears

blacknight · #7 (**permalink**) 23-08-2006, 10:12 AM

Quote:

Originally Posted by rgh

I wonder would it be possible to mod the Bad Behaviour script for use on forums to stop bad requests?

I think I'll be contacting a few guys about that

Even if you do use it though you still have to process the requests. Tying it in with iptables would be a better solution

frakilk · #8 (**permalink**) 23-08-2006, 11:39 AM

I use phpbb and I have been having a ton of list.ru registrations. I installed a little script that rejects any user trying to supply a website address during registration. Human visitors don't have the option to enter this when signing up. So if it is trying to be entered it is definitely a bot. I have not seen as many spam registrations since.

louie · #9 (**permalink**) 23-08-2006, 11:57 AM

how do you filtered websites addresses from the textarea submitted?

Are you using regex?

frakilk · #10 (**permalink**) 23-08-2006, 12:46 PM

Nothing as complex as that louie, the mod I used is here phpBB.com :: View topic - [2.0.19] Easy BotStopper

Simple but effective.

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode