banning this browser: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

In my htaccess file, I have some restrictions using the following
rule: RewriteCond %{HTTP_USER_AGENT}. One example would be:
RewriteCond %{HTTP_USER_AGENT} ^.*baidu.*$ [NC,OR], which restricts
any browser with baidu in it name.

How can I set up a rule to ban this particular browser: Mozilla/4.0
(compatible; MSIE 6.0; Windows NT 5.1; SV1) ? 95% of requests made to
my site by this browser is spam. I thought of using ^.*SV1.*$ but
unfortunately, there are legitimate browsers with SV1 in parts of
their name.

Thanks for the help.

--F

On May 16, 6:25 pm, "no.rep...@comcast.net" <no.rep...@comcast.net>
wrote:
> In my htaccess file, I have some restrictions using the following
> rule: RewriteCond %{HTTP_USER_AGENT}. One example would be:
> RewriteCond %{HTTP_USER_AGENT} ^.*baidu.*$ [NC,OR], which restricts
> any browser with baidu in it name.
>
> How can I set up a rule to ban this particular browser: Mozilla/4.0
> (compatible; MSIE 6.0; Windows NT 5.1; SV1) ? 95% of requests made to
> my site by this browser is spam. I thought of using ^.*SV1.*$ but
> unfortunately, there are legitimate browsers with SV1 in parts of
> their name.
>
> Thanks for the help.
>
> --F

when u say "ban" i assume you mean, "try to stop"
any real spammer will use a valid IE6 user agent. this method does not
work unfortunately.
better would be to use mod_security to rule the type of request out
quietly, and accept that any public website (or hospital or library)
will be a target, and take steps to deal with the results rather than
gradually close the doors to people based on the way they look, which
will only ban those with no skill to hurt your interests anyway.

On 16 May 2007 10:25:21 -0700 no.reply7@comcast.net said
> In my htaccess file, I have some restrictions using the following
> rule: RewriteCond %{HTTP_USER_AGENT}. One example would be:
> RewriteCond %{HTTP_USER_AGENT} ^.*baidu.*$ [NC,OR], which restricts
> any browser with baidu in it name.
>
> How can I set up a rule to ban this particular browser: Mozilla/4.0
> (compatible; MSIE 6.0; Windows NT 5.1; SV1) ? 95% of requests made to
> my site by this browser is spam. I thought of using ^.*SV1.*$ but
> unfortunately, there are legitimate browsers with SV1 in parts of
> their name.
>
> Thanks for the help.


Looks like a good UA to me, but if you want to stop it, try
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/4\.0 \(compatible; MSIE 6\.0; Windows NT 5\.1; SV1\)$

Sig

--
http://koiclubsandiego.org/comment/?r=8
3186fdb515784af427d7bd83018826b9

On May 16, 5:08 pm, sig <pwuldov...@kriocoudek.mailexpire.com> wrote:
> On 16 May 2007 10:25:21 -0700 no.rep...@comcast.net said
>
> > In my htaccess file, I have some restrictions using the following
> > rule: RewriteCond %{HTTP_USER_AGENT}. One example would be:
> > RewriteCond %{HTTP_USER_AGENT} ^.*baidu.*$ [NC,OR], which restricts
> > any browser with baidu in it name.
>
> > How can I set up a rule to ban this particular browser: Mozilla/4.0
> > (compatible; MSIE 6.0; Windows NT 5.1; SV1) ? 95% of requests made to
> > my site by this browser is spam. I thought of using ^.*SV1.*$ but
> > unfortunately, there are legitimate browsers with SV1 in parts of
> > their name.
>
> > Thanks for the help.
>
> Looks like a good UA to me, but if you want to stop it, try
> RewriteCond %{HTTP_USER_AGENT} ^Mozilla/4\.0 \(compatible; MSIE 6\.0; Windows NT 5\.1; SV1\)$
>
> Sig
>
> --http://koiclubsandiego.org/comment/?r=8
> 3186fdb515784af427d7bd83018826b9

I wished it worked but it didn't. It gave me a 500 error when I don't
use that browser.

Here is my problem: I get bombarded with bogus GET request with
sources such as
http://www.toyamall.com/
http://www.blackprofessional.co.uk/
http://www.jusoo.ch/
http://www.tscheiner-huette.info/
http://www.francorpconnect.com/
etc...etc...

A large portion come from Korea and all from the exact browser
mentioned above. Any thought on how to make it go away?

Any thought would be greatly appreciated.

Thanks.

--F

On May 17, 5:06 pm, "no.rep...@comcast.net" <no.rep...@comcast.net>
wrote:
> On May 16, 5:08 pm, sig <pwuldov...@kriocoudek.mailexpire.com> wrote:
>
>
>
> > On 16 May 2007 10:25:21 -0700 no.rep...@comcast.net said
>
> > > In my htaccess file, I have some restrictions using the following
> > > rule: RewriteCond %{HTTP_USER_AGENT}. One example would be:
> > > RewriteCond %{HTTP_USER_AGENT} ^.*baidu.*$ [NC,OR], which restricts
> > > any browser with baidu in it name.
>
> > > How can I set up a rule to ban this particular browser: Mozilla/4.0
> > > (compatible; MSIE 6.0; Windows NT 5.1; SV1) ? 95% of requests made to
> > > my site by this browser is spam. I thought of using ^.*SV1.*$ but
> > > unfortunately, there are legitimate browsers with SV1 in parts of
> > > their name.
>
> > > Thanks for the help.
>
> > Looks like a good UA to me, but if you want to stop it, try
> > RewriteCond %{HTTP_USER_AGENT} ^Mozilla/4\.0 \(compatible; MSIE 6\.0; Windows NT 5\.1; SV1\)$
>
> > Sig
>
> > --http://koiclubsandiego.org/comment/?r=8
> > 3186fdb515784af427d7bd83018826b9
>
> I wished it worked but it didn't. It gave me a 500 error when I don't
> use that browser.
>
> Here is my problem: I get bombarded with bogus GET request with
> sources such

sources? do you mean referers?
If so you are probably being log spammed. If you logs are public, the
referers are logged and your logs indexed by google, which increases
the ranking for these sites.
Implement a simple login/basic auth protection for your logs and they
will get bored.
However if you run an open proxy, and these requests are via your
server, lock it down immediately.


ashttp://www.toyamall.com/http://www.blackprofessional.co.uk/http://www.jusoo.ch/http://www.tscheiner-huette.info/http://www.francorpconnect.com/
> etc...etc...
>
> A large portion come from Korea and all from the exact browser
> mentioned above. Any thought on how to make it go away?
>

does your website sell to Korea, if not ban Korea's entire IP range,
this is similar to DNSBL for spam emails.
Use a dns tool to locate the network provider's IP block for this
user, and block them.
Implement a DNSBL script to test IPs, and dynamically ban them by
altering a precondition file (or .htaccess with a series of Deny from
xxx.xxx rules.
Get a list of bogus referers for this IP, and add them to a ban list.


> Any thought would be greatly appreciated.

Do the requests have a particular preferred time, implement time
dependant blocking of the smallest effective IP range.
If your site is itself involved with undesirable content, I would
imagine your site will always be a target for other similar sites, I
imagine to that sites like those dont always operate ethically. If
it's clean as a whistle then its probably just log spamming.
As you talk about "SPAM" do you mean these requests are just bogus
nonsense requests, or do you mean they are scanning you, or do you
mean they are sending spam through your forms, if the former two, you
could use a firewall in between your server and the internet to block
certain types of requests, Im not sure if you read my previous post
but even a software web application firewall like mod_security is
effective.
If the latter, then implement some kind of turing test, like a
captcha, and code with security in mind to keep unsophisticated
spammers out.

But ultimately, if your site is public and a target for sophisticated
attacks, (which it doesnt appear to be) then theres little you can do.
>
> Thanks.
>
> --F

On 17 May 2007 09:06:24 -0700 no.reply7@comcast.net said

>
> Here is my problem: I get bombarded with bogus GET request with
> sources such as
> http://www.toyamall.com/
> http://www.blackprofessional.co.uk/
> http://www.jusoo.ch/
> http://www.tscheiner-huette.info/
> http://www.francorpconnect.com/
> etc...etc...
>
> A large portion come from Korea and all from the exact browser
> mentioned above. Any thought on how to make it go away?
>
> Any thought would be greatly appreciated.

Order Deny,Allow
Deny from www.toyamall.com
Deny from www.blackprofessional.co.uk
Deny from www.jusoo.ch
Deny from www.tscheiner-huette.info
Deny from www.francorpconnect.com

Sig
--
http://koiclubsandiego.org/comment/?r=8

On May 17, 8:03 pm, sig <pwuldov...@kriocoudek.mailexpire.com> wrote:
> On 17 May 2007 09:06:24 -0700 no.rep...@comcast.net said
>
>
>
> > Here is my problem: I get bombarded with bogus GET request with
> > sources such as
> >http://www.toyamall.com/
> >http://www.blackprofessional.co.uk/
> >http://www.jusoo.ch/
> >http://www.tscheiner-huette.info/
> >http://www.francorpconnect.com/
> > etc...etc...
>
> > A large portion come from Korea and all from the exact browser
> > mentioned above. Any thought on how to make it go away?
>
> > Any thought would be greatly appreciated.
>
> Order Deny,Allow
> Deny fromwww.toyamall.com
> Deny fromwww.blackprofessional.co.uk
> Deny fromwww.jusoo.ch
> Deny fromwww.tscheiner-huette.info
> Deny fromwww.francorpconnect.com
>
> Sig
> --http://koiclubsandiego.org/comment/?r=8

this would only work if the requests are from these domains, but it
appears these are the refers because they have http:// in front.
Worrying about log files will get you ina fiddle wihtout really
acheiving anything, unless these guys are serious slurpers of
bandwidth/CPU or something else, who cares how many idiotic request
they make unless they are getting something out of it you dont want
them to. Just prevent the onvious forms of attacks and then you will
sleep at night no matter who gets on the end of your websites, which
are as I keep saying, public - potential millions of people can do
this, are you going to ban them all, and then stay up at night
worrying about your growing ban list, and how out of date its getting?!

On May 17, 12:13 pm, shimmyshack <matt.fa...@gmail.com> wrote:
> On May 17, 8:03 pm, sig <pwuldov...@kriocoudek.mailexpire.com> wrote:
>
>
>
>
>
> > On 17 May 2007 09:06:24 -0700 no.rep...@comcast.net said
>
> > > Here is my problem: I get bombarded with bogus GET request with
> > > sources such as
> > >http://www.toyamall.com/
> > >http://www.blackprofessional.co.uk/
> > >http://www.jusoo.ch/
> > >http://www.tscheiner-huette.info/
> > >http://www.francorpconnect.com/
> > > etc...etc...
>
> > > A large portion come from Korea and all from the exact browser
> > > mentioned above. Any thought on how to make it go away?
>
> > > Any thought would be greatly appreciated.
>
> > Order Deny,Allow
> > Deny fromwww.toyamall.com
> > Deny fromwww.blackprofessional.co.uk
> > Deny fromwww.jusoo.ch
> > Deny fromwww.tscheiner-huette.info
> > Deny fromwww.francorpconnect.com
>
> > Sig
> > --http://koiclubsandiego.org/comment/?r=8
>
> this would only work if the requests are from these domains, but it
> appears these are the refers because they have http:// in front.
> Worrying about log files will get you ina fiddle wihtout really
> acheiving anything, unless these guys are serious slurpers of
> bandwidth/CPU or something else, who cares how many idiotic request
> they make unless they are getting something out of it you dont want
> them to. Just prevent the onvious forms of attacks and then you will
> sleep at night no matter who gets on the end of your websites, which
> are as I keep saying, public - potential millions of people can do
> this, are you going to ban them all, and then stay up at night
> worrying about your growing ban list, and how out of date its getting?!- Hide quoted text -
>
> - Show quoted text -

My site is clean as a whistle so it might be log spamming. I'll look
into mod_secure. and yes, I was talking about referrers, not sources.

Thanks.

"no.reply7@comcast.net" <no.reply7@comcast.net> wrote:

>In my htaccess file, I have some restrictions using the following
>rule: RewriteCond %{HTTP_USER_AGENT}. One example would be:
>RewriteCond %{HTTP_USER_AGENT} ^.*baidu.*$ [NC,OR], which restricts
>any browser with baidu in it name.
>
>How can I set up a rule to ban this particular browser: Mozilla/4.0
>(compatible; MSIE 6.0; Windows NT 5.1; SV1) ? 95% of requests made to
>my site by this browser is spam. I thought of using ^.*SV1.*$ but
>unfortunately, there are legitimate browsers with SV1 in parts of
>their name.
>
>Thanks for the help.
>
>--F

I'd suggest checking to see if the spam comes from a single ip-addr
and banning the ip instead of the u-a. Anything to do with
user-agents is likely to bite you sooner or later, there are 'bots out
there that send null u-a strings, 'bots that make up random u-a
strings, all kind of crap going on that is sufficient to make trusting
anything u-a based kind of goofy.

--
contact via http://www.ren-prod-inc.com/hug_soft

shimmyshack wrote:
> On May 17, 5:06 pm, "no.rep...@comcast.net" <no.rep...@comcast.net>
> wrote:

<snip>
> sources? do you mean referers?
> If so you are probably being log spammed. If you logs are public, the
> referers are logged and your logs indexed by google, which increases
> the ranking for these sites.
> Implement a simple login/basic auth protection for your logs and they
> will get bored.

I'm not seeing much evidence that they get bored. My logs have never
been public, but I see log spam on a regular basis. One of the joys of
running an internet facing web server.

Jim