Subject: Wiki pulled down due to excessive spam

From: Shlomi Fish <[email-masked]>
Date: Mon, 23 May 2005 23:01:06 +0300

Hi all!

I had to disable the wiki due to an excessive amount of spam in the last 
couple of days. It seems that one spammer is using a network of zombie 
computers to spam our wiki. We are using rel="nofollow" and I monitor the RSS 
feed, but he still continues.

I hope this guy get caught and stuff. I can give some of the IPs that were 
used to spam the wiki if that's any help.

Regards,

	Shlomi Fish

---------------------------------------------------------------------
Shlomi Fish      [email-masked]
Homepage:        http://www.shlomifish.org/

Tcl is LISP on drugs. Using strings instead of S-expressions for closures
is Evil with one of those gigantic E's you can find at the beginning of 
paragraphs.

---

From: Arik Baratz <[email-masked]>
Date: Tue, 24 May 2005 02:54:57 +0300

> feed, but he still continues.

Shlomi,

Don't take it down, just make it subscribe-only.

-- Arik

---

From: Shlomi Fish <[email-masked]>
Date: Tue, 24 May 2005 07:48:13 +0300

On Tuesday 24 May 2005 02:54, Arik Baratz wrote:
> > computers to spam our wiki. We are using rel="nofollow" and I monitor the
> > RSS feed, but he still continues.
>
> Shlomi,
>

That I will do. But first I need to research how to do it. I also want to see 
if a subscription can involve confirmation by E-mail.

Regards,

	Shlomi Fish

---

From: Shlomi Fish <[email-masked]>
Date: Tue, 24 May 2005 08:30:38 +0300

On Tuesday 24 May 2005 07:48, Shlomi Fish wrote:
> > > I had to disable the wiki due to an excessive amount of spam in the
> > > last couple of days. It seems that one spammer is using a network of
> > > zombie computers to spam our wiki. We are using rel="nofollow" and I
> > > monitor the RSS feed, but he still continues.
> >
> > Shlomi,
> >
> That I will do. But first I need to research how to do it. I also want to
> see if a subscription can involve confirmation by E-mail.
>

OK. Done. The wiki has been restored. However, there doesn't seem to be a way 
to require an E-mail handshake right now. Time for some PHP hacking? ;-)

Regards,

	Shlomi Fish

---

From: [email-masked]
Date: Tue, 24 May 2005 15:53:51 +1000

> to require an E-mail handshake right now. Time for some PHP hacking? ;-)

In most wiki's I'm aware off, wacky character images (the ones which
look as if you see them through a badly damaged bottom of
whisky-bottle) are the most common "human authentication" way today.
I suspect this has become prevelant because spam bots became clever
enough to give a working temporary e-mail address and be able to
register automatically.

I can't find such tools right now (forgot the term used to reffer to
this trick),
but apparently the preactice which happened on your wiki is called
"wikispam" and you can find lots of stuff on the net by looking this term
up.
Here is a page I saw to be refferenced from a few places:
http://www.usemod.com/cgi-bin/mb.pl?WikiSpam

Cheers,

--Amos

---

From: Tal Kelrich <[email-masked]>
Date: Tue, 24 May 2005 11:15:11 +0300

> whisky-bottle) are the most common "human authentication" way today.

That's called a Captcha. [1]

BTW, WikiMedia has a couple of anti-spam features, including a URL
blacklist. [2]

[1] http://en.wikipedia.org/wiki/Captcha
[2] http://meta.wikimedia.org/wiki/Anti-spam_Features

-- 
Tal Kelrich
PGP fingerprint: 3EDF FCC5 60BB 4729 AB2F  CAE6 FEC1 9AAC 12B9 AA69
Key Available at: http://www.hasturkun.com/pub.txt
----
The heart has its reasons which reason knows nothing of. -- Blaise
Pascal
----

---

From: Arik Baratz <[email-masked]>
Date: Tue, 24 May 2005 13:23:22 +0300

> I can't find such tools right now (forgot the term used to reffer to

http://en.wikipedia.org/wiki/Captcha

I haven't seen a MediaWiki plugin that does that, but there are free
php implementations.

-- Arik

---

From: Shoshannah Forbes <[email-masked]>
Date: Tue, 24 May 2005 23:16:02 +0300

>  enough to give a working temporary e-mail address and be able to

They are called "CAPTCHA" and are a serious usability problem for 
people with vision related disabilities.
Hell, even without vision problems, I had enough of those reject me 
after what I though was an "l" turned out to be an "i" or a "1" or 
other similar problems.

BTW, spammers have figured out how to bypass those- they set up a "free 
porn site", that in order to enter, you need to answer a CAPTCHA- 
pulled from another site.
then the CAPTCHA and the answer are fed back to the original site- and 
they're in.

There are also ways to beat it using only computers. See here:
http://haacked.com/archive/2005/01/31/2060.aspx
---
Shoshannah Forbes
http://www.xslf.com

---

From: Shoshannah Forbes <[email-masked]>
Date: Tue, 24 May 2005 23:46:48 +0300

> http://haacked.com/archive/2005/01/31/2060.aspx

oops.. correct link:
http://www.brains-n-brawn.com/default.aspx?vDir=aicaptcha
he has links to other articles about computers solving CAPTCHAs.
---
Shoshannah Forbes
http://www.xslf.com

---

From: Tal Kelrich <[email-masked]>
Date: Wed, 25 May 2005 14:10:51 +0300

On Tue, 24 May 2005 23:16:02 +0300
Shoshannah Forbes <[email-masked]> wrote:

> >  In most wiki's I'm aware off, wacky character images (the ones
> >  which look as if you see them through a badly damaged bottom of
> >  whisky-bottle) are the most common "human authentication" way
> >  today. I suspect this has become prevelant because spam bots became
> >  clever enough to give a working temporary e-mail address and be
> >  able to register automatically.
> 

There are audio Captchas that deal with the problem, clearer Captchas
exist, and most sites allow one to bypass the system entirely by mailing
an administrator

> BTW, spammers have figured out how to bypass those- they set up a
> "free porn site", that in order to enter, you need to answer a
> CAPTCHA- pulled from another site.

That's a fairly easy fix, you make your tokens non-reuseable, dependent
on session, quick aging, etc.
then they can only attack it in real time.

-- 
Tal Kelrich
PGP fingerprint: 3EDF FCC5 60BB 4729 AB2F  CAE6 FEC1 9AAC 12B9 AA69
Key Available at: http://www.hasturkun.com/pub.txt
----
"I may be synthetic, but I'm not stupid" -- the artificial person, from
_Aliens_
----

---

From: "Nadav Har'El" <[email-masked]>
Date: Wed, 25 May 2005 20:18:53 +0300

> then they can only attack it in real time.

None of these idea help, because spammers *can* attack it in real time -
when a user goes into their porn site, they get a capcha from the wiki
(or whatever) site, and ask you to solve it for them.

Also, consider a captcha which has a 10% chance of being solved by a
good computer program. The attacker will need to run just 10 tries to
succeed in one in good probability.

So these CAPCHAs can help, but are certainly not the silver bullet against
site abuse by robots.

-- 
Nadav Har'El                        |    Wednesday, May 25 2005, 17 Iyyar 5765
[email-masked]             |-----------------------------------------
Phone +972-523-790466, ICQ 13349191 |How to become immortal: Read this
http://nadav.harel.org.il           |signature tomorrow and follow its advice.

---

From: Arik Baratz <[email-masked]>
Date: Wed, 25 May 2005 20:38:55 +0300

On 25/05/05, Nadav Har'El <[email-masked]> wrote:
[snip]
> So these CAPCHAs can help, but are certainly not the silver bullet against

I say let's use Captchas today, and worry about Captcha-cracking
monkeys or whatever becomes popular tomorrow - tomorrow. Shlomi - do
you need help with this?

-- Arik

---

From: Shoshannah Forbes <[email-masked]>
Date: Thu, 26 May 2005 01:01:03 +0300

On 25/05/2005, at 20:38, Arik Baratz wrote:

>  monkeys or whatever becomes popular tomorrow - tomorrow

Problem is, let's put it bluntly, that visual Captchas suck, and are a 
real pain for legitimate users (and they block legit users many times, 
not just blind and hard of seeing users either).

My point was that not only visual Captchas a usability and 
accessibility problem for legitimate users, they are not that effective 
for a "black hat" to brake, and with the wide spread use of them, this 
will happen sooner rather then later.

So- are they really necessary? Is the gain really worth the cost?
--
Shoshannah Forbes
http://www.xslf.com

---

From: Arik Baratz <[email-masked]>
Date: Thu, 26 May 2005 01:23:36 +0300

> not just blind and hard of seeing users either).

Make them only a part of the user registration process, so a user has
to deal with them only once. Have visually challanged people call
someone by phone to register a user. Once you have a user, that's it.

> So- are they really necessary? Is the gain really worth the cost?

Well, if you want the wiki to stay down than they are not worth it,
but if we want it up for the time being then definitely. As long as
there is easier pray, that will work. A home-grown captcha on our page
will be even harder, because someone will need to fit a
captcha-guessing engine for OUR site, which is even more effort. I say
we have a year until we need to think of something else. The wiki can
stay down during that year, or it can stay up with minimal effort.
What do you think?

-- Arik

---

From: Shoshannah Forbes <[email-masked]>
Date: Thu, 26 May 2005 01:43:56 +0300

On 26/05/2005, at 01:23, Arik Baratz wrote:
>  Have visually challanged people call

Visually challenged people are not the only ones that loose 
accessibility due to visual captcha's. There are also people with 
dyslexia, and even "normal" people with sum captcha's generated.

Although having a one time captcha is not that bad (but still a pain).

>  but if we want it up for the time being then definitely.

Ah, so it is a binary thing? Either captcha or nothing at all? What 
about email verification mentioned here before?

> A home-grown captcha on our page
>  captcha-guessing engine for OUR site, which is even more effort.

There are general captcha-beating scripts out there, that can probably 
beat many home-grown captcha systems. For example:
http://www.puremango.co.uk/cm_breaking_captcha_115.php
http://sam.zoy.org/pwntcha/

---
Shoshannah Forbes
http://www.xslf.com

---

From: Gadi Evron <[email-masked]>
Date: Thu, 26 May 2005 01:07:15 +0400

> used to spam the wiki if that's any help.

Sorry for not responding earlier.

DDoS'ing me is one of the stupidest thing anyone can do.. and
unfortunately I am a member of this list and take them DDoS'ing you
rather personally.

I will help, please provide me with as many IP's + timestamps.

A tcpdump sample log would also be nice.

	Gadi.

---

From: [email-masked]
Date: Thu, 26 May 2005 08:29:06 +1000

> captcha-guessing engine for OUR site, which is even more effort. I say

Which made me think - I don't know about developing a captcha in-house
(there's probably some mathematics and heuristics developed in this area)
but if the captcha is in Hebrew then I guess it will cut down the number of
possible spam bots who can get through it by 99% (the percentage of
hebrew-speaking people among the porn users).

> stay down during that year, or it can stay up with minimal effort.

I'd vote for at least a trial of a captcha (maybe hebrew-based).
Don't let what I percieve to be more theoretical arguments shoot this
option down.

--Amos

---

From: [email-masked]
Date: Thu, 26 May 2005 10:06:48 +1000

> Although having a one time captcha is not that bad (but still a pain).

When I brought the Captcha idea it was for registration only, not
for every time a user wants to login or post something.

> about email verification mentioned here before?

No. But captcha's seems to be the strongest practical tool against
spammers.

I raised the captcha idea after arguing that many spam bots easely
circumvent the e-mail registration verification with temporary e-mail
addresses.

> http://www.puremango.co.uk/cm_breaking_captcha_115.php

The first one simply relays on a basic bug in the captcha script logic
which allows multiple attempts - pretty similar to brute-force. It can be
easely fixed - limit the number of attempts to one or a very small number
(it doesn't make sense that a reasonable user will try the same captcha
more than about ten times).

> http://sam.zoy.org/pwntcha/

The "Escape from Captcha" link there indeed puts up some good arguments
against captcha - but not many solutions.

A couple of answers I though about what it says:

1. (Already suggested this): I gues that using Hebrew captcha should lock
out most of the robots and circumnavigation schemes.

2. A new idea - monitor the wiki:

   a. Create multi-level user accounts (two is enough?) - users who already
edited some pages fine or were verified in some other way will become unlimited.
   b. Large changes (complete page replacements, or additions which consist
mostly on many URL's) will have to be approved by an "unlimited user".

Just some ideas.

I'm trying not to loose the focus on what we want to achieve here:
on one hand accessibility to a WIKI (i.e. any non-melicious member of the
public should be able to participate) while avoiding automatic robots from
defacing the pages.

Cheers,

--Amos

---

From: [email-masked]
Date: Thu, 26 May 2005 13:37:51 +1000

> A tcpdump sample log would also be nice.

I'd be curios to hear war stories from this one if you may....

>         Gadi.

--A

---

From: Shoshannah Forbes <[email-masked]>
Date: Thu, 26 May 2005 17:40:54 +0300

>  circumvent the e-mail registration verification with temporary e-mail

I am not sure how they do that- these registration schemes tend to send 
the password for the logging (or an authentication token) to the mail 
box. Even if they use a temp email, they still need to check the 
mailbox to get the token/password.
---
Shoshannah Forbes
http://www.xslf.com

---

From: Arik Baratz <[email-masked]>
Date: Thu, 26 May 2005 18:08:40 +0300

> mailbox to get the token/password.

Step 1. Buy a domain, can be anything really (like 93763924bbb.com)
Step 2. set up MX record for the domain to a machine
Step 3. Install an SMTP server on the machine that drops every
incoming email message in a file
Step 4. Parse the files as RFC-2822 messages, and look for a URL in the body
Step 5. Visit every URL (i.e. perform the HTTP GET request and discard
the result)

And then run your robot and have it sign up for users and spam away.

Again, we can make it more complicated by (for example) having a
JavaScript algorithm on the confirm page that performs some action on
the client-side and sends the result or do other tricks to see if
there's a real browser on the other side of the connection. It's an
arms race, really.

--- Arik

---

From: Tal Kelrich <[email-masked]>
Date: Thu, 26 May 2005 18:49:47 +0300

On Thu, 26 May 2005 18:08:40 +0300

> there's a real browser on the other side of the connection. It's an

Better yet, give them a link for "I do not confirm, delete this user
now", should be clearly marked, though

-- 
Tal Kelrich
PGP fingerprint: 3EDF FCC5 60BB 4729 AB2F  CAE6 FEC1 9AAC 12B9 AA69
Key Available at: http://www.hasturkun.com/pub.txt
----
Noise proves nothing. Often a hen who has merely laid an egg cackles as
if she laid an asteroid. -- Mark Twain
----