Subject: CAPTCHA is NOT a Turing test, or even close

From: "IETF member Dave Aronson" <[email-masked]>
Date: Wed, 26 Sep 2007 13:12:24 +0000

Pars Mutaf [mailto:[email-masked]] writes:

 > On 9/26/07, John L <[email-masked]> wrote:
...
 > > approaches that depend on something like a CAPTCHA to
 > > work don't have much of a long term future.
 > 
 > I respect your opinion but it says that one day we won't be able to tell
 > humans and computers apart.

While that may or may not be true, it's not the only mechanism by which CAPTCHAs can be defeated.

First, many poor implementations aren't really all that difficult to OCR.

Second, many sites use a very limited set of images, whether static or generated, making it easy to fingerprint them and build a database of correct responses.

Third, the responses are generally short enough that the "keyspace" of correct responses is short enough to brute-force.  (Yes, I know it's usually changed after each try (though again some poor implementations don't), so it's not the typical dictionary-style of brute force attack.  Even so, each response stands the same chance of success, making infinite retries still viable.)  Remember, if it's automated, no attacker really cares how many tries it takes, so long as it is likely to succeed within a reasonable number of tries.  Lockouts and such can hellp with this, but again, a lot of sites don't bother.

Last, and most amusingly, I've seen rumors that some spambots and suchlike farm it out, by using CAPTCHAs that were, ahem, CAPTCHA'd from elsewhere, to control access to things such as porn sites, relying on the horndogs to solve them in close enough to real time that the originating site will accept it.  Even if this isn't really happening, or even feasible, it's a clever idea IMHO.

Upshot: CAPTCHAs are not to be relied upon for anything really important (such as preventing even a possibly-inadvertent DDoS attack on cellphone users' patience), not now and certainly not when designing a protocol that may be in use for decades to come.  Moore's Law will bite you HARD.

-Dave

-- 
Dave Aronson
"Specialization is for insects."  -Heinlein
Work: http://www.davearonson.com/
Play: http://www.davearonson.net/

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Joel Jaeggli <[email-masked]>
Date: Wed, 26 Sep 2007 08:28:06 -0700

> Upshot: CAPTCHAs are not to be relied upon for anything really important (such as preventing even a possibly-inadvertent DDoS attack on cellphone users' patience), not now and certainly not when de…

Captcha's that require visual acuity and pattern recognition more easily
discriminate against the visually impaired then they do robots. There
are other flavors of Captcha which have similar issues for the deaf,
color blind, non-print-literate, differing linguistic background from
the author, poor empathy skills etc.

The degenerate condition for turing tests is an arms race between the
robots and the captcha designers making it progressively harder to prove
that you are in fact a human, which just makes the process of
identifying yourself as such all the more annoying.

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Wed, 26 Sep 2007 10:24:04 -0700

CAPTCHA is by definition an attempt to create a Turing test, its what the T stands for.

The question is whether 1) a particular CAPTCHA is an effective Turing test and 2) whether an effective Turing test is an effective security measure.

The answer to the first question is usually yes and the answer to the second is, yes but only against a casual attacker. Professional Internet Criminals buy technology to defeat CAPTCHA from specialists. The prices paid are not very high given the amount of effort required. Note that this effect was anticipated by the original authors, the original subtitle of the paper is 'How lazy cryptographers do AI'.

http://www.captcha.net/captcha_cacm.pdf

>From a security point of view it absolutely does make a difference if we are dealling with a few tens of thousands of hard core professional criminals or several million script kiddies. Most car alarms can be defeated by the professional thief but they significantly reduce the ability/temptation of kids to steal a car to go joy riding.

If I have an asset that is worth $X to a professional criminal and a control that will cost >> X to defeat the asset is reasonably secure.

>From a security research point of view, yes anyone who puts up a CAPTCHA scheme and claims it is absolutely secure should expect to be hit by a few tomatoes. 

Yes I have been critical of a few particular CAPTCHA schemes and shown how the security claims made are not sustained. But please don't apply the security considerations we apply to the design of cryptographic security protocols as essential criteria that every security protocol must meet. Crypto is in a separate class because the tools we have are so good that we can reasonably expect to design protocols that are secure from cryptanalytic attack. That is not the case for security applications and in particular when we consider the user experience.

CAPTCHA can certainly be very effective if your objective is to stop ballot stuffing or casual trolling/spam. It is not a foolproof control against a professional criminal. But we have other ways to deal with those.

> -----Original Message-----
> From: IETF member Dave Aronson [mailto:[email-masked]] 
> Sent: Wednesday, September 26, 2007 9:12 AM
> To: [email-masked]
> Subject: CAPTCHA is NOT a Turing test, or even close
> 
>  > > approaches that depend on something like a CAPTCHA to  > 
>  > I respect your opinion but it says that one day we won't 
> be able to tell  > humans and computers apart.
> 
> While that may or may not be true, it's not the only 
> mechanism by which CAPTCHAs can be defeated.
> 
> Second, many sites use a very limited set of images, whether 
> static or generated, making it easy to fingerprint them and 
> build a database of correct responses.
> 
> Third, the responses are generally short enough that the 
> "keyspace" of correct responses is short enough to 
> brute-force.  (Yes, I know it's usually changed after each 
> try (though again some poor implementations don't), so it's 
> not the typical dictionary-style of brute force attack.  Even 
> so, each response stands the same chance of success, making 
> infinite retries still viable.)  Remember, if it's automated, 
> no attacker really cares how many tries it takes, so long as 
> it is likely to succeed within a reasonable number of tries.  
> Lockouts and such can hellp with this, but again, a lot of 
> sites don't bother.
> 
> Last, and most amusingly, I've seen rumors that some spambots 
> and suchlike farm it out, by using CAPTCHAs that were, ahem, 
> CAPTCHA'd from elsewhere, to control access to things such as 
> porn sites, relying on the horndogs to solve them in close 
> enough to real time that the originating site will accept it. 
>  Even if this isn't really happening, or even feasible, it's 
> a clever idea IMHO.
> 
> Upshot: CAPTCHAs are not to be relied upon for anything 
> really important (such as preventing even a 
> possibly-inadvertent DDoS attack on cellphone users' 
> patience), not now and certainly not when designing a 
> protocol that may be in use for decades to come.  Moore's Law 
> will bite you HARD.
> 
> -Dave
> 
> --
> Dave Aronson