Subject: I want to believe

From: Chen Shapira <[email-masked]>
Date: Tue, 20 Feb 2001 22:44:24 +0200

Hi,

Some of you may remember one of X-files slogans : "I want to believe"

Infact many people justify their faith in anything on that basis - their
need to believe.

An american cartoonist decided to have fun:

http://www.attackcartoons.com/believe.GIF

Anyway,
Its getting quite late, and here I am at work, doing stuff that don't really
need to be done, you know, some "nice to have" features.
I wonder if anyone could calculate the productivity gained by students
working late because they know that once they get home they'll have to start
studying for some test or another.

Oh, and one more thing, I took an Intro to Probability test today (went
fine, thanks) and it was of the multi-choice type.
We discussed why we hate this, and the fact that it doesn't measure
understanding.
The same professor that gave us this test, also gave a multi-choice test for
complex functions course, which is very extreme, I think.
Well, this particular test, had 18 questions, a right answers gives you 6
points, a wrong one substracts 2.
Lets assume I came to the test with 75% knowledge.
So each question I get right with 0.75 probability.
The expectancy of my score is quite reasonable - 72.
The variance on the other hand is 6048 (18 independent questions each with
variance of 12).
Unless I had an error of calculations, this is an extremely high variance,
meaning that a test score has low chance of reflecting my actual knowledge.

Oh well.

Good night all.

---

From: "Nadav Har'El" <[email-masked]>
Date: Tue, 20 Feb 2001 23:30:07 +0200

> meaning that a test score has low chance of reflecting my actual knowledge.

First, the square root of the variance (the standard deviation) gives a
more meaningful measure...

Second, I don't see what the high variance has anything to do with the
correlation between the score and your knowledge! I mean, if you answered
questions by random, then you're right - you'd expect a crazy distribution
of random scores, ranging from -36 to 108 with a high variance. But that's
not what happens: assume you have a intrinsic success fraction A (e.g., 0.75 in
your previous example), and a small chance of making errors on questions
you ought to know or succeeding in questions you ought not to know.
This small chance means that A is no longer a constant, but rather a random
variable with a narrow variance (not that it doesn't have anything to do
with the variance you calculated).

So, if A is gaussian with a mean 0.75 and standard deviation, say, 0.05,
then you can calculate your score to be roughly
	S = 18*6*A - 18*2*(1-A)
or
	S = 144*A - 36

If you had a success rate A=0 (a constant distribution), you'd get -36,
if you had a success rate A=1 (a constant distribution), you'd get 108
(your teacher invented these weird limits)

But in our example, where A is a gaussian with mean 0.75 and stdandard
deviation 0.05, you'd get that the expected value of S is 144*0.75-36,
or 72 (as you calculated), and a standard deviation of 144*0.05, or 7.2
points.

It's only slightly higher than the 5 point standard deviation you'd "expect"
when you chose 0.05 as a standard deviation for A, and it is higher only
because your range (-36..108) is higher than the range 0..100.

So I don't see why you think this test can't reflect your knowledge.

-- 
Nadav Har'El                        |     Tuesday, Feb 20 2001, 28 Shevat 5761
[email-masked]             |-----------------------------------------
Phone: +972-53-245868, ICQ 13349191 |I love deadlines. I love the whooshing
http://nadav.harel.org.il           |sound they make as they go by.