My Dominant Hemisphere

The Official Weblog of 'The Basilic Insula'

How Examinations And Diagnostic Tests Are Similar

with 4 comments

Source, Author and License

“You are more than a score”, or so the saying goes. But how much of that comes out as an emotional outburst as opposed to objective and rational thinking? Let’s try to see why the above is totally true, scientifically speaking.

In medicine, we’ve learned a lot about diagnostic tests, right? In fact everything investigative in nature can be considered a diagnostic test. Be it a screening exam for cervical cancer, that blood test for glucose, an X-ray for a broken arm, or your palpating hand feeling for that enlarged liver. Heck, even an entire research study could be considered a diagnostic test. The ‘null hypothesis’ technique often used in analytical research studies is nothing more than a diagnostic test of sorts.

When considering the dynamics of a diagnostic test, a fundamental underlying principle is that we separate what is observed via the test from the actual truth. In the case of tangible phenomena like death, disease and disability, it is quite easy to distinguish the actual truth from what the test predicts. Because of this, you have terms like ‘false positives’, ‘false negatives’ and the like. A pregnancy test for example could be positive, but you could easily compare that prediction to the actual outcome (pregnancy vs. non-pregnancy) and say that this particular test has got such and such false positive rates. More or less, all tests have the following attributes in this regard:-

  1. Sensitivity
  2. Specificity
  3. Positive Predictive Value
  4. Negative Predictive Value
  5. Validity/Accuracy
  6. Reliability/Precision

We ought to think about examinations such as the USMLE, etc. in this manner as well. Why? Well, because they are investigations too! Think of them as X-rays to diagnose your intelligence or whatever, if that metaphor helps. And as a consequence, notions about false positives, false negatives and all of the other things on that list also apply to them. Being the abstract intangible thing intelligence is, it is impossible to know its true value. And because there’s no way to compare prediction versus truth, it is impossible to say for sure what the false positive or negative rates (or any item on that list) for an exam are. And that’s why, ‘you are more than a score’ ! Statistically speaking, examinations are just so lame !

Do send in your comments!

Readability grades for this post:

Kincaid: 9.3
ARI: 9.8
Coleman-Liau: 12.9
Flesch Index: 58.0/100
Fog Index: 13.2
Lix: 42.9 = school year 7
SMOG-Grading: 12.0

Powered by BlogJet and Ubuntu Linux 7.04

Copyright © 2006 – 2008 Firas MR. All rights reserved.

4 Responses

Subscribe to comments with RSS.

  1. I hate academic Exams – especially First year and Second year University Exams which seem like an elimination round !

    Test in community colleges are more fun – the more tests you get to write, the better the average – provided you studied !

    I still don’t understand what you meant – but I’ll take your word for it. I think I trust doctors too much !

    Jaffer

    April 2, 2008 at 11:29 pm

  2. Jaffer: Thanks for commenting, dear buddy ol’ pal🙂 . LOL @ trusting doctors😀 . Boy, if only there could be more people like you, malpractice suits would be a scarcity in today’s day and age! LOL!! Seriously man, some doctors in the US have gotta spend nearly half of their incomes on malpractice insurance! Ob/Gyns and Neurosurgeons come to mind.

    See, the main idea I’ve tried put here is that exams should be treated just like any other statistical test. Because I’m engrossed in the medical world, I chose to give examples of medical tests to illustrate this point. Just as you can’t believe a pregnancy test blindly, knowing very well that it has a certain false positive rate (based on company handouts or independent research for example), you shouldn’t believe examinations either! Examinations are in fact a lot worse. That’s because when it comes to examinations, there isn’t a concrete way to compare a prediction versus the true value of what is being tested (in most cases the thing people are tested for is intelligence) for the simple reason that stuff like intelligence, etc. is intangible.

    Bottomline, if the label on a test doesn’t give you clear cut data on the 6 attributes mentioned in my post, it’s garbage! Now did ya understand why the pregnancy test is better than some lame exam😛 ? At least it’s got a clear cut label!

    Go ahead and ask yourself (and your professor) these questions the next time you appear for an exam. What’s the test’s sensitivity? What’s the false positive rate? And so on. Those head-pecking profs will find it very hard to come up with an answer! LOL!! I hope this has cleared your doubts🙂 and the next time you sit for an exam, you’ll feel like master of the universe.. lol!🙂

    Firas MR

    April 3, 2008 at 2:36 am

  3. Yup, I hate exams too…but then, who doesn’t?

    I’m not too sure I understood all that stuff abt diagnostic tests and false promises…er…false positives🙂 either.

    But you gotta admit, most tests are required just to establish certain baselines to proceed further, even if they do seem kinda dumb.

    Nice post.

    Noor

    April 11, 2008 at 8:49 pm

  4. Thanks for your comment ARN! Hmm…Let me try helping you understand this. The whole premise underlying probability and statistical tests is that you can’t know the 100% truth about something. At best, you can make estimates. More importantly, quantifiable estimates. It’s all like the Heisenberg uncertainty principle we learned at school.

    When you administer a test – medical test, examination, whatever – you are actually carrying out a statistical test to estimate the truth about an object of interest. Medical/diagnostic tests estimate disease. Examinations are meant to estimate intelligence.

    Because you know 100% truth can’t be known, it is important for you to set quality standards for your statistical tests. It is quite easy to compare how best your estimate compared to the actual truth if you became aware of it later. So for example, if a medical test predicted death in all of 100 patients and it was found later that only 60 died, the false-positive rate for this test is 40%. The next time you carry out this test you know what the margins of error and uncertainty are. The 6 attributes mentioned in my post describe these qualities in various forms. Predictions of any statistical test need to be considered with these borne in mind. If a test package doesn’t come with what the levels of uncertainty are, it’s good for nothing!

    What about examinations and intelligence? How do you set quality standards? The situation here is problematic. You are interested in knowing the truth about a guy’s intelligence. But there’s no way you can quantify such an abstract thing. All people are left with is quantifying and estimating practical impressions of intelligence. Does the guy seem like a dufus😛 ? Is his laugh goofy😛 ? Does he do well in calculating quadratic equations? And all that crap. lol!!🙂 There are no quality standards that can be applied to measuring intelligence as such. All you can do is set standards for measuring impressions of intelligent behavior, etc. Impressions, just like any other philosophical crap are debatable!

    I hope I’ve cleared your doubts.

    I do agree with you on the practicality of conducting exams, but people often forget that exams don’t really measure intelligence. There are, therefore, no real quality standards to assure you of their implications.

    Firas MR

    April 12, 2008 at 4:23 am


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: