What can you pull?
If we wanted to know how much weight you could lift, would you start with one kilo? No, this would be no test at all of your strength.
Would you start at 100 kilos? Again, for most folks, this would be too difficult and we're really not assessing anything.
Yet traditional tests do just that as they put a percentage of their questions either at the 'beginning' or at the 'advanced' levels of difficulty...wasting everyone's time.
Starting in the middle
We could start more in the middle...around 50 kilos...and depending on whether you could lift that amount, we could the increase or decrease the amount in 5 kilo increments.
If, after a couple of attempts, we find that you can lift around 40 kilos, we then we adjusts the increments to say 1/2 kilos to get a highly accurate assessment.
Computer-adaptive testing
Computerized adaptive testing (CAT) does just this: the exam adapts or adjusts to the examinee's individual ability level.
For example, if an examinee performs well on a test item of intermediate difficulty, they will then be presented with a more difficult question.
Or, if they get the question wrong, they would be presented with a simpler question.
Comparing CAT to traditional 'fixed' tests
Standardized tests categorize questions into different levels of complexity.
For example, if you were testing for English language prophecy, you would use the six levels of the CEFR. You would put more questions or weight your test toward the middle or intermediate levels and put less questions at the extreme ends, i.e. the beginning and advance levels.
Exam made under this traditional method need a lot of questions. One major English proficiency test asks 200 questions in an exam that last 150 minutes!
CAT tests are much shorter
According to Wikipedia, “…an adaptive test can typically be shortened by 50% and still maintain a higher level of precision than a fixed version.”
This translates into a time savings for the test-taker. Test-takers do not waste their time attempting items that are too hard or trivially easy.
It also means that the much less chance of “exam fatigue” where the candidate begins to perform at less than their optimal level because they are becoming less focused.
CAT test cost less
The testing organization benefits from the time savings as the cost of candidate seat time is substantially reduced. That means that they can also run more tests in the same amount of time.
A test bank of questions
One major Computer-adaptive language test asks the candidate 50 to 55 questions. And the questions are served up from a Test Bank containing approximately 10,000 questions.
Large test banks also reduce the risk of test item (question) exposure.
Greater security .
Large test banks also reduce the risk of test item (question) exposure.
After the first couple of introductory questions, candidates taking the exam at the same time are asked different questions as the exam begins for focus on their individual levels. It makes cheating that more difficult as it’s impossible to predict which questions will be a given to each individual candidate.
Do you want to see CAT in action?
The BULATS exam is a test of business English and it's fully computer-adaptive. You can learn more here.
About the Author
I’m the Managing Director at Vantage Siam, an authorized Cambridge English Exam Centre and management training firm.
We offer English language training via blended learning to both students and professionals in Thailand and also to remote learners throughout China.
