How to use the test item analysis calculator
- Paste one line per student with that student's score on each item, separated by spaces or commas. Right or wrong items use 1 and 0.
- Set the points an item is worth if the items are scored on more than 0 or 1. Every item must have the same maximum.
- Open the options to change the share of students in the top and bottom groups, which is 27% as an example.
- Read each item's difficulty, discrimination and item-total correlation, and the test's reliability and standard error of measurement.
Formula
difficulty = mean item score / points available
discrimination = (mean of top group - mean of bottom group) / points available
item-total correlation = Pearson correlation of the item with the sum of all other items
alpha = k / (k - 1) x (1 - sum of item variances / variance of total scores)
SEM = SD of total scores x sqrt(1 - alpha)
For right or wrong items, alpha equals the Kuder-Richardson formula 20. Students are ranked by total score, and the top and bottom groups hold the stated share of students.
Ten students, five questions
With the example scores, the totals are 4, 3, 3, 5, 1, 2, 3, 1, 4 and 1, so the mean is 2.70 and the standard deviation is 1.418. Alpha is 0.4972 and the standard error of measurement is 1.006 points. Item 1 was answered correctly by 8 of 10 students (difficulty 0.800) and item 4 by 3 of 10 (0.300).
With 27% of 10 students rounded to 3, the top three students and the bottom three are compared. Every item here has a discrimination of 0.667, meaning the top group scored two thirds of a point more per question than the bottom group. Item 1 has an item-total correlation of 0.396 and item 5 has 0.138.
Reading the three item measures
Difficulty tells you how many points students earned on the item: a high value is an easy item. Discrimination tells you whether students who did well overall also did well on this item. The item-total correlation says the same thing in a form that uses every student. A negative value means the top students did worse on the item than the bottom students, which is worth checking for a wrong key or a confusing question.
Reliability (alpha) says how consistently the items measure the same thing. It rises with more items and with items that agree. The Quiz Score Calculator totals scores, and the Grade Calculator turns them into course grades.
Limits
These are classical test statistics for the group you entered. With few students they change a lot from sample to sample, so use them to find items to look at again, not to judge a student, a class or a teacher. Ties at the edge of the top or bottom group are split by the order you entered.
More statistics tools
For related statistical tests and measures, try the Mann-Whitney U Test Calculator, Fisher Exact Test Calculator and P Value Adjustment Calculator.
Assumptions and limits
- Each row is one student and each column is one item, and every student has a score for every item.
- All items have the same maximum score.
- Top and bottom groups are the same size and are rounded to whole students.
- Example data are shown until you replace them.
Frequently asked questions
What is item difficulty in test analysis?
It is the average score on the item as a share of the points available. If 8 of 10 students got an item right, its difficulty is 0.80, so it is an easy item.
What is the discrimination index?
It is how much better the top-scoring students did on an item than the bottom-scoring students, as a share of the points. A value near 0 means the item does not separate them.
What is KR-20?
It is the Kuder-Richardson formula 20 for the reliability of a test whose items are scored right or wrong. It equals Cronbach alpha for items scored 1 and 0.
What does a negative discrimination mean?
The students with the highest totals scored lower on that item than those with the lowest totals. Check whether the answer key is wrong or the question is confusing.