Item Response Theory (IRT): What It Measures and What It Demands

Why a raw pass rate is not enough

A question's overall success rate says nothing about its quality: it may be badly worded, too easy to separate levels, or passed by guessing. Two items at 60% can behave in opposite ways depending on who gets them right.

The three parameters

Difficulty locates the ability level needed to answer correctly. Discrimination measures how well the item separates strong candidates from weak ones — the most useful parameter in practice. The third, pseudo-guessing, models success by chance, which matters most on multiple-choice items with few options.

What IRT demands before it produces anything

Credible calibration requires hundreds of responses per item, comparable testing conditions, and someone able to judge whether the model fits the data. On a cohort of thirty students, the estimated parameters are noise. A vendor advertising "automatic recalibration after every session" is mostly selling an unstable number.

What Educati provides, and what it does not

Educati computes no IRT parameters and announces no psychometric engine. The platform auto-marks multiple-choice items, records results per question and per student, and normalises every mark onto the Moroccan /20 scale. A team wanting IRT starts from those results and processes them in its own statistical tool.

A useful item review without an engine

First find the items your strongest candidates fail more often than the rest: that is almost always a wording flaw or a wrong answer key. Then the items everyone passes, which carry no information. That review needs neither a model nor a statistician, and fixes most of what is wrong with a question bank.