Validity, Reliability & FairnessApril 1, 2020 2020-05-30 4:23
Validity, Reliability & Fairness
The results are in: The GMAT™ exam more accurately predicts success in your program than grade point averages (GPAs).
Combining GMAT™ exam results and undergraduate GPAs is a powerful way to predict academic success.
For more than 50 years, the GMAT exam has been repeatedly analyzed, evaluated, and modified to ensure that it continues to meet the needs of graduate management programs worldwide.
What the Reliability of the GMAT Exam Means to You and Your Program
When you use a test to evaluate candidates’ potential to succeed in your program, you need to be confident that candidates with similar abilities will get similar scores. Reliability indicates the extent to which test scores are consistent over repeat sitting, and is a key reason why test scores have meaning.
Reliability is measured on a scale in which the lowest reliability is 0 and perfect reliability is 1. No test or human being is perfectly reliable, but the average reliability for GMAT exam scores is quite high, as shown below:
- Total score: 0.92
- Verbal score: 0.89
- Quantitative score: 0.90
What the Validity of the GMAT Exam Means for You and Your Program
You also need to be confident that your admission exam measures what you think it measures. GMAT exam scores have been shown to be strong predictors of first-year grades in graduate management programs, and GMAC continues to perform validity studies to statistically verify that the exam predicts success in your program. Validity correlations are measured on a scale of 0 to 1, in which 0.4 is typical for admission tests. Based on nearly 400 studies over the past 20 years, the median correlation between GMAT scores and mid-program graduate management school grades is much higher (0.48) than the relationship between undergraduate GPAs and mid-program graduate management school grades (0.30). The best predictions of candidate success come from combining data from GMAT scores and undergraduate GPA (0.53).
Making Better Decisions with the GMAC Validity Study Service
By analyzing your students’ GMAT scores, undergraduate GPAs and additional admission factors of your own, you can make your predictions even more accurate. Use the free Validity Study Service—available to schools and programs that accept the GMAT exam—to help pinpoint the relationship between GMAT scores, grades, and any other factors you choose to predict performance in your school’s program. This free service lets you know exactly how valid the GMAT exam is for your program and which factors can help you make better admission decisions.
How We Conduct Statistical and Sensitivity Analyses of Fairness
The GMAT exam is developed using best practice methods to ensure fairness for GMAT test-taker subgroups. To ensure that questions are not biased against any sub-population group of GMAT test takers, we at GMAC use three important procedures in the test-development process.
- GMAC thoroughly trains writers of GMAT questions to avoid topics and questions that may favor or disadvantage particular subgroups of test takers.
- Before including new questions in the GMAT exam, we have them reviewed by independent panels of fairness experts. If fairness reviewers identify potential issues of sensitivity or bias in a question, we revise or discard it.
- When we are considering a new question, we first ask it on a non-scoring basis on the GMAT exam and then analyze test-taker response data. Only after GMAT questions pass statistical criteria do we use them in calculating actual GMAT scores. For example, one statistical analysis we perform identifies questions that may favor test takers from one subgroup over another. Questions flagged by these methods are further reviewed by test development experts and then revised, discarded, or approved. Then we analyze the revised items again. Only approved questions are subsequently scored on the GMAT exam.
GMAT Validity Studies and Fairness
We at GMAC compile data based on validity studies and combine them to look at different subgroups. Outside experts in the field review the statistical analyses and findings prior to our publishing them. The studies we conduct include group analyses by gender, race/ethnicity for US students, language, nationality, and age. To date, each of the studies found that GMAT scores are not biased and predict well for all subgroups.