Need Advanced Reliability Testing? Try JRRE Calculator!

Jouve's Randomized Reliability Estimation (JRRE)

The Jouve's Randomized Reliability Estimation (JRRE) method refines the estimate of a test's consistency by averaging the reliability across multiple randomized test splits. The Fisher-Yates shuffle algorithm (Fisher & Yates, 1948) is utilized to ensure each split is truly random:

Randomly shuffle the items using the Fisher-Yates algorithm to ensure independence between splits.

For each shuffled sequence, split the test items into two halves.

Calculate the sum of scores for each individual on both halves.

Compute the Pearson correlation coefficient (r) between the two sets of scores for each split.

Apply the Spearman-Brown prophecy formula to each correlation coefficient to estimate the reliability of the full test.

Average the reliability estimates from all splits to obtain the final Jouve's Randomized Reliability Estimation.

The Fisher-Yates shuffle algorithm rigorously randomizes the order of elements in an array, ensuring that each permutation of the array is equally likely. The algorithm proceeds as follows:

Given an array \( a \) of \( n \) elements (indices \( 0 \) to \( n-1 \)):

Start from the last element of the array and move towards the first element, denoted as \( i \) (beginning with \( n-1 \) and decrementing by \( 1 \) each iteration).

For each \( i \), pick a random index \( j \) such that \( 0 \leq j \leq i \).

Swap the elements \( a[i] \) and \( a[j] \).

Continue this process until \( i \) reaches \( 1 \) (the second element of the array).

The detailed formula for each iteration \( i \) of the array is:

\[ \begin{align*} &\text{For } i = n-1 \text{ downto } 1: \\ &\quad \text{Choose a random integer } j \text{ such that } 0 \leq j \leq i \\ &\quad \text{Swap } a[i] \text{ with } a[j] \\ \end{align*} \]

After the algorithm completes, the array \( a \) is a random permutation of its original order, with each permutation having an equal probability of occurrence.

The Spearman-Brown prophecy formula (Cronbach, 1946) is utilized to extrapolate the reliability of a full test from the reliability of half of the test. This extrapolation is necessary because the reliability of half a test is typically less than that of the whole. The formula, applied to each split-half correlation, is detailed as follows:

\[ Reliability_{split} = \frac{2 \times Correlation_{split}}{1 + Correlation_{split}} \]

Where \( Correlation_{split} \) is the Pearson product-moment correlation coefficient between the two halves' scores for a given split. The coefficient is calculated as:

\[ Correlation_{split} = \frac{\sum_{i=1}^{n} (X_{i} - \overline{X})(Y_{i} - \overline{Y})} {\sqrt{\sum_{i=1}^{n} (X_{i} - \overline{X})^2 \sum_{i=1}^{n} (Y_{i} - \overline{Y})^2}} \]

Here, \( n \) is the number of observations, \( X_{i} \) and \( Y_{i} \) are the scores on the first and second halves for the \( i \)-th observation, and \( \overline{X} \) and \( \overline{Y} \) are the mean scores for the first and second halves, respectively.

The numerator represents the covariance between the two sets of scores, and the denominator is the product of the standard deviations of the two sets of scores. This formula effectively measures the degree to which two sets of scores vary together, and by extension, the consistency of the test.

The Spearman-Brown formula then doubles this correlation to adjust for the fact that the test length has been halved. The formula's denominator normalizes the doubled correlation, ensuring the reliability estimate does not exceed 1. This adjustment is crucial for providing an accurate estimate of the full test's reliability from a single split-half correlation.

The JRRE is derived by averaging the reliability estimates from multiple independent test splits. The detailed formula for calculating the average reliability across \( N \) splits is given by:

\[ JRRE = \frac{1}{N} \sum_{i=1}^{N} Reliability_{split_i} \]

The term \( Reliability_{split_i} \) represents the reliability estimate from the \( i \)-th split, calculated as:

\[ Reliability_{split_i} = \frac{2 \times Correlation_{split_i}}{1 + Correlation_{split_i}} \]

Where \( Correlation_{split_i} \) is the Pearson correlation coefficient for the \( i \)-th split. The average is calculated by summing these individual reliability estimates and then dividing by the total number of splits \( N \), as shown:

\[ JRRE = \frac{1}{N} \left( \sum_{i=1}^{N} \frac{2 \times Correlation_{split_i}}{1 + Correlation_{split_i}} \right) \]

This equation accounts for the variance in test reliability due to the random nature of the splits and ensures that the final estimate represents the overall consistency of the test.

To ensure the robustness of this estimate, each split's correlation coefficient is calculated using the scores from the randomized halves, and the Spearman-Brown prophecy formula is applied to adjust for the fact that each half is only part of the full test.