Here’s the link to the PDF version.

Current Biology, 16 May 2013
Copyright © 2013 Elsevier Ltd All rights reserved.
10.1016/j.cub.2013.04.045

Authors

Albert Snowball, Ilias Tachtsidis, Tudor Popescu, Jacqueline Thompson, Margarete Delazer, Laura Zamarian, Tingting Zhu, Roi Cohen Kadosh

Highlights

• Subjects received TRNS of the bilateral DLPFC while undergoing arithmetic training
• TRNS was uniquely coupled with NIRS, an optical brain imaging technique
• TRNS elicited short- and long-term improvements in trained and untrained material
• Hemodynamic responses suggested enhanced neurovascular coupling efficiency

Summary

Noninvasive brain stimulation has shown considerable promise for enhancing cognitive functions by the long-term manipulation of neuroplasticity [1,2,3]. However, the observation of such improvements has been focused at the behavioral level, and enhancements largely restricted to the performance of basic tasks. Here, we investigate whether transcranial random noise stimulation (TRNS) can improve learning and subsequent performance on complex arithmetic tasks. TRNS of the bilateral dorsolateral prefrontal cortex (DLPFC), a key area in arithmetic [4,5], was uniquely coupled with near-infrared spectroscopy (NIRS) to measure online hemodynamic responses within the prefrontal cortex. Five consecutive days of TRNS-accompanied cognitive training enhanced the speed of both calculation- and memory-recall-based arithmetic learning. These behavioral improvements were associated with defined hemodynamic responses consistent with more efficient neurovascular coupling within the left DLPFC. Testing 6 months after training revealed long-lasting behavioral and physiological modifications in the stimulated group relative to sham controls for trained and nontrained calculation material. These results demonstrate that, depending on the learning regime, TRNS can induce long-term enhancement of cognitive and brain functions. Such findings have significant implications for basic and translational neuroscience, highlighting TRNS as a viable approach to enhancing learning and high-level cognition by the long-term modulation of neuroplasticity.

The TLAP-R (Jouve, 2013a) is an untimed, non-verbal reasoning test prepared with perceptual material. It consists in 8 matrices, each one including 6 lines of 15 patterns of which 4 have been deleted. The examinee is asked to find the 4 missing patterns. The first version (limited to 5 matrices) of this test has been prepared by Xavier Jouve in mid 2000 for a research project.

The chiefly involved cognitive process mostly depends on fluid intelligence and the test is minimal-knowledge based. The material used to prepare the matrices is numerical but not heavily mathematically based so that the TLAP-R is suitable to assess any person with only basic arithmetic knowledge.

In order to complete the matrices, to find which patterns are missing, the TLAP-R requires inductive reasoning. The task requires figuring out a specific logical and governing rule from a chaotic situation. Spearman (1927) described this process and considered it as the requisite for the eductive part of g. However, the TLAP-R showed a strong relationship with crystallized intelligence even without measuring it directly (Jouve, 2013b).

Construct validity is the examination of the theoretical background of the test (Pennington, 2003) by the rationale of the author or through analysis of the empirical data collected. A test is always the empirical representation of the theoretical idea presented by the test designer. If the theoretical ideas of the person who created the test are not correct, construct validity will not be demonstrated. However, if the experiment has been prepared according to a proper rationale, it will demonstrate construct validity.

There are two ways to give an indication of a test’s construct validity. On the one hand, the first method that is issued form differential psychology, is an analytic method, which tends to give an approach based on dimensional measures. In this manner, the construct validity of the assessment is checked if the given measure is similar to the one from a test that is supposed to be tapping the same latent construct. Correlations with equivalent and non-equivalent known, usually published, psychometric tests and factor analytic studies are required to give support for the concrete validity.

On the other hand, the cognitive psychology method for establishing construct validity does not require empirical validity. In fact, the cognitive branch of psychology locates its main interest in the understanding of the functions and processes of the intellect. The specific procedure used by a cognitive psychologist is directed to establishing functional correspondence between the experimental test and another, which is supposed to be an equivalency. If both tests have sufficient equivalent parameters, one can conclude that the experimental test has good theoretical validity.

The rationale of the TLAP-R is above of all a measure of culture-fair reasoning and thus taps a non-verbal type of intellectual ability. We know that most of the non-verbal reasoning abilities are parts of the fluid part of g. In correspondence with this hypothesis, the TLAP-R therefore must be highly loaded on the fluid intelligence factor (2F). However, according to Gustaffson (1984) this factor is quite similar to the general intelligence factor (3G), consequently the TLAP-R must not be totally independent from the crystallized intelligence factor (2C) and other group factors (Carroll, 1993).

Moreover, as the group factors are ranked in the second stratum in the order of their correlation with 3G, the TLAP-R must have higher correlations with tests that are closer to it than tests that are more distant from it. For example, if we suppose that the TLAP-R is ranked in the second stratum close to the 2F factor, its correlations must be higher with a test measuring 2C than with a test of general memory (2M). Thus its loadings on factors such as fluid and crystallized intelligence, or on a corresponding first stratum vector, must be higher than its loading on any memory related latent trait.

Method

Participants

Table 1 displays background information for the samples used in this study. All samples were collected as a part of the reliability analyses and the validation of the TLAP-R. The information shows that the first two samples were of equivalent sizes including 67 and 66 persons each, while the third one has been of a smaller size with 22 participants.

In the first cohort of participants, 56.25% attended college. Among them, 10.94% had completed 5 to 8 years at university (Ph.D.’s represented 1.56% of them). As can been seen in Table 1, a slight majority of these people were females (56.71%). This sample was almost equally divided into students (49.21%) and persons having ended their formal education.

The majority of second sample subjects reported having completed or studying towards a college degree: 66.67%, among which, respectively 19.05% and 17.46% mentioned their highest educational level as being a first year, or a second year at university. Post-graduates were 7.94% Masters’ and 3.18% Ph.D.’s. An important part of this sample was composed of students from a variety of academic levels (70.79%).

The third and last group was mainly composed of males (66.67%). To the same extend and fortuitously the number of students (66.67%) was the largest among occupations given by the participants. 61.91% of subjects were pursuing their college degree or had ended studying after having completed a college programme. The highest education reported was a Bachelor degree (9.54%). The range of people used for this study was 16 – 24 years of age.

Measures

The criterions that have been chosen are widely known and are frequently used in differential and experimental psychology. They have a proper amount of academic literature to show evidence in their behalf.

Advanced Progressive Matrices (APM). The APM (Raven, Raven & Court, 1998a) is a nonverbal figurative reasoning test divided into two sets of items. Only the second set has been used for the purposes of this study. It consists in 36 matrices of 3 lines and 3 columns each. The last pattern of the third line is missing, and 6 choices are given to the subject. Only one of these can logically be chosen to complete the matrix.

The APM compose the test of hardest difficulty among the Raven’s Progressive Matrices: the Colored Progressive Matrices (CPM) (Raven et al. 1998b) are designed to assess children, the Standard Progressive Matrices (SPM) (Raven et al. 2004) aim at appraising average adults and the Advanced Progressive Matrices are dedicated to measure ability in above average adults .

Raven’s tests of matrices have provided with strong evidence over years of being very appropriate measures of fluid intelligence and are usually used as a criterion when the validity of an experimental questionnaire needs to be checked. According to the test manual, the APM scores Cronbach Alpha reliability was .87 in a cohort of 1,015 15 years old German students. Additional data from diverse researches reported split-half coefficients between .83 and .87 (Jensen, Larson & Paul, 1988; Lapsley & Enright, 1979; Paul, 1985; Poortinga, 1972). These are satisfactory values and indicate a good level of reliability in test scores (Aiken, 2000; Nunnally & Bernstein, 1994). Evidence for close relationships between the APM and other cognitive ability tests are numerous: as a relevant example, we can cite Paul (1985) who reported a correlation of .84 (corrected for restriction of range, N = 300) with the Performance IQ of the revised Wechsler Adult Intelligence Scale (WAIS-R) (Wechsler, 1981).

Scholastic Aptitude Test (SAT). The SAT (College Board, 2012) is a standardized, three-hour test that measures verbal and mathematical reasoning abilities that students develop over time, both in and out of school. Many colleges and universities use the SAT for admission purposes in order to help predicting successful performance in college. Moreover, the SAT, which was initially developed after an IQ test (Lemann, 1999) and despite of successive revisions, is still strongly correlated to traditional intelligence measures (Jouve, 2010, 2011). Frey and Detterman (2003) reported a correlation of .72 between the recentered SAT and the APM. Additionally, these authors indicated that the pre-recentered SAT correlated from .56 to .82 with intelligence measures, with most correlations being higher than .70 in magnitude. Likewise, a study conducted by Raz, Willerman, Ingmundson, and Hanlon (1983) resulted in a correlation between the SAT and the Culture Fair Intelligence Test (CFIT) (Cattell, Krug, & Barton, 1973) of .81.

The post recentering SAT, in use between 1995 and 2005 and from which the scores have been utilized in this study was divided into two sections, (i) a verbal part with emphasis on critical reading in which vocabulary was tested in the context of reading passages and in analogy and sentence-completion questions and (ii) a mathematical part with emphasis on data interpretation and applied math questions in which calculators were permitted but not required.

Short Term Memory Retention Test (STMRT). The STMRT (Jouve, 2000) was a test developed after Peterson and Peterson paradigm (1959) with the aim of measuring the retention of verbal items in short-term memory. The examinee needed to look at ten lines of four unrelated letters each during two and half minutes, with the instruction of memorizing those quadrigrams. Once the time was over and during an equivalent period of time, the subject was asked to take a paper-and-pencil task of symbol search as manner of distracting memorization before having to recall the letters as best possible. The symbol search part of the test was prepared with three given symbols to be encircled each time they were repeated into a 40×40 matrix of look-alike items.

Slosson Intelligence Test – Revised (SIT-R). The SIT-R (Slosson, Nicholson & Hibpsham, 1991; Slosson, 1998) is a test prepared for evaluating crystallized verbal intelligence in native English (children and adults). The 187 SIT-R items are derived from the following cognitive domains: Information, Comprehension, Arithmetic, Similarities and Differences, Vocabulary and Auditory Memory.

Standardized on 2,000 individuals, approximating the contemporary U.S. census, the SIT-R uses a deviation IQ (SD = 16). The SIT-R provides a complement to other educational assessments that look at learning ability, readiness or achievement. It has a noticeable scores reliability (Spearman-Brown = .97 in the entire sample) and gives an indication of a wide range of cognitive abilities. The technical manual for the test reports strong correlations (between .80 and .90) with other IQ scales such as the Weschler Intelligence Scale for Children (WISC) (Weschler, 1991) or the Stanford-Binet Intelligence Scale (SBIS) (Thorndike, Hagen & Sattler, 1986). Kunen, Overstreet & Salles (1996) reported a correlation of .92 with the recommended abbreviated battery of the SBIS in 191 mentally challenged patients. Furthermore, the pre-revised SIT was seen to very highly correlates with the SBIS Form LM (Terman & Merrill, 1960) with a Pearson product moment formula of .92 in 724 students (Armstrong, 1971) and showed a range-restriction corrected value of .73 (raw r = .61, N = 98; Baum, 1979) with the Wechsler Preschool and Primary Scale of Intelligence (WPPSI) (Weschler, 1967).

Results

Factor analysis is a common method of examining the patterns of relationships among a set of variables and is a widely used analytic approach in order to investigate the existence and the structure of any latent constructs among a set of items, or tests (Cronbach, 1990; Kamphaus, 2001). For exploring the presence of latent traits among the TLAP-R and the criterion measures, the method of principal components was chosen, with an orthogonal rotation for interpreting the results. The method employed for rotating factors was the varimax method (Kaiser, 1958), which is unquestionably the most frequently and most popular rotation method by far. Varimax rotation simplifies interpretation because each one of the variables is associated with a small number of large loadings and a large number of null to small loadings. As a matter of consequence, extracted factors only represent few variables.

The first unrotated factor is usually interpreted as g by researchers. However, the nature of any latent trait is determined by the nature of the clusters involved in the study: for example, a principal factor among a set of verbal tasks could be reasonably interpreted as general verbal ability.

Study 1. Table 2 shows the results of the principal components analysis (PCA) of the first sample. This sample included 67 scores from the SIT-R and the STMRT along with the TLAP-R. As can be seen, the loadings on the first factor of the TLAP-R and the SIT-R were somehow equivalent. The figures (.90 and .89 respectively) were very high and indicated that both measures were strongly loaded. At a lower level, the working memory task showed a moderate loading on the principal factor (.44). This factor explained 59.91 percent of the protocol variance. Figure 1 graphically illustrates these findings.

A second PCA has been performed with the same variables, but including also Age of the examinees. Factors eigenvalues were 1.82, 1.19, 0.74 and .25. The use of varimax rotation of the vectors helped exploring the results. These are plotted in Figure 2. In ascending order, loadings of the four variables on the first rotated factor were .94 for the TLAP-R, .88 for the SIT-R, .12 for the STMRT and .08 for the age of participants. The second factor of this analysis was a clear reflect of Age, in which it loaded at .98. Both the TLAP-R and the STMRT were not significantly loaded (-.10 each) by this trait while the SIT-R has been shown a low positive loading (.27). The third latent construct was correlated at .99 with the STMRT and from the fact was identified as a representation of short term memory retention.

Furthermore, the use of a two-dimensional factor structure made sense, and we observed a first factor on which loadings were as follows: .93 for the SIT-R, .87 for the TLAP-R, .35 for the STMRT and .27 for the age. The second factor was characterized by a strong positive loading of the age (.82) while the STMRT has been strongly loaded as well, but negatively (-.70).

Study 2. Plots from a PCA conducted on both the TLAP-R and the 40 minutes timed APM raw scores along with the age of 66 subjects are shown in Figure 3. Values resulted from a varimax rotation. Factors 1 and 3 portrayed the APM and the TLAP-R respectively. Loadings of both tests on their matching factor were .89, and each one loaded .45 on the factor that sketched the other one. The second factor was undoubtedly the age of persons gathered in this sample. Age loading on this one was perfect (1.00). However, it did not show any significant loading on the two other latent traits (.01 on factor 1 and .03 on factor 3).

Prior to apply varimax rotation, the APM and the TLAP-R appeared equivalently loaded on the principal factor with the same value of .95. Loading of the age of test-takers was .11. Eigenvalues suggested the use of an exploratory two-factorial solution as the value for the first factor was 1.81 (60.28% of the variability), and that of the second factor was 1.00 (33.15% of the variability).

Study 3. Another factor analysis has been performed on the scores of 22 individuals, aged 16-24, who took the SAT (recentered) prior to the TLAP-R. Results of confirmatory and exploratory analyses are displayed in Table 3. As shown in Figure 5, unrotated principal factor loadings yield a similar level of proximity for the APM (.92) and the TLAP-R (.90). Although a little less, the verbal reasoning scale of the SAT was also highly loaded (.66) on the principal factor.

Exploratory investigation of the latent constructs among this set of variables revealed a first factor on which the TLAP-R loaded the most (.90) and the mathematical reasoning scale of the SAT loaded significantly (.45). Factor 2 of this solution modeled the verbal reasoning scale of the SAT. In fact, the SAT V accounted for a value of .97.

After adding the age of persons whose scores were collected, a new exploratory factor analysis has been carried out. Eigenvalues were 2.09, 1.15, .56 and .20. Findings for a tri-factorial solution are exposed in Figure 4. The TLAP-R loaded at .93 and the SAT M loaded at .92 on the first factor. The SAT V showed only small positive correlation on it (.27). Nevertheless, the SAT V loaded at .96 on the third factor. The second factor matched almost completely the age of testees (.99).

The two-factorial solution gave enlightenment: the first factor represented almost equivalently the SAT M and the TLAP-R which the loading was .93 and .92 respectively. The second factor mimicked very well the age of participants, as age loading was .92. Concerning the SAT V, it showed comparable loadings on both, with .60 and .56.

Discussion

The purposes of this article were to examine the relationships between the TLAP-R and other measures of cognitive ability along with age of participants, with the aim of investigating the construct validity of the TLAP-R. In order to perform this duty, three independent studies using three different samples were conducted. Specifically, two questions were addressed. The main question was about the evidence of convergent validity between the TLAP-R and fluid intelligence factor as measured by the APM. The results presented in this article strongly suggest a very close linkage between both of them, as the TLAP-R and the APM were seen to tap on very closely related domains. In addition to the Pearson product moment correlation between the TLAP-R and the 40 minutes timed APM given by Jouve (2013b), this study provided with even more evidence of the TLAP-R being a measure of 2F.

The second question addressed in this research was the following: is there sufficient evidence to consider the TLAP-R as an indication of a wide range of mental abilities? The results did show evidence of a close relationship with crystallized intelligence as measured by the SIT-R and also with reasoning in scholastic domains, both verbally and non-verbally, as assessed by the college entrance SAT. The TLAP-R showed clear proximity with the mathematical reasoning scale of the SAT, and moderate proximity with verbal reasoning part of this battery. Moreover, findings suggested that the TLAP-R is a relatively fair instrument in regard to the age of examinees and to a negative effect induced by aging such the decline in short term memory. In other words, where the ability to retained information in short term memory and to recall it efficiently was inversely proportional to the age (both variables being at opposite extrema of the same factor), the TLAP-R remained unimpacted.

References

Aiken, L. R. (2000). Psychological testing and assessment (10th ed.). Needham Heights, MA: Allyn & Bacon.

Armstrong, R. (1971). Can Scores Obtained from the Slosson Intelligence Test be Used with as much Confidence as Scores Obtained from the Stanford-Binet Intelligence Scale? Paper presented at the Annual Meeting of the American Educational Research Association, New York, NY.

Baum, D. D. (1979). An Investigation of the Predictive Validity of the Slosson Intelligence Test with Learning Disabled Kindergarten Children. Educational an Psychological Measurement, 79, 1067-1072.

Carroll, J. B. (1993). Human Cognitive Abilities: A survey of factor-analytic studies. New York, NY: Cambridge University Press.

Cattell, R. B., Krug, S. E., & Barton, K. (1973). Technical Supplement for the Culture Fair Intelligence Tests, Scales 2 and 3. Champaign, IL: Institute for Personality and Ability Testing.

College Board (2012). SAT – College Board – The Most Widely Used College Admission Exam. Retrieved from http://sat.collegeboard.org/home.

Cronbach, L. J. (1990). Essentials of psychological testing (5th ed.). Boston, MA: Addison-Wesley.

Frey, M. C., & Detterman, D. K. (2003). Scholastic Assessment Test and g: The Relationship Between the Scholastic Assessment Test and General Cognitive Ability. Psychological Science, 15(6), 333-378.

Gustaffson, J. E. (1984). A Unifying Model for the Structure of Intellectual Abilities. Intelligence, 8, 179-203.

Jensen, A. R., Larson, G. E. & Paul, S. M. (1988). Psychometric g and mental processing speed on a semantic verification test. Personality and Individual Differences, 9(2), 243-255.

Jouve, X. (2000). Test de Rétention en Mémoire à Court Terme (Short Term Memory Retention Test, STMRT). Unpublished manuscript.

Jouve, X. (2010). Principal components factor analysis for the JCTI and SAT relationship. Retrieved from http://www.cogn-iq.org/archives/28

Jouve, X. (2011). Correlations between the JCCES and other measures. (2nd ed.). Retrieved from http://www.cogn-iq.org/archives/633

Jouve, X. (2013a). TLAP: Revised. Retrieved from http://www.cerebrals.org/tlap/

Jouve, X. (2013b). Correlations between the TLAP-R and other measures. Retrieved from http://www.cogn-iq.org/archives/770

Kaiser, H. F. (1958). The varimax criterion for analytic rotation in factor analysis. Psychometrika, 23 (3), 187-200.

Kamphaus, R. W. (2001). Clinical assessment of child and adolescent intelligence (2nd ed.). Needham Heights, MA: Allyn & Bacon.

Kunen, S., Overstreet, S., & Salles, C. (1996). Concurrent validity study of the Slosson Intelligence Test-Revised in mental retardation testing. Mental Retardation, 34(6), 380-386.

Lapsley, D. K., & Enright, R. D. (1979). The effects of social desirability, intelligence, and milieu on an American validation of the Conservatism scale. Journal of Social Psychology, 107, 9-14.

Lemann, N. (1999). The big test: The secret history of the American meritocracy. New York, NY: Farrar, Straus, & Giroux.

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric Theory (3rd ed.). New York, NY: McGraw-Hill.

Paul, S. M. (1985). The Advanced Raven’s Progressive Matrices: Normative data for an American University population and an examination of the relationship with Spearman’s g. Journal of Experimental Education, 54, 95–100.

Pennington, D. (2003). Essential Personality. New York, NY: Oxford University Press.

Peterson, L. R., & Peterson, M. J. (1959). Short Term Retention of Individual Verbal Items. Journal of Experimental Psychology, 58, 193-198.

Poortinga, Y. (1972). A comparison of African and European students in simple auditory and visual tasks. In L. J. Cronbach and P. D. Drenth (Eds.). Mental Tests and Cultural Adaptation. The Hague: Mouton.

Raven, J., Raven, J. C., & Court, J. H. (1998a). Manual for Raven’s Progressive Matrices and Vocabulary Scales. Section 4: The Advanced Progressive Matrices. San Antonio, TX: Harcourt Assessment.

Raven, J., Raven, J. C., & Court, J. H. (1998b). Manual for Raven’s Progressive Matrices and Vocabulary Scales. Section 2: The Coloured Progressive Matrices. San Antonio, TX: Harcourt Assessment.

Raven, J., Raven, J. C., & Court, J. H. (2004). Manual for Raven’s Progressive Matrices and Vocabulary Scales. Section 3: The Standard Progressive Matrices. San Antonio, TX: Harcourt Assessment.

Raz, N., Willerman, L., Ingmundson, P., & Hanlon, M. (1983). Aptitude-related differences in auditory recognition masking. Intelligence, 7(2), 71-90.

Slosson, R. L., Nicholson, C. L., & Hibpsham, T. H. (1991). Slosson Intelligence Test for Children and Adults. (Rev. ed. by C. L. Nicholson & T. H. Hibpsham). East Aurora, NY: Slosson Educational Publications, Inc.

Slosson, R. L. (1998). Slosson Intelligence Test Revised (SIT-R) For Children and Adults, Technical Manual, Calibrated Norms Tables. East Aurora, NY: Slosson Educational Publications, Inc.

Terman, L. M., & Merrill, M. A. (1960). Stanford–Binet Intelligence Scale: Manual for the Third Revision Form L–M with Revised IQ Tables by Samuel R. Pinneau. Boston, MA: Houghton Mifflin.

Thorndike, R. L., Hagen, E. P., & Sattler, J. M. (1986). Stanford-Binet Intelligence Scale (4th ed.). Itasca, IL: Riverside Publishing.

Wechsler, D. (1967). Manual for the Wechsler Preschool and Primary Scale of Intelligence. San Antonio, TX: The Psychological Corporation.

Wechsler, D. (1981). Wechsler Adult Intelligence Scale-Revised. San Antonio, TX: The Psychological Corporation.

Wechsler, D. (1991). The Wechsler Intelligence Scale for Children (3rd ed.). San Antonio, TX: The Psychological Corporation.

Draft of this article (pdf)

The TLAP-R (Jouve, 2013a) is an untimed, non-verbal reasoning test prepared with perceptual material. It consists in 8 matrices, each one including 6 lines of 15 patterns of which 4 have been deleted. The examinee is asked to find the 4 missing patterns. The first version (limited to 5 matrices) of this test has been prepared by Xavier Jouve in mid 2000 for a research project.

The chiefly involved cognitive process mostly depends on fluid intelligence and the test is minimal-knowledge based. The material used to prepare the matrices is numerical but not heavily mathematically based so that the TLAP-R is suitable to assess any person with only basic arithmetic knowledge.

In order to complete the matrices, to find which patterns are missing, the TLAP-R requires inductive reasoning. The task requires figuring out a specific logical and governing rule from a chaotic situation. Spearman (1927) described this process and considered it as the requisite for the eductive part of g. However, the TLAP-R showed a strong relationship with crystallized intelligence even without measuring it directly (Jouve, 2013b).

Item Response Theory (IRT) relates characteristics of items (item parameters) and characteristics of individuals (latent traits) to the probability of a positive response. A variety of IRT models could be applied for dichotomous as well as polytomous data. In each case, the probability of answering correctly or endorsing a particular response category can be represented graphically by an item response function (IRF). This function represents the non-linear regression of a response probability on a latent trait, such as inductive reasoning, spatial ability, verbal ability, or any other psychological factor (Hulin, Drasgow & Parsons, 1983).

In the Two-Parameter Logistic Model (2PLM) as expressed by Birnbaum (1968), the first parameter is the discrimination parameter referred to with letter a, which is proportional to the slope of the ICC at the ability scale point b, which is the second parameter and named difficulty parameter. This model is the generalization of the One-Parameter Logistic Model (1PLM) firstly introduced by Rasch (1960) that only takes into account the item difficulty as given by the b-parameter.

Additionally, it also exists a model that includes a third parameter for guessing, called the c-parameter. However, according to Hambleton, Swaminathan, & Jane Rogers (1991) the Three-Parameter Logistic Model (3PLM) is only of proper application for multiple-choice items in which the examinee needs to choose among diverse given alternatives. In fact, for free-response items like those of the TLAP-R, in which the examinee needs to engender an own original answer, the assumption of no guessing is quite probable and thus the model does not require the c-parameter.

Participants. The analyses performed in this investigation are based on the data collected from 1,003 respondents over a two-years period, between 2000 and 2002. The mean age of the individuals whose data were used was 24.43 years (SD = 8.11). This cohort included 45% female, 53.8% male and 1.2% missing gender data.

Method. The characteristic curves, for both the Item Empirical Functions (IEF) and the Item Response Functions (IRF) were drawn for each of the TLAP-R 48 matrix lines are all shown in Figures 1 to 2 respectively. These Item Response Functions (IRF) and could eventually be used in order select, discard or revised questions that would not match with psychometric standards. Test items must be those with steeper slopes because they are more useful to discriminate subjects and separate them into different ability levels. As a matter of consequence, a high a-parameter value results in a slope having a sharp inclination.

Estimation of the parameters has been performed with the Normal Ogive by Harmonic Analysis Robust Method (NOHARM; Fraser, 1986; Fraser & McDonald, 1988). This process of parameters estimation is a variant of the one described in McDonald (1982, 1985). In a general point of way, it is close to that of Christofferson (1975), with a main difference in the use of ordinary least squares where Christofferson utilizes generalized least square (McDonald, 1997). NOHARM yields some stable parameter estimates (Ackerman, 1988; Miller, 1991).

The calculation of the quadrature nodes and weights for empirical values followed the Levine & Drasgow (1988) method. On one hand, the theta scale of the IEF was divided into twenty-five points corresponding to the equally spaced percentile ranks from 2 to 98: -2.05, -1.56, -1.29, -1.08, -.92, -.77, -.64, -.52, -.41, -.31, -.20, -.10, 0, .10, .20, .31, .41, .52, .64, .77, .92, 1.08, 1.29, 1.56 and 2.05. On the other hand, the IRF theta scale that represents ability was sectioned into sixteen equally separated intervals of .5, ranging from -4 to 4. The midpoints of each interval were -3.75, -3.25, -2.75, -2.25, -1.75, -1.25, -.75, -.25, .25, .75, 1.25, 1.75, 2.25, 2.75, 3.25 and 3.75.

Results & discussion. According to their ICCs, the TLAP-R lines, each one representing an item, appeared suitable to discriminate examinees along the ability scope. Although the psychometric efficiency of test lines seemed varied, and not perfectly homogenous, the TLAP-R did not show any unappropriated item. Apparently, some lines of the TLAP-R were producing noise in the low-end of the proficiency scale, especially in empirical data.

This might be explained because in some lines of the test matrices, the examinees could be tempted to reproduced the patterns occurring with the highest frequency in the entire problem, and earned positive responses without the correct reasoning. In a such case, the test-taker’s behavior might be interpreted as a form of guessing.

Additionally, as can be seen in the two figures, some items are very similar the ones to the others. This finding suggests that they are redundant. However, this is seen as inevitable because of the inherent nature of the design of the TLAP-R matrices. In fact, in some of them, the person who is administered the test needs to look at the matrix globally, making an effort of abstraction, in order to complete the problem as a whole, i.e. to respond to more than a single line at once.

References

Ackerman, T. A. (1988). Comparison of multidimensional IRT estimation procedures using benchmark data. Paper presented at the ONR Contractors’ meeting, Iowa City,IA.

Birnbaum, A. (1968). Some latent traits models and their use in inferring an examinee’s ability. In F. M. Lord, & M. R. Novik, Statistical theories of mental test scores. Reading, MA: Addison-Wesley.

Christofferson, A. (1975). Factor analysis of dichotomized variables. Psychometrika, 14(40), 5-32.

Fraser, C. (1986). NOHARM: An IBM PC Program for Fitting Both Unidimensional and Multidimensional Normal Ogive Models of Latent Trait Theory. Amindale, Australia: The University of New England.

Fraser, C., & McDonald, R. P. (1988). NOHARM: Least Squares item factor analysis. Multivariate Behavioural Research, 23, 267-269.

Hambleton, R. K., Swaminathan, H., & Jane Rogers, H. (1991). Fundamentals of Item Response Theory. Newbury Park, CA: Sage Publications, Inc.

Hulin, C. L., Drasgow, F., & Parsons, C. K. (1983). Item Response Theory: Applications to Psychological Measurement. Homewood, IL: Dow Jones Irwin Inc.

Jouve, X. (2013a). TLAP: Revised. Retrieved from http://www.cerebrals.org/tlap/

Jouve, X. (2013b). Correlations between the TLAP-R and other measures. Retrieved from http://www.cogn-iq.org/archives/770

Levine, M. V., Drasgow, F. (1988). Optimal appropriateness measurement. Psychometrika, 53, 161-176.

McDonald, R. P. (1982). Linear versus nonlinear models in item response theory. Applied Psychological Measurement, 6(4), 379-396.

McDonald, R. P. (1985). Unidimensional and multidimensional models for item response theory. In D. J. Weiss (Ed.), Proceedings of the 1982 Item Response and Computerized Adaptive Testing Conference (pp 65-87). Minneapolis, MN: University of Minnesota.

McDonald, R. P. (1997). Normal Ogive Multidimensional Model. In W. J. van der Lindenand R. K. & Hambleton (Eds.), Handbook of Modern Item Response Theory (pp. 257–269). New York, NY: Springer-Verlag.

Miller, T. (1991). Empirical Estimation of Standard Errors of Compensatory MIRT Model Parameters Obtained from the NOHARM Estimation Program (Research report ONR91-2). The American College Testing Program.

Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen: Danish Institute for Educational Research.

Spearman, C. (1927). The abilities of man. New York, NY: McMillan.

The TLAP-R (Jouve, 2013) is an untimed, non-verbal reasoning test prepared with perceptual material. It consists in 8 matrices, each one including 6 lines of 15 patterns of which 4 have been deleted. The examinee is asked to find the 4 missing patterns. The first version (limited to 5 matrices) of this test has been prepared by Xavier Jouve in mid 2000 for a research project.

The chiefly involved cognitive process mostly depends on fluid intelligence and the test is minimal-knowledge based. The material used to prepare the matrices is numerical but not heavily mathematically based so that the TLAP-R is suitable to assess any person with only basic arithmetic knowledge.

In order to complete the matrices, to find which patterns are missing, the TLAP-R requires inductive reasoning. The task requires figuring out a specific logical and governing rule from a chaotic situation. This process has been described by Spearman (1927) who considered it as the requisite for the eductive part of g. However, the TLAP-R showed a strong relationship with crystallized intelligence even without measuring it directly.

The TLAP-R has been administered along with several other tests in order to investigate its validity. Correlations are reported in Table 1.

Advanced Progressive Matrices. The main study has been conducted with the Advanced Progressive Matrices (APM; Raven, Raven & Court, 1998). The APM is a nonverbal figurative reasoning test divided into two sets of items. Only the second set has been used for the purposes of this study. It consists in 36 matrices of 3 lines and 3 columns each. The last pattern of the third line is missing, and 6 choices are given to the subject. Only one of these can logically be chosen to complete the matrix.

The APM compose the test of hardest difficulty among the Raven’s Progressive Matrices: the Colored Progressive Matrices are designed to assess children, the Standard Progressive Matrices aim at appraising average adults and the Advanced Progressive Matrices are dedicated to measure ability in above average adults. However, experience has shown that the APM are of relatively modest difficulty and are more suitable in testing average intelligent individuals rather than the intellectually gifted.

Raven’s tests of matrices have provided with strong evidence over years of being very appropriate measures of fluid intelligence and are usually used as criterion when the validity of an experimental questionnaire needs to be checked.

Both tests were administered to two samples of 62 and 65 individuals respectively. In the first sample, the APM were used under 30 minutes limited time conditions, and in the second one, examinees were asked to work on the APM during no more than 40 minutes.

As expected, the TLAP-R shown high correlations with the Raven’s test with a closer relationship when the APM are less strictly timed. In fact, the more restrictive the time limit, the more important the part taken by cognitive speediness. The TLAP-R and the APM correlated above .70 in both groups. The TLAP-R and 30 min timed APM correlation was the lowest at .73. When administered in less than 40 minutes, the correlation between these two nonverbal tests increased to .80. This is seen as very high and indicates the measurement of very closely related constructs.

Further analysis might be done using the APM without limiting the time of test-taking. Based on actual results, we may reasonably expect an even greater figure.

College admission tests. The Scholastic Aptitude Test (SAT) is a standardized, three-hour test that measures verbal and mathematical reasoning abilities that students develop over time, both in and out of school. Many colleges and universities use the SAT for admission purposes because it helps to predict successful performance in college. Moreover, the SAT, which was initially developed after an IQ test (Lemann, 1999) and despite of successive revisions, is still strongly correlated to traditional intelligence measures (Frey & Detterman, 2003; Jouve, 2010, 2011).

The recentered SAT (SAT I), used between 1995 and 2005, was divided into two sections, (i) a verbal part with emphasis on critical reading in which vocabulary was tested in the context of reading passages and in analogy and sentence-completion questions and (ii) a mathematical part with emphasis on data interpretation and applied math questions in which calculators were permitted but not required.

The American College Test (ACT) is a battery that intends to assess students acquired educational and self-developed competencies. It is composed of four subtests: English and Reading as measures of verbal domains, Mathematics and Science as measures of quantitative domains.

Scaled on a 36 points standardized metric, each subtest yield with a domain score and a composite score is also provided for evaluating the student’s overall performance. Correlations between the ACT and the SAT are repeatedly reported as being very high.

A group of 25 persons, mostly students, were administered the SAT I before taking the TLAP-R. All correlations were adjusted for restriction of range (McNemar, 1969). The TLAP-R – SAT I Verbal was the lowest at .40. The raw TLAP-R score correlated at .72 with the SAT I composite but the highest value was observed with the Mathematical reasoning scale of the SAT I (.80).

Another cohort of 20 examinees reported their ACT scores. Unfortunately, not all the subtest scores were known consequently the correlations between the TLAP-R and the ACT subtests have not been analyzed. Nonetheless, the TLAP-R and the ACT composite were seen to correlate highly. The result was .79, which is comparable to that of the TLAP-R – SAT I composite relationship.

Because of the relatively small samples, the results of this study would benefit from the gathering of more testees. However, the correlations that were obtained are somewhat coherent with the content of the TLAP-R. These findings suggest that a significant proximity exists between the latent construct measured by the TLAP-R and achievement or more generally, crystallized intelligence.

Culture Fair Intelligence Tests & Bonnardel ”Loi des Séries”. The Culture Fair Intelligence Tests (CFIT; Cattell, Krug & Barton, 1973) were developed to measure fluid intelligence according to their author’s theory of mental abilities (Cattell, 1971). Two scales of CFIT exists: scale 2 aiming at assessing children and adults of average ability and scale 3 that is suitable to evaluate higher levels of intelligence. Both scales are split into two forms. Only form A of the second (CFIT-2A) and the third scale (CFIT-3A) have been used in this study.

Each form of the CFIT scales consists in four types of visual problems: Series, Classifications, Matrices and Conditions. The CFIT are severely timed and include from the fact a measure of cognitive speediness along with the desired evaluation of fluid intelligence: for example, the 50 items of the CFIT-3A are to be completed within 12 and half minutes.

The Bonnardel ”Loi des Séries” (BLS4; Thiébaut, 2000) or Law of Sequences is just like the CFIT, a measure of nonverbal figurative reasoning that uses a short time to be administered (10 min). Among Bonnardel’s tests, the BLS4 is the one dedicated to the assessment of highly able individuals. In subjects of average ability, the B53 is more appropriate.

The BLS4 consists in 30 visual sequences arranged in ascending order of difficulty. The examinee has to choose among 6 answers the one that correctly follows a 3 patterns sequence. Each given answer includes 2 additional patterns.

Correlations between the TLAP-R and these tests of nonverbal ability are all three of similar magnitude. Given the fact that the Bonnardel’s and both Cattell’s tests are very close in their respective contents and test-taking conditions; a comparable level of relationships was expected. We observed mid-.60 values of Pearson coefficient: the TLAP-R – BLS4-2T correlation was the highest (.67) while the TLAP-R – CFIT-2A being the lowest (.64). These findings indicate that the TLAP-R has a strong predictive value for non-language ratiocination when a quick response is required.

Slosson Intelligence Test. The Slosson Intelligence Test-Revised (SIT-R; Slosson, 1998) is a test prepared for evaluating crystallized verbal intelligence in natives English (children and adults). The 187 SIT-R items are derived from the following cognitive domains: Information, Comprehension, Arithmetic, Similarities and Differences, Vocabulary and Auditory Memory.

Standardized on 2,000 individuals, approximating the contemporary U.S. census, the SIT-R uses a deviation IQ (SD = 16). The SIT-R provides a complement to other educational assessments that look at learning ability, readiness or achievement.

As can be seen in Table 1, the raw scores from both the TLAP-R and the SIT-R were seen to correlate highly (.69) in a cohort of 69 persons. This finding supports the use of the TLAP-R as a measure of general cognitive functioning rather than solely an indication of one’s non-language reasoning.

Jouve Numerical Appraisal Test, Jouve “Test des Proverbes” & Short Term Memory Retention Test. Three other relationships were investigated with ad-hoc, circumstantially prepared tests in samples of native French speakers.

The Jouve Numerical Appraisal Test (JNAT; Jouve, 2002) was prepared with 28 numerical sequences of mixed mathematically demanding skill levels. It was to be taken within less than 10 minutes, making it hard to finish. Although not many studies were conducted on the JNAT, it was seen to be a reliable measure: the corrected split-half coefficient for the JNAT raw scores was .89 (N = 65), which is high and appropriate for psychological measurement (Aiken, 2000; Nunnally & Bernstein, 1994).

The Jouve “Test des Proverbes” (JPROV; Jouve, 2003), or Test of Proverbs was a verbal comprehension and knowledge test designed with 3 distinct grids, each one including 10 proverbs along with 10 meanings. The subject was asked to link proverbs and corresponding meanings together during a 10-min period. The JPROV proved to be suitable for assessing in low to average ranges of mental ability: an unselected subject would approximately score 21 or 22 out of 30.

Psychometric properties of the JPROV items have been analyzed with the 2-Parameters Logistic Model and Multi-Dimensional Scaling. Furthermore, the raw score shown a very satisfactory reliability level (Spearman-Brown = .91, N = 212), and as a matter of validity, it correlated significantly with that of the CFIT-2A (r = .53, N = 175).

A third experimental measure served to perform this study: the Short-Term Memory Retention Test (STMRT; Jouve, 2000). It was was a test developed after Peterson and Peterson’s paradigm (1959) with the aim of measuring the retention of verbal items in short-term memory. The examinee needed to look at 10 lines of 4 unrelated letters each during 2 and half minutes, with the instruction of memorizing those quadrigrams. Once the time was over, the subject was asked, during an equivalent period of time, to take a paper-and-pencil task of symbol search as manner of distracting memorization before having to recall the letters as best possible. The symbol search part of the test was prepared with three given symbols to be encircled each time they were repeated into a 40×40 matrix of look-alike items.

The TLAP-R demonstrated significant linkage with both the JNAT and the JPROV. Interestingly, although the numerical content of the TLAP-R would have led to expect a slightly higher correlation with the JNAT, which is numerically oriented; the contrary was observed with a TLAP-R – JNAT correlation of .40, and a TLAP-R – JPROV correlation of .45. About the STMRT, as presented in Table 1, a low positive correlation (.19) was obtained by studying the relationship with the TLAP-R. These findings suggest that the TLAP-R can predict to some extent one’s ability in verbally and mathematically oriented domains, and does not appear to be particularly mathematically biased. Moreover, the TLAP-R is apparently independent to working memory.

References

Aiken, L. R. (2000). Psychological testing and assessment (10th ed.). Needham Heights, MA: Allyn & Bacon.

Cattell, R. B. (1971). Abilities: Their Structure, Growth and Action. Boston, MA: Houghton Mifflin.

Cattell, R. B., Krug, S. E., & Barton, K. (1973). Measuring Intelligence with the Culture Fair Tests. Champaign, IL: Institute for Personality and Ability Testing.

Frey, M. C., & Detterman, D. K. (2003). Scholastic Assessment Test and g: The Relationship Between the Scholastic Assessment Test and General Cognitive Ability. Psychological Science, 15(6), 333-378.

Jouve, X. (2000). Test de Rétention en Mémoire à Court Terme (Short Term Memory Retention Test, STMRT). Unpublished manuscript.

Jouve, X. (2002). Jouve Numerical Appraisal Test (JNAT). Unpublished manuscript.

Jouve, X. (2003). JPROV. Le Test des Proverbes : Brochure. Unpublished manuscript.

Jouve, X. (2010). Principal components factor analysis for the JCTI and SAT relationship. Retrieved from http://www.cogn-iq.org/archives/28

Jouve, X. (2011). Correlations between the JCCES and other measures. (2nd ed.). Retrieved from http://www.cogn-iq.org/archives/633

Jouve, X. (2013). TLAP: Revised. Retrieved from http://www.cerebrals.org/tlap/

Lemann, N. (1999). The big test: The secret history of the American meritocracy.
New York, NY: Farrar, Straus, & Giroux.

McNemar , Q. (1949). Psychological statistics. New York, NY: Wiley.

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill.

Peterson, L. R., Peterson, M. J. (1959). Short Term Retention of Individual Verbal Items. Journal of Experimental Psychology, 58, 193-198.

Raven, J., Raven, J. C., & Court, J. H. (1998). Raven Manual: Sec. 4. Advanced Progressive Matrices (1998 ed.). Oxford: Oxford Psychologists Press.

Slosson, R. L. (1998). Slosson Intelligence Test Revised (SIT-R) For Children and Adults, Technical Manual, Calibrated Norms Tables. East Aurora, NY: Slosson Educational Publications, Inc.

Spearman, C. (1927). The abilities of man. New York, NY: McMillan.

Thiébaut, E. (2000). Les Bonnardel : Les tests de raisonnement. Paris: Editions et Applications Psychologiques.

“The RIAS is an individually administered test of intelligence appropriate for ages 3 through 94 years, which includes a co-normed, supplemental measure of memory. The RIAS includes a two-subtest Verbal Intelligence Index (VIX), a two-subtest Nonverbal Intelligence Index (NIX), and a Composite Intelligence Index (CIX). The VIX assesses verbal intelligence by measuring verbal problem solving and verbal reasoning where acquired knowledge and skills are important. Administration of the four intelligence subtests by a trained, experienced examiner requires approximately 20 to 25 minutes”. (Reynolds & Kamphaus, 2003)

Reliability of scores yielded by the RIAS is seen as excellent with Cronbach’s alphas ranged from .90 to .94 for the subtests, and from .94 to .96 for the Indexes (N = 2,438). The subtests’ raw scores are converted to age-referenced T-scores (M = 50; SD = 10). Then, these T-scores are summed in order to obtain the indexes by comparison with the manual norm tables. The RIAS indexes use the same metric than most of the contemporary cognitive ability batteries, with a mean of 100 and 15 points per standard deviation. This makes them relatively comparable to IQ or other indexes from the Wechsler’s scales or the most recent Stanford-Binet revisions. According to Reynolds & Kamphaus, the RIAS VIX correlated at .86 and at .78 with the WISC-III VIQ and FSIQ respectively (N = 54). These authors also reported correlations of .71 and .70 between the VIX and both the VIQ and the FSIQ (N = 31). In both the WISC and the WAIS samples, the means and SDs of the VIX are respectively lower and tighter than those of the Wechsler’s scales. The significantly lower correlations observed in adults between the VIX and the WAIS IQs could be a relative consequence of unexpected answers to verbal problems. Although the keys given by the manual are fairly appropriate in adolescents even in gifted subjects, these expected solutions could appear insufficient in gifted adults, requiring the examiner to perform researches. The authors encourage psychologists who are used to administer the RIAS to consult reference materials to decide on the scoring of unusual responses.

Participants. The sample of participants consisted in 125 individuals. Of these, 42 (33.6%) were females, 83 (66.4%) were males and 100 (80%) of them indicated to have completed at least one college degree. The average age of the sample was 33.54 (SD = 12.53).

Factor analysis. Factor analysis is a common method of examining the patterns of relationships among a set of variables and is a widely used analytical approach in order to evaluate the existence and the structure of any latent constructs among a set of items, or tests (Cronbach, 1990; Kamphaus, 2001). For exploring the presence of latent traits among the JCCES and the RIAS verbal subtests, the method of principal components was chosen, with a varimax rotation (Kaiser’s normalization) for interpreting the results. The first unrotated factor has an engeinvalue of 3.60 and accounted for 71.90% of trace. This trait is usually interpreted as g by researchers. However, the nature of any latent trait is determined by the nature of the clusters involved in the study. In this analysis, variables only represent cognitive abilities that are related to knowledge and its usage. Consequently, this general factor is believed to be an enlightenment of general crystallized intelligence, i.e. Gc, rather than pure g. Loadings on Gc were all very similar for verbal measures and ranged from .87 to .88. The only variable to show a lower saturation was the Mathematical Problems (MP) subtest of the JCCES at .72. However, this value is high. A two factors solution was then investigated. The second factor accounted for 12.07% of the trace and its engeinvalue was .60. As can be seen in Table 1, in this solution the first factor could be reasonably interpreted as language development (LD). The two subtests, General Knowledge (GK; .89) and Guess What (GWH; .90), which are prepared with information content are the more loaded. The second factor, very strongly appearing in the MP questionnaire (.95), could possibly be seen as quantitative knowledge, but with a significant relationship to reasoning. In fact, the verbal analogies loaded at .44 and .43 on this factor.

Characteristics

The FE has been seen as a rise in IQ test raw scores at various rates in virtually every corner of the world.  In the U.S. the usually cited effect size is 3 points per decade.  In Estonia, the gains have been about 1.65 points per decade, but have accelerated in more recent years.  Japan and Korea have experienced rates around 7.7 points per decade, but the Korean gains were delayed by almost 30 years.

Numerous studies of the FE have found that score gains were larger in the lower half of the IQ distribution.  For example this was reported for Denmark, Britain, Turkey and Spain.  These observations have led to explanations related to improvements in nutrition and education.  But some countries have shown greater FE gains in the upper half: Brazil and the U.S. (in the National Longitudinal Study of Youth data).  Using data sets of about 1.7 million scores, Wai showed that the FE was present in the top 5% of the U.S. IQ distribution.

Some studies have shown no FE gains; Cotton found none in Australian children.  Scandinavian countries have shown rapid FE gains followed by an end to the gains, and by reversal (negative FE) in some cases.

Various studies have shown that the full amount of score gains has been observed in children from age 4 to 6.  Likewise studies of developmental quotients (DQ) have shown gains that are similar to IQ gains.  [DQs reflect rates of maturation--hold up head, sit up, stand, walk, jump, etc.  They are measured by the Griffiths Test and Bayley Mental Scales.]

A large number of studies have reported that FE gains were greater on abstract test items than on scholastic items.  This can be stated as a bias towards tests of fluid intelligence and away from crystallized intelligence.  The highly abstract Raven’s Progressive Matrices tests (Color, Standard, and Advanced), have shown strong FE gains.  The Wechsler has shown FE gains that are almost as large as for the Raven tests, and the performance component is almost twice the magnitude of the verbal component.

The Raven tests have shown gains of 18-20 IQ points per generation in many industrialized countries. Dutch gains were 21 points over 30 years.  Urban Chinese gained 22 points between 1936 and 1986.

At the same time that IQ scores have been rising, academic performance has been declining in the US and Britain.  SAT scores have been declining, even after correction for the change in demographics of those taking the test.  A quarter of the decline remains after the correction.

Hypothetical causes

Among the causes that have been proposed to explain the FE are these:

 Education                                   Decreased family size
 Increased exposure to testing               Heterosis
 Exposure to artificial light                More complex visual environment
 Nutrition                                   Child rearing practices
 and the use of Classical Test Theory versus Item Response Theory.

Education

It is possible that improved education has accounted for some test score gains, although such gains would most likely have no g loading.  The finding that FE gains are seen in preschool children (at the same magnitude as seen in adults) suggests that education is not a primary cause of the FE.

Increased exposure to testing

Two mechanisms have been proposed:  1)  Brand suggested that the use of timed tests has caused students to work faster by guessing more frequently (multiple choice).  While this may be a factor, FE gains are seen on tests that are untimed and on tests that do not use multiple choice.  2) Jensen mentioned “increasing test wiseness from more frequent use of tests.”  His point was that frequent testing may have the same sort of impact on test scores as the increase associated with test-retest.  This is the same process that is observed with learning and shows up in situations where test training has been used (as is common with the SAT). Both Brand’s and Jensen’s ideas would presumably cause test scores to increase without showing gains on g.

Nutrition and medical care

DQs have gained 3.7 points per decade, while IQ  gains of 3.9 points per decade have been seen in preschool children (age 4-6).  These and gains in the lower part of the IQ distribution are consistent with the argument that improved nutrition has contributed to the FE.  Other factors also agree: increased birth weight; head size measured in 1 year olds has increased by about 1.5 cm from 1930 to 1985 [head size to brain size correlation = 0.80]; and height gains that have increased by about 1 SD over 50 years (similar to DQ gains).

Arguments against nutrition as a cause include: studies of nutrition have shown that neither vitamins nor supplements have had any impact on intelligence; nutrition is unlikely to have declined over the past 20 years in those countries that have a negative FE.  Height did not decline in those countries; and contrary to the intelligence gains seen in Norway, height gains from 1969 to 2002 were mostly in the upper half of the intelligence range.

Exposure to artificial light

Artificial light stimulates the pineal gland in animals.  The pineal gland appears to play a major role in sexual development, hibernation, metabolism, and seasonal breeding.  The effect of stimulating growth is used by poultry farmers to increase their output.   There does not seem to be any data available for whether this effect happens in humans, but the speculation is that it might.  There has been an obvious increase in the use of electric lighting by humans over much of the time that the FE has been observed.

Decreasing family size

Low IQ people statistically have more children than high IQ people.  The high heritability of intelligence, therefore, is a source of dysgenic pressure.  If the average family size decreases, the reduced numbers of low IQ children should produce a net increase in the mean, which would show up as a FE gain.

In a very large study of Norwegian conscripts, the previously debated birth order effect was shown to be real, although not large.  If family size is declining in various groups, there must be a positive contribution to mean IQ due to fewer low IQ children being born.

Heterosis

Mingroni has argued that since the effects of the environment  (on intelligence) are so small, the possibility of a genetic effect should be investigated. Lynn argued that heterosis is unlikely for three reasons:
1 – There was little immigration in Europe before 1950 (the FE was present before that date).
2 – The FE for IQs and DQs are just as large in Europe as in other places.
3 – Studies of heterosis have shown little positive effect on IQ.

Perhaps the most important consideration in determining whether there is a heterosis effect was pointed out by Mingroni: If the FE is found within-families, the cause is not genetic.  The FE, however, has been shown to exist within families (conscripts in Norway).

Enriched visual environment

Greenfield and others suggested that the FE gains are caused by the ever increasing shift from verbal communication to visual and interactive media.  This is seen globally in the increased presence of movies, television, photography, video games, computers, puzzles, mazes, exploded views, etc.

The mechanism for this hypothesis is that the shift towards visual representations removes some of the novelty from tests, especially the culture reduced tests that have shown about double the FE gains as found in other tests.  This is particularly convincing for tests such as the Raven’s, which presents abstract figures in a matrix.  Several decades ago these figures may have been more baffling than they are today.

Child rearing practices

The FE has been seen throughout the world, in both developed and undeveloped countries where child rearing practices certainly vary greatly.  It is unlikely that this hypothesis is a significant factor, not only because of the cultural variation in child rearing practices, but also because the shared environment has essentially no impact on adult intelligence (per prior discussion).

Classical Test Theory (CCT) versus Item Response Theory (IRT)

Most studies in the literature are based on CTT and are presented without passing along the test item data.  This practice hides some of the information that could be extracted from a data set.  Test scores are given, but the latent constructs they are designed to measure cannot be examined.  IRT allows the researcher to examine the changes in underlying latent ability.  Thus, CTT can show differences in scores, even when there is no change in the latent variable.  An increase may be due to a general gain in real intelligence, or a decrease in the levels of difficulty of test items.

Alex Beaujean’s results using CCT and IRT to measure FE gains:

Peabody Picture Vocabulary Test-Revised
CCT       0.44 points per year
IRT        0.06 points per year

Peabody Individual Achievement Test-Math
CCT       0.27 points per year
IRT        0.13  points per year

The results clearly show that the FE essentially vanishes for the PPVT-R when IRT is used.  The PIAT-M gains are cut to half using IRT.  Ergo, the FE gains are determined by the methodology, leading to the concern that much of the literature has reported findings that might be quite different if IRT had been used.

Is the FE invariant?

Multigroup confirmatory factor analyses of several data sets showed that they were not invariant, meaning that FE gains were not gains on the latent variables that the tests were supposed to measure.  Besides providing insight as to the nature of the FE gains, the rejection of factorial invariance demonstrates that subtest score interpretations are necessarily different over time.

Real or hollow gains?

When Flynn begain his study and reports on the secular gains, he gave numerous examples of how extreme the gains have been, questioning that they could possibly be real.  For example the large gains in The Netherlands would mean that, by 1982 standards, the Dutch mean IQ in 1952 would have been 79.  Flynn commented “Has the average person in The Netherlands ever been near mental retardation?”  “Does it make sense to assume that at one time almost 40% of Dutch men lacked the capacity to understand soccer, their most favored national sport?”  He noted that there are not more gifted Dutch school children now and that patented inventions have shown a sharp decline.  He presented a number of similar arguments, all of which questioned the possibility that such large changes could have happened and, therefore, the score gains must be meaningless.

If the secular gains are real, they must show a g loading (this is called a Jensen Effect).  Numerous studies of the g loading of FE gains have shown that the gains were not on g.  The usual test for a Jensen Effect is the use of the method of correlated vectors.  When applied to data showing a FE, it has not shown a Jensen Effect.

Rushton used principal components analysis to show that data exhibiting a FE forms a cluster, thereby indicating that it is a real effect.  But the cluster does not overlap with the clusters formed by racial differences,  inbreeding depression scores (purely genetic), and g  factor  loadings  (largely genetic). The secular increase is, therefore, unrelated to g and other heritable measures.

As with virtually every aspect of the FE, different data sets produce different results.  Colom found that tests of crystallized intelligence did not show gains in g, but tests of fluid intelligence did.

Predictive bias

Jensen commented that the definitive test of whether FE gains are hollow or not is to apply the predictive bias test.  This means that two points in time would be compared on the basis of an external criterion (real world measurement, such as school grades).  If the gains are hollow, the later time point would show underprediction, relative to the earlier time. This assumes that the later group has not been renormed. [Earlier IQ points would exceed the performance of the later generation for the same IQ.] In actual practice tests are periodically renormed so that the mean remains at 100.  The result of this recentering is that the tests maintain their predictive validity, indicating that the FE gains are indeed hollow.  If the gains were real and the tests were renormed, people at a given IQ would be getting smarter and this would show up in the predictive validity.

Summary

•          The FE exists between birth cohorts.
•          It is found within sibships.
•          It appears early in life (before school age).
•          There are presumably multiple causes.
•          The gains are all or mostly hollow (not Jensen Effects).
•          There are serious methodological issues to be resolved and which may be a major cause of the gains.
•          The FE is not invariant over time.

The foregoing review is a greatly condensed version of my paper, Understanding the Flynn Effect.  It contains more detailed discussions of all of the points mentioned above, with identification of the researchers and their papers, as they apply to each topic.  A full reference list is included.  Anyone wishing to read the full paper may find it here:

https://sites.google.com/site/thirdstratum/papers-1

Bob Williams