Cogn-IQ

The Flynn Effect – A Summary

Bob Williams — Fri, 23 Dec 2011 20:29:15 +0000

In the 1940s, Smith and Tuddenham independently noticed and reported that IQ test scores were rising over time. The phenomenon did not draw much attention at the time and was idle until Lynn and then Flynn independently rediscovered and reported it in the early 1980s. When The Bell Curve was published, Herrnstein and Murray named the secular rise the “Flynn Effect.” Subsequently the FE has been the subject of a great deal of study and speculation, but has remained elusive and enigmatic.

Characteristics

The FE has been seen as a rise in IQ test raw scores at various rates in virtually every corner of the world. In the U.S. the usually cited effect size is 3 points per decade. In Estonia, the gains have been about 1.65 points per decade, but have accelerated in more recent years. Japan and Korea have experienced rates around 7.7 points per decade, but the Korean gains were delayed by almost 30 years.

Numerous studies of the FE have found that score gains were larger in the lower half of the IQ distribution. For example this was reported for Denmark, Britain, Turkey and Spain. These observations have led to explanations related to improvements in nutrition and education. But some countries have shown greater FE gains in the upper half: Brazil and the U.S. (in the National Longitudinal Study of Youth data). Using data sets of about 1.7 million scores, Wai showed that the FE was present in the top 5% of the U.S. IQ distribution.

Some studies have shown no FE gains; Cotton found none in Australian children. Scandinavian countries have shown rapid FE gains followed by an end to the gains, and by reversal (negative FE) in some cases.

Various studies have shown that the full amount of score gains has been observed in children from age 4 to 6. Likewise studies of developmental quotients (DQ) have shown gains that are similar to IQ gains. [DQs reflect rates of maturation--hold up head, sit up, stand, walk, jump, etc. They are measured by the Griffiths Test and Bayley Mental Scales.]

A large number of studies have reported that FE gains were greater on abstract test items than on scholastic items. This can be stated as a bias towards tests of fluid intelligence and away from crystallized intelligence. The highly abstract Raven’s Progressive Matrices tests (Color, Standard, and Advanced), have shown strong FE gains. The Wechsler has shown FE gains that are almost as large as for the Raven tests, and the performance component is almost twice the magnitude of the verbal component.

The Raven tests have shown gains of 18-20 IQ points per generation in many industrialized countries. Dutch gains were 21 points over 30 years. Urban Chinese gained 22 points between 1936 and 1986.

At the same time that IQ scores have been rising, academic performance has been declining in the US and Britain. SAT scores have been declining, even after correction for the change in demographics of those taking the test. A quarter of the decline remains after the correction.

Hypothetical causes

Among the causes that have been proposed to explain the FE are these:

 Education                                   Decreased family size
 Increased exposure to testing               Heterosis
 Exposure to artificial light                More complex visual environment
 Nutrition                                   Child rearing practices
 and the use of Classical Test Theory versus Item Response Theory.

Education

It is possible that improved education has accounted for some test score gains, although such gains would most likely have no g loading. The finding that FE gains are seen in preschool children (at the same magnitude as seen in adults) suggests that education is not a primary cause of the FE.

Increased exposure to testing

Two mechanisms have been proposed: 1) Brand suggested that the use of timed tests has caused students to work faster by guessing more frequently (multiple choice). While this may be a factor, FE gains are seen on tests that are untimed and on tests that do not use multiple choice. 2) Jensen mentioned “increasing test wiseness from more frequent use of tests.” His point was that frequent testing may have the same sort of impact on test scores as the increase associated with test-retest. This is the same process that is observed with learning and shows up in situations where test training has been used (as is common with the SAT). Both Brand’s and Jensen’s ideas would presumably cause test scores to increase without showing gains on g.

Nutrition and medical care

DQs have gained 3.7 points per decade, while IQ gains of 3.9 points per decade have been seen in preschool children (age 4-6). These and gains in the lower part of the IQ distribution are consistent with the argument that improved nutrition has contributed to the FE. Other factors also agree: increased birth weight; head size measured in 1 year olds has increased by about 1.5 cm from 1930 to 1985 [head size to brain size correlation = 0.80]; and height gains that have increased by about 1 SD over 50 years (similar to DQ gains).

Arguments against nutrition as a cause include: studies of nutrition have shown that neither vitamins nor supplements have had any impact on intelligence; nutrition is unlikely to have declined over the past 20 years in those countries that have a negative FE. Height did not decline in those countries; and contrary to the intelligence gains seen in Norway, height gains from 1969 to 2002 were mostly in the upper half of the intelligence range.

Exposure to artificial light

Artificial light stimulates the pineal gland in animals. The pineal gland appears to play a major role in sexual development, hibernation, metabolism, and seasonal breeding. The effect of stimulating growth is used by poultry farmers to increase their output. There does not seem to be any data available for whether this effect happens in humans, but the speculation is that it might. There has been an obvious increase in the use of electric lighting by humans over much of the time that the FE has been observed.

Decreasing family size

Low IQ people statistically have more children than high IQ people. The high heritability of intelligence, therefore, is a source of dysgenic pressure. If the average family size decreases, the reduced numbers of low IQ children should produce a net increase in the mean, which would show up as a FE gain.

In a very large study of Norwegian conscripts, the previously debated birth order effect was shown to be real, although not large. If family size is declining in various groups, there must be a positive contribution to mean IQ due to fewer low IQ children being born.

Heterosis

Mingroni has argued that since the effects of the environment (on intelligence) are so small, the possibility of a genetic effect should be investigated. Lynn argued that heterosis is unlikely for three reasons:
1 – There was little immigration in Europe before 1950 (the FE was present before that date).
2 – The FE for IQs and DQs are just as large in Europe as in other places.
3 – Studies of heterosis have shown little positive effect on IQ.

Perhaps the most important consideration in determining whether there is a heterosis effect was pointed out by Mingroni: If the FE is found within-families, the cause is not genetic. The FE, however, has been shown to exist within families (conscripts in Norway).

Enriched visual environment

Greenfield and others suggested that the FE gains are caused by the ever increasing shift from verbal communication to visual and interactive media. This is seen globally in the increased presence of movies, television, photography, video games, computers, puzzles, mazes, exploded views, etc.

The mechanism for this hypothesis is that the shift towards visual representations removes some of the novelty from tests, especially the culture reduced tests that have shown about double the FE gains as found in other tests. This is particularly convincing for tests such as the Raven’s, which presents abstract figures in a matrix. Several decades ago these figures may have been more baffling than they are today.

Child rearing practices

The FE has been seen throughout the world, in both developed and undeveloped countries where child rearing practices certainly vary greatly. It is unlikely that this hypothesis is a significant factor, not only because of the cultural variation in child rearing practices, but also because the shared environment has essentially no impact on adult intelligence (per prior discussion).

Classical Test Theory (CCT) versus Item Response Theory (IRT)

Most studies in the literature are based on CTT and are presented without passing along the test item data. This practice hides some of the information that could be extracted from a data set. Test scores are given, but the latent constructs they are designed to measure cannot be examined. IRT allows the researcher to examine the changes in underlying latent ability. Thus, CTT can show differences in scores, even when there is no change in the latent variable. An increase may be due to a general gain in real intelligence, or a decrease in the levels of difficulty of test items.

Alex Beaujean’s results using CCT and IRT to measure FE gains:

Peabody Picture Vocabulary Test-Revised
CCT 0.44 points per year
IRT 0.06 points per year

Peabody Individual Achievement Test-Math
CCT 0.27 points per year
IRT 0.13 points per year

The results clearly show that the FE essentially vanishes for the PPVT-R when IRT is used. The PIAT-M gains are cut to half using IRT. Ergo, the FE gains are determined by the methodology, leading to the concern that much of the literature has reported findings that might be quite different if IRT had been used.

Is the FE invariant?

Multigroup confirmatory factor analyses of several data sets showed that they were not invariant, meaning that FE gains were not gains on the latent variables that the tests were supposed to measure. Besides providing insight as to the nature of the FE gains, the rejection of factorial invariance demonstrates that subtest score interpretations are necessarily different over time.

Real or hollow gains?

When Flynn begain his study and reports on the secular gains, he gave numerous examples of how extreme the gains have been, questioning that they could possibly be real. For example the large gains in The Netherlands would mean that, by 1982 standards, the Dutch mean IQ in 1952 would have been 79. Flynn commented “Has the average person in The Netherlands ever been near mental retardation?” “Does it make sense to assume that at one time almost 40% of Dutch men lacked the capacity to understand soccer, their most favored national sport?” He noted that there are not more gifted Dutch school children now and that patented inventions have shown a sharp decline. He presented a number of similar arguments, all of which questioned the possibility that such large changes could have happened and, therefore, the score gains must be meaningless.

If the secular gains are real, they must show a g loading (this is called a Jensen Effect). Numerous studies of the g loading of FE gains have shown that the gains were not on g. The usual test for a Jensen Effect is the use of the method of correlated vectors. When applied to data showing a FE, it has not shown a Jensen Effect.

Rushton used principal components analysis to show that data exhibiting a FE forms a cluster, thereby indicating that it is a real effect. But the cluster does not overlap with the clusters formed by racial differences, inbreeding depression scores (purely genetic), and g factor loadings (largely genetic). The secular increase is, therefore, unrelated to g and other heritable measures.

As with virtually every aspect of the FE, different data sets produce different results. Colom found that tests of crystallized intelligence did not show gains in g, but tests of fluid intelligence did.

Predictive bias

Jensen commented that the definitive test of whether FE gains are hollow or not is to apply the predictive bias test. This means that two points in time would be compared on the basis of an external criterion (real world measurement, such as school grades). If the gains are hollow, the later time point would show underprediction, relative to the earlier time. This assumes that the later group has not been renormed. [Earlier IQ points would exceed the performance of the later generation for the same IQ.] In actual practice tests are periodically renormed so that the mean remains at 100. The result of this recentering is that the tests maintain their predictive validity, indicating that the FE gains are indeed hollow. If the gains were real and the tests were renormed, people at a given IQ would be getting smarter and this would show up in the predictive validity.

Summary

•          The FE exists between birth cohorts.
•          It is found within sibships.
•          It appears early in life (before school age).
•          There are presumably multiple causes.
•          The gains are all or mostly hollow (not Jensen Effects).
•          There are serious methodological issues to be resolved and which may be a major cause of the gains.
•          The FE is not invariant over time.

The foregoing review is a greatly condensed version of my paper, Understanding the Flynn Effect. It contains more detailed discussions of all of the points mentioned above, with identification of the researchers and their papers, as they apply to each topic. A full reference list is included. Anyone wishing to read the full paper may find it here:

https://sites.google.com/site/thirdstratum/papers-1

Bob Williams

Correlations between the JCCES and other measures (update #2)

Xavier Jouve — Thu, 22 Dec 2011 15:28:02 +0000

Here is a new updated table of correlations that were observed between the Jouve-Cerebrals Crystallized Educational Scale (JCCES) and other measures of cognitive abilities or achievement.

Book review: Richard Lynn’s — The Chosen People

Bob Williams — Fri, 14 Oct 2011 18:59:55 +0000

The Chosen People: A Study of Jewish Intelligence and Achievement

Richard Lynn

Whitefish, MT: Washington Summit Publishers, 2011

Reviewed by Bob Williams

Richard Lynn has added yet another valuable collection of global data and historical insight to the literature. Various sources (Lynn, Herrnstein and Murray, Storfer, etc.) have reported studies showing high mean IQs for Ashkenazi Jews, which Lynn estimates to be 110, but these have not included a detailed study of Jewish intelligence on a global basis, nor a supportive set of data relating to real world accomplishments. The Chosen People provides both, along with a fascinating look at Jewish history. The result is a bit of a history book, combined with a truly massive quantity of data.

In the early part of the book, Lynn discusses the Mizrahim, Sephardim, Ashkenazim, and Ethiopian Jews as separate groups with different mean IQs. [Mean IQs in the prior order: 91, 99, 110, and 66] Lynn examines these four groups on a country-by-country basis and historical basis, sometimes going back 2,000 years. Proceeding alphabetically from Australia through 17 countries, ending with the United States the reader sees a consistent pattern of outstanding accomplishment in a wide variety of fields.

The pattern that emerges is consistent and becomes repetitious. Jews consistently show low fertility, low infant mortality, greater longevity, high educational achievement, high income, high SES, encouragement of eugenic practices, and extraordinary performance levels in a wide range of disciplines. To show the magnitude of achievement, Lynn presented data in the form of Achievement Quotients (AQ), which are simply the ratio of the percentage of Jews who met an achievement measurement (such as receiving the Nobel Prize), divided by the percentage of Jews in the country. For example, US data from 1922 to 1932 showed that 37% of virtuosi musicians were Jews. The 1927 population (3.6%) was taken as the reference, giving an achievement quotient of 10. In situations where the Jewish population is quite small, the AQs sometimes become large. For example, Switzerland has produced 17 Nobel Prize winners, three of whom were Jewish. During the 20^th century, Jews were about 0.3% of the Swiss population. This works out to an AQ of 60.

In the U.S. (1960), Jewish AQs by profession show a wide range of successes:

Psychiatrists               5.8                   Lawyers          3.3

Dentists                    4.0                   Architects       1.7

Mathematicians              3.8                   Engineers        1.1

Doctors                     3.7                   Artists          1.4

Writers                     3.4                   Military         0.5

Lynn interprets the high verbal scores of Jews as explaining the relative magnitudes of the above numbers. AQs above 3.0 are found in fields that depend strongly on verbal (and math) skills; those with lower AQs are fields that require strong visualization and spatial abilities.

United States

Of the country chapters, the one on the US is the most detailed and the one on Israel is of special interest, given the nature of the country. The U.S. discussion has a section on “Music and Hollywood” that is not paralleled by discussions of the other countries. No American would be surprised that the motion picture business is dominated by Jews, but the degree of the domination was beyond my prior knowledge. Studios such as Paramont, Fox, Universal, Goldwin, etc. were founded by and are still run by Jews. Some of the great performers (Streisand, Borge, Marx, Newman, and Hoffman, for example) retained their Jewish names, but many changed their names. Some of the highly recognizable names in Lynn’s longer list:

Konigsberg                 Woody Allen

Ullman                     Douglas Fairbanks

Kaminsky                   Danny Kaye

Schwartz                   Tony Curtis

Keisler                    Hedy Lamarr

Tuvim                      Judy Holiday

Goldenberg                 Edward Robinson

Kubelsky                   Jack Benny

Gumm                       Judy Garland

Birnbaum                   George Burns

Jews have likewise been generously represented in both classical and popular music as composers and performers. In the former category there are such names as Copland, Bernstein, Schnabel, Horowitz, Rubinstein, Heifetz, Milstein, Stern, and Menuhin.

The Jewish dominance has likewise shown up in the media, founding or controlling entities such as Time, Newsweek, U.S. News & World Report, Daily News, Atlantic Monthly, Commentary, The Public Interest, The New York Review of Books, New Republic, and Partisan Review.. Some AQs for media elites:

                                          % Jews           AQ

1975                Media elite           26               10.0

1980                Media elite           30               13.6

1980                Hollywood movies      66               30.0

1994                Hollywood TV          46               20.9

Israel

Israel has five major ethnic groups: Ashkenazim, Sephardic, Oriental, Ethiopian, and Arabs. The Arabs (as discussed in the last chapters) are the source group for Jews, about 2,000 years ago. Their mean IQ in Israel today is 86 (the mean outside of Israel is 84); they account for 20% of the population. Oriental Jews in Israel have a mean IQ of 85. As Lynn has shown in his other books, the genetic nature of intelligence is evident in mixed groups. In this case, high school graduates with both parents European have the largest percentage of their children above high IQ reference points (128, 120, and 110); families with one parent European and one Oriental had smaller percentages at these levels; when both parents were Oriental, the percentages were considerably lower at the reference IQs. Since all of these children were reared together in the same kibbutzim environment, the effect is presumably limited to genetic causes.

Ethiopian Jews in Israel have a mean IQ of about 69 (about the same as that for sub-Saharan Africans–67). Ethiopian Jews in Israel display the characteristics of other low IQ groups: high juvenile crime rate; high percentage of single-parent families; high dropout rates from high school; poor school achievement; low scores on matriculation exams; low employment rates (45% of men in 2003); high fertility (60% of families with 5 or more children); and a high HIV infection rate. The Brookdale Institute estimated that each Ethiopian immigrant costs the Israeli taxpayer about $100,000 over the course of his lifetime.

Achievement

There are a large number of measures that indicate very high performance in various fields. Among those used in the book:

Nobel Prize

Membership in the Royal Society and Membership in the British Academy (Britian)

Fields Medalist and Wolf Prize (mathematics)

Pulitzer Prize

Wealth (wealth quotients are calculated the same way as EQs)

Eminence (Charles Murray’s Human Accomplishment cited)

Education (educational quotients are calculated the same way as EQs)

Chess grandmasters and bridge champions

Jews, and particularly Ashkenazi Jews, have performed much better in these measures than would be expected on the basis of their group size. This leads to the obvious question of why? Throughout the book, Lynn points to high intelligence as a likely explanation for success in any task that is cognitively demanding. The 10 point IQ advantage (relative to the Northwest European standard, known as the Greenwich standard,” after the reference point for longitude) increases the percentage representation at the high end of the bell curve. Above IQ 130 the representation is four times that of a group with a mean of 100; above 145, it is six times as great. [My calculation agrees with Lynn’s at 130, but I found a 7.3 times as great number for 145.] The point is that various AQs are considerably larger than the number of high IQ individuals who might form a pool for success in the measures listed above. Charles Murray also commented that the higher mean IQ is not sufficient to explain the high rates of success of Jews.

Lynn presents discussions of other factors that may account for the unusually high achievements, such as cultural values and motivation. He offers the formula: IQ x Motivation x Opportunity = Achievement.

Causes for the High IQ of Ashkenazi Jews

Lynn discusses the history of the four Jewish groups and offers some speculation as to how they arrived at their present day numbers. The Ashkenazi Jews are decedents of Arabs, who presumably had the same mean IQ (84) as they have today, leading to the most interesting question of how they increased the mean by 26 points and what factors may have led to the higher verbal-math and lower visualization-spatial abilities.

Before getting into specific theories, Lynn discusses the genetic basis of intelligence and the present day understanding that the family environment has no long-term effect on intelligence. He builds a solid case that the high Ashkenazim mean is not the result of environmental factors, then addresses the possible explanations for the genetic boost.

Eugenic Hypothesis – The idea is that Jewish customs and practices favor the survival of the intelligent. The only argument that seems to support this line is that sometimes there were restrictions on the marriages of the poor. This happened when the controlling Gentiles limited the number of Jewish marriages allowed per year. The problem with this hypothesis is that it fails to explain why there were significantly greater gains among the Ashkenazim than among other Jews.

Persecution Hypothesis - The country by country history from Lynn shows that Jews were frequently expelled from one nation, then another, throughout Europe. At various times they were killed, sometimes in connection with the Inquisition and Crusades. It is not unreasonable to believe that this protracted and deadly stress factor caused disproportionate numbers of less intelligent Jews to die, in much the same way as the stress of cold weather is likely to have killed off less intelligent people who migrated north from the African Savannah.

Discrimination Hypothesis – Jews were sometimes limited to the jobs they were allowed to hold, limiting their options severely. One area in which Jews were successful was money lending (as a result of usury laws that applied to Gentiles). This was a sufficiently complex task that it is reasonable to believe that only the bright survived. At that time, wealth was a significant advantage to survival, as infant and child mortality rates were high.

Miscegenation Hypothesis – There is evidence of interbreeding between Jews and Gentiles who lived in the same locations. This practice created a path for higher IQ genes to boost the Jewish mean intelligence. Lynn put the gain limit at 6 points, moving the mean from 84 to 90 and requiring other factors to account for the additional gain of 20 points.

Apostasy Hypothesis – Proposed by Charles Murray, this hypothesis is based on a requirement from 64 AD that all Jewish boys attend school. The scholastic requirements were demanding and the argument is that individuals who did not possess high verbal ability became discouraged and renounced their faith. This is consistent with the decrease in the number of Jews from 4.5 million in the 1^st century to about 1.5 million in the 6^th century.

Lynn does not argue in favor of or against any of the five hypotheses, but points out that they are all possible and that several or all of these processes contributed to the IQ gains. He ends the book with three conclusions: (1) the high IQ of the Jews must be genetic; (2) eugenic customs contributed to high Jewish IQ, proving that eugenic practices work; and (3) a minority group with high IQ succeeds despite discrimination. These conclusions are strongly supported by the large body of independent research cited by Lynn, but all are counter to the PC thinking that is prevalent today.

The Future

Jewish populations are declining throughout the world, except in Israel. This has been due to low Jewish fertility, migration to Israel, loss of faith, and intermarriage with Gentiles. In the U.S. Jewish fertility is 1.16, which will result in an approximate halving of the population in each generation. Despite migration to Israel, the Central Intelligence Agency issued a report in March of 2009, predicting that Palestinians and Jews would merge in a single state and that, over time, the higher fertility of the Palestinians would lead to their majority status. That, combined with a predicted migration of Jews from Israel to Russia will likely result in Israel not surviving as a Jewish state.

Correlations between the JCCES and other measures (update)

Xavier Jouve — Sun, 07 Aug 2011 19:00:11 +0000

Here is an up to date table of correlations that were observed between the Jouve-Cerebrals Crystallized Educational Scale (JCCES) and other measures of cognitive abilities or achievement.

I Am a Word: Raw Score Interpretation Table

Xavier Jouve — Thu, 30 Jun 2011 16:37:37 +0000

This table is based on the study of 93 score pairs from persons who took the IAW and at least one of the following cognitive ability assessments: the verbal part of the WAIS and the verbal part of the RIAS.

The IAW and the verbal ability as measured by these other tests were correlated at .82. This indicates a high correlation.

The Spearman-Brown coefficient for the IAW raw score was .96 (N=286). This is seen as excellent and ensures an appropriate standard error of measurement (3.08).

Speed & Intelligence

Michael Chew — Tue, 24 May 2011 23:48:37 +0000

Speed of Information Processing

How many times have we heard references to someone we know, who is labeled as clever because he is fast at grasping a concept or quick on some mental arithmetic and phrases used to describe them like “speed of cop-on” when we associate speed with intelligence. It seems one could almost say that being fast is synonymous with being smart. Certainly in the field of psychometrics speed of information processing and its correlation with g, the proxy for intelligence is of much interest and the subject of numerous studies. Speed of information processing can easily be measured by elementary cognitive tasks (ECTs). An example of this, would be a task requiring a correct response to a visual stimuli (e.g. whether an arrow that appears on a screen is pointing left or right) of which the response is measured from the time the stimuli appears to the time a subject responds by pressing a button. It almost seems counterintuitive that such simple tasks, reducible to basic fundamental neuro-physiological functions, of which involves no cognitive operations can capture the complex and multi-dimensional quality of the brain i.e. intelligence. However many studies have been carried out since the 1980s, and have shown robust correlations (negative) between the reaction time (RT) of responses to ECTs and intelligence.

Typical correlations between single ECTs (elementary cognitive tasks) and IQ are moderate, ranging from -0.2 to -0.4 (Jensen 1998, Sheppard 2007) but increase dramatically when tests are carried out using a battery of ECTs or if the complexity of the ECTs increases by using competing tasks (Fogarty & Stankov 1995) or dual tasks (Jensen 1998), although the ECTs bear no resemblance to conventional psychometric tests. Jensen explains that the reason that a battery of ECTs correlate higher with g, is due to the fact that the “global” speed component is cumulative with every ECT added to the test battery while the unique non-cognitive speed component for each ECT is only added once. This is analogous to adding various items in a conventional IQ test which increases the test correlations with other tests. With respect to speed of processing between high and low ability groups, the RT difference between High IQ subjects and Low IQ subjects are in ways that are seemingly paradoxical. A small percentage of the fastest RTs of low IQ subjects are almost as fast as the fastest RTs of High IQ subjects and faster that the median RTs of the High IQ subjects (Jensen 1998). But where they differ most is in the longest RTs produced. The High IQ subjects almost never produce any RTs that are as slow as most of the RTs produced by the Low IQ subjects. More importantly, it is the differences in longer RTs that are highly correlated with IQ and not the shortest RTs as observed by Larson et al. (1990) and Krantzler (1992).

Fig 1- “The worst performance rule”: The RT data from the original experiment by Larson and Alderton (1990, Table 4) were divided into 16 bands, the mean of which was correlated with g. Mean RT from the slowest bands correlates more strongly with g than the mean RT from the fastest bands.

Test Taking Speed

A cursory look at a typical speeded IQ test like the Cattell Culture Fair III (CCFT) which is considered a good measure of fluid intelligence and highly speeded (50 items to be completed in a time frame of 12.5 minutes or an average of 15 seconds per item) seemingly indicates that speed is indeed an important component of intelligence or a part of intelligence if you like. Does this mean that a high IQ measured on the CCFT would require a high speed of information processing? Vernon (1989) carried out five studies of which he extracted a g factor from a battery of tests which included the WAIS + RAPM and the MAB (Multidimensional Aptitude Battery) of which he labeled IQ_g . He also extracted a general factor from a battery of ECTs from which he labeled RT_g. Paradoxically, he found that the correlations between the untimed version of the RAPM and RT_g was the highest and in contrast the digi-symbol, a speeded subtest of the WAIS battery had the lowest correlation with RT_g bearing in mind that the RAPM had no commonality in content with any of the ECTs. Further, Jensen (1998) found that the time taken by university graduates on the RAPM was not significantly correlated to the number of questions they answered correctly. Consequently, he distinguishes test taking speed with speed of information processing and considers these to be completely different abilities, with test taking speed more associated with personality and not cognitive factors.

Working Memory, Complexity & Speed

Theories on the limited capacity of working memory (WM) tend to lend support to the correlation between speed of processing and ability. It is theorized that working memory has two properties which make speed of information processing crucial for solving tasks in a given amount of time. WM is limited in its capacity to store information and the information entered into storage decays over time. An example of a WM task is remembering a 10-digit phone number and then adding the digits to produce an answer. The capacity of the WM in retaining the 10 digits is crucial before the individual can start the addition process. The cause of a subject failing to complete the task successfully may be attributed to two reasons; i) if his WM capacity is low, the 10-digit information is lost before the addition can be completed or even take place, or ii) the speed of which he adds the 10 digits is too slow and he fails to execute the adding operation before the information in WM eventually decays and has to be re-entered. Hence, the speed with which the information can be processed is a decisive factor to explain the differences in ability of individuals (Vernon 1983) since the information retained in working memory decays if not processed in time. Since task complexity increases the demand it places on the retention of the information in WM, then it stands to reason that the successful processing of the task would then be mediated by the speed of which it can be processed. In other words the heavier the burden on working memory, the more beneficial speed of processing becomes lending support to a speed dependent WM.

Vernon and Kantor (1986) predicted an increase in correlations between speed of processing and reasoning from timed to untimed conditions. They reasoned that if time constraints are relaxed for reasoning tasks (which correlates highly with WM) the correlations between speed of processing and reasoning should be higher since the measure under untimed conditions will contain more speed variance, due to the fact that under untimed conditions the subjects are able to work through all of the items in a test, and the higher ability subjects will be able to solve the hardest items (not usually possible under timed conditions), thereby introducing more variance in scores. The opposing hypothesis to this is that under timed conditions the more able will be able to go through more items with a higher percentage of correct responses, since they will work faster and more accurately through the items compared to the slower subjects. So placing time constraints will increase the correlations between speed of processing and speeded reasoning. Wilhelm et al. (2001), sets out to test this hypothesis by using 367 high school students. Test instruments using reasoning and processing speed subtests were obtained from the Berlin Model of Intelligence Test (BIS-4) of which were divided into 2 groups of reasoning tests and 1 group of speed test. The subjects were divided into 6 groups of which all were tested on both groups of reasoning tests under timed and untimed conditions, and 4 groups worked on the speed of processing tests. The correlations between tests taken by all 6 groups are summarized in table 4 reproduced below;

	Speed of Processing	Speeded Reasoning	Un-speeded Reasoning
Speed of Processing	0.82	0.49	0.34
Speeded Reasoning		0.69	0.64
Un-speeded Reasoning			0.69

The correlations between speed of processing and speeded reasoning is higher than that between speed of processing and un-speeded reasoning which runs counter to what was hypothesized by Vernon & Kantor based on the concept of a speed dependent WM, although the relatively lower correlations are still significant enough that a speed dependent WM cannot be dismissed outright.

Speed, Item Difficulty & Ability

What would the correlations look like if with increase task difficulty, and if the performance of individuals on the tasks were calibrated according to ability? Would we still get the negative correlations so ubiquitous in the numerous studies carried out on information processing tasks? Danthiir et. al. (2005) carried out studies on 186 undergraduate university students and subjects recruited from, the wider community, with Gs (processing speed tasks or ECTs), Gf and Gc tasks. CFA was carried out on the scores for the 3 types of tasks (4 each for GS, Gf and Gc tasks) to confirm the ability factors labeled as Gs, Gf and Gc. What they found was that as the item difficulty increase, the response times for both Gf and Gc items increased with larger latencies for Gf items compared to Gc items of similar difficulty. The findings for the easy item are as expected where the subjects higher on Gf perform faster for easy items but slower as the item difficulty increases. A similar relationship is found for Gc items. This means that the correlations start from negative, reducing as the level of difficulty increases to a point where positive correlations are obtained for the hardest items. The negative correlations are highest for the easiest items, with negative correlations higher on Gc items compared to Gf items. Processing speed seems to play a more important role for easier tasks (more so for Gc items) compared to more difficult items. Correlations between Mean Correct Response Latencies for the easiest items and Ability Factor Scores, are -0.48 with Gc, -0.38 with Gf and -0.25 with Gs for Gc items, and -0.20 with Gc, -0.26 with Gf and -0.22 with Gs for Gf items. See figure 2 (from the original paper).

A re-analysis of the data was carried out, this time with the response latencies calibrated according to ability of the subjects. The scores were divided into 3 ability groups for each of the Gf and Gc items with the results on table 4 re-tabulated below;

Table 4:Mean correct response latencies (SD) for ability groups, with item difficulty calibrated for ability.
Ability	Gf Items				Gc Items
Ability	Easy	Mid-Easy	Medium	Hard	Easy	Mid-Easy	Medium	Hard
	latency (ms)	latency(ms)	latency(ms)	latency(ms)	latency(ms)	latency(ms)	latency(ms)	latency (ms)
1	12470	28820	33982	40409	8530	9711	11285	12068
1	(3162)	(10456)	(11666)	(19884)	(1731)	(1878)	(2703)	(3211)
2	24845	32565	53384	56856	9457	11007	11983	14081
2	(8966)	(8325)	(20174)	(21867)	(2066)	(3206)	(2871)	(4776)
3	31355	35861	57428	69627	9186	10834	11628	13256
3	(7931)	(10961)	(17898)	(28932)	(2025)	(2879)	(3583)	(3728)
Note: Statistics reported for Gf item difficulties are with participants grouped according to Gf ability; statistics reported for Gc item difficulties are with participants grouped according to Gc ability. Response latencies are in ms. 1 = below average, 2 = average, 3 = above average.

For the Gf items, the latencies increase from easy to difficult items across ability groups with the highest latencies for the highest ability group. The latencies are highest for the high ability group even for the easiest items. For the Gc items the latencies increase from easy to the hardest items with the highest latencies demonstrated among the average group. The latencies are much higher in general for the Gf items compared to the Gc items. What is obvious from the results are i) the data does not suggest that the subjects higher on Gf and Gc are faster on the easy items nor does data suggest that ii) the difference in latencies reduces the more difficult the items become. This pattern or rather the rank order of latencies with respect to item difficulty still holds, although to a much lesser extent for the Gc items. An important point to note is that the difficulty of the items used for the higher ability group are harder than that for the lower ability group, and hence what can be correctly concluded is that the high ability group uses more time for each increment in difficulty of the items, compared to the low ability group. The findings are different from that showed in table 2, where the high ability subjects (on both Gf and Gc items) are faster on the easy items. It would seem evident from this study, that the speed of information processing or the reciprocal measure i.e. the magnitude of the latencies are mediated by item difficulty and when calibrated for ability, are dependent on the ability level of the subjects. Also the results of this study, seemingly provides conclusions that run counter to what other studies have indicated with faster processing speed for higher ability subjects. Interestingly, in testing how mental speed relates to real-world criteria such as school performance, Rindermann & Neubauer (2002) with their structural equation models obtained correlations of school performance (on 271 students between years 9 to 11) with IQ of 0.5, processing speed and school performance of 0.35 (0.39 for high ability subjects compared to 0.26 for average ability subjects). They concluded that although processing speed can “index” performance such as school grades and cognitive abilities, it cannot substitute psychometric intelligence or g since the correlations between school performance and speed of processing cannot be as high as that between school performance and IQ.

It seems then that the differences in general ability is more pronounced when measured using more complex cognitive operations and tasks and that IQ tests are still a more reliable measure of cognitive ability compared to applying a common index graded along a single continuum of measurement, speed.

References:

1. Danthiir V. Wilhelm O. & Schacht A. Decision speed in intelligence tasks: correctly an ability?-Psychology Science, Volume 47,2005 (2), P.200-229.

2. Fogarty G., & Stankov L. (1995). Challenging the Law of Diminishing Returns. Intelligence, 21 (2), 157-176.

3. Jensen A. R. (1998)- The G Factor, The Science of mental Ability (Praeger).

4. Kranzler John H (1992) – A test of Larson and Alderton’s (1990) worst performance rule of reaction time variability. Personality and Individual Differences Vol 13, Issue 3 March 1992 Pages 255-261 .

5. Kyllonen, P. C., & Christal, R. E. (1990). Reasoning ability is (little more than) working-memory capacity?! Intelligence, 14, 389–433.

6. Larson, G. E., & Alderton, D. L. (1990). Reaction time variability and intelligence: A worst performance analysis of individual differences. Intelligence, 14, 309-325.

7. Neckar, E. (1992). (1992). Cognitive analysis of intelligence: The significance of working memory processes. Personality and Individual Differences, 13 (9), 1031-1046.

8. Rindermann H., Neubauer A. C. – Processing speed, intelligence, creativity, and school performance: Testing of causal hypotheses using structural equation models. Intelligence 32 (2004) 573–589

9. Sheppard Leah D., Philip A. Vernon – Intelligence and speed of information-processing: A review of 50 years of research. Personality and Individual Differences 44 (2008) 535–551

10. Vernon Phillip A. (1989) – The generality of g. Personality and Individual Differences Volume 10, Issue 7, 1989, Pages 803-804

11. Vernon, PA (1983a). Recent finding in the nature of g. Journal of Special Education, 17 (3), 388-400.

12. Vernon, PA (1983b).Vernon, PA (1983b).

13. Vernon, P. A., & Kantor, L. (1986). Reaction time correlations with intelligence test scores obtained under either timed or untimed conditions. Intelligence, 10, 315–330.

14. Wilhelm O., Shulze R., The relation of speeded and unspeeded reasoning with mental speed. Intelligence 30 (2002) 537-554.

The Generality of Spearman’s Law of Diminishing Returns

Michael Chew — Tue, 15 Mar 2011 00:21:13 +0000

Diminishing Returns

Spearman (1927) discovered that when he compared the test scores of 2 groups of children in a correlation matrix, where one group composed of “normal children” and the other group composed of children with low abilities, the mean test score correlations were lower for the normal group (+0.466) compared to the low ability group (+0.782). Spearman used the term “diminishing returns” to describe this phenomenon, alluding to the fact that the benefits accorded to persons with higher g, diminishes the more g he possesses and since then several studies substantiated what he called the Law of Diminishing Returns now aptly named Spearman’s Law of Diminishing Returns (SLODR). Among these were Detterman & Daniel’s (1989) study using the Wechsler scales on large samples divided into 5 ability levels, Legree et al.(1996) on the 1980 Armed Services Vocational Aptitude Battery (ASVAB) normative sample, and that by Martin Evan’s (1999) using confirmatory factor analyses on large samples of test scores from the ASVAB and the Otis-Lennon Mental Ability test. All three “rediscovered” SLODR with lower correlations for high ability groups. Several hypotheses were put forward to explain this phenomenon and many subscribe to the fact that the cause of this is due to the more specialized abilities of high ability subjects. In other words the higher the level of g, the more is invested in specialized skills, trained to a point where if confronted with a demand for that skill to solve a problem, the high ability subject will provide a response that is automated and would not rely on his high g level but instead rely on his non-g related specialized ability. This explanation is called the “ability level differentiation” or the “differentiation hypothesis”. Figure 1 below by Andrea Kuszewski (2009) best describes this phenomenon;

Does the law infer that the g construct for high ability groups is different than that for low ability groups? What is the cause of this phenomenon and could it be due to problems with the test measurement but more importantly how general is it?

Test Complexity

Detterman (1991) uses the following metaphor to describe g: “Each person can be thought of as having a set of independent abilities related to each other by a set of weights specifying each ability’s relationship to other abilities in the performance of a particular task or test; g arises from this set of weights in combination with a person’s independent abilities”.Fogarty and Stankov (1995) challenges this notion by explaining that the metaphor used by Detterman means that a weak person attempting to carry these moderately heavy weights will benefit significantly by an increase in energy. However a strong person will not exhibit a similar benefit simply because he will extend minimal effort to lift the weight thereby not drawing out his true higher abilities. They then extend the metaphor by stating: “Consider the situation where the weights to be lifted are heavy. A weak person cannot lift the weights at all and does not benefit from a boost of additional energy. A strong person, however, will go close to lifting these weights under normal conditions and will probably benefit considerably from the same energy boost. The Law of Diminishing Returns will not apply under these conditions.” What they are alluding to is that most standardized tests are constructed to measure abilities around the mean. If sufficiently harder test items were chosen to discriminate at higher levels, the positive manifold will be exhibited at the higher end. However care must be taken when selecting items of greater difficulty so that these items do not tap into non-g related specific abilities. For instance, if a test is composed on numerical items and we wish to increase the difficulty of the test, we risk increasing the demands on specific mathematical skills if we make the test more difficult “mathematically”, requiring say complex mathematical operations for instance, which load heavily on non-g mathematical skills.

For the first study, Fogarty and Stankov selected 15 year-old students, drawn from a private school in Australia. The sample were grouped according to their differential aptitude test (DAT) results, and were divided with the low-ability group having a mean IQ level of below 88 and above 112 for the high ability group. They administered 8 subtests consisting of 4 types of single tests containing several items each, tapping into low order perceptual, visualization, auditory and fluid capabilities. For tests with items requiring higher cognitive demand, 4 combinations of 2 single tests were used with a total of 4 single tasks and 4 competing tasks. The competing tasks were presented simultaneously, requiring equal attention thereby imposing a higher cognitive load suitable for higher ability subjects. The term “complexity” is preferred over “difficulty” for the competing tasks since the individual tasks making up the competing tasks were not difficult before combination and simply increased in cognitive demand after combination. Descriptive statistics (table 2) in the paper shows that the standard deviations and reliability coefficients for each of the sub-test scores are very similar for both groups which means that if any differences in the inter-test correlations are discovered cannot then be attributed to range restriction nor the reliability of the test items.

Note : Jensen (1998 – pg 587) states that if one wants to compare the inter-test correlations between high and low ability groups, the range restriction (of both groups) has to be dealt with and that the two groups should be selected such that the respective test scores on a selection test, show similar SDs and reliability coefficients. The reasoning behind this is that if we do not select the 2 groups with similar SD’s and reliabilities, and say the SD of the test scores are smaller in the higher ability, then we cannot be certain if SLODR is simply due to the lower variance in test scores for the high ability group. So the question: “are the inter-correlations higher in the lower ability group?” should only be answered after the two groups are selected on test scores of similar or equal variance.

The composite DAT scores were used to estimate the g saturations of the sub-test scores and were included in the inter-test correlation matrix. The subtest correlation matrix (table 3 of the paper), shows the overall level of inter-correlations were higher for the high ability group compared to the low ability group. Difference in correlations between the high & low ability group were r = .35 versus r = -.00, (Table 4a) for composite scores of single tasks and r = .44 versus r = -.02, (Table 4b) for composite scores of competing tasks. The result not only shows no presence of SLODR, but shows evidence against it. From the first study it appears then that the lack of test complexity is a possible cause for low correlations and conversely increased complexity causes higher correlations at the high end. The corollary of this is that if the test item complexity is moderated or consists of items which have the same discriminatory power for both ability groups, the resulting test-correlations for both groups should be about the same. Since perceptual speed items have the same discriminatory power for high and low ability groups (they are easy for all ability levels) Fogarty and Stankov (1995) used these tests in their second study to confirm if the test-score correlations would be similar for both groups. A total of 179 subjects were selected with 149 of them consisting of 1^styear psychology students from the University of Sydney and the rest from the general population. The groups were divided into those scoring lower than 1 SD (IQ 92 to 103) below the mean and more than 1SD (IQ > 125) above the mean, screened by using a battery of 21 tests. The high and low ability groups finally consisted of 29 and 28 subjects each with the total of 179 representing the whole population. Both groups were then subjected to 4 perceptual speed tests consisting of Comparison, Search, Stroop and Pairing of Digit-Symbols tasks. The results of the second study (table 5-correlation matrices) showed very similar correlations and standard deviations indicating similar amounts of a common variance for both groups and the total population. The second study not only complements the first in confirming that test item difficulty controls test-score correlations for higher ability subjects, but also does not support SLODR since there is no decrease in correlations for the high ability group.

Colom et al.,(2002) provides a similar explanation with respect to test item complexity, albeit in an indirect way. What they found with their study on the WAIS-III was that the higher we go up the ability scale, the lower the g-loading on the sub-test scores. They postulated that the higher the ability, the less complex each test item appears to high ability subjects. It follows that the lower relative complexity of the test due to higher ability, the lower the g loading and less of the variance in test scores will be attributable to g.

SLODR on the WISC-IV (2003) and WAIS-III (1997) Normative Samples

The study by Saklofske et al. (2008) on the WISC-IV and WAIS-III standardization data of which samples are drawn from the US, Canada, Australia and the Netherlands yielded results that did not support SLODR. For the WISC-IV (N=4151), the subtest scores were compared in two ways, between two age groups (6-8 & 9-16) and also between high and low ability groups using a cut-off score of 495. The high and low ability groups were screened using an IRT-based scoring method on the item-level score data available. The average mean score of the low and high ability groups were 488 and 504 with average standard deviations of 5.10 and 5.33 respectively. For both comparisons the g loadings for the higher age group and higher ability group showed comparatively higher loadings for nearly all of the sub-test scores, (tables 1.3 and 2.3). For example the US sample shows an average inter-correlation of 0.32 versus 0.43 for the low compared with the high ability group. Interestingly table 2.1 of the paper shows a high correlation between age and ability, which is expected and typically found in studies demonstrating SLODR with lower g loadings for the higher age group. However a higher g loading for the higher age group was found instead, findings of which are the reverse of SLODR.

The second study on the WAIS-III (N=4522), with subjects from the US, Canada, Australia and the Netherlands. The scores were compared between age group and ability with the split of age groups at 16-44 and 45-89. Average scores (table 4.2- for the US and similar for the other 4 countries) for the 16-44 age group were 502.63 with an SD of 6.83 and 497.52 with an SD of 7.41 for the 45-89 age group. The average scores for the high and low ability group were 491.97 SD 5.03 and 504.63 SD 4.15 respectively. The g loadings for the two ability groups were mixed (table 4.3). For example the US data shows an average inter-correlation of 0.30 versus 0.18 for the low ability group compared with the high ability group, but 0.16 versus 0.22 for the Netherlands sample, showing weaker and therefore inconclusive evidence for SLODR. Interestingly, the Wechsler series which contain a large battery of diverse subtests makes it likely for SLODR to be found as did Detterman and Daniel (1989) since it allows the high ability subjects to demonstrate their varied abilities on the variety of subtests. However SLODR is not supported in this study.

BØrge Priens PrØve on 2 Large Danish Samples

Hartmann & Teasdale (2004) analysed two large sets of data (N=38,333-1988, N=25,020-1998) on Danish military recruits ages 18-19 tested with the BØrge Priens PrØve (BPP) test. The BPP has a correlation of 0.82 with the WAIS as reported by Mortensen, Reinisch and Teasdale (1989) and a correlation of 0.57 with the RAPM. The BPP contains four sub-tests of Letter Matrices, Verbal Analogies, Number Sequences and Geometric Figures. Samples were divided between low and high ability based on two cut-off scores (35 and 38) and were screened for almost equal SDs for each subtest. Care was taken in the selection of the samples based on subtest SDs such that the differences between high and low ability groups were as small as possible to ensure proper inter-correlations can be carried out and to avoid inflated values on the g loading i.e. to reduce the restricted range effects. The two cut-off points were used to compare the sub-test SDs between high and low ability groups, again with the aim of minimizing the between group sub-test SD difference. Two methods of factor analyses were used; principal component analysis and total explained variance for the first principal axis factor (PC and PAF1). The key results are reproduced on the tables 1 & 2 below;

The between ability group difference in total score SD were not only much higher for the “35” group, but also higher for the high ability group, compared to lower SDs for high ability subjects of the “38” group. Further, although the sub-test SDs were similar for both the “35” and “38” group, the results from the “35” group showed a larger difference in total explained variance in scores (see table 1 & 2 above, for inter-correlations between low and high ability). This is somewhat unexpected since a higher effect of SLODR in the “38” group was expected because of the difference in ability (between high and low subjects) is larger for the “38” group compared to the “35” group. However, not only is the higher effect of SLODR not found, a higher g loading was discovered for the higher ability group (for both the “38” and “35” group) instead with a larger increase in g for the “35 group”, meaning that the reverse of SLODR was observed. Hartmann & Teasdale explained that for the SLODR to manifest itself, non-g factors have to be present for the subjects to exhibit their varied abilities. Hence one possible reason why SLODR was not present is because the BPP test consists of only 4 highly g-loaded subtests and not much non-g factors, which did not allow the subjects to demonstrate their varied abilities. This explanation would not hold though, if we look at the results for the study carried out by Saklofske et al. (2008) on the WISC-IV series where higher g saturations were found for the high ability group.

Summary

The above studies seem to indicate that if we want to test high ability subjects and maintain high test-score inter-correlations or to test for SLODR, methodological considerations such as equating SDs for subtest scores between ability groups, test complexity, type of subtests, may be determining factors. One significant difference between the 3 studies above compared to the studies carried out by Detterman and Daniel (1989), Legree et al.,(1996) and Evans M.G. (1999), is that no corrections to the correlations were made for restriction of range for each of the ability groups. Instead sample scores were selected for comparable SDs and reliability ensuring that any difference in correlations between the ability groups would not be due to the smaller variance in scores of the high ability group.

The “law” of diminishing returns have been supported by several studies, e.g. Abad et al. (2003), Deary et al. (1996), Detterman & Daniel 1989), Der & Deary (2003), Evans M. G. (1999), Jensen (2003), Legree et al. (1996), te Nijenhuis &Hartmann (2006). But how general is this “law”? A review by Hartman and Nyborg (2004) of the published literature on SLODR revealed that within an ability span of 1.5SD, only about two thirds of the studies showed a tendency towards SLODR and only about 50% of these were statistically significant. If we look at the 3 studies above, Fogarty and Stankov (although using a small N) can demonstrate that with the inclusion of more complex test items which do not rely on specific abilities, will result in higher score inter-correlations for high ability subjects. Further, the analyses carried out by Hartmann and Teasdale which are based on a large sample size, findings against SLODR were obtained. The study by Saklofske et al.(2008) show similar findings against SLODR for the WISC-IV, but inconclusive findings for the WAIS-III. Interestingly, Detterman & Daniel carried out a similar study using the WISC-R as well, but arrived at opposite conclusions. The three studies above which cover ability spans well over 2 SD of which two are based on data of large samples do not support SLODR and in some cases exhibits the opposite effect, putting into question the generality of the “law”.

References:

1. Abad, F.J., Colom, R., Juan-Espinosa, M., & García, L.F. (2003).Intelligence differentiation in adult samples. Intelligence, 31,157–166.

2. Colom et. al., (2002)- Education, Wechsler’s Full Scale IQ, and g- Intelligence 30 (2002) 449–462.

3. Detterman, D. K., & Daniel, M. H. (1989). Correlates of mental tests with each other and with cognitive variables are highest for low IQ groups. Intelligence, 13, 349-359.

4. Detterman, D. K. (1991). Reply to Deary and Pagliari: Is g intelligence or stupidity? Intelligence, 15, 251-255.

5. Deary, I.J., Egan, V., Gibson, G.J., Austin, E.J., Brand, C.R., & Kellaghan, T. (1996). Intelligence and the differentiation hypothesis. Intelligence, 23, 105–132.

6. Der, G., & Deary, I.J. (2003). IQ, reaction time, and the differentiation hypothesis. Intelligence, 31, 491–503.

7. Evans, M. G. (1999). On the asymmetry of g. Psychological Reports, 85, 1059-1069.

8. Fogarty,G., & Stankov, L. (1995). Challenging the Law of Diminishing Returns. Intelligence, 21 (2), 157-176.

9. Hartman P & Teasdale T.(2004)- A test of Spearman’s Law of Diminishing Returns in two large samples of Danish military draftees-Intelligence 32 (2004) 499–508.

10. Hartmann, P., & Nyborg, H. (2004). Spearman’s Law of Diminishing Returns: A critical eye on a century of methods, results, and current standing on the theory. Unpublished work.

11. Jensen, A.R. (2003). Regularities in Spearman’s law of diminishing returns. Intelligence, 31, 95–105.

12. Kuszewski Andrea (2009) – The Divergence Hypothesis – It’s effects on the analysis on complex traits.

13. Legree, P.J., Pifer, M.E., & Grafton, F.C. (1996). Correlations among cognitive abilities are lower for higher ability groups. Intelligence, 23, 45–57.

14. Mortensen, E. L., Reinisch, J. M. & Teasdale, T. W. (1989). Intelligence as measured by the Wais and a military draft board group test. Scandinavian Journal of Psychology, 30, 315–318.

15. Saklofske Donald H. , Yang Z., Zhu J., Austin J. Elizabeth Spearman’s Law of Diminishing Returns in Normative Samples for the WISC-IV and WAIS-III. Journal of Individual Differences 2008; Vol. 29(2):57–69.

16. Spearman, C. (1927), The Abilities of Man: Their Nature and Measurement.New Yor: Macmillan.

17. te Nijenhuis, J., Hartmann, P. (2006). Spearman’s “law of diminishing returns” in samples of Netherlands and immigrant children and adults. Intelligence, 34, 437-447.

Insights From Residuals

Bob Williams — Fri, 11 Mar 2011 17:59:56 +0000

Insights From Residuals

Bob Williams – March 2011

Most of us are aware that the predictive validity of an IQ test is all, or almost all, due to its g loading. Jensen (1998) presented numerous studies showing that as test g loading increases, its predictive validity increases. He also showed that non-g factors contribute virtually nothing to the validity of the test.

When g is removed from a correlation matrix, the remaining factors are known as residuals. [There is a good discussion of how to factor g out of a correlation matrix in Jensen (1980) -- Chapter 6.] When a residual matrix is tested for predictive validity, the result is a validity coefficient that lies near zero. For example, Ree and Earles (1994) examined the ASVAB for 78,049 airmen and found that the g factor scores had an average predictive validity coefficient of +.76, while the non-g portion of the test had an average predictive validity of -.02. The difference between the validity coefficient for g and the coefficient for g plus non-g factors is known as the incremental validity. Olea and Ree (1994) studied 1,400 navigator trainees and 4,000 pilot trainees using data from the AFOQT and found incremental non-g validity coefficients of +.020 (navigators) and +.084 (pilots).

Various sources identify some specific non-g factors that make small contributions to incremental validity. Jensen (1998) identifies spatial and psychomotor abilities and notes that physical scientists have both high g and high spatial ability. Jackson and Rushton (2006) found a modest residual correlation (r = .12) for mathematics and found negligible contributions from other residual components. Within the top range of intelligence, spatial ability shows an incremental validity for individuals who pursue STEM degrees (Wai, et al., 2009).

What are the components of non-g factors?

When g is removed, the remaining variance includes group factors, specific factors, and error variance. Group factors are broad abilities, such as verbal, numerical and spatial. Specific factors are those that are unique to tests and, when combined with random error, are referred to as uniqueness. Although error cannot be extracted as a factor, it can be determined by other means (see Chapter 7, Jensen, 1980).

Gains from practice (and presumably learning in general) cause tests to lose g loading, while specificity increases. The thing that can be trained is specificity, not g (Fleishman and Hempel, 1955).

In the Wechsler battery, about 40% of the total variance is specific to each subtest and about 10% is measurement error. The g factor scores, obtained from the subtests, are correlated more than .95 with the whole test full scale IQ (Jensen, 1998). This points to the great importance of g as the overall worth of the test.

Residuals reveal sex differences that are otherwise obscured by g

Given the previously mentioned lack of predictive validity for residuals, one might conclude that they contribute little to the understanding of intelligence. Indeed, the focus of research has been overwhelmingly on g and its many manifestations in the study of cognitive performance. But Johnson and Bouchard (2007) took a close look at residuals and found sex differences and abilities that behave somewhat as a zero-sum (as one factor increases, the other decreases).

Various sex differences have been reported that relate to means of g: IQ, brain and hypothalamus size, math ability, verbal ability, coding, structure of the corpus callosum, brain mapping, and the impact of white matter integrity. The literature on these items is too large to list. Some of the findings have been inconsistent, others have generated debate, while others are consistent and beyond argument.

The Johnson-Bouchard study was based on data from a battery of 42 mental ability tests administered to the participants of The Minnesota Study of Twins Reared Apart (MISTRA) (Bouchard, et al., 1990). Age, sex, and g-effects were held constant. The VPR (Visual-Spatial-Image Rotation) model (Johnson and Bouchard, 2005) was used with 7 second stratum factors: verbal, scholastic, fluency, content memory, perceptual speed, spatial, and image rotation. The number factor was discarded because of low loadings on it.

The residuals from the above showed negative correlations between image rotation and verbal; and between focus of attention and diffusion of attention. These negative correlations mean that individuals with high scores in one of these abilities tended to have low scores at the other end of the continuum. For example, high image rotation scores were statistically paired with low verbal scores; high focus of attention was typically paired with low diffusion of attention.*

*Note: The focus–diffusion dimension is not a single variable that goes from focus to diffusion. It is a dimension of two abilities (focus and diffusion) which apply to different cognitive items. Residual verbal, scholastic, spatial, and rotation abilities benefit from focus of attention, while residual fluency, content memory, and perceptual speed benefit from diffuse attention to multiple cues.

The trade-off along a dimension of rotation-verbal exists when the variables are held constant, as noted above. This condition applies for both sexes. Males have an advantage in rotation and females have an advantage in verbal, so the trade-off dimension involving these two abilities turns out to be a sex difference. Similarly, males have an advantage in focus, while females have an advantage in diffusion, which leads to a second sex difference, when all of the data are taken together.

As mentioned earlier, the variances associated with residual factors are small, when compared to g. In highly intelligent people, a low ability in one of the previously mentioned residual factors would presumably not be noticed because those people would be able to resolve the associated tasks by using their g resources. For example, a high-g person with a low residual rotational factor would be able to perform rotation tasks by the application of g, thus making the low residual ability inconsequential. But a low intelligence person with a low residual ability factor may not be able to “cover up” the low non-g ability. Thus, the trade-offs between the two pairs of factors can be seen by examining residuals, but may be masked (at least for more intelligent subjects) when g is not factored out.

Johnson and Bouchard did not find invariance between the sexes nor in the two residual dimensions. This suggests that the tasks in question elicit different cognitive approaches to problem solving that may reflect differences in brain structure and function. Future work in brain imaging may clarify these differences.

An unexpected residual strength

Coyle and Pillow (2008) examined the SAT and ACT data from a university group and the 1997 National Longitudinal Study of Youth. Both the SAT and ACT are marketed as achievement tests and both have demonstrated utility in predicting college GPA. Coyle and Pillow wanted to see if these tests would predict college GPA, after g was factored out.

They estimated g from various combinations of the Wonderlic, WAIS, Raven’s Advanced Progressive Matrices and the ASVAB. Structural Equation Modeling (SEM) was used to determine the relationships between g, GPA, and the two achievement tests and to determine the degree of prediction with and without g factored out. They predicted that the SAT and ACT would be highly g loaded and that the predictive validity of the non-g factors (unique variance loading) would be very low.

The SAT and ACT scores were positively correlated with each other at r = .83. The g loadings of the two tests were approximately .91. The subtest scores from the SAT and ACT were highly correlated (about .90 and above) with the composite scores of the tests. Surprisingly, when g was factored out, the residuals were effective in predicting GPA. The SEM path coefficients from SAT or ACT to GPA were about the same as the path coefficients from g to GPA. This suggests that, in these two tests, the non-g factors were about as effective in predicting GPA as was g.

The strongest correlations found in the study were between the Wonderlic and the ACT/SAT. This is consistent with the finding that all three tests are highly g loaded. While the unique variance of the ACT and SAT had moderate predictive validity with respect to GPA, the unique variance of the Wonderlic showed a negative path coefficient with GPA. Coyle and Pillow explained that this may be the result of the strong Wonderlic to g relationship (coefficient = .78) and the moderate relationship of g to GPA (coefficient = .28).

No indication of non-g factors in cortex thickness

In a recent, and somewhat related study, Karama et al. used MRI and a battery of 7 cognitive tests to study 207 children, ages 6 to 18.3. The test battery was designed to produce three factors (cognitive domains): Academic Skills, Verbal Reasoning, and Spatial Reasoning. From these three factors, g was extracted and accounted for 39% of the total variance. When cortical thickness was correlated against g, the result was variable as a function of position, but ranged from 0.15 to 0.34 for statistically significant foci. When g was factored out, the residuals did not show significant correlations with any of the cognitive domains or specific ability tests. This result is parallel to the usual finding that non-g factors display essentially no predictive validity. In this case, there is clearly an association between g and cortical thickness that is not seen in the non-g factors. If nothing else, this at least means that there is a physiological property of g that does not extend to non-g domains.

Summary

Sex differences can be seen on non-g factors favoring males in image rotation and favoring females in verbal.
Likewise sex differences exist in non-g factors favoring males in focus and favoring females in diffusion.
The residual dimensions: Rotation-Verbal and Focus-Diffusion exhibit trade-offs such that higher scores in one factor are statistically paired with lower scores on the other.
The non-g components of Rotation-Verbal and Focus-Diffusion are masked by g in more intelligent individuals.
The predictive validity of IQ tests is almost entirely due to its g loading.
Both the SAT and ACT achievement tests are highly g loaded and, therefore, are reasonable measurements of intelligence.
The SAT and ACT achievement tests are predictive of college level GPA.
SAT and ACT predict GPA about equally well from their g loadings and their non-g loadings.
The non-g factors of the Wonderlic are not predictive of GPA.

Abbreviations

AFOQT           Air Force Officer Qualifying Test

ASVAB 	        Armed Services Vocational Aptitude Battery

STEM 		Science, Technology, Engineering, and Mathematics

VPR  		Verbal-Perceptual-Image Rotation

WAIS 		Wechsler Adult Intelligence Scale

References

Bouchard, T. J., Lykken, D. T., McGue, M., Segal, N. L., & Tellegen, A. (1990). Sources of human psychological differences: The Minnesota Study of Twins Reared Apart. Science, 250, 223-228.

Coyle, T. R. and Pillow, D. R. (2008). SAT and ACT predict college GPA after removing g. Intelligence 36 719–729.

Fleishman, E. A. and Hempel, W. C. Jr. (1955). The relation between abilities and improvement with practice in a visual discrimination task. Journal of Experimental Psychology, 49, 301-312.

Jackson, D. N., and Rushton, J. P. (2006). Males have greater g: Sex differences in general mental ability from 100,000 17- to 18-year-olds on the Scholastic Assessment Test. Intelligence, 34, 479-486.

Jensen, A.R. 1980. Bias in mental testing, Free Press, New York, NY (1980).

Jensen, A.R. 1998. The g factor: The science of mental ability, Praeger, Westport, CT.

Johnson, W., & Bouchard, T. J. (2005). Constructive replication of the Visual–Perceptual–Image Rotation (VPR) Model in Thurstone’s (1941) Battery of 60 Tests of Mental Ability. Intelligence, 33, 417-430.

Johnson, W., & Bouchard, T. J. (2007). Sex differences in mental abilities: g masks the dimensions on which they lie. Intelligence, 35, 197-209.

Karama, S., Colom, R., Johnson, W., Deary, I. J., Haier, R., Waber, D. P., Lepage, C., Ganjavi, H., Jung, R., Evans, A. C., and The Brain Development Cooperative Group (2011). Cortical thickness correlates of specific cognitive performance accounted for by the general factor of intelligence in healthy children aged 6 to 18. NeuroImage (in press).

Olea, M. M. and Ree, J. M. (1994). Predicting pilot and navigator criteria: Not much more than g. Journal of Applied Psychology, 79, 845-851.

Ree, M. J. and Earles, J. A. (1994). The ubiquitous predictiveness of g. In Rumsey, M. G., Walker, C. B. and Harris, J. H. (Eds.), Personnel selection and classification. Hillsdale, NJ: Erlbaum.

Wai, J., Lubinski, D., and Benbow, C. P. (2009). Spatial Ability for STEM Domains: Aligning Over 50 Years of Cumulative Psychological Knowledge Solidifies Its Importance. Journal of Educational Psychology Vol. 101, No. 4, 817–835.

Thinking About Mathematical Thinking at PsycCRITIQUES

Xavier Jouve — Thu, 03 Mar 2011 16:43:58 +0000

An interesting review authored by Gordon Pitz.

http://psyccritiquesblog.apa.org/2010/07/thinking-about-mathematical-thinking.html

The Flynn Effect on the RAPM for High Ability Groups

Michael Chew — Mon, 28 Feb 2011 16:00:59 +0000

The generational increase in raw IQ test scores reported by Flynn have since elicited many studies on the subject of which the phenomenon had been attributed to several causes such as improvements in nutrition, greater access to education, test familiarity and exposure to modern living. Flynn (1987) reported that the score increases over time are attributed to problems in the test measurement of intelligence. Many studies have been carried out on the Flynn Effect on scores around the mean, but there has been a dearth of studies on the higher ranges of IQ. A few of the studies carried out on the FE at the higher end of the scale are that by Herman H. Spitz (1989) on the Wechsler Scales and recently by Colom et al. (2005) using the data on the Pressey’s Graphic Test. Spitz demonstrated in his paper that the score discrepancy between the WAIS and WAIS-R (which are normed 25 years apart) showed increase in scores across the IQ scale, decreasing towards the left and right of the mean, with almost no increase at IQ 125 and above. Colom on the other hand, used the Pressey’s Graphic test to demonstrate a difference in distribution of scores (between1979 to 1999) with the mean shifting right although showing almost zero gain at the 99th percentile. These two studies it seems indicates that massive gains in raw scores are conditional upon the level of IQ.

Studies Carried Out with the Ravens Advanced Progressive Matrices

There are several studies carried out on university samples with the Ravens Advanced Progressive Matrices (RAPM) for various research purposes, of which I believe could offer some insights, and if indeed the Flynn Effect has afflicted the RAPM, the raw score gains would be quite obvious over a span of more than 35 years. The RAPM is typically administered to the top 20% of the general population and usually within university settings, making the test ideal, if one wants to observe any gains in test scores for a high ability group. The table below lists the test results using the 40 minute timed version of the RAPM (Set II-36 item) carried out over a span of more than 35 years, on various university populations. The study carried out by Paul in 1985 however is based on the untimed version but included in table 1 for comparison.

The age data of the samples listed in the table above is incomplete. However it is reasonable to assume that since the samples are primarily from university populations and many of which were carried out on 1^st year students, the possibility for depression of scores due to age effects can mostly be ruled out. A cursory look at the table reveals almost no significant increase in raw scores over a period of almost 4 decades. If we plot the weighted mean of the raw scores for each study (of the male & female scores) and plot it against the year each study was carried out, we get a scatter plot as shown on the graph below;

Note : S.M. Paul’s data is excluded as the study was carried out based on the untimed version of the RAPM

The raw scores over 37 years indicates that the RAPM is internally consistent and stable over time and somewhat evident that the trend-line appears quite flat with no evidence of any significant increase in raw scores since Yates and Forbes’ (1967) study. In fact, Stough & Nettlebeck(1993) arrived at the same conclusion by comparing the mean raw score of 24.4 based on their study at the University of Adelaide, compared to the mean raw score of 23.17 in Yates and Forbes’ study at the University Western Australia in 1965.

Raw Scores for A University Sample Compared to the 1992/1993 Standardization Normative Data

In 2002/03 Francis Van Dam & Raven (2008) carried out a study on a University of Louvain sample using the RAPM. The study was carried out on the same cohort about 30 years apart, the aim being to investigate if increases in age had the effect of lowering test scores. However in the process, some curious findings were discovered when the test scores were compared against the 1992/93 UK/US standardized normative data. Due to the fact that the Louvain study used the 48-item version of the RAPM, we need to convert the raw scores from the 48-item test to make comparisons with scores on the 36-item RAPM. Graph 4 in the editions of the (British) Guide to the Use of the Advanced Progressive Matrices (Raven, J. C., 1965) identifies the items eliminated when Set II was reduced from 48 to 36 items in 1962. Items 1-8 and 17 were eliminated because everyone got them right. Item 11 was a bad item. And items 44 and 46 were too difficult. This means that to convert the 36-item test to the 48-item test, one can add 8 to scores on the new test that lie between 1 and 2, 9 to scores between 3 and 7, 10 to scores up to 33 and 12 above 33. Conversely if one wants to convert the 48-item scores to the present 36-item test, we could take off the scores accordingly. The normative data from the UK standardization in 1992 (Raven J., 2000) at the 90^th percentile, is a raw score of 31 for 20 year-olds and 29 for 50 year-olds. The scores in 1962 are re-tabulated in table 2 below;

Table 2 – RAPM Raw Score (untimed) 1992 Standardization (Dumfries)

Age Group	20			50
Year	1962	1970/71 est.	1992	1962 est.	1970/71 est.	1992	2002/03 est.
90^th percentile	21	24	31	17	21	29	33

Notes: i) The normative data are from the UK Standardization in 1992, see tables 10 & 12 (Raven J., 2000). ii) The raw score for 40 year olds is 17 in 1962, and for simple comparison, is taken as 17 for 50 year olds. iii) The raw score for 50 year olds in 2002 is extrapolated based on the score increase between 1962 and 1992. iv)The 1970/71 score is interpolated between the 1962 and 1992 normative scores.

If we look at the scores of the Louvain sample in 1970/71 and the re-test scores 30 years later in 2002/2003, the raw score for the same cohort of 99 respondents fell from 35.9 to 33.4. If we convert these to the 36-item test equivalent we arrive at 25.9 (35.9-10) and 23.4 (33.4-10), that is a score of 25.9 for 20 year-olds in 1970 and a score of 23.4, 30 years later as 50 year-olds, keeping in mind that these scores are from the same cohort, 30 years apart. Now if you look at table 2 again, the normative data from the 1992 standardization indicates an increase (24 to 33), if you were to compare the raw score for 20-year olds in 1970/71 vs that for the 50 year-olds in 2002/03 for the untimed version of test. However the Louvain data indicates a decline in scores between the 2 age groups which are 30 years apart! Since the Louvain sample used the 40-minute timed version of the test and the standardized data uses the untimed version, direct score comparisons cannot be made. However, significantly, one sample shows a decline, whereas the standardized data shows a significant increase for essentially the same test. Van Dam & Raven (2008) makes a similar comparison, using the untimed normative data extrapolations with the mean Louvain score, although a direct comparison of raw scores between timed and untimed versions of the test are not really comparable.

If we go back to Table 1 and select the scores closest to the period between1962 and 1992 to make comparisons with the 1992 standardized data. The closest comparison will be the study carried out at the University of Western Australia by Yates & Forbes and that carried out 27 years later at the University of Adelaide. The weighted average (between male and female scores) mean raw score by Yates and Forbes (1967) in 1965 is at 23.5 of which Stough & Nettlebeck (1993) estimates this at IQ 125 or the 95^th percentile. The raw score in 1992 for the University of Adelaide is at 24.4. Now let’s look at the data from the 1992 Standardization again (see table 3).

Table 3 – RAPM Raw Score (untimed) 1992 Standardization (Dumfries)

Age Group	20
Year	1962	1965 est.	1992
95^th percentile	24	25	33

Notes: i) The normative data are from the UK Standardization in 1992 see tables 10 & 12 (Raven J., 2000). ii) The 1965 score is interpolated between the 1962 and 1992 normative data.

The normative data from the 1992 standardization is for the untimed version of the test, and the raw score increase between 1965 and 1992 is quite large (8 raw points). However if we look at the score difference between the Australian university samples for the 40-minute timed version of the test, the raw score increase is only about 1 raw point. It is reasonable to assume that the standard deviations for the timed and untimed version of the test at the 95^th percentile cannot be that far apart since we are basically looking at very similar cohorts in terms of ability, then the difference in raw scores for both the time and untimed version of the test, 27 years apart should not be too different in magnitude. However that is not what we get.

In sum, it seems one can conclude there appears to be no Flynn effect on the raw IQ scores of the timed version of the RAPM for university population samples from 1965 through to 1992 or even up to 2004. Paradoxically however, the 1992 standardized scores of the untimed version of the RAPM do not exhibit the same temporal stability over pretty much the same period, even at the 95^th percentile.

References:

Abad F. J., Colom R., Rebollo I., Escorial S. (2004) -Sex differential item functioning in the Raven’s Advanced Progressive Matrices: evidence for bias- Pergamon Personality and Individual Differences 36 (2004) 1459–1470.
Colom et al (2005) – The generational intelligence gains are caused by decreasing variance in the lower half of the distribution: Supporting evidence for the nutrition hypothesis – Intelligence 33 (2005) 83–91.
Day & Arthur (2004) – Ability based pairing strategies in the team-based training of a complex skill.
Douglas A. Bors and Tonya L. Stokes (1998) – Raven’s Advanced Progressive Matrices: Norms for First-Year University Students and the Development of a Short Form -Educational and Psychological Measurement 1998; 58; 382.
Flynn, J. R. (1987). Massive IQ gains in 14 nations: What IQ tests really measure. Psychological Bulletin, 101, 171-191
Irwing, P. & Lynn, R. (2005) -Sex differences in means and variability on the progressive matrices in university students: A meta-analysis -British Journal of Psychology (2005), 96, 505–524.
Jensen A.R., Dennis P. Saccuzzo and Gerald E. Larson (1988) -Equating the Standard and Advanced Forms of the Raven Progressive Matrices -Educational and Psychological Measurement 1988; 48; 1091.
Raven J. (2000)– The Raven’s Progressive Matrices: Change and Stability over Culture and Time - Cognitive Psychology 41, 1–48 (2000).
Stough, C. & Nettlebeck, T. (1993) – Raven’s advanced Progressive Matrices and increases in intelligence – Personality and Individual Differences Volume 15, No.1 pp 103-104 1993.
Spitz Herman H (1989) –Variations in Wechsler Interscale IQ Disparities at Different Levels of IQ - Intelligence 13, 157-167 (1989).
Teasdale & Owen (2005) A long-term rise and recent decline in intelligence test performance: The Flynn Effect in reverse.
Van Dam, F. & Raven J. (2008) – Uses and Abuses of Intelligence Studies Advancing Spearman and Raven’s Quest for Non-Arbitrary Metrics – Chapter 9 Does the “Flynn Effect” Invalidate the Interpretation Placed on Most of the Data Previously Believed to Show a Decline in Intellectual Abilities with Age?
Yates, A. J. & Forbes, A. R. (1967). Raven’s Advanced Progressive Matrices (1962): Provisional Manual for Australia and New Zealand. Hawthorn, Victoria: Australian Council for Educational Research.