Predictive Validity of Non-g Residuals of Tests: More Than g

Abstract

This comment argues that an important issue in intelligence research is to identify constructs with validity beyond g, and that non-g residuals of tests represent a promising target.

1. Introduction

I thank Paul De Boeck for the opportunity to contribute a comment to the special issue. My comment provides a short and long answer to the questions posed by Paul [1].

Short answer:

• What are the most important scientific issues in the domain of human intelligence? Identifying constructs with validity beyond g.
• What are the most promising new ideas and approaches in the study of human intelligence? Investigating the predictive validity of non-g residuals of tests.

Long answer: A construct central to intelligence is g, reflected by positive correlations among mental tests. g has been found to contribute strongly to a test’s predictive validity at school and work [2]. In contrast, non-g factors of tests, obtained after removing g, have been found to contribute trivially to predictive validity. These findings suggest that the predictive validity of tests is attributable to “not much more than g” [3]. In contrast to this claim, non-g factors of some tests, based on non-g residuals, have been shown to predict outcomes beyond g [4,5]. As noted below, these non-g factors can be obtained with widely used tests (SAT and ACT), and are relevant to theories of intelligence.

1.1. Non-g Residuals

Non-g factors can be inferred from correlations with non-g residuals, obtained after removing g from tests. Non-g residuals are assumed to reflect specific abilities (unrelated to g), such as verbal and math abilities. Using structural equation modeling, non-g residuals were obtained for the SAT (formerly, Scholastic Aptitude Test) and ACT (formerly, American College Testing), two college admissions tests [4,5]. The non-g residuals of these tests predicted college grade point average (GPA) almost as well as g, despite being non-g factors (Figure 1). In addition, the non-g residuals of the math and verbal subtests correlated positively with similar abilities (math residuals and math ability) but negatively with competing abilities (math residuals and verbal ability) (Figure 2). The effects of the residuals were generally moderate in size (mean beta ≈ 0.30).

Figure 1. Relation of SAT non-g residuals with college GPA. g was based on the Armed Services Vocational Aptitude Battery (ASVAB), a diverse battery of 12 tests. The SAT residuals (e13) predicted GPA (beta = .29) almost as well as g (beta = .32), indicating that non-g factors influenced the predictive validity of the SAT. The ACT residuals produced similar effects. Figure adapted from Coyle and Pillow [4].

Figure 1. Relation of SAT non-g residuals with college GPA. g was based on the Armed Services Vocational Aptitude Battery (ASVAB), a diverse battery of 12 tests. The SAT residuals (e13) predicted GPA (beta = .29) almost as well as g (beta = .32), indicating that non-g factors influenced the predictive validity of the SAT. The ACT residuals produced similar effects. Figure adapted from Coyle and Pillow [4].

Figure 2. Relation of SAT math non-g residuals with verbal ability. g was based on the ASVAB, which also estimated four specific abilities (verbal, math, speed, and shop). The SAT/ACT factor was based on the math (SATm and ACTm) and verbal (SATv and ACTv) subtests. The SAT math residuals (e16) negatively predicted verbal ability (beta = -.34; shown in figure) and positively predicted math ability (beta = .30; not shown). In contrast, the math residuals were weakly (and not reliably) related to speed and shop abilities (betas ≈ .01). The ACT residuals produced similar effects. Figure adapted from Coyle et al. [5].

Figure 2. Relation of SAT math non-g residuals with verbal ability. g was based on the ASVAB, which also estimated four specific abilities (verbal, math, speed, and shop). The SAT/ACT factor was based on the math (SATm and ACTm) and verbal (SATv and ACTv) subtests. The SAT math residuals (e16) negatively predicted verbal ability (beta = -.34; shown in figure) and positively predicted math ability (beta = .30; not shown). In contrast, the math residuals were weakly (and not reliably) related to speed and shop abilities (betas ≈ .01). The ACT residuals produced similar effects. Figure adapted from Coyle et al. [5].

1.2. Theories

The pattern of effects for the non-g residuals supports investment theories [6] and niche-picking theories [7], both of which focus on the development of specific abilities (unrelated to g). Investment theories assume that differential investment of time and effort strengthens some abilities over others. This investment produces positive relations among residuals and similar abilities, and negative relations among residuals and competing abilities. Niche-picking theories assume that differential selection of opportunities (compatible with preferences) also strengthens some abilities over others, yielding the same pattern of relations. Both theories are two sides of the same coin. Niche-picking preferences are assumed to influence differential investment of time and effort, and differential investment is assumed to influence preferences for some abilities over others.

1.3. Future Research

Following suggestions by Hunt and Jaeggi [8], future research could examine the predictive validity of non-g residuals at work and school. If non-g residuals reflect differential investment, then the residuals of math and verbal tests may differentially predict outcomes. Math residuals may predict math-loaded outcomes such as college degrees and jobs in science, technology, engineering, and math (STEM). Verbal residuals may predict verbally-loaded outcomes such as degrees and jobs in the humanities (e.g., English and fine arts). Following Hunt and Jaeggi’s [8] suggestions, these predictions could be tested with Big Data surveys (e.g., National Longitudinal Survey of Youth), which contain work and school outcomes.

2. Conclusion

The predictive validity of tests has been attributed to “not much more than g” [3; see also, 2]. Contrary to this claim, the non-g residuals of the SAT and ACT predict college GPA almost as well as g. In addition, the non-g residuals of both tests predict verbal and math abilities in a pattern consistent with investment theories. The pattern suggests that the residuals of the SAT and ACT reflect specific abilities (unrelated to g) and are not random error (in a psychometric sense). Future research should examine whether the effects of the residuals generalize to tests other than the SAT and ACT, and whether the effects increase over time with differential investment in specific abilities.