What is g?

"Too long, likely won't read" summary: this is an explanation about how the concept of general intelligence, or IQ, is justified statistically, and some general criticisms about it. Tried to keep it simple, but that means keeping it long.

Took a class in multivariate statistics this spring. Among other things we learned about Principal Components Analysis and Factor Analysis, a family of procedures invented by the psychometrician Charles Spearman at the turn of the last century for the express purpose of studying intelligence. In fact the entire concept of "general intelligence", or g, is essentially founded in factor analysis.

Today, PCA and Factor Analysis have all kinds of applications that reach beyond justifications for IQ tests and conservative policies. But learning about it even in a general sense demystified what IQ is to me to some extent, and I wanted to share.

A few disclaimers- I am by no means an expert in factor analysis, and even what I know will be simplified for the sake of getting the general point across without getting mired in technical details. If you want a more involved explanation, check out this blog post by an academic at University of Michigan, and if you want to get down to the linear equations and matrix algrebra, get my teachers' text, which is apparently as loathed by past students reviewing it on amazon as it is thorough.

I was talking about with a friend today and realized you need to go back to ground 0 to really explain this to everybody, so I'll start with the basic-basics, correlation. You did it in 9th grade. Y is a vertical line, x is a horizonal one. On the horizontal axis, you might have an independent predictor variable, like the $ amount of money you sink into a theme park. On the vertical Y axis you have your dependent variable, say number of visitors to the theme park. If money spent on theme parks increases visitors, you should see a linear relationship, and a diagonal line at a 45 degree angle on the graph: for every buck you spend, one more visitor comes, etc. This would be a correlation of 1. No relationship whatsoever gives you a correlation of 0. A perfect negative relationship (each dollar spent repels an existing visitor from the park) is negative 1.

In real life, of course, the relationships are rarely this clear-cut. If you have a perfect relationship of 1, its almost certainly just because you screwed up your data and just have two different looking measures of the thing (toe size in shoes increases with shoe size, for example). In real life, even .8 is a great correlation. Even .4 can be important, though on the graph it just looks like a smear of data points angled in a general upward direction to the naked eye.

It was observed by Spearman (you can read the original paper here) that all school tests tend to correlate, geology with math, art with history, German with geology. (I see this myself: at work we look for tests that correlate with the TOEIC Bridge, a test of ability in English language. As it is, everything correlates with it, including scores on math tests).

Spearman developed a technique for studying these intercorrlelations between multitudes of tests, which begat Principal Components Analysis and later Factor Analysis.

A brief segway to explain the process: To put it as simply as I can, these procedures group things that correlate with one another together. As an example, suppose you gave people a survey about things they liked to do. Half the activities mentioned on it were things like going out to meet friends, meeting new people, etc, and half were stuff like staying home and reading a book.

You might find that the activities that involved going out and meeting new people tended to correlate with one another, and the activities that involved being alone tended to correlate with one another. Looking at how people answer these questions about activities, you might see 2 "factors" emerge: an extroversion factor, made up of the going out questions, which correlate with one another highly, and an introversion factor, of activities that involve being alone (come to think of it, you might just find that these two personalities made up polar opposites of just one big factor...but you get my drift).

Anyway: Factor Analysis is quite good at isolating groups of questions/tests etc that are correlated with one another (Group A, math problems), and therefore share some kind of relation, with groups of unrelated questions that also correlate well with one another (Group B: questions designed to measure love of food) but do not correlate with the questions in group A (because you can love food and hate math, love math and food, love math and be skinny and not care about food, etc etc. No correlation).

However, as I said, all those school tests do tend to correlate, even, say, English and science, which you would think would be pretty different.

People I talk to object to this idea. Surely, there are great writers that just don't get science and vice versa, right? Well, yes and no. And in the long run, more or less just no. We tend to look at extremes and exceptions, but as a general rule, a kid who's a great writer will tend to be not too shabby in math class either, even if he won't be the next Einstein. And the math nerd will probably be an at least competent writer, even if he never becomes Shakespeare. And conversely, to hold down the other end of the correlation, there are plenty of people that won't be particularly great at either of those subjects, too.

Spearman found one big factor that accounted for most of the variance, which came to be known as g, or general intelligence. Spearman posited that much in the same way all those intercorrelated questions about going out and meeting people represent an "extroversion" factor, the big factor of intercorrelations between various mental tests represented "general intelligence", in other words, how smart you supposedly are. The variance unique to each test that couldn't be explained by the first big general factor was presumed to be unique to each test, but what could be explained by that first factor was "general intelligence": a general tendency to succeed on paper and pencil and paper tests.

Of course, since then there have been all kinds of alterations, challenges, and more alterations to the theory, and Spearman's original methodology has been tweaked and improved upon in countless ways, with literally thousands of pages written on the subject. And yet, the theory, in essence, remains the same: one general factor accounts for success on more or less any kind of mental test. You might expect that there would be at least, say, two, one for verbal intelligence and one for mathematical intelligence. But no. Quibbling about first and second order factors aside, one essentially describes them all quite well. Tweaking that theory might improve your model fit in Confirmatory Factor Analysis (which I won't get in to here) a bit, but bottom line, one does it.

And so, IQ tests are designed to tap that general factor, that general correlation between any other test you might take. It works as a fairly good predictor of how you'll do on a test of economics, or a flight school exam, or a rock and roll trivia quiz, precisely because its designed to tap that statistical overlap between them.

So the apparently statistically unassailable correlation between IQ tests and other paper and pencil tests did a lot to justify the serious study of IQ. If there are "multiple intelligences", why can't anyone ever find them in factor analysis? At the heart of it, this is the trump card IQ supremacists like to play when they say science is on their side.

And it sounds pretty convincing. But here's the thing: I've spent this spring doing factor analyses and PCA on all kinds of data. Prices of companies on the stock market. Countries' performances on Olympic events. Various brands of cereals.

And nearly all of them can be explained to a fair degree by a large first factor.

Oh yes, I know there are all kinds of qualifications to be made here. Yes, large first factors are more common in PCA, which is now frowned upon for analysis of latent variables. Yes, maximum likelihood extraction will account for less variance (meaning smaller factors, first or otherwise) and explain more of the correlation matrix. Yes, rotations can make large first factors vanish, or at least diminish.

And yet, to only somewhat greater and lesser degrees, those big first factors loom over almost any data you subject to these procedures. In the case of stock prices, it's a "general index factor", meaning the overall health of the stock market (secondary factors are unique to specific industries such as finance, energy, etc). In the case of Olympic decathlons, its "general athletic ability".

The point is, we tend to mystify study of the mind. If we're trying to see something in our heads that can't be seen by the naked eye, say, the inspiration that led the paintings on the Sistine Chapel, we can tend to make out that first factor to be something of striking importance. And in all fairness, it may indeed be a useful method for summarizing our data. But even if it does help guide our understanding of intelligence, let's be clear about what it is, statistically speaking: Its an attempt to organize and explain the data derived from a battery of tests. And when you apply the same logic and reasoning to other things, it suddenly becomes a lot clearer, and a suddenly seems a lot less exciting and insightful.

As an example:

Suppose a small nation is about to enter the olympics for the first time. This hypothetical small nation has 3000 athletes that have only played one sport before now, but they need to pick the top athletes and assign them to different event(say 24).

So this is what they do: they each compete in the 24 sports. Times and scores are recorded, and a factor analysis is performed on the data.

A first factor that explains 45% of the data is derived. This first factor is labelled g.a, general athletic ability. It is the commonality that explains success across the board with all those sports. It correlates well with all those sports because it is, in fact, the creation of those correlations. Each athlete receives a factor score, which shows how highly they load on it.

And you know what? It works quite well, statistically speaking. Want to predict who will do well at javelin throwing? Well, lets look at who loaded highest on that general factor. Hey, he's good!

Want to see who will do well at the 300m swim? Check the factor scores. Not bad!
Since this data seems statistically unassailable, it is decided: athletes with high loading on that factor will get to go to the olympics, participating in randomly selected sports. Because that general factor makes the specific sport irrelevant, you see.

Just one problem: while that factor score may well be a good indicator, statistically, if you want to get someone who's really good at javelin throwing...just look at the javelin throw test you did, and see who threw furthest.

If you really want to see who will do best at the 300m swim...just check the race you held, and look who came in first.

Depressingly often, people that did best in those events may not have even have had extremely high scores for that factor at all.

The same is true of mental tests. If you really, really want to mash together an economics test with a flight test with a rock and roll trivia quiz, you can do it, statistically. But if you want to see who does best at each, that general factor is a poor replacement for the original info. By the same token- the traits that make a good stockbroker really do differ from the traits that make a good scientist, regardless of that overlap.