Are personality tests accurate? Reliability, retest results and the Barnum effect

What “accurate” means for a personality test

Psychologists split accuracy into two questions:

• Reliability: does the test measure consistently? Do items meant to tap the same trait agree with each other (internal consistency), and do you get a similar score if you retake it (test–retest reliability)?
• Validity: does it measure the right thing? Do scores agree with other evidence about you, such as ratings by people who know you, and do they predict behaviour and life outcomes?

A test can be reliable without being valid (a bathroom scale that is always two kilos off), but it cannot be valid without being reliable.

Reliability: are the scores consistent?

Internal consistency is usually reported as Cronbach’s alpha (α), from 0 to 1. Values around .80 or above are generally considered good for research scales. The public-domain IPIP scales that 64 Types uses have α between .79 and .89 in our norm samples of about 20,000 people each; full figures are on our science page. Published MBTI studies also report generally strong internal consistency for its four scales, though values vary between studies.^[8]

Test–retest stability over years is high but far from perfect. Across 152 longitudinal studies, the correlation between personality scores measured about 6.7 years apart rose from .31 in childhood to around .74 between ages 50 and 70.^[9] Some of that instability is genuine change: average trait levels shift over adulthood.^[10] See Can your personality type change?

Why type results seem unreliable

If scales are reliable, why do so many people get a different type on retest? Because of what happens near the cut-off. Personality scores are center-weighted, with most people near the middle, and not split into two humps.^[4][11] Anyone close to the midpoint can flip a letter from a small, normal fluctuation. With four letters, the chance that at least one flips adds up. Reviews of the MBTI have flagged this classification instability as a key reason for caution.^[5]

Here is a simple illustration. By definition, 20% of people fall between the 40th and 60th percentile on any trait. If the six traits in a 64-type code were completely independent, the share of people with no trait in that zone would be 0.8⁶, about 26%, so roughly three in four would have at least one borderline letter. (The traits are somewhat correlated, so the real figure differs a little, but the point stands.)

Validity: do other people see you the same way?

One strong check is whether people who know you rate you similarly. A set of meta-analyses covering 44,178 people rated across 263 samples examined how well observers’ ratings agree with each other and with self-ratings. Accuracy improved as the rater became closer to the person, especially for less visible traits such as emotional stability. Ratings by others also predicted academic achievement and job performance substantially better than self-ratings did, and added to them.^[1]

Self and others each have blind spots. In one study comparing self-, friend- and stranger-ratings with behavioural measures, the self was the best judge of neuroticism-related traits, friends were best for intellect-related traits, and everyone was about equally good at judging extraversion.^[12]

Validity: do scores predict real life?

Big Five traits predict mortality, divorce and occupational attainment roughly as well as socioeconomic status and cognitive ability do.^[2] Conscientiousness predicts job performance across occupations,^[13] neuroticism predicts anxiety and depressive disorders,^[14] and personality predicts many other outcomes.^[15]

Those findings hold up. A preregistered project re-tested 78 published trait–outcome links with large samples and found that 87% replicated, although the effects were typically about 77% as strong as originally reported.^[3] So the right summary is “real, consistent, modest”: the trait–outcome correlations in major meta-analyses mostly fall between about .10 and .30, for example .11 between conscientiousness and longevity and .31 between extraversion and leadership.^[16][17] That is useful for understanding tendencies across many people and much weaker for predicting what any one person will do.

The Barnum effect: why every result feels accurate

In 1949, psychologist Bertram Forer gave his students a personality test and, a week later, what each believed was a personal profile. In fact everyone received the same text, assembled largely from a newsstand astrology book, with lines such as “you have a great need for other people to like and admire you”. On average the students rated it as a highly accurate description of themselves.^[6]

A later review of decades of follow-up studies found that people are more likely to accept such feedback when it is favourable, when it is general enough to fit almost anyone, and when they believe it was derived specifically for them.^[7]

Type descriptions often tick all three boxes. That does not make them worthless, but it means that “this sounds exactly like me” cannot distinguish a valid test from a horoscope. The checks above, consistency, agreement with others and prediction, can.

A checklist for judging any personality test

1. Does it say where its questions come from? Research tests use published item sets, such as the public-domain IPIP.^[18]
2. Does it compare you with real people? Percentiles based on a stated norm sample mean more than “high” or “low” on an arbitrary scale.
3. Does it report reliability? Look for figures such as Cronbach’s α.
4. Does it show scores, not just a label? Continuous scores reflect how traits are actually distributed.^[11]
5. Does it say when you are near the middle? Borderline results are the main source of type flips.^[4]
6. Does it cite evidence for its claims about careers or relationships? And does it report effect sizes honestly?
7. Does it state its limits? A self-report test is not a clinical diagnosis or a hiring tool.

How 64 Types measures up

We built 64 Types around this checklist: 62 public-domain IPIP items, percentiles against roughly 20,000 real responses per scale, published reliabilities, borderline flags with alternative codes, and cited claims throughout. Its limitations are stated plainly too: the norm samples are online volunteers who skew young and English-speaking. Details in How the 64 Types test works and on our science page.

Frequently asked questions