Methodology

Methodology & Sources

What IQ Mosaic is (and isn't)

IQ Mosaic is an original cognitive-challenge app for iPhone, built for entertainment and education. It presents 50 mixed-domain reasoning questions across five domains — abstract, verbal, logic, quantitative, and spatial — plus two short optional subtests (a working-memory digit-span task and a 60-second processing-speed task). From your performance it produces a self-referenced Cognitive Profile: a picture of how you did across the app's own sections, with an estimated score shown alongside an honest margin of error.

We want to be upfront about the limits, because they matter:

It is not accredited. No body has reviewed or certified it.
It is not clinically validated and must not be used for diagnosis, screening, or any clinical purpose.
It is not population-normed. Today's score is self-referenced and provisional — it compares you against the app's own scale, not against a representative sample of the general population.
It is not the Raven's Progressive Matrices, the WAIS/Wechsler scales, or any other proprietary test. Those instruments are copyrighted and are not administered, reproduced, or implied to be endorsed here. Every item in IQ Mosaic is original, written in the tradition of — and modeled on — well-studied public paradigms.

What follows is an honest account of how the questions are actually built and quality-controlled, and of the established cognitive science each question type draws on. Where we cite a foundational test (e.g., Raven's, Wechsler's), we cite it to credit the paradigm our item formats follow — never to claim we are that instrument.

How we build the questions

Every question in the app comes from one master pool (currently 260 deduplicated items across 95 item families), assembled into the bundled bank the app ships. Two generation tracks feed that pool, and both are gated by the same correctness philosophy: an item only ships if its answer can be defended and a blind solver who never sees the key still picks it.

The five domains and their families

The pool spans all five domains (live counts: quant 47, logic 48, abstract 61, spatial 56, verbal 48), at three author-labeled difficulty tiers (easy 91 / medium 107 / hard 62). Representative families:

Abstract — text: letter-sequence, number-step, odd-one-out (numeric), letter-analogy, interleaved series, accelerating-letter series, symbol substitution, compound number series, dual-transformation analogy. Visual (rendered figures): matrix-rotation, matrix-count, figure-series, figure-analogy, odd-one-out (figure).
Logic — transitive comparison, immediate class inference, categorical syllogism, multi-clue ordering, relational kinship, odd-one-out by rule, conditional/contrapositive, set/Venn inclusion-exclusion, knights-and-knaves, plus invented forms (river-crossing, day-of-week deduction, seating-grid, balance-weighing, self-referential count).
Quantitative — percent-of, fraction-share, mean-of-set, unit-rate, solve-for-x, speed/distance/time, reverse-percent, ratio division, factor counting, series sums, weighted average, combined work-rate, plus invented forms (clock-angle, mixture/dilution, dice probability, modular/remainder).
Spatial — text: turn-sequence bearing, two-step coordinate transform (rotate + reflect), cube-net opposite-face, glued-cube visible-faces. Visual: arrow-rotation, mirror-arrow, clock-rotation, mental rotation (chiral figure), hole-punch (fold-and-punch).
Verbal — word-ladder, anagram, hidden-word, odd-meaning-out, semantic gradients, compound-word logic, prefix/suffix sense, idiom meaning. (Verbal is text-only.)

Two short subtests sit outside the 50 and are reported as separate raw indices, never folded into the main score: Working Memory (backward digit span) and Processing Speed (60-second symbol search).

Track 1 — Deterministic generation (answers correct by construction)

A fixed-seed deterministic generator builds the deterministic items. The defining property: the generator computes every answer rather than authoring it. Percentages, means, ratios, alphabet-position sequences, coordinate transforms, and cube-face counts are all derived in code, so the key is correct by construction. Each family is a function returning a question; a declared PLAN sets how many of each family and difficulty to make, and the driver retries, discarding any item whose options collide.

This track also produces all the visual items, because LLMs cannot reliably emit valid figure-spec JSON. Visual items are emitted as a compact FigureSpec (shape kind, rotation, fill, size, placement, mirror, dashed) matching the app's Swift Codable schema and rendered by a dependency-free SwiftUI Canvas.

Track 2 — AI item-invention loop (text only)

An automated process runs an "inventor" agent per domain that proposes new text-item families. An explicit avoid-list steers it away from families already covered, so each round reaches for fresher forms; it loops until a target batch survives verification. This track is text-only by design.

Correctness QA: blind double-solve

The central correctness gate is independent blind re-solving, applied to both tracks:

Invented items: every candidate is re-solved by two blind solver agents who never see the answer key — one told to "answer precisely and literally," the other cast as "a skeptical editor hunting ambiguity." An item survives only if both pick the labeled answer and neither flags it ambiguous, and only if it is not a duplicate of anything already in the pool.
Deterministic text items: an automated verification pass blind-re-solves every item with the key held out and computed separately. A first literal pass collects any disagreement-with-key or ambiguity flag; a second independent solver re-answers only the flagged items. An item is called "likely-wrong" only if both solvers converge on the same non-key answer. Result on the deterministic text bank: 0 likely-wrong.
Deterministic computed correctness: because these answers are computed, the blind re-solve functions as a cross-check against bad distractors and ambiguity rather than as the primary key.

Guards against the ways items go wrong

Ambiguity guards. An automated check covers a broad set of rival predicates (even / prime / square / consecutive-product / triangular / power-of-two / multiples 2–12) across every odd-one-out set and rejects any set where a second rule could pick a different element — this caught and fixed a real "12/30/56/24" trap. The text builder returns nothing (forcing regeneration) rather than padding with weak distractors when it can't produce four genuinely distinct options. Mental-rotation items use a genuinely chiral figure with distinct corner markers, so a correct rotation can never coincide with a mirror-image distractor.
Render-collision check (visual). A render-collision check mirrors the Swift renderer's clamps (e.g., a half-filled triangle clamps to solid; striping at the smallest size clamps to solid; rotational symmetry of circles/squares) so two figure tiles that look identical when drawn are treated as duplicates and rejected — even if their JSON differs. This caught a real bug where a half-filled-triangle distractor rendered identically to the correct solid triangle.
Answer-position de-biasing. Assembly re-inserts each correct option so that A/B/C/D are used as evenly as possible across the bank (verified flat), removing position as a cue.
Stable item identity. Each item gets a content-hash identifier (derived from the prompt, figure, options, and correct answer) that survives bank rebuilds and the position-rebalance, so any future calibration stays attached to the same item. An IRT calibration sidecar is wired but currently empty (no IRT has run).
Assembly validation. Assembly validates 0 ≤ answerIndex < 4, rejects duplicate choices or figures, and refuses to install if any errors exist.

Variety and difficulty controls

Surface variety. Large, diverse pools of names (~120, drawn least-used-first so no name dominates), comparison relations, ordering scenes, syllogism triples, class pairs, kinship templates, and varied prompt phrasings keep items from recycling the same surface content.
Difficulty tiers. Every item carries an author difficulty label; the PLAN sets per-family difficulty mixes, and difficulty is also a weight in scoring and a target for the empirical difficulty cross-check.
Global dedup. Deduplication runs across all families and at merge time — text items keyed on prompt, visual items on figure content plus correct answer — so genuinely different items survive but true duplicates drop.
In-test family diversity. When the app assembles each 50-item form, the sampler tracks used families and prefers a fresh family within each difficulty bucket. Verified by autoplay runs: 47–48 distinct families per 50-item form, with no family appearing three or more times. The larger pool also means retakes draw fresh items with no repeats across sittings.

What each question type measures — and what it's modeled on

For every type below: the items are original, written in the tradition of the named paradigm. They are not that paradigm's actual test, and naming a paradigm implies no endorsement by its authors or publishers.

Abstract / matrix reasoning

Our item: a 2×2 or 3×3 grid of abstract figures with the bottom-right cell missing; you infer the rule(s) governing change across rows and columns and pick the figure that completes the matrix. Difficulty rises as more simultaneous rules must be tracked. All original artwork.

What it taps: fluid reasoning (Gf) — perceiving abstract relations, inferring governing rules, and applying them to novel visual problems with little reliance on verbal or cultural knowledge. Matrix reasoning is among the strongest single indicators of the general factor g.

Modeled on: the matrix / pattern-completion tradition of Raven's Progressive Matrices (Penrose & Raven, 1936; Raven, 1938), with item design following the rule-based cognitive account of Carpenter, Just & Shell (1990) and the construct-driven generative approach of Embretson (1998).

Verbal reasoning

Our item: four-term verbal analogies ("A is to B as C is to ?"), synonym/antonym choices, and semantic-relation items (category, part-whole, function, degree), with distractors built from near-synonym traps and surface-feature lures.

What it taps: crystallized / verbal-comprehension ability (Gc) — the breadth of acquired language knowledge and the capacity to perceive and re-apply the relation linking a word pair.

Modeled on: the componential analogy paradigm of Sternberg (1977) — encoding, inference, mapping, application — and the structure-mapping account of analogy of Gentner (1983), organized within the Cattell-Horn-Carroll framework (Cattell, 1963; Carroll, 1993).

Logic / deductive reasoning

Our item: short deductive problems — categorical syllogisms, conditional (if-then) arguments testing valid forms (modus ponens, modus tollens) against classic fallacies (affirming the consequent, denying the antecedent), and "which conclusion necessarily follows" items. Distractors are built from common reasoning errors (illicit conversion, plausible-but-invalid conclusions).

What it taps: deductive / relational reasoning — drawing conclusions that necessarily follow from premises and distinguishing valid inference from seductive fallacy; a core component of fluid reasoning.

Modeled on: the mental-models theory of syllogistic reasoning of Johnson-Laird (1983), the conditional-reasoning tradition of Wason (1968), and the deduction-paradigm review of Evans (2002), which informs our realistic distractor design.

Quantitative reasoning

Our item: text-based number problems — number-series completion, proportional/ratio reasoning, arithmetic word problems, and quantitative comparisons, each with one defensibly correct answer.

What it taps: numerical/quantitative reasoning — detecting and applying numeric rules and proportional relationships; novel number series tap inductive/fluid reasoning, while computation taps learned number facility.

Modeled on: the number-facility and induction factors identified by Thurstone (1938) and catalogued in the ETS Kit of Factor-Referenced Cognitive Tests (Ekstrom, French, Harman & Dermen, 1976), within the fluid-reasoning framework of Cattell (1963) and Carroll (1993).

Spatial reasoning

Our item: (a) mental-rotation sets where you judge whether block figures are the same object rotated versus a mirror image, and (b) paper-folding / hole-punch items where you pick the correct unfolded result. All figures are app-generated original art; rotation items include a deliberate mirror-reflection distractor.

What it taps: visuospatial ability (Gv) — mentally rotating, folding, and transforming objects and holding the result in the mind's eye; a factor distinct from verbal and quantitative ability and a strong predictor of STEM achievement.

Modeled on: the mental-rotation paradigm of Shepard & Metzler (1971) and its group-administered multiple-choice format from Vandenberg & Kuse (1978); and the Paper Folding (VZ-2) visualization format from the ETS Kit (Ekstrom et al., 1976) and Thurstone's Space factor (Thurstone, 1938). The real-world relevance of spatial ability is documented by Wai, Lubinski & Benbow (2009), and its trainability by Uttal et al. (2013) — paradigm-level findings about the literature, not validated claims about this app's scores.

Working memory (backward digit span)

Our subtest: digits are shown one at a time (each ~0.85s on, ~0.3s gap); you then enter them in reverse order. It is adaptive — starting at span 3, two trials per level, advancing one digit on at least one success, capped at span 9 — and the score is the longest span reversed correctly.

What it taps: working memory — holding a sequence in mind while actively reordering it, recruiting the phonological loop and central-executive control.

Modeled on: the backward digit-span task historically associated with the Wechsler scales (Wechsler, 1939), interpreted through the multicomponent working-memory model of Baddeley & Hitch (1974; Baddeley, 2003). As Conway et al. (2005) note, simple backward span is a lighter working-memory measure than full complex-span tasks — so we describe it as "in the tradition of" working-memory assessment, not a definitive capacity measure.

Processing speed (symbol search)

Our subtest: a fixed 60-second speeded block — a target symbol on the left, a row of symbols on the right; you tap Yes/No whether the target appears, as fast and accurately as you can. Symbols reuse the app's visual-figure renderer. The score is net correct (correct minus errors) over the timed block.

What it taps: processing speed — efficiency of visual scanning, attention, and rapid same/different decisions under time pressure; how quickly simple cognitive operations are executed.

Modeled on: the symbol-search / digit-symbol-coding family of timed tasks historically associated with the Wechsler scales (Wechsler, 1939) and the processing-speed theory of cognition of Salthouse (1996).

The frameworks behind the design

The whole app is organized by the mainstream science of cognitive abilities. The general factor g (Spearman, 1904) explains why diverse reasoning tasks correlate — the basis for an overall profile. The fluid-versus-crystallized distinction (Cattell, 1963; Horn & Cattell, 1966) explains why we separate novel-reasoning items (abstract, logic, quantitative series, spatial) from knowledge-based items (verbal). And the Cattell-Horn-Carroll (CHC) three-stratum taxonomy (Carroll, 1993; synthesized by McGrew, 2009) gives the hierarchy we use to label domains: g over broad abilities (Gf, Gc, Gv, quantitative reasoning) over specific item types.

For how the items are produced and audited, the app follows the established discipline of Automatic Item Generation (AIG): turning a cognitive model of a construct into a template ("item model") with controlled variables that a computer instantiates into many parallel items. This is a peer-reviewed methodology (Bejar, 2002; Embretson & Yang, 2006; Gierl, Lai & Turner, 2012; Gierl & Haladyna, 2013), not an ad-hoc shortcut. The quality-control layer is modeled on standard psychometrics — classical item analysis via difficulty p-values and point-biserial discrimination (Crocker & Algina, 1986), internal-consistency reliability via Cronbach's alpha (Cronbach, 1951), the standard error of measurement and true-score reasoning of classical test theory (Lord & Novick, 1968), and item response theory for difficulty/discrimination modeling (Embretson & Reise, 2000).

Finally, as a precedent for generated and openly-licensed cognitive items, we point to the International Cognitive Ability Resource (ICAR) (Condon & Revelle, 2014) — a credible, public-domain item set. We use ICAR only as a conceptual benchmark for item format and open practice; we do not administer ICAR's items, nor any proprietary battery, and we claim no affiliation with any of them.

How we score

Today's score is self-referenced and provisional — and the app says so on screen.

The app computes a difficulty-weighted percent correct. It reads a bundled norms file, but the shipped artifact has status "none" (no real data yet), so scoring falls back to provisional constants: it converts your weighted percent to a z-score against placeholder values, scales it to an IQ-style number, and derives a percentile from a normal-curve approximation. The results screen frames this honestly as an estimated score, scaled against app users rather than the general population or any clinical sample, and the on-screen note says "placeholder scoring" while the status is "none."

We show the uncertainty rather than hide it. The app uses a deliberately conservative provisional reliability of 0.80, which yields a standard error of measurement and a wide 95% confidence band shown with every score. This is a placeholder until reliability is measured from real data.

The path to real norms is built but data-gated. When a real norms file (built from collected sittings) is bundled, scoring switches automatically — z-scored against the observed mean and spread, with percentiles from the empirical distribution. Sample-size gates keep claims honest: status stays "none" below ~200 sittings, "provisional" (clearly labeled) up to ~1,500, and only then moves toward "stable." A full psychometric pipeline — item analysis, reliability (alpha as a conservative lower bound, split-half, SEM), and IRT calibration — is built but inactive today (calibration is empty; no IRT has run). Pre-registered data-cleaning rules are fixed in code before any norms could be computed.

On privacy: the shipped app collects and transmits nothing. Research logging is off; the response log, uploader, ingest server, and analysis pipeline are dormant scaffolding for a possible future opt-in, consent-gated research mode. Everything above about real norms is contingent on that future, consented data — and even then, results would always be app-user-referenced, never census-representative.

The paid report, when generated, derives only self-referenced reads from a single sitting: per-domain weighted score and median response time, your strongest domain versus relative blind spot, a speed-versus-accuracy quadrant relative to your own median pace, and a difficulty curve. It is explicit that with roughly 10 items per domain these are directional reads of one sitting, not fixed traits.

What this is — and what it isn't

It is an original cognitive-challenge app for entertainment and education: 50 mixed-domain reasoning items plus two short subtests, producing a self-referenced Cognitive Profile.
It is not a diagnosis or a clinical, medical, employment, or educational-placement tool. Do not use it for any high-stakes decision.
It is not accredited, not clinically validated, and not population-normed. Today's score is provisional and self-referenced, with a deliberately wide confidence band.
The reliability figure (0.80) is a conservative placeholder, not a measured value; IRT calibration has not run; the subtests have no norms (raw scores only).
All items are original. Question formats are modeled on / in the tradition of public, well-studied paradigms (matrix reasoning, mental rotation, paper-folding, digit span, symbol search, syllogisms, number and letter series).
It does not administer or reproduce any proprietary test (Raven's Progressive Matrices, WAIS/Wechsler, ETS kits, Vandenberg-Kuse, ICAR, or others), and it is not endorsed by or affiliated with those instruments, their authors, or their publishers.
The cited science describes the established paradigms our formats draw on. It does not mean this app has been validated against, normed on, or equated to any of those instruments.

References

1. Baddeley, A. D., & Hitch, G. J. (1974). Working memory. In G. H. Bower (Ed.), The Psychology of Learning and Motivation (Vol. 8, pp. 47–89). Academic Press.
2. Baddeley, A. (2003). Working memory: Looking back and looking forward. Nature Reviews Neuroscience, 4(10), 829–839. https://doi.org/10.1038/nrn1201
3. Bejar, I. I. (2002). Generative testing: From conception to implementation. In S. H. Irvine & P. C. Kyllonen (Eds.), Item Generation for Test Development (pp. 199–217). Lawrence Erlbaum Associates.
4. Carpenter, P. A., Just, M. A., & Shell, P. (1990). What one intelligence test measures: A theoretical account of the processing in the Raven Progressive Matrices Test. Psychological Review, 97(3), 404–431.
5. Carroll, J. B. (1993). Human Cognitive Abilities: A Survey of Factor-Analytic Studies. Cambridge University Press.
6. Cattell, R. B. (1963). Theory of fluid and crystallized intelligence: A critical experiment. Journal of Educational Psychology, 54(1), 1–22. https://doi.org/10.1037/h0046743
7. Condon, D. M., & Revelle, W. (2014). The International Cognitive Ability Resource: Development and initial validation of a public-domain measure. Intelligence, 43, 52–64.
8. Conway, A. R. A., Kane, M. J., Bunting, M. F., Hambrick, D. Z., Wilhelm, O., & Engle, R. W. (2005). Working memory span tasks: A methodological review and user's guide. Psychonomic Bulletin & Review, 12(5), 769–786. https://doi.org/10.3758/BF03196772
9. Crocker, L., & Algina, J. (1986). Introduction to Classical and Modern Test Theory. Holt, Rinehart and Winston.
10. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/BF02310555
11. Ekstrom, R. B., French, J. W., Harman, H. H., & Dermen, D. (1976). Manual for Kit of Factor-Referenced Cognitive Tests. Educational Testing Service.
12. Embretson, S. E. (1998). A cognitive design system approach to generating valid tests: Application to abstract reasoning. Psychological Methods, 3(3), 380–396.
13. Embretson, S. E., & Reise, S. P. (2000). Item Response Theory for Psychologists. Lawrence Erlbaum Associates.
14. Embretson, S. E., & Yang, X. (2006). Automatic item generation and cognitive psychology. In C. R. Rao & S. Sinharay (Eds.), Handbook of Statistics, Vol. 26: Psychometrics (pp. 747–768). Elsevier/North-Holland.
15. Evans, J. St. B. T. (2002). Logic and human reasoning: An assessment of the deduction paradigm. Psychological Bulletin, 128(6), 978–996. https://doi.org/10.1037/0033-2909.128.6.978
16. Gentner, D. (1983). Structure-mapping: A theoretical framework for analogy. Cognitive Science, 7(2), 155–170. https://doi.org/10.1207/s15516709cog0702_3
17. Gierl, M. J., & Haladyna, T. M. (Eds.). (2013). Automatic Item Generation: Theory and Practice. Routledge/Taylor & Francis.
18. Gierl, M. J., Lai, H., & Turner, S. R. (2012). Using automatic item generation to create multiple-choice test items. Medical Education, 46(8), 757–765. https://doi.org/10.1111/j.1365-2923.2012.04289.x
19. Horn, J. L., & Cattell, R. B. (1966). Refinement and test of the theory of fluid and crystallized general intelligences. Journal of Educational Psychology, 57(5), 253–270. https://doi.org/10.1037/h0023816
20. Johnson-Laird, P. N. (1983). Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness. Harvard University Press.
21. Lord, F. M., & Novick, M. R. (1968). Statistical Theories of Mental Test Scores. Addison-Wesley.
22. McGrew, K. S. (2009). CHC theory and the human cognitive abilities project: Standing on the shoulders of the giants of psychometric intelligence research. Intelligence, 37(1), 1–10. https://doi.org/10.1016/j.intell.2008.08.004
23. Penrose, L. S., & Raven, J. C. (1936). A new series of perceptual tests: Preliminary communication. British Journal of Medical Psychology, 16(2), 97–104. https://doi.org/10.1111/j.2044-8341.1936.tb00690.x
24. Raven, J. C. (1938). Progressive Matrices: Sets A, B, C, D, and E. H. K. Lewis. (Standard Progressive Matrices, 60 items in 5 sets.)
25. Salthouse, T. A. (1996). The processing-speed theory of adult age differences in cognition. Psychological Review, 103(3), 403–428. https://doi.org/10.1037/0033-295X.103.3.403
26. Shepard, R. N., & Metzler, J. (1971). Mental rotation of three-dimensional objects. Science, 171(3972), 701–703. https://doi.org/10.1126/science.171.3972.701
27. Spearman, C. (1904). "General Intelligence," Objectively Determined and Measured. The American Journal of Psychology, 15(2), 201–293. https://doi.org/10.2307/1412107
28. Sternberg, R. J. (1977). Component processes in analogical reasoning. Psychological Review, 84(4), 353–378.
29. Thurstone, L. L. (1938). Primary Mental Abilities (Psychometric Monographs No. 1). University of Chicago Press.
30. Uttal, D. H., Meadow, N. G., Tipton, E., Hand, L. L., Alden, A. R., Warren, C., & Newcombe, N. S. (2013). The malleability of spatial skills: A meta-analysis of training studies. Psychological Bulletin, 139(2), 352–402.
31. Vandenberg, S. G., & Kuse, A. R. (1978). Mental rotations, a group test of three-dimensional spatial visualization. Perceptual and Motor Skills, 47(2), 599–604. https://doi.org/10.2466/pms.1978.47.2.599
32. Wai, J., Lubinski, D., & Benbow, C. P. (2009). Spatial ability for STEM domains: Aligning over 50 years of cumulative psychological knowledge solidifies its importance. Journal of Educational Psychology, 101(4), 817–835.
33. Wason, P. C. (1968). Reasoning about a rule. Quarterly Journal of Experimental Psychology, 20(3), 273–281. https://doi.org/10.1080/14640746808400161
34. Wechsler, D. (1939). The Measurement of Adult Intelligence. Williams & Wilkins.