Reliability
This is three different AI models looking at the same painting nine times each, and how often they saw the same thing. Every artwork in the collection gets scored by 3 independent vision-language models, three passes apiece, and this page reports where those 3 models agree with each other across all 36 fingerprinted artworks—and, just as important, where they don't.
What these numbers are, and aren't
- This measures agreement between models, not agreement with human experts. No humans are in this loop.
- Models that agree may share a training bias. Agreement here is not evidence of correctness—three models converging because they learned from overlapping data is a weak signal about the painting, whatever it feels like.
- This is not Krippendorff's alpha, and it is not inter-rater reliability in the formal sense either term implies. Those require human coders, and this page has none.
- The sample size behind every number below is n = 36 artworks, shown per principle since it varies. That's a small corpus. Read the percentages accordingly.
- That agreement needs a floor to stand on. Shuffle the principle labels inside each model's own vector—same scores, same per-model habits, but no connection left between a score and the principle it was given for—and the three models still land within a point of each other 51% of the time, because the scale has only four values and most answers sit on two of them. Observed, they agree 97% of the time. The gap between those two numbers is the result worth reading; the 97% on its own isn't.
- The same test on the ordering, rather than the score: ask how closely two models rank all 58 principles from loudest to quietest in one painting. Against shuffled labels they match at 0.00—no relationship at all. On the same painting, 0.76. On two different paintings they still match at 0.44, which is worth knowing—a fair part of this agreement is about Western art in general, not the picture in front of you. The distance from 0.44 to 0.76 is the part that belongs to the individual work. Both baselines were measured offline across 30 artworks in August 2026; everything else on this page is computed live.
Every principle, weakest agreement first
Sorted by how often the three models contradicted each other on this principle. Click a column to re-sort. n is how many artworks actually had a measurement for that principle—never padded, never guessed.
| Principle | Tier | n | Stable | Thin | Contradicted | Detected |
|---|---|---|---|---|---|---|
| #21 Transparency/Opacity | B | 36 | 83% | 6% | 11% | 86% |
| #30 Optical vs. Physical Color Mixing | B | 36 | 86% | 3% | 11% | 58% |
| #17 Focal Blur/Depth of Field | B | 36 | 83% | 8% | 8% | 28% |
| #2 Color Opponent Channels | A | 36 | 89% | 3% | 8% | 89% |
| #55 Agency Attribution | A | 36 | 92% | 0% | 8% | 61% |
| #49 Gaze Direction / Social Attention | A | 36 | 86% | 8% | 6% | 64% |
| #26 Binocular vs. Monocular Depth Cues | A | 36 | 89% | 6% | 6% | 81% |
| #6 Color Temperature | B | 36 | 92% | 3% | 6% | 94% |
| #8 Symmetry/Asymmetry | B | 36 | 92% | 3% | 6% | 100% |
| #45 Economy of Means | C | 36 | 92% | 3% | 6% | 100% |
| #51 Visual Pop-out / Pre-attentive Features | A | 36 | 92% | 3% | 6% | 100% |
| #7 Atmospheric Perspective | A | 36 | 94% | 0% | 6% | 58% |
| #39 Dynamic Symmetry and Proportional Systems | D | 36 | 94% | 0% | 6% | 28% |
| #56 Postural Affect | A | 36 | 94% | 0% | 6% | 72% |
| #28 Specularity/Surface Reflection | A | 36 | 92% | 6% | 3% | 67% |
| #9 Scale Relationships | B | 36 | 94% | 3% | 3% | 100% |
| #11 Texture | B | 36 | 94% | 3% | 3% | 100% |
| #22 Local vs. Atmospheric Color | B | 36 | 94% | 3% | 3% | 94% |
| #36 Simultaneity vs. Sequence | C | 36 | 94% | 3% | 3% | 42% |
| #42 Visual Tension | B | 36 | 94% | 3% | 3% | 100% |
| #44 Interval Relationships | B | 36 | 94% | 3% | 3% | 94% |
| #14 Directional Force/Implied Movement | B | 36 | 97% | 0% | 3% | 100% |
| #27 Cropping/Framing | B | 36 | 97% | 0% | 3% | 100% |
| #31 Color Weight and Advancement | B | 36 | 97% | 0% | 3% | 94% |
| #37 Progressive Rhythm | B | 36 | 97% | 0% | 3% | 75% |
| #43 Anchoring and Reference Points | B | 36 | 97% | 0% | 3% | 100% |
| #52 Peak Shift / Supernormal Stimuli | B | 36 | 97% | 0% | 3% | 33% |
| #54 Canonical Views | B | 36 | 97% | 0% | 3% | 86% |
| #58 Empathic Resonance | C | 36 | 97% | 0% | 3% | 69% |
| #23 Linear Perspective | A | 36 | 94% | 6% | 0% | 50% |
| #41 Emergence | B | 36 | 94% | 6% | 0% | 97% |
| #25 Color Harmony Systems | C | 36 | 97% | 3% | 0% | 94% |
| #47 Face Detection | A | 36 | 97% | 3% | 0% | 67% |
| #1 Edge Detection | A | 36 | 100% | 0% | 0% | 100% |
| #3 Pattern Completion/Closure | B | 36 | 100% | 0% | 0% | 97% |
| #4 Figure-Ground Relationships | A | 36 | 100% | 0% | 0% | 100% |
| #5 Grouping | A | 36 | 100% | 0% | 0% | 100% |
| #10 Rhythm and Repetition | B | 36 | 100% | 0% | 0% | 100% |
| #12 Saturation | B | 36 | 100% | 0% | 0% | 94% |
| #13 Overlap/Occlusion | A | 36 | 100% | 0% | 0% | 100% |
| #15 Closure/Negative Space | A | 36 | 100% | 0% | 0% | 94% |
| #16 Light Source Logic | A | 36 | 100% | 0% | 0% | 78% |
| #18 Pattern vs. Figure | B | 36 | 100% | 0% | 0% | 89% |
| #19 Visual Weight/Balance | B | 36 | 100% | 0% | 0% | 100% |
| #20 Simultaneous Contrast | A | 36 | 100% | 0% | 0% | 100% |
| #24 Value Structure/Massing | B | 36 | 100% | 0% | 0% | 100% |
| #29 Contour Quality | B | 36 | 100% | 0% | 0% | 100% |
| #32 Chromatic Aberration | C | 36 | 100% | 0% | 0% | 0% |
| #33 Elevation in Picture Plane | A | 36 | 100% | 0% | 0% | 92% |
| #34 Detail Density Gradient | B | 36 | 100% | 0% | 0% | 100% |
| #35 Implied Time/Narrative Sequence | C | 36 | 100% | 0% | 0% | 75% |
| #38 Unity vs. Variety Tension | B | 36 | 100% | 0% | 0% | 100% |
| #40 Figure-Ground Reversal | B | 36 | 100% | 0% | 0% | 8% |
| #46 Complexity and Visual Density | B | 36 | 100% | 0% | 0% | 100% |
| #48 Biological Motion Detection | A | 36 | 100% | 0% | 0% | 53% |
| #50 Common Fate | A | 36 | 100% | 0% | 0% | 25% |
| #53 Embodied Perspective / Eye-Level Effects | B | 36 | 100% | 0% | 0% | 89% |
| #57 Embodied Simulation | B | 36 | 100% | 0% | 0% | 75% |
Where this instrument should not be trusted yet
The collection agrees on most things most of the time—these are the exceptions: the principles where the three models most often failed to agree with each other, relative to everything else on this page. Not "wrong"—there's no ground truth here to be wrong against—but not something to cite as a stable reading either.
- B Transparency/Opacity contradicted on 4 of 36 artworks (11%)
- B Optical vs. Physical Color Mixing contradicted on 4 of 36 artworks (11%)
- B Focal Blur/Depth of Field contradicted on 3 of 36 artworks (8%)
- A Color Opponent Channels contradicted on 3 of 36 artworks (8%)
- A Agency Attribution contradicted on 3 of 36 artworks (8%)
- A Gaze Direction / Social Attention contradicted on 2 of 36 artworks (6%)
- A Binocular vs. Monocular Depth Cues contradicted on 2 of 36 artworks (6%)
- B Color Temperature contradicted on 2 of 36 artworks (6%)
A coverage note on the newest principles
Principles 55 through 58 were added later than the rest—social-cognition principles introduced after the collection already had fingerprinted work in it. An artwork fingerprinted before that vector grew simply has no measurement for those four, which is a gap in coverage, not a finding about the principle. In the current collection every artwork post-dates that change, so there's no gap to see yet—but the mechanism is worth stating before it reappears.
Method
Every fingerprinted artwork is scored independently by 3 vision-language models—Claude (Anthropic), Gemini (Google), GPT (OpenAI), three passes each, blind: Pass 1 sees only the image, never the artist, title, or any other metadata. That's nine raw scores per principle per artwork before anything here runs.
Those nine scores collapse to one tier per principle per artwork. Each provider's own three passes are combined into that provider's median score; the three providers' medians are then compared to each other: if all three sit within a single point of each other, the read is stable. If exactly one provider is a clear outlier while the other two agree, it's thin. If there's no clean pair that agrees, it's contradicted. This page reports what share of the collection lands in each bucket, principle by principle.
Full list of what each principle measures: Principles.