01 / Leaderboard
Leaderboard
| Rank | Model | Organization | Configuration | Run time | Tool calls | Run date | Recording |
|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2Anthropic | Anthropic | Single shot · High | 30 min 43 s | 92 | 2026-11-10 | Watch the Claude Fable 5.2 recording on YouTube |
| 2 | Claude Opus 5.5Anthropic | Anthropic | Single shot · High | 44 min 42 s | 53 | 2026-09-23 | Watch the Claude Opus 5.5 recording on YouTube |
| 3 | Claude Fable 5.1Anthropic | Anthropic | Single shot · High | n/a | n/a | n/a | Watch the Claude Fable 5.1 recording on YouTube |
| DQ | Gemini 3.8 FlashGoogle | Single shot · High | 18 min 29 s | 105 | 2026-09-26 | Watch the Gemini 3.8 Flash recording on YouTube | |
| 4 | GPT-6 AstraOpenAI | OpenAI | Single shot · High | n/a | n/a | n/a | Watch the GPT-6 Astra recording on YouTube |
| DQ | DeepSeek V4.1 FlashDeepSeek | DeepSeek | Single shot · High | 43 min 33 s | 199 | 2026-09-26 | Watch the DeepSeek V4.1 Flash recording on YouTube |
| 5 | Kimi K3Moonshot AI | Moonshot AI | Single shot · High | 67 min 16 s | 116 | 2026-09-26 | Watch the Kimi K3 recording on YouTube |
| 6 | GPT-6 SolOpenAI | OpenAI | Single shot · High | 23 min 07 s | 50 | 2026-09-25 | Watch the GPT-6 Sol recording on YouTube |
| 7 | Qwen3.8 MaxAlibaba | Alibaba | Single shot · High | 50 min 11 s | 105 | 2026-09-26 | Watch the Qwen3.8 Max recording on YouTube |
| 8 | Claude Opus 4.7Anthropic | Anthropic | Single shot · High | 26 min 28 s | 48 | 2026-08-09 | Watch the Claude Opus 4.7 recording on YouTube |
| 9 | Grok 4.7xAI | xAI | Single shot · High | 9 min 31 s | 138 | 2026-09-26 | Watch the Grok 4.7 recording on YouTube |
| 10 | Claude Sonnet 5Anthropic | Anthropic | Single shot · High | 15 min 06 s | 147 | 2026-09-26 | Watch the Claude Sonnet 5 recording on YouTube |
| 11 | Claude Haiku 4.5Anthropic | Anthropic | Single shot · High | 1 min 43 s | 25 | 2026-09-26 | Watch the Claude Haiku 4.5 recording on YouTube |
| 12 | DeepSeek V4 ProDeepSeek | DeepSeek | Single shot · High | 13 min 11 s | 55 | 2026-09-26 | Watch the DeepSeek V4 Pro recording on YouTube |
| 13 | Mistral Medium 3.5Mistral AI | Mistral AI | Single shot · High | 4 min 04 s | n/a | 2026-09-27 | Watch the Mistral Medium 3.5 recording on YouTube |
- DQ: disqualified for a rule violation. Listed where the original authors of the reference demo (members of Jumalauta) placed it, without a rank; the reason is on its result page.
- Ranking: an order of preference set by the original authors of the reference demo (members of Jumalauta). This benchmark produces a ranking only. There are no scores, by design.
02 / All versions side by side
All versions side by side
03 / Key takeaways
Key takeaways
- The original authors of the demo ranked thirteen reimplementations: 1. Claude Fable 5.2, 2. Claude Opus 5.5, 3. Claude Fable 5.1, 4. GPT-6 Astra, 5. Kimi K3, 6. GPT-6 Sol, 7. Qwen3.8 Max, 8. Claude Opus 4.7, 9. Grok 4.7, 10. Claude Sonnet 5, 11. Claude Haiku 4.5, 12. DeepSeek V4 Pro, 13. Mistral Medium 3.5.
- Two runs were disqualified for fetching the original from the web, which the rules forbid: Gemini 3.8 Flash downloaded the original demo from scene.org, and DeepSeek V4.1 Flash read its info file on pouet.net. Their outputs are published, marked as rule violations, without a rank.
- DeepSeek V4 Pro does not accept image input, so it never saw the reference capture and worked from the soundtrack and average frame colours. The task rewards models that can look at the video.
- According to the original authors, the amount of detail in the Claude Fable 5.2 version is staggering.
- According to the original authors, the Claude Opus 5.5 version has noticeably higher fidelity to the original than the other entries.
- Visual polish alone did not decide the order. According to the original authors, the GPT-6 Astra version looks better at first glance, but Claude Fable 5.1 understood the essence of the demo, and they weighted that above surface quality.
- The benchmark measures interpretation and an understanding of the spirit of a work, not visual quality alone. It produces a ranking by design: there are no scores.
- All fifteen models produced a full-length audiovisual work from a single prompt.
- Every evaluated output is published as a side-by-side recording against the original, so readers can check the judgment for themselves.
04 / Recordings
Recordings
05 / Evaluators' notes
Evaluators' notes
The original authors gave the following reasons for their order. They are reported here as the authors' views, not as findings of this site. No other evaluation comments have been published.
- Rank 1
Claude Fable 5.2
According to the original authors, the amount of detail in the Claude Fable 5.2 version is staggering.
- Rank 2
Claude Opus 5.5
According to the original authors, the Claude Opus 5.5 version has noticeably higher fidelity to the original than the other entries.
- Rank 3
Claude Fable 5.1
According to the original authors, Claude Fable 5.1 understood the essence of the demo. Their example is the credits, where the model credited itself under an invented handle in the style of the group's own credits. The output is fully procedural, as its own credits state.
gfx: taas yks tekoaely joka luulee osaavansa piirtaeae
gfx: yet another AI that thinks it can drawRecording, about 1:28 gfx: ei yhtaeaen kuvaa, kaikki proseduraalista
gfx: not a single image, everything proceduralRecording, about 1:40 - Rank 4
GPT-6 Astra
According to the original authors, the GPT-6 Astra version looks better at first glance: the model used image generation models for its assets, and the output contains four generated images. The authors weighted an understanding of the spirit of the work above visual polish.
- Rank 5
Kimi K3
According to the original authors, Kimi K3 places narrowly above GPT-6 Sol, on the strength of its visuals.
- Rank 8
Claude Opus 4.7
According to the original authors, the Claude Opus 4.7 version looks noticeably rougher than the Claude Fable 5.1 and GPT-6 Astra entries.
- Rank 9
Grok 4.7
According to the original authors, the code error in the demo and the resulting frozen visuals lower its placement.
- Rank 10
Claude Sonnet 5
According to the original authors, the Claude Sonnet 5 version goes off in directions of its own instead of following the original.
- Rank 12
DeepSeek V4 Pro
According to the original authors, the DeepSeek V4 Pro effects are better than Claude Haiku 4.5's, but the demo shows no trace of the spirit of the original production.
06 / The task
The task
Models receive the same video capture and rebuild the work for the web. The prompt deliberately leaves room for interpretation: composition, effects, assets and creative choices belong to the model.
Kiinalaisen saatanan palvontademo
- Released
- 2001-06-16, Proxy 20016th place, combined demo competition
- Reference capture
- 169.62 s (2 min 50 s)
- Target platform
- HTML5 Canvas / WebGL
- Original soundtrack
- Permitted
- Code, assets and effects
- AI-generated
07 / Prompt
Prompt
So here's an old demoscene demo. what i would like is that you'd recreate this demo in demo_capture.mp4 rules: - use the capture as reference but it can be inspired-by/your interpretation partially and not frame-by-frame accurate - music can be used as is but code and assets and effects need to be generated by AI - technology: Web (HTML5 Canvas/WebGL) - demo's length must match the original's length
demo_capture.mp4Same reference capture for every modelThis is the complete prompt, reproduced verbatim. Every model receives exactly this text together with the listed inputs.
To test another model, copy the prompt and follow the shared conditions in the evaluation protocol.
A benchmark kit containing the prompt, the reference capture, the protocol and the tooling used to produce the comparison recordings exists. It is not yet public; publication is planned.
08 / Evaluation protocol
Evaluation protocol
Every model receives the same two inputs: the prompt, reproduced verbatim on this page, and the reference
capture demo_capture.mp4. The model produces its reimplementation in a single shot at High reasoning
effort, in its maker's own agent harness in a vanilla setup: the default configuration, with no customisation.
A result therefore describes the model together with its harness. There is no human
follow-up: nobody corrects, steers or re-prompts the model after the initial prompt. A single run per model
is used. No retries, no best-of-N selection.
Runs are made in a reference container: a fresh Docker environment with Debian 13, Node.js 26, Python 3 with NumPy and Pillow, FFmpeg, ImageMagick, Git and common build tools, plus the harness installed with its official installer. No web browser is preinstalled. The model works as an unprivileged user with passwordless sudo and internet access, so it may install anything it needs, including a browser for checking its own output. The harness starts from factory settings, with no plugins, skills, memories or instruction files. Permission prompts are approved automatically; the container is the sandbox, and only the task directory is shared with the host. The time limit is 240 minutes. Every model gets the same image, so the tooling a model sets up for itself is part of its result. Each result page states whether the run used the reference container.
Fetching the original production or its files, for example from scene.org or pouet.net, is a rule violation, even when nothing from it ends up in the output. Every run is checked before publication: its transcript for web searches and downloads, and its output for files identical to or named like those of the original release. From 2026-09-26 on, each run also records its network connections outside the model's reach. A violating run is published with a label on its recording; the original authors decide whether it is ranked or disqualified.
The evaluators are the original authors of the reference demo: members of Jumalauta, the group that made it in 2001. This is the core of the benchmark and a deliberate design decision. The authors know the intention and the cultural context of the work better than anyone else, and the benchmark measures how well a model understands exactly that.
The authors place the reimplementations in order of preference, judging each as a whole work: interpretation, creativity and implementation together. A frame-by-frame reconstruction is not required. Interpretation is explicitly permitted by the prompt: original credits or invented creator names, for example, can be intentional creative decisions within the work and are treated as such.
The result is an ordinal ranking. By design there are no scores, no sub-scores and no confidence intervals.
Each output is documented as a side-by-side comparison recording: the original on the left, the model's version on the right, with the audio taken from the original. The recordings are 1920x1080, open with a 5-second title card and run about 2 min 55 s.
Publication status
- Prompt
- Published verbatim
- Reference work
- Published
- Recordings
- 15 of 15 models
- Ranking
- 13 of 15 models placed
- Evaluators' notes
- 9 of 15 models
- Scores
- None, by design
- Evaluators
- The original authors of the reference demo (members of Jumalauta)
- Number, names and roles of evaluators
- Not published
- Harness names, run dates, cost and tokens
- 15 of 15 models
- Benchmark kit
- Not yet public
Shared conditions
- Generation
- Single shotOne initial prompt. No subsequent human direction.
- Reasoning effort
- HighExtra-high and max settings are excluded.
- Harness setup
- VanillaDefault configuration of the harness, no customisation. The harness of each run is listed on its result page.
- Human follow-up
- NoneThe model's first complete output is the evaluated output.
- Runs per model
- OneA single run per model is used. No retries, no best-of-N selection.
- Implementation
- HTML5 Canvas / WebGLThe work must run on the web.
- Code, visual assets and effects
- AI-generatedEverything except the soundtrack must be produced by the model.
- Original soundtrack
- PermittedThe music of the reference may be reused as is.
- Duration
- Must match the referenceThe reference capture file is 169.62 seconds long.
- Time, cost and token budgets
- Not specifiedVersion 1.0 of the protocol sets no budget limits.
09 / Limitations
Limitations
- An author-centred view, by design
- The evaluators are the original authors of the reference demo. Their ranking reflects how well each output captures the work as they understand it. It is a subjective view and not a general verdict on quality.
- A small group of evaluators
- The evaluators are a small group. Their number, names and roles have not been published, and the evaluation is not claimed to be blind.
- An ordinal ranking only
- The result is an order of preference. It does not say how large the differences between the works are, and there is no score to compare across versions or benchmarks.
- One task, one run per model
- The benchmark consists of a single audiovisual task built on a single reference work, and each model was run once. Run-to-run variance has not been measured, so a different sample from the same model could look different.
- Fifteen models
- Only fifteen models have been evaluated. The ranking says nothing about models that were not run.
- Possible training data contamination
- The reference demo has been publicly available online for years. It may be present in the training data of the evaluated models.
- Not a measure of general capability
- The results describe performance on this benchmark only. They say nothing about how the models compare elsewhere.
10 / FAQ
FAQ
- Isn't this just taste?
- Yes, and deliberately so. The outputs are creative works, and the benchmark asks how well a model understood the work it was given. That is a judgment, not a measurement. The benchmark makes the judgment inspectable instead: the prompt, the conditions and every recording are on this page.
- Why do the original authors judge? Isn't that a conflict of interest?
- It is the central design decision of this benchmark. The authors know the intention and the cultural context of the original better than anyone else, and understanding exactly that is what the benchmark measures. The price is that the result is an author-centred, subjective view and not a general verdict on quality. According to the authors, their order did not follow visual polish, which is the kind of distinction this choice of evaluators is meant to capture.
- Why is there no score?
- The evaluators put the works in order of preference as whole works. There is no rubric, no sub-scores and no confidence interval, and none are planned for this benchmark. A ranking says which work the authors preferred. It does not say by how much.
- Is the original demo in the training data?
- Possibly. The demo was released in 2001 and has been publicly available online since. This cannot be ruled out for any of the evaluated models, and it is listed under Limitations.
- Why only fifteen models?
- These are the models evaluated so far. Any model can be run with the same prompt, the same reference capture and the same conditions.
- Was each model run more than once?
- No. A single run per model is used. No retries, no best-of-N selection.
11 / Version history
Version history
- v1.0Active
- Initial publication with three evaluated models.
- Side-by-side comparison recordings published for every model.
- Full ranking by the original authors published, with the authors' notes.
- Claude Opus 5.5 evaluated and added to the ranking.
- GPT-6 Sol evaluated and added to the ranking.
- Mistral Medium 3.5 evaluated and added to the ranking.
- Claude Haiku 4.5 evaluated and added to the ranking.
- Claude Sonnet 5 evaluated and added to the ranking.
- Kimi K3 and Qwen3.8 Max evaluated and added to the ranking.
- DeepSeek V4 Pro evaluated and added to the ranking. Gemini 3.8 Flash disqualified for a rule violation.
- Grok 4.7 evaluated and added to the ranking. DeepSeek V4.1 Flash disqualified for a rule violation.
12 / How to cite
How to cite
Cite the benchmark together with its version. Results from different major versions are not comparable.
@misc{jumalaita2026creativereimplementation,
author = {{JumalAIta Intelligence}},
title = {Creative Reimplementation Benchmark},
year = {2026},
url = {https://ai.jumalauta.org/benchmarks/creative-reimplementation/},
note = {Version 1.0}
}The principles that apply to every JumalAIta Intelligence benchmark are described in the general methodology. Changes to this site are recorded in the changelog.
















