Release · November 2026
A molecule, written as text.A 4B model works it out.
Arxo Valentia ChemIQ 0.1 reaches 74.1% on ChemIQ without tools. GPT-5 is at 70.2%. The benchmark’s own checker did the scoring.
The task
Answer organic chemistry questions from a SMILES string, without tools.
Counting atoms and rings, shortest paths, atom mapping, IUPAC names, reaction products.
Valentia ChemIQ
Correct
74.1%
605 of 816 questions.
ChemIQ, all 816 questions. One training run.Open the repository
Next to published results
ChemIQ, all 816 questions, scored by the benchmark’s own parser and answer checker. The circle is the model’s size: its area is the number of parameters.
Valentia ChemIQTrained on the task · the repository
74.1%
GPT-5Not trained on the task · post by the benchmark’s author
70.2%
o3-mini, high reasoningNot trained on the task · ChemIQ paper
56.7%
Gemini 2.5 ProNot trained on the task · ChemIQ paper
55.5%
DeepSeek-R1Not trained on the task · ChemIQ paper
50.2%
Arxo Valentia was trained on questions of the same kinds, built from molecules outside the test set; the other rows are general models that answer from the prompt alone. Sizes are the ones in the model names; a dash means the name gives none.
By category
Eight kinds of question. Under each: o3-mini with high reasoning, Gemini 2.5 Pro and DeepSeek-R1, from the model responses released with the benchmark, scored with the same checker.
Carbon counting50 questions · o3-mini 92.0% · Gemini 2.5 Pro 82.0% · DeepSeek-R1 90.0%
100%50 / 50
Ring counting48 questions · o3-mini 100% · Gemini 2.5 Pro 93.8% · DeepSeek-R1 93.8%
100%48 / 48
Shortest path108 questions · o3-mini 72.2% · Gemini 2.5 Pro 63.0% · DeepSeek-R1 59.3%
100%108 / 108
Atom mapping184 questions · o3-mini 56.0% · Gemini 2.5 Pro 46.2% · DeepSeek-R1 54.3%
78.3%144 / 184
SMILES to IUPAC name200 questions · o3-mini 28.5% · Gemini 2.5 Pro 38.5% · DeepSeek-R1 20.0%
70.5%141 / 200
Structure-activity (Free-Wilson)40 questions · o3-mini 100% · Gemini 2.5 Pro 95.0% · DeepSeek-R1 95.0%
85.0%34 / 40
Reaction product90 questions · o3-mini 60.0% · Gemini 2.5 Pro 55.6% · DeepSeek-R1 45.6%
88.9%80 / 90
NMR structure elucidation96 questions · o3-mini 38.5% · Gemini 2.5 Pro 51.0% · DeepSeek-R1 38.5%
0%0 / 96
The model was not trained on NMR questions and solves none of them. On the other seven categories it scores 605 of 720 (84.0%).
The benchmark’s checker rejects any answer longer than 200 characters. An atom mapping for a molecule of 23 or more atoms is longer than that, so 30 of the 184 atom-mapping questions cannot be credited to any answer. With that limit lifted the same answers score 629 of 816 (77.1%), and atom mapping 168 of 184.
Scope
Seven of eight categories
The model covers seven of the eight ChemIQ categories and the question wording of the benchmark. NMR structure elucidation is outside this release. On other chemistry questions it is untested.
Names
A name counts as correct when OPSIN parses it to the intended structure, as ChemIQ scores it. The model writes a systematic name of that kind; it is not always the preferred IUPAC name.
Molecules
Those of the benchmark’s range: neutral organic molecules of up to about 30 heavy atoms, without stereochemistry.
A research release
One training run. Greedy decoding, thinking switched off, a limit of 3,900 new tokens.
How it was taught
Kept.
What is public
The model is on Hugging Face; the repository holds what it answered and how each answer was scored. Arxo Valentia follows the Arxo Materia releases on MedCalc-Bench and EHRSQL.
- Arxo-Valentia-4B-ChemIQ-0.1
- Hugging Face
- What the model answered on all 816 questions
- Public.
- Per-question results from the benchmark’s checker
- Public.
- Checksums of every file
- Public.