Calculemus

Machines have learned to speak. They speak about everything, without rest, and almost always well. It is the achievement of this decade, and it is half of one.

Fluency is half of it

Ask a frontier model the same thing twice. Ask it which words its answer rests on. Ask it where it guessed. Each is a way of asking whether anything holds the answer in place.

Scale does not remove this. A larger model is a better reader. It is not a better ledger. No amount of fluency adds up to exactness. It has to be built.

The other half

We built it. Arxo takes a text that people must answer to and makes it answer back. Not an opinion about the text. The text itself, carried to its conclusion. Every answer can be checked against the text.

Leibniz had the word three centuries ago: calculemus, let us calculate. He meant that a dispute could end not when one side tired, but when the sum came out.

We are not going to describe how it is done.

What comes next

We set out to make answers exact. Something else happened. We are not ready to say what. One hint: everyone can make a machine speak, and almost no one can say what theirs knows.

We have been taking Arxo to public benchmarks: the ones where someone else does the grading. The first numbers are in. We will publish them one at a time.

The numbers are public. What it is, we keep.

We open Arxo to a few teams at a time, on their own texts. If something you run depends on a text, write to us.

Calculemus.

← All notes