The baseline of 2026-11-02
The fourth baseline of deduxit's engines, the engines of 0.2.0, taken by
bench/baseline.sh on 2026-11-02 from 14:34 for 2 h 23 min, at commit
a917eaf4772d, in a release build on an Intel Core Ultra X9 388H (16
cores, 62 GB). A time is the median of up to three runs. Every proof of
the baseline passed the checker, and no verdict differs from the third
baseline's on any problem both decide. The tables of every run are
bench/RESULTS.md,
the rows bench/results/2026-11-02/, and the four baselines compared
bench/COMPARISON.md,
from which the tables below are taken.
It was taken by day, beside other work, and not on the third's cores:
- the sequential runs went in four streams at once, each pinned to a CPU
of its own, CPUs 4, 5, 2 and 3 where the third's were 1, 0, 2 and 3:
the passes under one bias (
lltp-rarer,lltp-forward), the long runs and two of the last stage's reruns (lltp-intuitionistic-generous,lltp-classical-generous) went on two efficiency cores, the bias passes taking about 1.3 to 1.6 times the third's time for that; the families, the engines and the other LLTP passes ran on the third's cores; - other processes could share the streams' CPUs (46 problems waited
for a CPU for over 1 % of their time,
†in the tables); - the parallel stage ran on 16 threads over the whole machine while two other projects' sessions worked on it, so it is reported apart below and is not comparable with the third baseline's.
The families
The generated families, on one thread under a limit of 300 s
(families.csv, CPU 2 as in the third baseline); a run past 300 s went
again under 1 200 s (long-1.csv, long-2.csv):
| family | the fourth baseline | the third |
|---|---|---|
| 3-partition-no (bins of b) | every size in under 1 ms (b = 5: 320 µs) | b = 5: 346 µs |
| partition-yes (n items) | n = 24: 71.6 s, 28 > 1 200 s | n = 24: 74.4 s |
| partition-no | n = 12: 3.2 s, 14 and 15 > 300 s | n = 12: 3.8 s |
| qbf (n variables, four instances) | n = 48: three of four, 142–185 s (#0 at the memory bound); qbf/44#3 at the memory bound | n = 48: 162–222 s |
| wide-m1 (k pairs) | k = 2 048: 203 ms (the focused engine) | 523 ms (the net engine) |
| wide-m3 (k literals) | k = 2 048: 163 ms | 173 ms |
| wide-m4 | k = 2 048: 154 ms | 167 ms |
| mix (k pairs) | k = 11: 270 ms (the inverse method's turn) | k = 10: 62.5 s, 11 > 1 200 s |
| counter (n tokens) | n = 64: 53 µs | 55 µs |
| counter-over | refuted at every size, 64 in 45 µs | the same |
| chain (k clauses) | k = 256: 276 ms | 275 ms |
| additive (depth d) | d = 16: 192 ms | 182 ms |
| growing (copy bound n) | refuted at every size, 1 024 in 113 ms (the classical reading) | the copy bound or the recursion limit |
The families new in 0.2.0 ran at their listed sizes here; their growing sets are Families that grow, 2026-11-02.
The families' time against their size. The wall time of every
generated family's instances under the engine the command chooses for
it, the median of the runs of families.csv above (one thread, 300 s,
CPU 2), both axes logarithmic, a line per engine; a size where most runs
of some instance ended undecided, at the time limit or a bound, is a
hollow marker on the limit's line. Against the third baseline's figure:
mix is now the inverse method's, flat at about 0.27 s from 8 pairs on
where the focused engine rose to a minute at 10; wide-m1 and
wide-m2 are now the focused engine's, below the net engine's old
line; growing is refuted at every size instead of ending at a bound;
partition-yes, partition-no and qbf still rise ever more steeply.
The new families take their panels: contraction, euler,
pigeonhole, reach-fo, lambek-fo and witness-fo under the
two-sided engine, duplicate under the Horn engine, chain-fo under
the focused engine.
The focused engine against the net engine. The same measure for the
families of engines.csv, where the baseline forces each engine in
turn (one thread, 60 s, CPU 3 as in the third): the focused engine
decides every instance; the net engine, with its symmetry breaking and
its balance check new in 0.2.0, now decides every 3-partition-mll-no
instance (8 bins in 3 ms, past 60 s before from 6), partition-yes up
to 6 items (0.48 s) and partition-no up to 4 (9.6 ms), and meets the
limit beyond; on wide-m1 the two grow alike, the focused engine about
three times ahead at 2 048 pairs (209 against 599 ms). The net engine
refuses additive, which has no proof nets.
The LLTP library
Problems solved within 5 s on one thread, per pass (the counts of
bench/RESULTS.md's "Solved within the time limit", summed per file):
| pass | solved | the third |
|---|---|---|
lltp-intuitionistic (4 495 problems, copy bound 3; CPU 2) | 3 663 | 3 610 |
lltp-classical (4 512; CPU 3) | 3 674 | 3 621 |
lltp-rarer (the backward search alone; CPU 4, an efficiency core) | 3 662 | 3 608 |
lltp-forward (the forward search alone, 30 copies; CPU 5, an efficiency core) | 4 135 | 4 026 |
lltp-copies-10 (1 003 that ended at the copy bound; CPU 2) | 713 | 592 |
lltp-recursion (995 that ended at the recursion limit; CPU 3) | 895 | 895 |
The Petri nets are unchanged, 3 071 of 3 137 in each pass; the gains
are in the SYJ and KLE collections, by two things new in 0.2.0, as the
rows' engines and verdicts say: the refutation by the sequent's
classical reading after a search that ended at its copy bound (53 more
in the default and the classical copy-bound-3 passes, 54 under --bias rarer), and the inverse method's turns beside the
two-sided engine on a goal with an exponential (100 of lltp-forward's
problems that ended at the time limit before, and with the classical
reading 121 of lltp-copies-10's). Each such refutation is an
assignment falsifying the classical reading, so it holds whatever the
file's header says; the one new verdict against a header is KLE083+1
of KLE-01, whose other translations were refuted before.
The parallel stage, not comparable
The runs on more than one thread, parallel.csv (the families on 1, 2,
4, 8 and 16 threads, 120 s), parallel-net.csv (the net engine's),
lltp-all-cores.csv (1 102 LLTP problems on every core, 5 s) and
lltp-default.csv (the library under the command's default, 2 s), ran
on 16 threads over the whole machine (the rows' jobs) while other
work used it, where the third baseline's ran at night: their times,
speedups and the problems decided within the limit measure that load as
much as the engines. A rerun of this stage on a quiet machine replaces
these four files and this section.
| pass | solved | the third |
|---|---|---|
lltp-all-cores (1 102, every core, 5 s) | 993 | 993 |
lltp-default (4 495, the command's default: no copy bound, one thread for 0.1 s and then a pool of the other fifteen, 2 s) | 4 124 | 4 024 |
Under the default, 104 problems that ended at the limit before are
decided by the inverse method's turns, and four that the third decided
within 2 s end at the limit (SYJ205+1.008 and SYJ212+1.012 to
.014 of the cbn translation), one of them with about 6.5 of its 16
threads' worth of CPU time. One thing in parallel.csv holds whatever
the load: on mix, one thread decides 8 to 11 pairs in about 0.27 to
0.30 s by the inverse method's turn, which the dispatch gives only to a
search of one thread, while 2 to 16 threads race the focused engine
alone and take 0.8 s at 8 pairs, about a minute and more at 10, and
past the 120 s limit at 11.