0.7.0 release card: permutation invariance#

Release card

Permutation invariance

0.7.0

We compare GES and the seven methods 0.7.0 supports, GIES, exact search, CD-NOD, MultiGroupDirectLiNGAM, GFCI, GIN, and CALM, with the packages that implement them: causal-learn and lingam in Python, pcalg in R, and Tetrad in Java. GES runs more than 60x faster than causal-learn at 50 to 150 variables, and more than 300x at 150, with the same graph on all 15 datasets. GIN runs more than 70x faster than Tetrad and more than 2x faster than causal-learn at every size. CD-NOD runs more than 15x faster than causal-learn at 20 and 50 variables, and CALM more than 15x faster with its default schedule, both with fewer errors on more datasets. GFCI runs faster than Tetrad from 50 variables, more than 3x at 200, with fewer errors at every size.

Speed is not accuracy. On the largest graphs, the errors of GES and GIES approach the empty graph’s in both packages (see Limits). Every dataset lists its variables in a random order, so the column order tells no method anything about the causal order (see Method). You can find speedups and accuracy in the summary. These results apply to the measured workloads. Speed and accuracy depend on the data, settings, and hardware.

Summary#

We compare each method with every package that implements it with the same test or score. We report every paired speed ratio, regardless of accuracy, except for GES at 200 variables, where both packages’ errors reach the empty graph’s; its table gives that row.

Speed.

Family	Method	causal-learn	Tetrad	pcalg	lingam	Andrey 0.4.0
Score-based	GES	68.1-325x	-	-	-	18.4-65.7x
	GIES	-	-	1.62-2.62x	-	-
	Exact search	2.35-2.57x	-	-	-	-
	CALM	19.4-26.2x*	-	-	-	-
Constraint-based	CD-NOD	16.0-31.1x	-	-	-	-
	GFCI	-	1.27-3.65x	-	-	-
LiNGAM	MultiGroupDirectLiNGAM	-	-	-	2.51-3.30x	-
Hidden causes	GIN	3.84-7.14x	79.7-146x	-	-	-

* Andrey’s default schedule; on the paper’s schedule, which causal-learn uses, 2.44-2.95x.

Each cell is the range of Andrey’s speedup over that package across the method’s sizes; each method’s section below gives every size. A dash marks a package we did not compare.

Accuracy.

Family	Method	causal-learn	Tetrad	pcalg	lingam	Andrey 0.4.0
Score-based	GES	100%	-	-	-	100%
	GIES	-	-	100%	-	-
	Exact search	100%	-	-	-	-
	CALM	80%	-	-	-	-
Constraint-based	CD-NOD	85%	-	-	-	-
	GFCI	-	95%	-	-	-
LiNGAM	MultiGroupDirectLiNGAM	-	-	-	100%	-
Hidden causes	GIN	100%	100%	-	-	-

Each cell is the share of datasets on which Andrey’s result has no more errors than that package’s: fewer or the same number. Against Andrey 0.4.0, both builds return the same graph on every dataset. Each method’s section gives the datasets with the same graph and those on which each package has fewer errors.

Method#

Data. Each method runs on data of the kind it is for. GES, exact search, and CALM run on linear-Gaussian data from an Erdos-Renyi DAG with mean degree 2, standardized, with n = 10 d rows. GIES adds interventions, simulated as in Hauser and Bühlmann (2012): round(0.4 d) variables are each set by an intervention in their own block of rows, and every fit gets the rows’ intervention targets. CD-NOD pools 2,000 rows from four domains, in which one variable in three shifts its mean. MultiGroupDirectLiNGAM gets three groups of 1,000 rows over the same DAG, linear with uniform noise and weights of their own sign in each group. GFCI’s data hide one node in ten, and d counts the hidden nodes. GIN’s data have m latent variables, each a cause of every later one and each measured by three indicators, with 1,000 rows of the 3 m indicators. There are 5 datasets per size, one per seed, and 20 for CD-NOD.

Shuffled columns. Every dataset lists its variables in an order drawn from its seed, with the true graph permuted the same way, so the column order carries nothing of the causal order. GIN’s data list the indicators in a random order of their own.

Accuracy. Each method’s errors count what it returns against the truth it can identify: the SHD between CPDAGs for GES, exact search, and CALM; between interventional essential graphs for GIES; between the CPDAG that the domain index orients for CD-NOD; and summed over the groups’ DAGs for MultiGroupDirectLiNGAM. The SHD counts one for each pair of variables whose edge is missing, extra, or oriented differently, so the empty graph’s errors are the number of true edges. GFCI’s errors are the endpoints whose mark differs from the true PAG’s, so the empty graph’s are twice the true edges. GIN’s error count is the latents whose indicators it does not return as one cluster; the empty graph misses every latent. Errors are medians over a size’s datasets. “Same graph” means the two packages return the same result.

Timing. Times are seconds per fit, the median over a size’s datasets. For each dataset, we divide the other package’s time by Andrey’s, then report the median of these paired ratios. This need not equal the ratio of the displayed median times. We show no speedup for a size at which both packages’ median times are under 0.1 s, and leave it out of the summary. Tetrad’s time covers the search inside the Java virtual machine, excluding startup and data reading. pcalg’s time includes converting the rows’ intervention targets into its own. Andrey’s time excludes Python’s startup and import. Times and ratios have three significant figures.

Hardware. Every fit ran on an AMD EPYC 7713 with 4 cores per fit and 4 BLAS and OpenMP threads; a parallel mode runs 4 single-threaded worker processes. Fits ran one at a time on an exclusive node, with 1 timed fit per dataset and a 30,000 MB memory cap. Every method but GIN and CALM had 1 untimed warm-up fit. The wall cap was 1,800 s, and 3,600 s for CALM.

Builds and versions#

Comparison	Andrey build	Measured package versions
GIN	Pre-release 0.7.0, October 18, 2026	Tetrad (tetrad-lib 7.6.10), causal-learn 0.1.4.8
GES against Andrey 0.4.0	Pre-release 0.7.0, October 17, 2026	Andrey 0.4.0
GFCI (with its FCI and parallel-mode rows), Andrey’s rows	Pre-release 0.7.0, October 19, 2026, with order-free possible-d-separation	Andrey only
CD-NOD, Andrey’s rows	Pre-release 0.7.0, October 19, 2026	Andrey only
GFCI and CD-NOD, Tetrad’s and causal-learn’s rows	Pre-release 0.7.0, October 17, 2026	Tetrad (causal-cmd 1.12.0, tetrad-lib 7.6.3), causal-learn 0.1.4.8
Every other comparison	Pre-release 0.7.0, October 17, 2026	causal-learn 0.1.4.8, lingam 1.13.0, pcalg 2.7.12

Andrey ran on Python 3.13 with NumPy 2.5.4, SciPy 1.16.3, and torch 2.14.1 (CPU). lingam ran in its own Python 3.12 environment, pcalg in R 4.6.1, and Tetrad on OpenJDK 21. GIN’s references ran on the build of October 16, 2026, with the same GIN harness and data as Andrey’s GIN. The summary contains one row per comparison and size. Each row gives median seconds, the paired ratio, the datasets both packages finished (paired), and those with the same result (same_graph). It also gives each package’s errors beside the empty graph’s, the datasets on which each has fewer errors, and F1.

GES#

GES searches over equivalence classes, adding, then removing, the edges that most improve the score. Both packages use the BIC with a penalty of log(n) per parameter and at most 4 parents, and return a CPDAG; causal-learn halves the penalty’s scale, so its lambda_value=0.5 is Andrey’s 1. Andrey runs its NumPy backend in one process.

d	n	Andrey (s)	causal-learn (s)	Speedup	Datasets	Same graph	Errors, Andrey	Errors, causal-learn	Errors, empty graph
50	500	0.107	5.95	68.1x	5 of 5	5 of 5	47	47	50
100	1,000	0.229	41.9	182x	5 of 5	5 of 5	79	79	100
150	1,500	0.549	193	325x	5 of 5	5 of 5	140	140	150
200	2,000	1.05	584	629x	3 of 5	3 of 3	199	199	200

GES returns the same graph as causal-learn on every dataset. At 200 variables, causal-learn did not finish 2 of the 5 datasets within the cap, so the speedup is the median over the 3 it finished. With each unfinished fit counted at 900 s, the longest a timed fit could take under the cap, the speedup is at least 749x.

Against the released Andrey 0.4.0, on the same datasets, in one process and in parallel:

d	n	Backend	Andrey 0.7.0 (s)	Andrey 0.4.0 (s)	Speedup	Same graph
50	500	NumPy, one process	0.107	1.69	18.4x	5 of 5
100	1,000	NumPy, one process	0.229	9.34	41.3x	5 of 5
150	1,500	NumPy, one process	0.549	36.1	65.7x	5 of 5
200	2,000	NumPy, one process	1.05	90.6	87.0x	5 of 5
50	500	NumPy, parallel	0.102	1.63	15.9x	5 of 5
100	1,000	NumPy, parallel	0.303	4.95	16.3x	5 of 5
150	1,500	NumPy, parallel	0.964	14.6	15.0x	5 of 5
200	2,000	NumPy, parallel	1.68	33.5	19.9x	5 of 5

The two builds return the same graph on every dataset.

Supported methods#

GIES#

GIES learns from data that mix observation with interventions. Both packages optimize the same BIC (pcalg’s GaussL0penIntScore with its default penalty), run the forward, backward, and turning phases until none improves, with no degree cap, and return the interventional essential graph.

d	n	Andrey (s)	pcalg (s)	Speedup	Datasets	Same graph	Errors, Andrey	Errors, pcalg	Errors, empty graph	Targets
10	100	0.00928	0.0161	-	5 of 5	5 of 5	2	2	10	4
20	200	0.0178	0.0234	-	5 of 5	5 of 5	9	9	17	8
50	500	0.0621	0.0819	-	5 of 5	5 of 5	25	25	51	20
100	1,000	0.249	0.392	1.62x	5 of 5	5 of 5	72	72	96	40
200	2,000	0.997	2.65	2.62x	5 of 5	5 of 5	181	181	198	80

GIES returns the same graph as pcalg on every dataset.

Exact search#

Exact search finds a DAG with the lowest BIC by A* search, with no parent limit. Where several DAGs share the lowest BIC, the packages may return different ones, so we compare their CPDAGs.

d	n	Andrey (s)	causal-learn (s)	Speedup	Datasets	Same graph	Errors, Andrey	Errors, causal-learn	Errors, empty graph	Same BIC
5	50	0.00255	0.00431	-	5 of 5	5 of 5	1	1	5	5 of 5
10	100	0.112	0.261	2.35x	5 of 5	5 of 5	3	3	10	5 of 5
15	150	6.41	16.5	2.57x	5 of 5	5 of 5	7	7	15	5 of 5

Exact search reaches the same BIC and returns the same graph as causal-learn on every dataset.

CD-NOD#

CD-NOD learns from data pooled across domains, with the domain index as one more variable that orients the edges it reaches. Both packages test with Fisher-Z at alpha 0.05 with a stable skeleton search. Andrey orients conflicting colliders by the p-values that separated their ends (collider_order="p_value"). causal-learn puts a collider wherever the middle node is in no separating set, in the order it visits the triples.

d	n	Andrey (s)	causal-learn (s)	Speedup	Datasets	Same graph	Errors, Andrey	Errors, causal-learn	Errors, empty graph	Andrey ahead	causal-learn ahead
10	2,000	0.00612	0.0470	-	20 of 20	16 of 20	1.5	1.5	10	3	1
20	2,000	0.0166	0.263	16.0x	20 of 20	5 of 20	5	5.5	20	9	5
50	2,000	0.200	6.33	31.1x	20 of 20	0 of 20	12	15.5	50	14	3

The two collider rules agree unless wrong independence decisions make two colliders conflict. Then each keeps a different one, so the graphs differ more often as the graphs grow.

MultiGroupDirectLiNGAM#

MultiGroupDirectLiNGAM learns one causal order shared by several groups of data and a DAG for each group. Both packages fit three groups. The errors sum over the groups, so the empty graph’s are three times the true edges.

d	n	Andrey (s)	lingam (s)	Speedup	Datasets	Same graph	Errors, Andrey	Errors, lingam	Errors, empty graph
5	1,000	0.0539	0.0958	-	5 of 5	5 of 5	0	0	15
10	1,000	0.228	0.571	2.51x	5 of 5	5 of 5	0	0	30
20	1,000	1.22	4.04	3.30x	5 of 5	5 of 5	1	1	60

MultiGroupDirectLiNGAM returns the same graphs as lingam on every dataset.

GFCI#

GFCI runs GES, removes edges with independence tests, and orients a PAG with FCI’s rules. Both packages test with Fisher-Z at alpha 0.05 and score with a BIC penalty of 1. Tetrad runs the possible-d-separation step and the complete rule set. The table adds the observed variables, Andrey’s FCI on the same datasets, and the speed of Andrey’s parallel mode relative to one process.

d	n	Andrey (s)	Tetrad (s)	Speedup	Datasets	Same graph	Errors, Andrey	Errors, Tetrad	Errors, empty graph	Andrey ahead	Tetrad ahead	Observed	Errors, Andrey’s FCI	Parallel mode
20	200	0.0286	0.0355	-	5 of 5	0 of 5	21	22	48	4	0	18	23	0.905x
50	500	0.162	0.298	1.27x	5 of 5	0 of 5	30	51	100	4	1	45	46	0.987x
100	1,000	0.577	1.04	1.90x	5 of 5	0 of 5	86	114	218	5	0	90	118	0.894x
200	2,000	2.88	10.5	3.65x	5 of 5	0 of 5	197	325	444	5	0	180	354	0.810x

Andrey’s GFCI has fewer errors than Tetrad’s on 18 of 20 datasets and fewer than Andrey’s FCI at every size.

GIN#

GIN finds latent variables, each measured by a group of observed indicators, and orders the latents. Andrey’s GIN tests independence with HSIC and a gamma approximation. Tetrad’s runs its KCI test under its defaults, of the same family with its own bandwidth rule. causal-learn’s runs the same HSIC test as Andrey’s, with one fix: its GIN tested the columns at each loop counter instead of the variables it names (causal-learn issue 271). The tables give the latents each misses, the adjusted Rand index (ARI) of the clusters, and the share of latent pairs ordered right (“Order”).

Against Tetrad:

Latents (m)	Indicators	Andrey (s)	Tetrad (s)	Speedup	Datasets	Same result	Latents missed, Andrey	Latents missed, Tetrad	Latents missed, empty	ARI, Andrey	ARI, Tetrad	Order, Andrey	Order, Tetrad
2	6	0.905	72.1	79.7x	5 of 5	3 of 5	0	0	2	1.00	1.00	1.00	1.00
3	9	2.62	302	118x	5 of 5	1 of 5	0	0	3	1.00	1.00	1.00	0.667
4	12	5.30	805	146x	5 of 5	1 of 5	1	1	4	0.891	0.831	1.00	0.667

Against causal-learn:

Latents (m)	Indicators	Andrey (s)	causal-learn (s)	Speedup	Datasets	Same result	Latents missed, Andrey	Latents missed, causal-learn	Latents missed, empty	ARI, Andrey	ARI, causal-learn	Order, Andrey	Order, causal-learn
2	6	0.905	3.48	3.84x	5 of 5	3 of 5	0	0	2	1.00	1.00	1.00	1.00
3	9	2.62	14.6	5.58x	5 of 5	1 of 5	0	0	3	1.00	1.00	1.00	0.667
4	12	5.30	38.1	7.14x	5 of 5	1 of 5	1	1	4	0.891	0.891	1.00	0.333

CALM#

CALM learns a DAG by continuous optimization. causal-learn runs the authors’ implementation with its defaults, the paper’s schedule: up to 100 rounds of 40,000 steps. Andrey’s default schedule runs up to 25 rounds of up to 3,000 steps and drops weights under 0.3. We also run Andrey on the paper’s schedule.

With Andrey’s default schedule:

d	n	Andrey (s)	causal-learn (s)	Speedup	Datasets	Same graph	Errors, Andrey	Errors, causal-learn	Errors, empty graph	Andrey ahead	causal-learn ahead
10	100	17.9	498	26.2x	5 of 5	0 of 5	6	8	10	3	2
20	200	39.7	817	21.9x	5 of 5	0 of 5	8	12	20	4	1
50	500	59.0	1,150	19.4x	5 of 5	0 of 5	15	20	50	4	0

With the paper’s schedule in both packages:

d	n	Andrey (s)	causal-learn (s)	Speedup	Datasets	Same graph	Errors, Andrey	Errors, causal-learn	Errors, empty graph	Andrey ahead	causal-learn ahead
10	100	174	498	2.89x	5 of 5	0 of 5	6	8	10	3	2
20	200	282	817	2.95x	5 of 5	0 of 5	9	12	20	4	1
50	500	472	1,150	2.44x	5 of 5	0 of 5	12	20	50	4	0

CALM has fewer errors than causal-learn at every size with either schedule.

Limits#

Data coverage. We measure one simulated regime per method, with mean degree 2, up to 200 variables. These results do not establish performance on measured data or on other mechanisms.
GES at 200 variables. Both packages’ graphs have about as many errors as the empty graph (199 against 200), so that row’s speedup appears in its table only.
GIES on large graphs. The errors grow toward the empty graph’s with size: at 200 variables, they average 0.89 of the empty graph’s, and one dataset of five has more errors than the empty graph, in both packages. pcalg returns the same graphs.
GFCI’s parallel mode. Running its GES step in four worker processes was slower than one process at every size measured (0.810-0.987x).
GFCI’s order-free step. On the benchmark graphs, its order-free possible-d-separation step takes as long as the column-order mode (possible_dsep="fast"). On some graphs with 400 or more variables, its visiting order needs more tests: on one GFCI graph of mean degree 2 at 400 variables, it took 406-417 s against 120-336 s for the column-order mode across five column orders.
CALM’s time. At 50 variables, a fit takes about a minute with Andrey’s default schedule and about 8 minutes with the paper’s.
CD-NOD against causal-learn. The two orient conflicting colliders by different rules, so they return the same graph on 16 of 20 datasets at 10 variables and on none at 50.
GIN’s latent order. With 3 and 4 latents, Andrey’s result differs from both references’ on 8 of 10 datasets, in the order of the latents on every one. The rules differ: Andrey’s GIN runs each root test once per cluster member and takes the root candidate whose lowest pooled p-value is highest, and causal-learn’s and Tetrad’s take the first candidate in list order that passes. Andrey’s median order accuracy is 1 at 3 latents, against 0.667 for both references, and 1 at 4, against Tetrad’s 0.667 and causal-learn’s 0.333, but on 1 dataset Andrey leaves every latent unordered where both references order them.
Hardware. Ratios are specific to the measured hardware, core budget, and settings.

Reproduce#

You can find environment setup, campaign drivers, and scores in the benchmark harness guide. pcalg runs in the harness’s R environment, and Tetrad through the driver described in the guide. To rerun the comparisons, install Andrey 0.7.0 (andrey-core), causal-learn 0.1.4.8, lingam 1.13.0, pcalg 2.7.12, causal-cmd 1.12.0, and tetrad-lib 7.6.10, then run the drivers below. GIN’s causal-learn side takes the column-index patch the harness applies (benchmarks/patch_causal_learn.sh), and lingam runs in its own environment (benchmarks/start_lingam.sh). Every driver shuffles the columns by default and runs each fit on 4 cores with a 30,000 MB memory cap. Times will differ with the hardware and build; graphs should match closely.

Comparison	Driver	d	n	Datasets x repeats	Wall cap (s)	Warm-up
GES against causal-learn	`ges_cpu.py`	50, 100, 150, 200	10 per variable	5 x 1	1,800	1
GIES against pcalg	`gies_cpu.py`	10, 20, 50, 100, 200	10 per variable	5 x 1	1,800	1
Exact search against causal-learn	`exact_cpu.py`	5, 10, 15	10 per variable	5 x 1	1,800	1
CD-NOD against causal-learn	`cdnod_cpu.py`	10, 20, 50	2,000	20 x 1	1,800	1
MultiGroupDirectLiNGAM against lingam	`multi_group_cpu.py`	5, 10, 20	1,000	5 x 1	1,800	1
GFCI against Tetrad	`gfci_latent.py`	20, 50, 100, 200	10 per variable	5 x 1	1,800	1
GIN against Tetrad and causal-learn	`gin_cpu.py`	2, 3, 4 latents	1,000	5 x 1	1,800	0
CALM against causal-learn	`calm_cpu.py`	10, 20, 50	10 per variable	5 x 1	3,600	0

common="--cores 4 --repeats 1 --cap-mem-mb 30000"
for d in 50 100 150 200; do
  $ANDREY_BENCH_PYTHON benchmarks/ges_cpu.py --out outputs/ges/d$d --ladder $d --seeds 5 \
      --cap-wall-s 1800 $common
done
$ANDREY_BENCH_PYTHON benchmarks/gies_cpu.py --out outputs/gies --ladder 10 20 50 100 200 \
    --seeds 5 --warmup 1 --cap-wall-s 1800 $common
$ANDREY_BENCH_PYTHON benchmarks/exact_cpu.py --out outputs/exact --ladder 5 10 15 --seeds 5 \
    --warmup 1 --cap-wall-s 1800 $common
$ANDREY_BENCH_PYTHON benchmarks/cdnod_cpu.py --out outputs/cdnod --ladder 10 20 50 --seeds 20 \
    --n 2000 --warmup 1 --cap-wall-s 1800 $common
$ANDREY_BENCH_PYTHON benchmarks/multi_group_cpu.py --out outputs/multi_group --ladder 5 10 20 \
    --seeds 5 --n 1000 --warmup 1 --cap-wall-s 1800 $common
$ANDREY_BENCH_PYTHON benchmarks/gfci_latent.py --out outputs/gfci --ladder 20 50 100 200 \
    --seeds 5 --warmup 1 --cap-wall-s 1800 $common
$ANDREY_BENCH_PYTHON benchmarks/gin_cpu.py --out outputs/gin --ladder 2 3 4 --seeds 5 --n 1000 \
    --warmup 0 --cap-wall-s 1800 $common
for d in 10 20 50; do
  $ANDREY_BENCH_PYTHON benchmarks/calm_cpu.py --out outputs/calm/d$d --ladder $d --seeds 5 \
      --warmup 0 --cap-wall-s 3600 $common
done

calm_cpu.py runs Andrey’s CALM with its default schedule and with the paper’s in the same run. ges_cpu.py also runs gCastle, which we left out of the summary because a single fit above 50 variables passes the wall cap. The rows against Andrey 0.4.0 ran the released Andrey 0.4.0 in a separate environment on the same datasets, right after each 0.7.0 fit.