0.6.0 release card: mixed and categorical data#

Release card

Mixed and categorical data

0.6.0

We compare PC, GES, and hill climbing on categorical and mixed data with the packages that implement them: causal-learn and pgmpy in Python, bnlearn in R, and Tetrad in Java. On categorical data, GES with the BDeu score runs more than 50x faster than causal-learn at every size and returns the same graph on every dataset. Hill climbing runs more than 20x faster than pgmpy at every size. On mixed data, PC with the conditional Gaussian test runs more than 10x faster than Tetrad at 10 and 20 variables, and GES with the conditional Gaussian score more than 9x faster than Tetrad’s FGES at every size. We support these tests and scores in 0.6.0.

Speed is not accuracy. Andrey and pgmpy maximize the same hill climbing score, yet pgmpy’s graphs have fewer errors on more datasets (see Limits). You can find speedups and accuracy in the summary, and measurement and scoring definitions in Method. These results apply to the measured workloads. Speed and accuracy depend on the data, settings, and hardware.

Summary#

We compare each method with every package that implements it with the same test or score. We report every paired speed ratio, regardless of accuracy.

Speed.

Method	Data	Compared with	Variables (`d`)	Andrey speedup
PC, chi-square	Categorical	causal-learn	20-40	2.30-2.63x
PC, chi-square	Categorical	bnlearn	20-40	3.03-5.47x
PC, G-square	Categorical	causal-learn	20-40	2.65-2.76x
PC, G-square	Categorical	bnlearn	20-40	3.16-5.68x
GES, BDeu	Categorical	causal-learn	10-40	51.9-64.7x
Hill climbing, BDeu	Categorical	pgmpy	10-40	24.1-32.6x
Hill climbing, BDeu	Categorical	bnlearn	40	1.16x
Hill climbing, BIC	Categorical	pgmpy	10-40	20.5-36.3x
Hill climbing, BIC	Categorical	bnlearn	40	1.13x
PC, conditional Gaussian	Mixed	Tetrad	10-40	8.96-12.2x
PC, conditional Gaussian	Mixed	bnlearn	20-40	1.25-1.87x
PC, degenerate Gaussian	Mixed	Tetrad	10-40	261-1,160x
GES, conditional Gaussian	Mixed	Tetrad	10-40	9.51-15.8x
GES, degenerate Gaussian	Mixed	Tetrad	10-40	0.802-3.39x
Hill climbing, conditional Gaussian	Mixed	bnlearn	40	1.35x
Hill climbing, degenerate Gaussian	Mixed	bnlearn	40	2.00x

Accuracy.

Method	Data	Compared with	Datasets	Same graph	Andrey ahead	Other ahead
PC, chi-square	Categorical	causal-learn	15	15 of 15	0	0
PC, chi-square	Categorical	bnlearn	15	1 of 15	12	1
PC, G-square	Categorical	causal-learn	15	15 of 15	0	0
PC, G-square	Categorical	bnlearn	15	1 of 15	12	1
GES, BDeu	Categorical	causal-learn	15	15 of 15	0	0
Hill climbing, BDeu	Categorical	pgmpy	15	1 of 15	3	9
Hill climbing, BDeu	Categorical	bnlearn	15	15 of 15	0	0
Hill climbing, BIC	Categorical	pgmpy	15	1 of 15	6	8
Hill climbing, BIC	Categorical	bnlearn	15	15 of 15	0	0
PC, conditional Gaussian	Mixed	Tetrad	18	1 of 18	12	4
PC, conditional Gaussian	Mixed	bnlearn	30	1 of 30	26	3
PC, degenerate Gaussian	Mixed	Tetrad	25	1 of 25	19	2
GES, conditional Gaussian	Mixed	Tetrad	30	11 of 30	11	7
GES, degenerate Gaussian	Mixed	Tetrad	30	14 of 30	9	4
Hill climbing, conditional Gaussian	Mixed	bnlearn	30	17 of 30	12	0
Hill climbing, degenerate Gaussian	Mixed	bnlearn	30	3 of 30	-	-

“Ahead” counts the datasets on which that package’s graph has fewer errors than the other’s. “Other” is the compared package. On the rest, both have the same number of errors. We give no count for hill climbing with the degenerate Gaussian score. bnlearn has no such score, so it runs its conditional Gaussian search, which uses a different objective.

Method#

Data. Categorical data come from discrete_er: a discrete Bayesian network on an Erdos-Renyi DAG with mean degree 2 (andrey.data.sample_discrete). Every variable has 2 or 3 categories, and each row of its conditional probability table is drawn from a flat Dirichlet. Mixed data come from mixed_er: a conditional linear Gaussian network on the same kind of DAG (andrey.data.sample_mixed). The first half of the variables in causal order are categorical, and a categorical variable has only categorical parents. Each continuous variable is a linear function of its continuous parents, with a shift for each category of its categorical parents and Gaussian noise. d is the number of variables and n the number of rows, 2,000 throughout. There are 5 datasets per size of categorical data, one per seed, and 10 of mixed data: 5 seeds with 2 to 3 categories per categorical variable and 5 with 3 to 5.

Columns. Every package reads a column as categorical when it holds at most 10 distinct whole numbers: Andrey with max_categories=10, Tetrad with --numCategories 10, and bnlearn by turning such columns into factors. On these datasets, the rule identifies exactly the generated categorical columns.

Accuracy. Errors are the SHD between the estimate’s CPDAG and the true DAG’s CPDAG: one for each pair of variables whose edge is missing, extra, or oriented differently. We compare DAG estimates, which bnlearn’s and pgmpy’s hill climbing return, through their CPDAGs. The empty graph’s errors are the number of true edges. PC against bnlearn on mixed data counts skeleton errors only (see Mixed PC). Errors are medians over a size’s datasets. “Same graph” means the two packages’ CPDAGs are equal.

Timing. Times are seconds per fit, the median over a size’s datasets. For each dataset, we divide the other package’s time by Andrey’s, then report the median of these paired ratios. This need not equal the ratio of the displayed median times. We show no speedup for a size at which both packages’ median times are under 0.1 s, and leave it out of the summary. Tetrad’s time covers the search inside the Java virtual machine, excluding startup and data reading. Andrey’s time excludes Python’s startup and import. Times and ratios have three significant figures.

Hardware. Every fit ran on an Intel Core i7-7820X with 4 cores per fit. Fits ran one at a time, with 1 timed fit per dataset and a 30,000 MB memory cap. Categorical data had 1 untimed warm-up fit and an 1,800 s wall cap; mixed data had no warm-up and a 2,400 s cap.

Builds and versions#

Comparison	Andrey build	Measured package versions
PC and GES, mixed data	Pre-release 0.6.0, October 13, 2026	bnlearn 5.2.1, Tetrad (causal-cmd 1.12.0, tetrad-lib 7.6.3)
Hill climbing, mixed data	Pre-release 0.6.0, October 14, 2026	bnlearn 5.2.1
Categorical data	Pre-release 0.6.0, October 16, 2026	causal-learn 0.1.4.8, bnlearn 5.2.1, pgmpy 1.1.2

bnlearn ran in R 4.6.1, and Tetrad on OpenJDK 21.0.6. The summary contains one row per comparison and size. Each row gives median seconds, the paired ratio, the datasets both packages finished (paired), and those with the same graph (same_graph). It also gives each package’s errors beside the empty graph’s, the datasets on which each has fewer errors, and F1.

Categorical data#

Categorical PC#

PC tests whether two variables are independent given others, removes the edges its tests cannot keep, and orients what the remaining colliders allow. Andrey’s chi-square ("chisq") and G-square ("gsq") tests compare the counts of two variables with independence within each configuration of the conditioning set. Both packages test at alpha 0.05 with no limit on the conditioning set. bnlearn’s PC runs its -adf tests (x2-adf, mi-adf). These exclude empty rows and columns from a table’s degrees of freedom, as Andrey’s tests do.

With the chi-square test, against causal-learn:

d	Andrey (s)	causal-learn (s)	Speedup	Datasets	Same graph	Errors, Andrey	Errors, causal-learn	Errors, empty graph
10	0.0132	0.0390	-	5 of 5	5 of 5	5	5	10
20	0.0502	0.126	2.30x	5 of 5	5 of 5	7	7	20
40	0.178	0.469	2.63x	5 of 5	5 of 5	14	14	40

With the G-square test, against causal-learn:

d	Andrey (s)	causal-learn (s)	Speedup	Datasets	Same graph	Errors, Andrey	Errors, causal-learn	Errors, empty graph
10	0.0134	0.0437	-	5 of 5	5 of 5	5	5	10
20	0.0512	0.121	2.65x	5 of 5	5 of 5	7	7	20
40	0.175	0.469	2.76x	5 of 5	5 of 5	16	16	40

With the chi-square test, against bnlearn:

d	Andrey (s)	bnlearn (s)	Speedup	Datasets	Same graph	Errors, Andrey	Errors, bnlearn	Errors, empty graph	Andrey ahead	bnlearn ahead
10	0.0132	0.0633	-	5 of 5	1 of 5	5	7	10	3	0
20	0.0502	0.154	3.03x	5 of 5	0 of 5	7	10	20	4	1
40	0.178	0.957	5.47x	5 of 5	0 of 5	14	26	40	5	0

With the G-square test, against bnlearn:

d	Andrey (s)	bnlearn (s)	Speedup	Datasets	Same graph	Errors, Andrey	Errors, bnlearn	Errors, empty graph	Andrey ahead	bnlearn ahead
10	0.0134	0.0637	-	5 of 5	1 of 5	5	7	10	3	0
20	0.0512	0.140	3.16x	5 of 5	0 of 5	7	10	20	4	1
40	0.175	0.991	5.68x	5 of 5	0 of 5	16	24	40	5	0

PC returns the same graph as causal-learn on every dataset. bnlearn’s graphs differ for two reasons given in Limits.

Categorical GES#

GES searches over equivalence classes, adding, then removing, the edges that most improve the score. Both packages use the BDeu score with an equivalent sample size of 1 and at most 4 parents.

d	Andrey (s)	causal-learn (s)	Speedup	Datasets	Same graph	Errors, Andrey	Errors, causal-learn	Errors, empty graph
10	0.0231	1.67	64.7x	5 of 5	5 of 5	3	3	10
20	0.101	7.49	62.0x	5 of 5	5 of 5	4	4	20
40	0.595	29.7	51.9x	5 of 5	5 of 5	6	6	40

GES returns the same graph as causal-learn on every dataset.

Categorical hill climbing#

Hill climbing starts from the empty graph. It takes the best edge addition, removal, or reversal until none improves the score, with no parent limit and no restarts. With the BDeu score, every package uses an equivalent sample size of 1 and a structure prior of one expected parent per variable. pgmpy has no such prior, so the harness adds Andrey’s through pgmpy’s own hook. The BIC score penalizes log(n) / 2 per parameter in every package.

With the BDeu score, against pgmpy:

d	Andrey (s)	pgmpy (s)	Speedup	Datasets	Same graph	Errors, Andrey	Errors, pgmpy	Errors, empty graph	Andrey ahead	pgmpy ahead
10	0.0116	0.289	24.1x	5 of 5	1 of 5	5	1	10	0	4
20	0.0461	1.08	25.9x	5 of 5	0 of 5	8	4	20	2	3
40	0.151	5.05	32.6x	5 of 5	0 of 5	18	14	40	1	2

With the BIC score, against pgmpy:

d	Andrey (s)	pgmpy (s)	Speedup	Datasets	Same graph	Errors, Andrey	Errors, pgmpy	Errors, empty graph	Andrey ahead	pgmpy ahead
10	0.0141	0.289	20.5x	5 of 5	1 of 5	6	4	10	2	2
20	0.0378	1.16	31.8x	5 of 5	0 of 5	8	7	20	1	4
40	0.147	5.30	36.3x	5 of 5	0 of 5	20	18	40	3	2

With the BDeu score, against bnlearn:

d	Andrey (s)	bnlearn (s)	Speedup	Datasets	Same graph	Errors, Andrey	Errors, bnlearn	Errors, empty graph
10	0.0116	0.0459	-	5 of 5	5 of 5	5	5	10
20	0.0461	0.0688	-	5 of 5	5 of 5	8	8	20
40	0.151	0.164	1.16x	5 of 5	5 of 5	18	18	40

With the BIC score, against bnlearn:

d	Andrey (s)	bnlearn (s)	Speedup	Datasets	Same graph	Errors, Andrey	Errors, bnlearn	Errors, empty graph
10	0.0141	0.0461	-	5 of 5	5 of 5	6	6	10
20	0.0378	0.0670	-	5 of 5	5 of 5	8	8	20
40	0.147	0.150	1.13x	5 of 5	5 of 5	20	20	40

Hill climbing returns the same graph as bnlearn on every dataset. Against pgmpy, the two scores agree on every move, but the graphs differ on 28 of 30 datasets (see Limits).

Mixed data#

Mixed PC#

The conditional Gaussian test ("cond_gauss") fits the continuous variables within each combination of categories with a Gaussian of its own mean and covariance. The degenerate Gaussian test ("degen_gauss") replaces each categorical variable of k categories with k - 1 indicator columns and treats every column as Gaussian. Both compare the likelihoods of the models with and without the dependence. Andrey and Tetrad run order-independent PC at alpha 0.05 with no depth limit. Both keep the first of two conflicting colliders.

With the conditional Gaussian test, against Tetrad:

d	Andrey (s)	Tetrad (s)	Speedup	Datasets	Same graph	Errors, Andrey	Errors, Tetrad	Errors, empty graph	Andrey ahead	Tetrad ahead
10	0.0427	0.498	12.2x	8 of 10	1 of 8	5	4	10	3	3
20	0.131	1.36	10.0x	6 of 10	0 of 6	3.5	6.5	20	5	1
40	0.421	4.11	8.96x	4 of 10	0 of 4	8.5	17.5	40	4	0

With the conditional Gaussian test, against bnlearn:

d	Andrey (s)	bnlearn (s)	Speedup	Datasets	Same graph	Skeleton errors, Andrey	Skeleton errors, bnlearn	Skeleton errors, empty graph	Andrey ahead	bnlearn ahead
10	0.0427	0.0556	-	10 of 10	0 of 10	1.5	3.5	10	8	2
20	0.131	0.177	1.25x	10 of 10	1 of 10	1	5.5	20	9	0
40	0.421	0.760	1.87x	10 of 10	0 of 10	5	15	40	9	1

With the degenerate Gaussian test, against Tetrad:

d	Andrey (s)	Tetrad (s)	Speedup	Datasets	Same graph	Errors, Andrey	Errors, Tetrad	Errors, empty graph	Andrey ahead	Tetrad ahead
10	0.0421	11.8	261x	10 of 10	1 of 10	5	8	10	5	2
20	0.108	98.7	736x	10 of 10	0 of 10	7	26	20	9	0
40	0.376	505	1,160x	5 of 10	0 of 5	18	62	40	5	0

Tetrad’s PC with the conditional Gaussian test stopped with an error on 12 of 30 datasets because it could not evaluate a likelihood. We do not score those fits. With the degenerate Gaussian test, Tetrad exceeded the 2,400 s limit on 5 of 10 datasets at 40 variables. bnlearn’s pc.stable with its conditional Gaussian test forbids every edge from a continuous to a categorical variable. It orients each mixed edge by that rule. Andrey’s PC orients every edge from the data, so we count only skeleton errors in this comparison.

Mixed GES#

Andrey’s GES and Tetrad’s FGES search with the matching score, with a penalty of 1 and no structure prior.

With the conditional Gaussian score:

d	Andrey (s)	Tetrad (s)	Speedup	Datasets	Same graph	Errors, Andrey	Errors, Tetrad	Errors, empty graph	Andrey ahead	Tetrad ahead
10	0.0584	0.873	15.8x	10 of 10	6 of 10	1.5	1.5	10	2	1
20	0.242	2.28	9.77x	10 of 10	4 of 10	4	4	20	3	3
40	1.16	11.0	9.51x	10 of 10	1 of 10	6.5	6.5	40	6	3

With the degenerate Gaussian score:

d	Andrey (s)	Tetrad (s)	Speedup	Datasets	Same graph	Errors, Andrey	Errors, Tetrad	Errors, empty graph	Andrey ahead	Tetrad ahead
10	0.0416	0.158	3.39x	10 of 10	7 of 10	1.5	1.5	10	1	2
20	0.125	0.260	2.08x	10 of 10	5 of 10	2	2	20	4	0
40	0.749	0.574	0.802x	10 of 10	2 of 10	8	10	40	4	2

Mixed hill climbing#

Both packages search from the empty graph with no move limit and no restarts, penalize log(n) / 2 per parameter, and never give a categorical variable a continuous parent. bnlearn’s conditional Gaussian BIC is the comparison score for both of Andrey’s scores.

With the conditional Gaussian score:

d	Andrey (s)	bnlearn (s)	Speedup	Datasets	Same graph	Errors, Andrey	Errors, bnlearn	Errors, empty graph	Andrey ahead
10	0.0300	0.0228	-	10 of 10	7 of 10	3.5	3.5	10	3
20	0.0531	0.0635	-	10 of 10	7 of 10	7	7	20	2
40	0.142	0.191	1.35x	10 of 10	3 of 10	11	13	40	7

With the degenerate Gaussian score, against bnlearn’s conditional Gaussian search:

d	Andrey (s)	bnlearn (s)	Speedup	Datasets	Same graph	Errors, Andrey	Errors, bnlearn	Errors, empty graph	Andrey ahead	bnlearn ahead
10	0.0249	0.0228	-	10 of 10	1 of 10	3	3.5	10	-	-
20	0.0409	0.0635	-	10 of 10	2 of 10	6	7	20	-	-
40	0.0936	0.191	2.00x	10 of 10	0 of 10	12.5	13	40	-	-

The second comparison uses different objectives, so we count neither package as ahead.

Limits#

Data coverage. We measure one simulated regime for each kind of data, with mean degree 2 and 2,000 rows, up to 40 variables. These results do not establish performance on measured data, on more categories, or on other mechanisms.
Hill climbing against pgmpy. The two searches maximize the same score, but their graphs differ on 28 of 30 datasets: pgmpy’s has fewer errors on 17 and Andrey’s on 9. On those 17, the searches agree on every move until adding x -> y and adding y -> x tie exactly. Andrey breaks that tie by column order, and pgmpy by its own move order; from there, the two reach different local optima.
PC against bnlearn on categorical data. bnlearn’s -adf tests return p = 1 when a table has fewer than 5 rows per cell on average, so its PC removes edges that Andrey’s PC tests. At 2,000 rows, this starts at about four conditioning variables. On the same skeleton, the two also orient colliders differently: bnlearn keeps the first separating set it finds, and Andrey the union of the sets that separated the pair.
Tetrad’s PC. With the conditional Gaussian test, Tetrad stopped with an error on 12 of 30 datasets, so its speedups use 8, 6, and 4 datasets at 10, 20, and 40 variables. With the degenerate Gaussian test, Tetrad’s graphs have more errors than the empty graph on most datasets from 20 variables, and it exceeded the time limit on half of the datasets at 40. Tetrad’s degenerate Gaussian test statistic is twice the likelihood ratio.
GES with the degenerate Gaussian score. At 40 variables, Andrey’s GES is slower than Tetrad’s FGES (0.802x).
Hill climbing timing. On categorical data and on mixed data, hill climbing at 10 and 20 variables takes under 0.1 s in both Andrey and bnlearn, so those sizes show no speedup.
Hardware. Ratios are specific to the measured hardware, core budget, and settings.

Reproduce#

You can find environment setup, campaign drivers, and scores in the benchmark harness guide. bnlearn runs in the harness’s R environment. Tetrad runs through causal-cmd 1.12.0 with the driver described in the guide. To rerun the comparisons, install Andrey 0.6.0 (andrey-core), causal-learn 0.1.4.8, pgmpy 1.1.2, bnlearn 5.2.1, and causal-cmd 1.12.0, then run the drivers below. Times will differ with the hardware and build; graphs should match closely.

Comparison	Driver	d	n	Datasets x repeats	Wall cap (s)	Cores
Categorical data	`discrete_cpu.py`	10, 20, and 40	2,000	5 x 1	1,800	4
Mixed data	`mixed_cpu.py`	10, 20, and 40	2,000	10 x 1	2,400	4

$ANDREY_BENCH_PYTHON benchmarks/discrete_cpu.py --out outputs/discrete-cpu --cores 4 \
    --cap-mem-mb 30000
$ANDREY_BENCH_PYTHON benchmarks/mixed_cpu.py --out outputs/mixed-cpu --categories 2 3 \
    --warmup 0 --cap-wall-s 2400 --cap-mem-mb 30000 --cores 4
$ANDREY_BENCH_PYTHON benchmarks/mixed_cpu.py --out outputs/mixed-cpu-35 --categories 3 5 \
    --warmup 0 --cap-wall-s 2400 --cap-mem-mb 30000 --cores 4