Interventional data#

Cover: engraved dominoes, those before the amber one standing, those after it falling, drawn in navy lines

TL;DR#

  • What it is: data from experiments. Some rows are observed as they come; in others, a variable was changed from outside by an intervention. Where observation leaves the direction of an edge open, an experiment on either end can settle it.

  • What Andrey has: GIES and IGSP when you know what each experiment changed, GnIES and UT-IGSP when you do not, CD-NOD for data from several domains, and ICP for the causes of one variable, all taking one env and targets, and a simulator for interventions.

  • As accurate as the authors’ code: GnIES is worse than the authors’ package in none of 21 settings and about 34 times faster; UT-IGSP and IGSP are worse than the authors’ library in none of 112 and 114 comparisons; ICP accepts the same sets as the authors’ R package on every paired fit with two environments, 41 to 760 times faster.


An experiment answers a question that observation cannot: what happens to the rest of a system when one part of it is changed. Interventional data mix observed rows with rows from such experiments, and they show which way an effect runs.

Observational data#

A structural causal model writes each variable as a function of its parents and a noise term of its own. From observational data alone, graphs that differ in the direction of some edges imply the same independences, so PC and GES return those edges undirected. A functional causal model breaks the tie by assuming the shape of each effect. Interventional data break it with an experiment.

Interventions and the do operator#

An intervention changes the mechanism of one variable, its target, from outside, and leaves every other variable’s mechanism as it was. Pearl writes a hard intervention with the do operator: do(xj=a)\mathrm{do}(x_j = a) replaces the equation of xjx_j with xj=ax_j = a, so xjx_j no longer listens to its parents, and the graph loses every arrow into xjx_j (Pearl, 2009). A soft intervention changes the mechanism and keeps the parents: it shifts the mean of the noise, changes its spread, or changes the weights of the parents.

Intervening is not conditioning. P(y∣x=a)P(y \mid x = a) is the distribution of yy in the rows where xx happens to equal aa, and it carries every path between xx and yy, including a common cause. P(y∣do(x=a))P(y \mid \mathrm{do}(x = a)) is the distribution of yy under the intervention, and it changes only through what xx causes. Observational data give the first; an experiment gives the second.

Data from several experiments come in environments: the rows of one environment share one set of interventions, and its targets are the variables they change. An observational environment has no targets.

Take the smallest graph, x→yx \to y, with y=0.8 x+ey = 0.8\,x + e and xx and ee both Gaussian. The figure draws the observed rows (a), and the rows of two experiments: do(x)\mathrm{do}(x) (b) and do(y)\mathrm{do}(y) (c), each drawing its target from outside, centered at 2.5. Under (b) and (c), a two-sample test asks whether the other variable changed.

(a) Observationxyx -> y and y -> x fit alike(b) do(x)xyy changes, p < 0.001(c) do(y)xyx does not change, p = 0.34
One pair in which x causes y. In (b) and (c), the gray points are the observational rows of (a), and the blue points the rows of the intervention.

In (a), the data are a symmetric Gaussian cloud, and the models x→yx \to y and y→xy \to x fit them alike. No method can tell the direction from these rows.

In (b), do(x)\mathrm{do}(x) moves the cloud along the line: yy follows xx, and the test finds that yy changed. In (c), do(y)\mathrm{do}(y) moves the cloud up, but xx keeps its place and spread, and the test finds that xx did not change. An intervention changes what lies downstream of its target and leaves the rest alone, so an experiment on either end of the edge orients it.

With experiments on several variables, the data identify the graph up to a smaller class, its interventional equivalence class (Hauser and Bühlmann, 2012): every edge at a target is directed, and so is every edge those directions imply.

Interventional data in Andrey#

Observational data are one table, a row per sample and a column per variable. Interventional data add two things: the environment of each row, as one more column or an array, and the targets of each environment. Here are two rows of each of three environments: observed, do(x1=2)\mathrm{do}(x_1 = 2), and the noise of x3x_3 tripled.

data.py#
import pandas as pd

import andrey
import andrey.data as data
from andrey.data.interventions import hard, soft

if __name__ == "__main__":  # worker processes re-import this script
    scm = data.SCM(
        graph=data.graphs.erdos_renyi(d=4, avg_degree=2),
        functional=data.functional.linear(),
        noise=data.noise.gaussian(),
    )
    # Three environments: observed, do(x1 = 2), and the noise of x3 tripled.
    envs = [{}, {1: hard(2.0)}, {3: soft(noise_scale=3.0)}]
    ds = scm.sample(n=[500, 500, 500], seed=0, envs=envs)

    # One table: a column per variable, and a column naming each row's environment.
    names = {0: "observed", 1: "do(x1 = 2)", 2: "noise x3"}
    df = pd.DataFrame(ds.data, columns=["x0", "x1", "x2", "x3"]).round(2)
    df["env"] = [names[e] for e in ds.env]
    print(df.groupby("env", sort=False).head(2).to_string())

    # And each environment's targets, by column name.
    targets = {"observed": [], "do(x1 = 2)": ["x1"], "noise x3": ["x3"]}
    result = andrey.gies(df, "env", targets=targets)
    print(result.structure.oriented_edges())
data.py#
        x0    x1    x2    x3         env
0     1.54  1.70  1.91 -1.31    observed
1     0.44  0.69 -0.29  0.03    observed
500  -1.06  2.00  1.89 -1.98  do(x1 = 2)
501  -0.62  2.00  1.20 -2.11  do(x1 = 2)
1000 -0.54 -0.46  0.30 -1.93    noise x3
1001  0.04  0.84  0.07 -0.63    noise x3
[('x0', 'x1', 'directed'), ('x0', 'x3', 'directed'), ('x1', 'x2', 'directed'), ('x1', 'x3', 'directed')]

Under do(x1=2)\mathrm{do}(x_1 = 2), x1x_1 is 2.00 in every row, and GIES recovers the true graph.

data is a pandas DataFrame or a NumPy array, or the environments’ tables apart. env and targets take these forms:

Argument

Form

Meaning

env

an array, one label per row

each row’s environment; the labels are strings or numbers, not both

env

a column name, such as "env"

the DataFrame column that holds the labels; it is not a variable

targets

a mapping {label: [variables]}

each environment’s targets: [] for an observational environment, a list for known targets, None for unknown ones

targets

a column name

a DataFrame column whose cells give each row’s targets, empty for an observational row

targets

a 0/1 array of shape (rows, variables)

1 where the row’s environment intervenes on the variable

data

a list or dict of tables, one per environment

each environment’s rows apart, labeled by position in a list and by key in a dict; env is left out, and targets maps those labels

Variables are named by column label or 0-based position. With targets as a column or an array, env can be left out: the rows with the same targets form one environment. GIES and IGSP need every environment’s targets; GnIES and UT-IGSP take None for the ones you do not know and learn them; ICP needs no targets but uses them, leaving out the environments that intervened on its response; CD-NOD takes env alone. GIES, IGSP, GnIES, UT-IGSP, CD-NOD, and ICP also take the tables apart:

andrey.gies([observed, knockout], targets={0: [], 1: ["x"]})

Targets known and unknown#

An experiment does not always say what it changed. A drug can act on more than the protein it was made for, and a gene knockout can have side effects. Each method in Andrey assumes its own kind of experiment and learns what it is not told:

Method

Data

Targets

Output

GIES

an observational environment and experiments

known

interventional essential graph

IGSP

an observational environment and experiments

known

interventional essential graph

GnIES

environments that differ in the spread of some variables’ noise

learned

interventional essential graph and the targets

UT-IGSP

an observational environment and experiments

known in part, the rest learned

interventional essential graph and each experiment’s targets

CD-NOD

domains whose mechanisms differ

not needed

CPDAG

ICP

environments that leave one variable’s mechanism alone

not needed

that variable’s causal parents

GIES (Hauser and Bühlmann, 2012) runs GES’s search over interventional equivalence classes, scoring the rows of each environment with the targets left out of their own fit. IGSP (Wang, Solus, Yang, and Uhler, 2017) searches variable orders for the sparsest graph the tests allow, and uses each experiment to decide which directions it supports.

GnIES (Gamella, Taeb, Heinze-Deml, and Bühlmann, 2022) learns the targets with the graph, when every experiment changes the spread of its targets’ noise. It fits one set of targets for all environments: the variables whose noise differs in some environment. UT-IGSP (Squires, Wang, and Uhler, 2020) takes the targets you know and learns the rest, for each experiment, from tests of which variables’ mechanisms changed. It accepts any change of mechanism: a shift, a new spread, new weights, or a hard intervention.

CD-NOD (Huang et al., 2020) reads the environment as one more variable that can cause the others, and needs no targets. ICP (Peters, Bühlmann, and Meinshausen, 2016) asks a narrower question: which variables are the causes of one response. It accepts the sets of variables whose regression of the response stays the same in every environment, and returns the variables in all of them.

A simulated example#

We simulate ten variables on a known graph: 1,000 observed rows, and 1,000 rows from each of three experiments that triple the noise of one variable. We ask GES for the graph from the observed rows, GIES with the targets given, and GnIES and UT-IGSP without them.

example.py#
import andrey
import andrey.data as data
from andrey.data.interventions import soft
from andrey.metrics import shd

if __name__ == "__main__":  # worker processes re-import this script
    # Ten variables on a known graph, observed in environment 0 and changed by one intervention
    # each in environments 1 to 3: the noise of one variable grows threefold.
    scm = data.SCM(
        graph=data.graphs.erdos_renyi(d=10, avg_degree=2),
        functional=data.functional.linear(),
        noise=data.noise.gaussian(),
    )
    envs = [{}, {2: soft(noise_scale=3.0)}, {5: soft(noise_scale=3.0)}, {7: soft(noise_scale=3.0)}]
    truth = scm.sample(n=[1000] * 4, seed=0, envs=envs)
    print(f"true graph: {len(truth.graph.oriented_edges())} edges, targets {truth.targets}")

    def undirected(result):
        return sum(kind == "undirected" for *_, kind in result.structure.oriented_edges())

    # GES on the observational rows alone returns the equivalence class.
    observed = truth.data[truth.env == 0]
    ges = andrey.ges(observed)
    print(f"ges:     {shd(ges.structure, truth.graph)} errors, {undirected(ges)} edges undirected")

    # GIES is told each environment's targets.
    gies = andrey.gies(truth.data, truth.env, targets=truth.targets)
    print(
        f"gies:    {shd(gies.structure, truth.graph)} errors, {undirected(gies)} edges undirected"
    )

    # GnIES and UT-IGSP are not told the targets, and learn them. GnIES learns the variables
    # whose noise changes in some environment; UT-IGSP learns each environment's targets.
    gnies = andrey.gnies(truth.data, truth.env)
    print(
        f"gnies:   {shd(gnies.structure, truth.graph)} errors, {undirected(gnies)} edges "
        f"undirected, targets {gnies.metadata['changing_variables']}"
    )
    unknown = {0: [], 1: None, 2: None, 3: None}
    ut_igsp = andrey.ut_igsp(truth.data, truth.env, targets=unknown)
    print(
        f"ut_igsp: {shd(ut_igsp.structure, truth.graph)} errors, {undirected(ut_igsp)} edges "
        f"undirected, targets {ut_igsp.metadata['targets']}"
    )
example.py#
true graph: 10 edges, targets {0: [], 1: [2], 2: [5], 3: [7]}
ges:     2 errors, 2 edges undirected
gies:    0 errors, 0 edges undirected
gnies:   0 errors, 0 edges undirected, targets [2, 5, 7]
ut_igsp: 0 errors, 0 edges undirected, targets {0: [], 1: [2], 2: [5], 3: [7]}

GES finds the right connections but leaves two of them undirected: the observed rows cannot orient them. GIES, told the targets, recovers the true graph. GnIES and UT-IGSP recover it too, and name the three targets they were not told.

Simulating interventions#

andrey.data draws interventional data. hard(value) is do(xj=value)\mathrm{do}(x_j = \text{value}): it fixes the variable and cuts it from its parents; soft(shift=..., weight_scale=..., noise_scale=...) changes its mechanism and keeps its parents. SCM.do is the do operator: it applies them to a model and returns the intervened model. envs= on SCM.sample draws several environments, each with its own interventions, into one dataset whose env and targets the methods take directly:

import andrey.data as data
from andrey.data.interventions import hard, soft

scm = data.SCM(graph=data.graphs.erdos_renyi(d=8, avg_degree=2),
               functional=data.functional.linear(),
               noise=data.noise.gaussian())
ds = scm.sample(n=[500, 200, 200], seed=0, envs=[{}, {3: hard(2.0)}, {5: soft(shift=1.0)}])
ds.targets  # {0: [], 1: [3], 2: [5]}

Checked against the authors’ code#

We compared each method with its authors’ code on simulated data from the experiments it assumes.

  • GnIES: the authors’ gnies package. Worse in none of 21 settings, better in 2 on fresh data, and about 34 times faster.

  • UT-IGSP: the authors’ library. Worse in none of 112 comparisons. It is 1.7 times slower per fit at 20 variables; at 100 variables it takes 15 s against the library’s 17 s.

  • IGSP: the authors’ library, and a copy of it that follows the paper’s definition where a setting has two or more targets. Worse in none of 114 comparisons at Andrey’s defaults, and in none of 684 at three fixed levels. It is slower in 84 of 87 settings, up to 8.0 times per fit, at most about 2 s at these sizes.

  • ICP: the authors’ R package. The same accepted sets on all 5,220 paired fits with two environments, and every other difference traced to a documented choice. 41 to 760 times faster.

The release card has every comparison.

Limits#

The experiments must fit the method. GnIES assumes linear effects, Gaussian noise, and experiments that change only the spread of their targets’ noise; UT-IGSP and IGSP assume that the observational environment is among the data.

Few rows per experiment. With 10 rows per environment, GnIES beats the empty graph in some settings and not in others, and so does the authors’ package.

Dense graphs. At 50 variables with mean degree 5, neither UT-IGSP nor the authors’ library makes fewer errors than the empty graph.

Interventions on ICP’s response. ICP’s guarantee needs environments that leave its response’s mechanism alone. Where an experiment hits the response, a set holding one of its children can look invariant, and the share of fits that name a wrong parent rises to 0.18, in the authors’ package too. Pass the targets you know, and ICP leaves those environments out.

Experiments that change nothing. An intervention can leave a variable without causes as it was, for example by drawing its noise again from the same distribution; such an experiment says nothing about the arcs out of it. IGSP still marks those arcs and reverses about one in five, as the authors’ library does; GIES reverses almost none.

Try it#

pip install -U andrey-core

The score-based search example runs GnIES, and the permutation-based search example runs IGSP and UT-IGSP.

Help us build it#

Andrey is in alpha. The API may change between releases. When you try it on your own data, tell us what breaks or which method you need. We also welcome pull requests. The contributing guide explains how you can set up the development environment.

References#

  • Gamella, J. L., Taeb, A., Heinze-Deml, C., and Bühlmann, P. (2022). Characterization and greedy learning of Gaussian structural causal models under unknown interventions. arXiv:2211.14897.

  • Hauser, A., and Bühlmann, P. (2012). Characterization and greedy learning of interventional Markov equivalence classes of directed acyclic graphs. JMLR 13, 2409-2464. jmlr.org.

  • Huang, B., Zhang, K., Zhang, J., Ramsey, J., Sanchez-Romero, R., Glymour, C., and Schölkopf, B. (2020). Causal discovery from heterogeneous/nonstationary data. JMLR 21(89), 1-53. jmlr.org.

  • Pearl, J. (2009). Causality: Models, Reasoning, and Inference, 2nd ed. Cambridge University Press. doi:10.1017/CBO9780511803161.

  • Peters, J., Bühlmann, P., and Meinshausen, N. (2016). Causal inference by using invariant prediction: identification and confidence intervals. Journal of the Royal Statistical Society Series B 78(5), 947-1012. doi:10.1111/rssb.12167.

  • Squires, C., Wang, Y., and Uhler, C. (2020). Permutation-based causal structure learning with unknown intervention targets. UAI, PMLR 124, 1039-1048. proceedings.mlr.press.

  • Wang, Y., Solus, L., Yang, K. D., and Uhler, C. (2017). Permutation-based causal inference algorithms with interventions. NeurIPS 30. proceedings.neurips.cc.