API reference

andrey.data.load_dataset#

andrey.data.load_dataset(name, *, n=None, seed=None, discrete=False, shuffle=True, as_frame=True, data_home=None)[source]#

Load a real dataset, or a published network with data simulated on it, and its graph.

The first load downloads the dataset’s file and keeps it in the cache directory; later loads read the cached copy and need no network. Every load checks the file’s SHA-256, so a changed or truncated file is downloaded again rather than read.

Parameters:

  • name (str) –

    The dataset; list_datasets() returns every name.

    • "sachs": the protein-signaling measurements of Sachs et al. (2005) [1], 853 observational cells by 11 proteins and phospholipids, with the 17-arc consensus network.

    • "asia" (8 variables, 8 arcs), "alarm" (37, 46), "hepar2" (70, 123), "andes" (223, 338), and "diabetes" (413, 602): published Bayesian networks [2]. The graph is the network’s structure; the data are simulated on it by a linear-Gaussian structural causal model, standardized, or, with discrete=True, sampled from the network’s own conditional probability tables.

  • n (int | None) – Rows to simulate on a published network; None gives 1000. Measured data such as "sachs" take no n.

  • seed (int | None) – Seed of the simulation’s weights and noise, and of the column order; None gives 0. The same seed and n give the same rows. Measured data take no seed.

  • discrete (bool) – Sample a published network from its own conditional probability tables instead of a linear-Gaussian model: each variable is drawn from its table’s row for its parents’ categories, in causal order. A DataFrame column holds the network’s category names as a pandas category, in the file’s order; an array holds the codes 0..r-1 in that order. Measured data take no discrete.

  • shuffle (bool) – List a published network’s variables in an order drawn from the seed, with feature_names and the graph’s labels in the same order. A network file lists its variables in its authors’ order, which often follows the causal order, and a method whose result depends on the column order would gain or lose accuracy from it. The rows are the same as with shuffle=False, which keeps the file’s order, for example to show the network as published. Measured data keep the file’s column order.

  • as_frame (bool) – Return the observations as a pandas DataFrame named by variable, so a method run on them names its nodes. False returns a float64 NumPy array in the same column order.

  • data_home (str | PathLike[str] | None) – The cache directory. None uses ANDREY_DATA_DIR when set, else $XDG_CACHE_HOME/andrey (~/.cache/andrey when XDG_CACHE_HOME is unset).

Returns:

data, graph (the reference DAG over the same names), feature_names, and description, which gives the source and the citation.

Return type:

RealDataset

Raises:

  • ValueError – If name is not a dataset, n, seed, or discrete is given for measured data, n is not an integer of at least 2, or seed is not a non-negative integer. These are checked before any download.

  • OSError – If no source gives the file with its recorded SHA-256, for example without network access. The message lists each URL tried and the path to put the file at by hand, and names a cached file that fails the check; that file is left in place.

Notes

Each dataset’s description gives its source, license, and citation.

Limitations. A sample from a network’s own tables can leave a category unseen, or a variable with one category, in few rows: some of these networks’ table rows put almost all their mass on one category. The categorical settings then read that variable as having fewer categories, and refuse a column that holds one value.

References

Examples

>>> from andrey.data import load_dataset
>>> sachs = load_dataset("sachs")
>>> sachs.data.shape, len(sachs.graph.oriented_edges())
((853, 11), 17)
>>> alarm = load_dataset("alarm", n=2000, seed=1)
>>> alarm.data.shape, len(alarm.graph.oriented_edges())
((2000, 37), 46)
>>> asia = load_dataset("asia", n=5000, discrete=True)
>>> asia.data["either"].cat.categories.tolist()
['yes', 'no']