{ "cells": [ { "cell_type": "markdown", "id": "quick_start-0", "metadata": {}, "source": [ "# Quick start\n", "\n", "[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Abel-ai-lab/andrey/blob/main/examples/quick_start.ipynb)\n", "\n", "You call every method in Andrey the same way: pass it your data, and read the result it returns.\n", "This notebook loads the protein measurements of Sachs et al. (2005), learns a graph from them\n", "with four classic methods, PC, GES, FCI, and DirectLiNGAM, reads each result, and draws each\n", "graph." ] }, { "cell_type": "markdown", "id": "quick_start-1", "metadata": {}, "source": [ "## Start\n", "\n", "Choose a CPU runtime and run this cell first. In Colab it installs `andrey-core==0.17.1` from PyPI;\n", "local kernels use the installed environment." ] }, { "cell_type": "code", "execution_count": 1, "id": "quick_start-2", "metadata": { "tags": [ "colab-setup" ] }, "outputs": [], "source": [ "import os\n", "import sys\n", "\n", "IN_COLAB = \"google.colab\" in sys.modules\n", "\n", "if IN_COLAB:\n", " os.environ[\"ANDREY_DEVICE\"] = \"cpu\"\n", " !pip install --quiet \"andrey-core[viz]==0.17.1\" pandas" ] }, { "cell_type": "code", "execution_count": 2, "id": "quick_start-3", "metadata": {}, "outputs": [], "source": [ "import andrey\n", "import andrey.viz\n", "from andrey.data import load_dataset" ] }, { "cell_type": "markdown", "id": "quick_start-4", "metadata": {}, "source": [ "## The data\n", "\n", "Sachs and colleagues (2005) measured 11 proteins and phospholipids in 853 single human immune\n", "cells. Biologists had already mapped how these molecules signal to one another, so a graph\n", "learned from the measurements can be checked against a known network of 17 links.\n", "`load_dataset` downloads the measurements on first use and caches them.\n", "\n", "Each row is one cell and each column one molecule. For your own data, pass a pandas DataFrame\n", "or a NumPy array of the same shape: one row per sample, one column per variable." ] }, { "cell_type": "code", "execution_count": 3, "id": "quick_start-5", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
rafmekplcpip2pip3erkaktpkapkcp38jnk
026.413.28.8218.3058.806.6117.0414.017.0044.940.0
135.916.512.3016.808.1318.6032.5352.03.3716.561.5
259.444.114.6010.2013.0014.9032.5403.011.4031.919.5
373.082.823.1013.501.295.8311.8528.013.7028.623.1
433.719.85.199.7324.8021.1046.1305.04.6625.781.3
\n", "
" ], "text/plain": [ " raf mek plc pip2 pip3 erk akt pka pkc p38 jnk\n", "0 26.4 13.2 8.82 18.30 58.80 6.61 17.0 414.0 17.00 44.9 40.0\n", "1 35.9 16.5 12.30 16.80 8.13 18.60 32.5 352.0 3.37 16.5 61.5\n", "2 59.4 44.1 14.60 10.20 13.00 14.90 32.5 403.0 11.40 31.9 19.5\n", "3 73.0 82.8 23.10 13.50 1.29 5.83 11.8 528.0 13.70 28.6 23.1\n", "4 33.7 19.8 5.19 9.73 24.80 21.10 46.1 305.0 4.66 25.7 81.3" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sachs = load_dataset(\"sachs\") # 853 cells, 11 proteins, and the known network\n", "data = sachs.data\n", "data.head()" ] }, { "cell_type": "markdown", "id": "quick_start-math", "metadata": {}, "source": [ "## In math\n", "\n", "Each column of `data` is a variable $X_j$, here $d = 11$ of them, and each row is one draw from a\n", "distribution $P$ over $V = \\{X_1, \\dots, X_d\\}$. We model each variable as a function of its\n", "parents in a directed acyclic graph (DAG) $G$, its direct causes, and of its own noise:\n", "\n", "$$X_j = f_j(X_{\\mathrm{pa}_G(j)}, N_j), \\qquad j = 1, \\dots, d,$$\n", "\n", "with the noise terms $N_j$ jointly independent, so no hidden variable causes two columns. A method\n", "estimates $G$, or the part of $G$ the data identify: PC and GES below return a class of DAGs, FCI\n", "drops the independence of the noise to allow hidden common causes, and DirectLiNGAM identifies $G$\n", "itself by assuming linear $f_j$ and non-Gaussian noise. The\n", "[concepts page](https://andrey.abel.ai/docs/guides/concepts.html) states each assumption and what\n", "each graph claims." ] }, { "cell_type": "markdown", "id": "quick_start-6", "metadata": {}, "source": [ "## PC\n", "\n", "PC starts with every pair of molecules linked. It removes a link when a statistical test finds\n", "the two independent, either alone or given some of their other neighbors, and orients a link\n", "only when a pattern in the tests forces its direction." ] }, { "cell_type": "code", "execution_count": 4, "id": "quick_start-7", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "pc cpdag | 11 nodes | 8 edges (6 undirected, 2 directed)\n", "\n", " raf -- mek\n", " plc -- pip3\n", " pip2 -- pip3\n", " erk -- akt\n", " erk -- pka\n", " akt -- pka\n", " p38 -> pkc\n", " jnk -> pkc\n", "\n", " indep_test: fisherz\n", " tie_break: rank\n" ] } ], "source": [ "pc = andrey.pc(data)\n", "print(pc)" ] }, { "cell_type": "markdown", "id": "quick_start-8", "metadata": {}, "source": [ "The first line names the method and the kind of graph. This one is a CPDAG: it stands for\n", "every causal graph that independence tests cannot tell apart. A directed edge (`->`) points\n", "the same way in all of them, and an undirected edge (`--`) can point either way.\n", "\n", "You can read the same parts in code. `pc.structure` is the graph: its `kind` says how to read\n", "the edges, and `oriented_edges()` lists each edge as (source, target, type), with your column\n", "names. `pc.metadata` records the settings of the run." ] }, { "cell_type": "code", "execution_count": 5, "id": "quick_start-9", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "kind: cpdag\n", "directed: [('p38', 'pkc', 'directed'), ('jnk', 'pkc', 'directed')]\n", "test: fisherz\n" ] } ], "source": [ "graph = pc.structure\n", "print(\"kind:\", graph.kind)\n", "print(\"directed:\", [edge for edge in graph.oriented_edges() if edge[2] == \"directed\"])\n", "print(\"test:\", pc.metadata[\"indep_test\"])" ] }, { "cell_type": "markdown", "id": "quick_start-10", "metadata": {}, "source": [ "`andrey.viz.draw` renders the graph as an SVG, which the notebook shows. A directed edge ends\n", "in an arrowhead; an undirected edge has none and is drawn in amber." ] }, { "cell_type": "code", "execution_count": 6, "id": "quick_start-11", "metadata": {}, "outputs": [ { "data": { "text/html": [ "rafmekplcpip2pip3erkaktpkapkcp38jnk" ], "text/plain": [ "'rafmekplcpip2pip3erkaktpkapkcp38jnk'" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "andrey.viz.draw(pc.structure)" ] }, { "cell_type": "markdown", "id": "quick_start-12", "metadata": {}, "source": [ "## GES\n", "\n", "GES, greedy equivalence search, reaches a graph a different way: it searches for the graph\n", "with the best score, instead of testing independence. You call it the same way." ] }, { "cell_type": "code", "execution_count": 7, "id": "quick_start-13", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "ges cpdag | 11 nodes | 8 edges (6 undirected, 2 directed)\n", "\n", " raf -- mek\n", " plc -- pip3\n", " pip2 -- pip3\n", " erk -- akt\n", " erk -- pka\n", " akt -- pka\n", " p38 -> pkc\n", " jnk -> pkc\n", "\n", " score: 6.692e+04\n", " score_func: bic\n", " penalty_discount: 1\n", " ebic_gamma: 1\n", " tie_break: rank\n" ] } ], "source": [ "ges = andrey.ges(data)\n", "print(ges)" ] }, { "cell_type": "markdown", "id": "quick_start-14", "metadata": {}, "source": [ "GES returns a CPDAG too, and on these data it is the same graph as PC's. Its metadata holds\n", "the score of the graph, where lower is better, and the settings that produced it." ] }, { "cell_type": "code", "execution_count": 8, "id": "quick_start-15", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "same edges as PC: True\n" ] }, { "data": { "text/html": [ "rafmekplcpip2pip3erkaktpkapkcp38jnk" ], "text/plain": [ "'rafmekplcpip2pip3erkaktpkapkcp38jnk'" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "print(\"same edges as PC:\", ges.structure.oriented_edges() == pc.structure.oriented_edges())\n", "andrey.viz.draw(ges.structure)" ] }, { "cell_type": "markdown", "id": "quick_start-16", "metadata": {}, "source": [ "## FCI\n", "\n", "PC and GES assume that every common cause of two measured variables is measured too. FCI does\n", "not: it allows hidden common causes, and says less about each edge in return." ] }, { "cell_type": "code", "execution_count": 9, "id": "quick_start-17", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "fci pag | 11 nodes | 8 edges (6 circle, 2 partially_directed)\n", "\n", " raf o-o mek\n", " plc o-o pip3\n", " pip2 o-o pip3\n", " erk o-o akt\n", " erk o-o pka\n", " akt o-o pka\n", " p38 o-> pkc\n", " jnk o-> pkc\n", "\n", " indep_test: fisherz\n", " tie_break: rank\n" ] } ], "source": [ "fci = andrey.fci(data)\n", "print(fci)" ] }, { "cell_type": "markdown", "id": "quick_start-18", "metadata": {}, "source": [ "FCI returns a PAG. A circle (`o`) at an end of an edge means the tests do not say whether that\n", "end is an arrowhead or a tail. So `p38 o-> pkc` says that pkc does not cause p38, and leaves\n", "open whether p38 causes pkc or a hidden variable causes both. The drawing shows each circle as\n", "a hollow circle." ] }, { "cell_type": "code", "execution_count": 10, "id": "quick_start-19", "metadata": {}, "outputs": [ { "data": { "text/html": [ "rafmekplcpip2pip3erkaktpkapkcp38jnk" ], "text/plain": [ "'rafmekplcpip2pip3erkaktpkapkcp38jnk'" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "andrey.viz.draw(fci.structure)" ] }, { "cell_type": "markdown", "id": "quick_start-20", "metadata": {}, "source": [ "## DirectLiNGAM\n", "\n", "DirectLiNGAM assumes that each variable is a linear function of its causes plus noise that is\n", "not Gaussian. When that holds, the data identify one graph, so it returns a DAG: every edge has\n", "a direction." ] }, { "cell_type": "code", "execution_count": 11, "id": "quick_start-21", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "direct_lingam dag | 11 nodes | 11 edges (11 directed)\n", "\n", " mek -> raf (+1.2)\n", " mek -> pip2 (+0.285)\n", " plc -> pip2 (+0.583)\n", " plc -> pip3 (+0.236)\n", " pip2 -> pip3 (+0.0952)\n", " erk -> akt (+1.37)\n", " erk -> pka (+1.84)\n", " pka -> akt (+0.017)\n", " pkc -> p38 (+1.28)\n", " jnk -> pkc (-0.0526)\n", " jnk -> p38 (+0.0582)\n", "\n", " order: erk -> plc -> mek -> pip2 -> pip3 -> pka -> jnk -> akt -> pkc -> p38 -> raf\n", "\n", " tie_break: rank\n" ] } ], "source": [ "lingam = andrey.direct_lingam(data)\n", "print(lingam)" ] }, { "cell_type": "markdown", "id": "quick_start-22", "metadata": {}, "source": [ "The number beside an edge is its weight, the coefficient of the cause in the linear equation\n", "for the effect, and `order` is the causal order the method found. These directions rest on the\n", "method's assumption, so treat them as hypotheses to test." ] }, { "cell_type": "code", "execution_count": 12, "id": "quick_start-23", "metadata": {}, "outputs": [ { "data": { "text/html": [ "rafmekplcpip2pip3erkaktpkapkcp38jnk" ], "text/plain": [ "'rafmekplcpip2pip3erkaktpkapkcp38jnk'" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "andrey.viz.draw(lingam.structure)" ] }, { "cell_type": "markdown", "id": "quick_start-24", "metadata": {}, "source": [ "## Next\n", "\n", "The [Methods notebooks](https://andrey.abel.ai/docs/examples/index.html#methods) each take one\n", "family of methods further: its other methods, its settings, and the data it suits." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.14.4" } }, "nbformat": 4, "nbformat_minor": 5 }