Stage_0

{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Process Mining with sberpm" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The __`sberpm`__ python library is designed to analyze business processes and solve related problems using machine learning and process mining techniques." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Table of Contents" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[DataHolder](#DataHolder)\n", "\n", "I. [Miners and graph visualization](#I.-Miners-and-graph-visualization)\n", " 1. [SimpleMiner](#1.-SimpleMiner)\n", " 2. [CausalMiner](#2.-CausalMiner)\n", " 3. [HeuMiner](#3.-HeuMiner)\n", " 4. [AlphaMiner](#4.-AlphaMiner)\n", " 5. [AlphaPlusMiner](#5.-AlphaPlusMiner)\n", " 6. [InductiveMiner](#6.-InductiveMiner)\n", " \n", "II. [Metrics](#II.-Metrics)\n", "- [Metrics + Graphs](#Metrics-+-Graphs)\n", "\n", "III. [BPMN](#III.-BPMN)\n", "\n", "[Machine Learning](#Machine-Learning)\n", "\n", "IV. [Process Vectorizing](#IV.-Process-Vectorizing)\n", "\n", "V. [Graph Embedding](#V.-Graph-Embedding)\n", "\n", "VI. [Clustering](#V.-Clustering)\n", "\n", "VII. [Autoinsights](#VI.-Autoinsights)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# DataHolder" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The data of the business process should be an event log which stores the information about the sequence of activities in the process. Example of an event log: $W = \\{(a, b, c, d), (a, c, b, d), (a, e, d) \\}$, where the activities $ a $, $ b $, $ c $, $ d $ and $ e $ are sorted by their timestamps." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`DataHolder` is the data structure that contains the event log and its metainformation. It is used as an input argument by many PM algorithms.\n", "\n", "In order to create a `DataHolder` object, it is possible either to specify a path to a log-file in the `DataHolder`'s constructor or provide the log in the form of a pandas DataFrame. It is also obligatory to set an __id_column__ and an __activity_column__. Moreover, for most Process Mining algorithms present in the library at least one of the timestamp columns (__start_timestamp_column__, __end_timestamp_column__) and a user column (__user_column__) are needed. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Parameters of DataHolder \n", "\n", "- **data (str or pd.DataFrame)** – path to a log-file (.csv, .xls(x), .txt) or pandas.DataFrame \n", "- **id_column (str)** – column that represents an ID of the event trace/case\n", "- **activity_column (str)** – column that represents an activity\n", "- __*start_timestamp_column (str)__ – timestamp that represents the beginning of activity execution\n", "- __*end_timestamp_column (str)__ – timestamp that represents the end of activity execution\n", "- __user_column (str)__ – column that represents the users who performed tasks in the process\n", "- __text_column (str)__ – column with the textual data (e.g., comments)\n", "- __duration_column (str)__ – column that contains the activity durations (If not given, it is calculated as *end_timestamp_column - start_timestamp_column*. If only one timestamp column is given, it is calculated as a difference between the timestamp of two consecutive activities, this leads to the duration of either the first or the last activity in the event trace to be NaN).\n", "- __duration_unit (str)__ – unit of the values in a *duration_column* (needed only if the *duration_column* is given)\n", "\n", "- __sep (str, default=',')__ – delimiter (used only if reading the log from file)\n", "- __encoding (str)__ – encoding (used only if reading the log from file)\n", "- __nrows (int)__ – number of rows to read from file (used only if reading the log from file)\n", "\n", "- __preprocess (bool, default=True)__ – preprocessing of the data (sorting, removing/filling None-values, timestamp conversion), so that the library algorithms are able to work with the data\n", "- __time_format (str)__ – format of the timestamp columns (strongly recommended to specify this parameter for correct time conversion). Time format syntax: https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes)\n", "- __num_cpu (int, default=1)__ – maximum number of processors to use (for some operations that support multiprocessing)\n", "\n", "\n", "__*__ For most algorithms it is needed to specify at least one of the timestamp columns. If it is unknown whether the timestamp column of the event log represents the start or end timestamp, it can be set as a __start_timestamp_column__. It is strongly recommended to specify __time_format__ parameter for correct time conversion." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Creation of DataHolder Object" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### – using pandas DataFrame" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "pycharm": { "is_executing": true, "name": "#%%\n" } }, "outputs": [], "source": [ "from sberpm import DataHolder\n", "import pandas as pd\n", "\n", "df = pd.DataFrame({\n", " 'id_column': [1, 1, 2, 2, 3, 3],\n", " 'activity_column': ['st1', 'st2', 'st1', 'st3', 'st1','st2'],\n", " 'start_timestamp_column': ['10.05.2020', '10.09.2020', '10.03.2020', '10.04.2020', '10.05.2020', '10.05.2020']\n", "})\n", "\n", "data_holder = DataHolder(data=df, \n", " id_column='id_column', \n", " activity_column='activity_column', \n", " start_timestamp_column='start_timestamp_column', \n", " time_format='%d.%m.%Y')" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "#### – using file path" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "pycharm": { "is_executing": true, "name": "#%%\n" } }, "outputs": [], "source": [ "path = 'example.csv'\n", "data_holder = DataHolder(data=path, \n", " id_column='id', \n", " activity_column='stages', \n", " start_timestamp_column='dt', \n", " user_column='users', \n", " time_format='%Y-%m-%d')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When reading a log from a file, one can specify the parameters like __sep__, __encoding__, __nrows__ in the constructor of `DataHolder` if needed." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Attributes of DataHolder\n", "`DataHolder` stores the names of the key columns of the event log and gives an easy access to them:\n", "- id_column\n", "- activity_column\n", "- start_timestamp_column\n", "- end_timestamp_column\n", "- user_column\n", "- text_column\n", "- duration_column\n", "\n", "Moreover, `DataHolder` stores the original preprocessed log and its version grouped by *id_column* with some other columns aggregated to tuples (calculated in a lazy way when needed by some algorithms or if one calls a specific method) in the form of pandas DataFrames:\n", "- data\n", "- grouped_data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Methods of DataHolder\n", "- __check_or_calc_duration( )__ – calculate the execution time of each activity (in seconds) if not calculated before\n", "- __get_grouped_data(...)__ – return the data (log) grouped by *id_column* with the given columns aggregated to tuples (e.g., activity_column or start_timestamp_column)\n", "- __get_unique_activities( )__ – return the list of unique activities\n", "- __get_columns( )__ – return the names of the columns in the event log\n", "- __get_text( )__ – return the name of the column with the textual data if it was specified" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "data": { "text/html": [ "

\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "

	id	stages	dt	users
0	0.0	Stage_0	2019-12-01 00:00:00+00:00	Oliver
1	0.0	Stage_3	2019-12-02 00:00:00+00:00	Liam
2	0.0	Stage_5	2019-12-05 00:00:00+00:00	Emma
3	0.0	Stage_2	2019-12-11 00:00:00+00:00	Patricia
4	0.0	Stage_7	2019-12-17 00:00:00+00:00	Callum
...	...	...	...	...
95	116.0	Stage_3	2019-12-15 00:00:00+00:00	Noah
96	116.0	Stage_9	2019-12-20 00:00:00+00:00	Barbara
97	117.0	Stage_0	2019-12-01 00:00:00+00:00	Oliver
98	117.0	Stage_1	2019-12-08 00:00:00+00:00	Thomas
99	117.0	Stage_3	2019-12-14 00:00:00+00:00	Amelia

\n", "

100 rows × 4 columns

\n", "

" ], "text/plain": [ " id stages dt users\n", "0 0.0 Stage_0 2019-12-01 00:00:00+00:00 Oliver\n", "1 0.0 Stage_3 2019-12-02 00:00:00+00:00 Liam\n", "2 0.0 Stage_5 2019-12-05 00:00:00+00:00 Emma\n", "3 0.0 Stage_2 2019-12-11 00:00:00+00:00 Patricia\n", "4 0.0 Stage_7 2019-12-17 00:00:00+00:00 Callum\n", ".. ... ... ... ...\n", "95 116.0 Stage_3 2019-12-15 00:00:00+00:00 Noah\n", "96 116.0 Stage_9 2019-12-20 00:00:00+00:00 Barbara\n", "97 117.0 Stage_0 2019-12-01 00:00:00+00:00 Oliver\n", "98 117.0 Stage_1 2019-12-08 00:00:00+00:00 Thomas\n", "99 117.0 Stage_3 2019-12-14 00:00:00+00:00 Amelia\n", "\n", "[100 rows x 4 columns]" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data_holder.data.head(100)" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [], "source": [ "data_holder.check_or_calc_duration()" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "data": { "text/html": [ "

	stages	dt	users	duration
0	Stage_0	2019-12-01 00:00:00+00:00	Oliver	86400.0
1	Stage_3	2019-12-02 00:00:00+00:00	Liam	259200.0
2	Stage_5	2019-12-05 00:00:00+00:00	Emma	518400.0

\n", "

" ], "text/plain": [ " id stages dt users duration\n", "0 0.0 Stage_0 2019-12-01 00:00:00+00:00 Oliver 86400.0\n", "1 0.0 Stage_3 2019-12-02 00:00:00+00:00 Liam 259200.0\n", "2 0.0 Stage_5 2019-12-05 00:00:00+00:00 Emma 518400.0" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data_holder.data.head(3)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "data": { "text/html": [ "

	id	stages	dt
0	0.0	(Stage_0, Stage_3, Stage_5, Stage_2, Stage_7)	(2019-12-01 00:00:00+00:00, 2019-12-02 00:00:0...
1	1.0	(Stage_0, Stage_2, Stage_4, Stage_3, Stage_8)	(2019-12-01 00:00:00+00:00, 2019-12-06 00:00:0...
2	10.0	(Stage_0, Stage_2, Stage_5, Stage_7)	(2019-12-01 00:00:00+00:00, 2019-12-06 00:00:0...
3	100.0	(Stage_2, Stage_3, Stage_8)	(2019-12-01 00:00:00+00:00, 2019-12-09 00:00:0...
4	101.0	(Stage_0, Stage_2, Stage_5, Stage_3, Stage_7, ...	(2019-12-01 00:00:00+00:00, 2019-12-10 00:00:0...

\n", "

" ], "text/plain": [ " id stages \\\n", "0 0.0 (Stage_0, Stage_3, Stage_5, Stage_2, Stage_7) \n", "1 1.0 (Stage_0, Stage_2, Stage_4, Stage_3, Stage_8) \n", "2 10.0 (Stage_0, Stage_2, Stage_5, Stage_7) \n", "3 100.0 (Stage_2, Stage_3, Stage_8) \n", "4 101.0 (Stage_0, Stage_2, Stage_5, Stage_3, Stage_7, ... \n", "\n", " dt \n", "0 (2019-12-01 00:00:00+00:00, 2019-12-02 00:00:0... \n", "1 (2019-12-01 00:00:00+00:00, 2019-12-06 00:00:0... \n", "2 (2019-12-01 00:00:00+00:00, 2019-12-06 00:00:0... \n", "3 (2019-12-01 00:00:00+00:00, 2019-12-09 00:00:0... \n", "4 (2019-12-01 00:00:00+00:00, 2019-12-10 00:00:0... " ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data_holder.get_grouped_data(data_holder.activity_column, data_holder.start_timestamp_column).head()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# I. Miners and graph visualization" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Having the event log of a business process, one can build the process model in the form of a graph by using the algorithms called *miners* and visualize it." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In order to do this, the library provides a number of algorithms that are stored in the module __`sberpm.miners`__ and they all have the only method:\n", "- __apply( )__ – create a graph of the process that is stored in the *graph* attribute" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1. SimpleMiner" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`SimpleMiner` displays all edges found in the log (no filtering is applied).\n", "\n", "Using the formal Process Mining language:\n", "> The situation when activity $X$ is directly followed by $Y$ in at least one trace in the event log ($...XY...$) is called the _follows_ relation and denoted as $X>Y$ ($Y$ follows $X$).\n", "\n", "SimpleMiner depicts the edges between the activities $X$ and $Y$, if $X>Y$." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [], "source": [ "from sberpm.miners import SimpleMiner" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [], "source": [ "# Creation of the miner object. It takes DataHolder and the parameters of the algorithm\n", "# (this miner does not have any additional parameters).\n", "simple_miner = SimpleMiner(data_holder)\n", "\n", "# Calculation.\n", "simple_miner.apply()\n", "\n", "# Getting the result\n", "graph = simple_miner.graph" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Visualization of the Graph\n", "For the purpose of visualization `GraphvizPainter` from the module __`sberpm.visual`__ should be used." ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "pycharm": { "is_executing": true, "name": "#%%\n" } }, "outputs": [], "source": [ "%matplotlib inline\n", "from sberpm.visual import GraphvizPainter" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Class `GraphvizPainter` has the following methods:\n", "- __apply(...)__ – take a graph and prepare it for being visualized\n", "- __write_graph(...)__ – save the graph in the given format (pdf, svg, gv, png)\n", "- __show( )__ – show the graph in Jupyter Notebook" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "pycharm": { "is_executing": true, "name": "#%%\n" }, "scrolled": true }, "outputs": [ { "data": { "text/html": [ "\r\n", "\r\n", "\r\n", "\r\n", "\r\n" ], "text/plain": [ "" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "painter = GraphvizPainter()\n", "painter.apply(graph)\n", "#painter.write_graph(\"causal_miner.png\", format='png')\n", "painter.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2. CausalMiner" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`CausalMiner` filters the edges.\n", "> Other types of relations derived from $X>Y$:\n", "- _causal_ relation ($X \\to Y$) – if $Х>Y$ and not $Y>X$\n", "- _parallel_ relation ($X\\parallel Y $) – if $Х>Y$ and $Y>X$\n", "- _independent_ ($X\\#Y$) – if not $X>Y$ and not $Y>X$\n", "\n", "CausalMiner depicts the edge between $X$ and $Y$if $X\\to Y$." ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "pycharm": { "is_executing": true, "name": "#%% md\n" } }, "outputs": [], "source": [ "from sberpm.miners import CausalMiner" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "data": { "text/html": [ "\r\n", "\r\n", "\r\n", "\r\n", "\r\n" ], "text/plain": [ "" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Miner\n", "causal_miner = CausalMiner(data_holder)\n", "causal_miner.apply()\n", "graph = causal_miner.graph\n", "\n", "# Visualization\n", "painter = GraphvizPainter()\n", "painter.apply(graph)\n", "painter.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3. HeuMiner" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`HeuMiner` is a Heuristic Miner. It removes *unimportant* edges whose *unimportance* is below the given threshold. \n", "The **threshold** parameter must be **between 0 and 1**. The bigger it is, the less edges are depicted (the remaining ones are considered to be more important).\n", "\n", "Source: https://www.researchgate.net/publication/229124308_Process_Mining_with_the_Heuristics_Miner-algorithm" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "pycharm": { "is_executing": true, "name": "#%%\n" } }, "outputs": [], "source": [ "from sberpm.miners import HeuMiner" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "pycharm": { "is_executing": true }, "scrolled": false }, "outputs": [ { "data": { "text/html": [ "\r\n", "\r\n", "\r\n", "\r\n", "\r\n" ], "text/plain": [ "" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Miner\n", "heu_miner = HeuMiner(data_holder, threshold=0.8)\n", "heu_miner.apply()\n", "graph = heu_miner.graph\n", "\n", "# Visualization\n", "painter = GraphvizPainter()\n", "painter.apply(graph)\n", "painter.show()" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "## 4. AlphaMiner" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`AlphaMiner` takes into consideration causal, parallel and independent relations and produces a Petri net that contains the information about parallel and XOR-like executions of the activities." ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "pycharm": { "is_executing": true, "name": "#%%\n" } }, "outputs": [], "source": [ "from sberpm.miners import AlphaMiner" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "data": { "text/html": [ "\r\n", "\r\n", "\r\n", "\r\n", "\r\n" ], "text/plain": [ "" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Miner\n", "alpha_miner = AlphaMiner(data_holder)\n", "alpha_miner.apply()\n", "graph = alpha_miner.graph\n", "\n", "# Visualization\n", "painter = GraphvizPainter()\n", "painter.apply(graph)\n", "painter.show()" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "## 5. AlphaPlusMiner" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`AlphaPlusMiner` is an implementation of Alpha+ miner. It is a modification of Alpha Miner that can handle one-loop cases (logs where $X\\to X$ relation is present)." ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "pycharm": { "is_executing": true, "name": "#%%\n" } }, "outputs": [], "source": [ "from sberpm.miners import AlphaPlusMiner" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "data": { "text/html": [ "\r\n", "\r\n", "\r\n", "\r\n", "\r\n" ], "text/plain": [ "" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Miner\n", "alpha_miner_plus = AlphaPlusMiner(data_holder)\n", "alpha_miner_plus.apply()\n", "graph = alpha_miner_plus.graph\n", "\n", "# Visualization\n", "painter = GraphvizPainter()\n", "painter.apply(graph)\n", "painter.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 6. InductiveMiner" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`InductiveMiner` creates a Process Tree structure. Its leaf nodes are real activities of the process and all the other nodes are the operators. There are four types of operators: \n", "- SEQUENTIAL (`->`), \n", "- EXCLUSIVE_CHOICE/OR (`X`), \n", "- PARALLEL/AND (`||`), \n", "- LOOP (`*`).\n", "\n", "There is an additional 'operator' that sais that it is impossible to detect an appropriate operator:\n", "- FLOWER_MODEL ('`?`')\n", "\n", "*Note*: some of the leaf nodes might be *hidden activities* represented by black rectangles. They are not real activities and exist only to preserve the correct process tree structure.\n", "\n", "For example, a log containing two event traces $W = \\{(a, b, c), (a, c)\\}$ can produce following the Process Tree: \n", "`SEQUENTIAL(a, EXCLUSIVE_CHOICE(b, hidden_activity), c)`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If during an iteration the algorithm cannot find an appropriate cut of the graph corresponding to one of the operators, it is possible to do a trick: if there exists such activity, so that the graph without is has an appropriate cut, the algorithm returns the following process tree: `PARALLEL(EXCLUSIVE_CHOICE(act, hidden_activity), graph_without_act)` - roughly speaking, the activity is considered to be parallel to the graph.\n", "\n", "\n", "This additional check can be turned on or off by **parallel_activity** argument of `InductiveMiner` class." ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [], "source": [ "from sberpm.miners import InductiveMiner" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "data": { "text/html": [ "\r\n", "\r\n", "\r\n", "\r\n", "\r\n" ], "text/plain": [ "" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Miner\n", "inductive_miner = InductiveMiner(data_holder)\n", "inductive_miner.apply()\n", "graph = inductive_miner.graph\n", "\n", "# Visualization\n", "painter = GraphvizPainter()\n", "painter.apply(graph)\n", "painter.show()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "# II. Metrics" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A module __`sberpm.metrics`__ offers five basic types of metrics:\n", "1. `ActivityMetric` – metrics by activities (group by activity_column)\n", "2. `TransitionMetric` – metrics by transitions (two consecutive activities) (group by unique transitions)\n", "3. `IdMetric`– metrics by ids (group by id_column)\n", "4. `TraceMetric` – metrics by event traces (group by unique traces)\n", "5. `UserMetric` – metrics by users (group by user_column)" ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [], "source": [ "from sberpm.metrics import ActivityMetric, TransitionMetric, IdMetric, TraceMetric, UserMetric" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Parameters:\n", "- __data_holder__ – object of type DataHolder to calculate metrics for\n", "- __time_unit__ – unit of time, by default time metrics are calculated in days\n", "- __round__ – number of digits in float numbers after decimal point" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Methods for all classes:\n", "- __apply(...)__ – calculate all available metrics\n", "- __calc_metrics(...)__ – calculate given metrics (correspond to method names/column names in DataFrame after apply())\n", "- __calculate_time_metrics(...)__ – calculate time metrics\n", "\n", "\n", "- __total_duration__ – calculate the total duration\n", "- __min_duration__ – calculate the minimum duration\n", "- __max_duration__ – calculate the maximum duration\n", "- __mean_duration__ – calculate the mean duration\n", "- __median_duration__ – calculate the median duration\n", "- __std_duration__ – calculate the standard deviation of duration\n", "- __var_duration__ – calculate the variance of duration\n", "\n", "\n", "\n", "Extra methods:\n", "- ActivityMetric\n", " - __count__ - number of occurrences in the event log\n", " - __unique_ids__ - unique IDs corresponding to this object\n", " - __unique_ids_num__ - number of unique IDs corresponding to this object\n", " - __aver_count_in_trace__ - average number of occurrences of this object in the event trace\n", " - __loop_percent__ - loop percent (from 0 to 100)\n", " - __throughput__ - number of execution of this object per time unit\n", " - __unique_users__ - unique users working on this object\n", " - __unique_users_num__ - number of unique users working on this object\n", " - __success_rate(...)__ - id rate, that end successfully (end with 'successfull' activities)\n", " - __failure_rate(...)__ - id rate, that end unsuccessfully (end with 'unsuccessfull' activities)\n", "\n", "\n", "\n", "- IdMetric\n", " - __trace__ - event trace (list of activities)\n", " - __trace_length__ - length of trace (number of activities)\n", " - __unique_activities__ - unique activities in trace\n", " - __unique_activities_num__ - number of unique activities in trace\n", " - __loop_percent__ - loop percent (from 0 to 100)\n", " - __unique_users__ - unique users working on this object\n", " - __unique_users_num__ - number of unique users working on this object\n", "\n", "\n", "\n", "- TraceMetric\n", " - __count__ - number of occurrences in the event log\n", " - __ids__ - (unique) IDs corresponding to this object\n", " - __trace_length__ - length of trace (number of activities)\n", " - __unique_activities__ - unique activities in trace\n", " - __unique_activities_num__ - number of unique activities in trace\n", " - __loop_percent__ - loop percent (from 0 to 100)\n", " - __unique_users__ - unique users working on this object\n", " - __unique_users_num__ - number of unique users working on this object\n", " \n", " \n", "- TransitionMetric\n", " - __count__ - number of occurrences in the event log\n", " - __unique_ids__ - unique IDs corresponding to this object\n", " - __unique_ids_num__ - number of unique IDs corresponding to this object\n", " - __aver_count_in_trace__ - average number of occurrences of this object in the event trace\n", " - __loop_percent__ - loop percent (from 0 to 100)\n", " - __throughput__ - number of execution of this object per time unit\n", " - __unique_users__ - unique users working on this object\n", " - __unique_users_num__ - number of unique users working on this object\n", " - __success_rate(...)__ - id rate, that end successfully (end with 'successfull' activities)\n", " - __failure_rate(...)__ - id rate, that end unsuccessfully (end with 'unsuccessfull' activities)\n", "\n", "\n", "- UserMetric\n", " - __count__ - number of occurrences in the event log\n", " - __unique_activities__ - unique activities that user worked on\n", " - __unique_activities_num__ - number of unique activities that user worked on\n", " - __unique_ids__ - unique IDs corresponding to this object\n", " - __unique_ids_num__ - number of unique IDs corresponding to this object\n", " - __throughput__ - number of activity execution per time unit\n", " - __workload__ - rate of all activities in the event log executed by user\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 1. ActivityMetric" ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "pycharm": { "is_executing": true, "name": "#%%\n" } }, "outputs": [ { "data": { "text/html": [ "

	count	unique_ids	unique_ids_num	aver_count_in_trace	throughput	unique_users	unique_users_num	total_duration	mean_duration	median_duration	max_duration	min_duration	variance_duration	std_duration
Stage_0	758	{640.0, 490.0, 136.0, 968.0, 575.0, 584.0, 46....	758	1.0	0.200053	{Oliver, Jake}	2	3789.0	5.079088	5.0	9.0	1.0	568935.172394	2.566106
Stage_2	672	{640.0, 490.0, 136.0, 968.0, 575.0, 46.0, 900....	672	1.0	0.203575	{Mason, Patricia, Olivia}	3	3301.0	5.070661	5.0	9.0	1.0	556191.654102	2.537204
Stage_8	633	{640.0, 490.0, 136.0, 575.0, 584.0, 46.0, 900....	633	1.0	0.595484	{Mary, Isabella}	2	1063.0	5.288557	6.0	9.0	1.0	555910.358654	2.536562
Stage_5	613	{640.0, 490.0, 362.0, 136.0, 968.0, 584.0, 46....	613	1.0	0.245396	{Emma, Margaret}	2	2498.0	4.976096	5.0	9.0	1.0	580655.011825	2.592401
Stage_3	577	{640.0, 968.0, 584.0, 46.0, 900.0, 5.0, 906.0,...	577	1.0	0.211821	{Amelia, Liam, Noah}	3	2724.0	5.082090	5.0	9.0	1.0	573913.299176	2.577308

\n", "

" ], "text/plain": [ " count unique_ids \\\n", "Stage_0 758 {640.0, 490.0, 136.0, 968.0, 575.0, 584.0, 46.... \n", "Stage_2 672 {640.0, 490.0, 136.0, 968.0, 575.0, 46.0, 900.... \n", "Stage_8 633 {640.0, 490.0, 136.0, 575.0, 584.0, 46.0, 900.... \n", "Stage_5 613 {640.0, 490.0, 362.0, 136.0, 968.0, 584.0, 46.... \n", "Stage_3 577 {640.0, 968.0, 584.0, 46.0, 900.0, 5.0, 906.0,... \n", "\n", " unique_ids_num aver_count_in_trace loop_percent throughput \\\n", "Stage_0 758 1.0 0.0 0.200053 \n", "Stage_2 672 1.0 0.0 0.203575 \n", "Stage_8 633 1.0 0.0 0.595484 \n", "Stage_5 613 1.0 0.0 0.245396 \n", "Stage_3 577 1.0 0.0 0.211821 \n", "\n", " unique_users unique_users_num total_duration \\\n", "Stage_0 {Oliver, Jake} 2 3789.0 \n", "Stage_2 {Mason, Patricia, Olivia} 3 3301.0 \n", "Stage_8 {Mary, Isabella} 2 1063.0 \n", "Stage_5 {Emma, Margaret} 2 2498.0 \n", "Stage_3 {Amelia, Liam, Noah} 3 2724.0 \n", "\n", " mean_duration median_duration max_duration min_duration \\\n", "Stage_0 5.079088 5.0 9.0 1.0 \n", "Stage_2 5.070661 5.0 9.0 1.0 \n", "Stage_8 5.288557 6.0 9.0 1.0 \n", "Stage_5 4.976096 5.0 9.0 1.0 \n", "Stage_3 5.082090 5.0 9.0 1.0 \n", "\n", " variance_duration std_duration \n", "Stage_0 568935.172394 2.566106 \n", "Stage_2 556191.654102 2.537204 \n", "Stage_8 555910.358654 2.536562 \n", "Stage_5 580655.011825 2.592401 \n", "Stage_3 573913.299176 2.577308 " ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "activity_metric = ActivityMetric(data_holder, time_unit='day')\n", "activity_metric.apply().head()" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "### 2. TransitionMetric" ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "pycharm": { "is_executing": true, "name": "#%%\n" }, "scrolled": true }, "outputs": [ { "data": { "text/html": [ "

	count	unique_ids	unique_ids_num	aver_count_in_trace	throughput	unique_users	unique_users_num	total_duration	mean_duration	median_duration	max_duration	min_duration	variance_duration	std_duration
(Stage_0, Stage_2)	313	{968.0, 575.0, 46.0, 900.0, 5.0, 906.0, 58.0, ...	313	1.0	0.193210	{Oliver, Jake}	2	1620.0	5.175719	5.0	9.0	1.0	572320.076759	2.573728
(Stage_2, Stage_3)	242	{476.0, 224.0, 4.0, 931.0, 733.0, 968.0, 140.0...	242	1.0	0.200165	{Mason, Patricia, Olivia}	3	1209.0	4.995868	5.0	9.0	1.0	526610.094939	2.468810
(Stage_7, Stage_8)	219	{224.0, 640.0, 911.0, 490.0, 392.0, 136.0, 413...	219	1.0	0.198370	{Callum, Harry}	2	1104.0	5.041096	5.0	9.0	1.0	605443.122537	2.647158
(Stage_3, Stage_5)	184	{476.0, 19.0, 733.0, 523.0, 511.0, 867.0, 257....	184	1.0	0.195329	{Amelia, Liam, Noah}	3	942.0	5.119565	5.0	9.0	1.0	579147.448015	2.589034
(Stage_5, Stage_7)	178	{640.0, 490.0, 441.0, 755.0, 236.0, 46.0, 444....	178	1.0	0.199105	{Emma, Margaret}	2	894.0	5.022472	5.0	9.0	1.0	544567.605100	2.510551

\n", "

" ], "text/plain": [ " count unique_ids \\\n", "(Stage_0, Stage_2) 313 {968.0, 575.0, 46.0, 900.0, 5.0, 906.0, 58.0, ... \n", "(Stage_2, Stage_3) 242 {476.0, 224.0, 4.0, 931.0, 733.0, 968.0, 140.0... \n", "(Stage_7, Stage_8) 219 {224.0, 640.0, 911.0, 490.0, 392.0, 136.0, 413... \n", "(Stage_3, Stage_5) 184 {476.0, 19.0, 733.0, 523.0, 511.0, 867.0, 257.... \n", "(Stage_5, Stage_7) 178 {640.0, 490.0, 441.0, 755.0, 236.0, 46.0, 444.... \n", "\n", " unique_ids_num aver_count_in_trace loop_percent \\\n", "(Stage_0, Stage_2) 313 1.0 0.0 \n", "(Stage_2, Stage_3) 242 1.0 0.0 \n", "(Stage_7, Stage_8) 219 1.0 0.0 \n", "(Stage_3, Stage_5) 184 1.0 0.0 \n", "(Stage_5, Stage_7) 178 1.0 0.0 \n", "\n", " throughput unique_users unique_users_num \\\n", "(Stage_0, Stage_2) 0.193210 {Oliver, Jake} 2 \n", "(Stage_2, Stage_3) 0.200165 {Mason, Patricia, Olivia} 3 \n", "(Stage_7, Stage_8) 0.198370 {Callum, Harry} 2 \n", "(Stage_3, Stage_5) 0.195329 {Amelia, Liam, Noah} 3 \n", "(Stage_5, Stage_7) 0.199105 {Emma, Margaret} 2 \n", "\n", " total_duration mean_duration median_duration \\\n", "(Stage_0, Stage_2) 1620.0 5.175719 5.0 \n", "(Stage_2, Stage_3) 1209.0 4.995868 5.0 \n", "(Stage_7, Stage_8) 1104.0 5.041096 5.0 \n", "(Stage_3, Stage_5) 942.0 5.119565 5.0 \n", "(Stage_5, Stage_7) 894.0 5.022472 5.0 \n", "\n", " max_duration min_duration variance_duration \\\n", "(Stage_0, Stage_2) 9.0 1.0 572320.076759 \n", "(Stage_2, Stage_3) 9.0 1.0 526610.094939 \n", "(Stage_7, Stage_8) 9.0 1.0 605443.122537 \n", "(Stage_3, Stage_5) 9.0 1.0 579147.448015 \n", "(Stage_5, Stage_7) 9.0 1.0 544567.605100 \n", "\n", " std_duration \n", "(Stage_0, Stage_2) 2.573728 \n", "(Stage_2, Stage_3) 2.468810 \n", "(Stage_7, Stage_8) 2.647158 \n", "(Stage_3, Stage_5) 2.589034 \n", "(Stage_5, Stage_7) 2.510551 " ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "transition_metric = TransitionMetric(data_holder, time_unit='day')\n", "transition_metric.apply().head()" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "### 3. IdMetric" ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "data": { "text/html": [ "

	trace	trace_length	unique_activities	unique_activities_num	unique_users	unique_users_num	total_duration	mean_duration	median_duration	max_duration	min_duration	variance_duration	std_duration
0.0	(Stage_0, Stage_3, Stage_5, Stage_2, Stage_7)	5	{Stage_2, Stage_7, Stage_3, Stage_5, Stage_0}	5	{Callum, Liam, Patricia, Emma, Oliver}	5	16.0	4.000000	4.5	6.0	1.0	388800.0	2.121320
1.0	(Stage_0, Stage_2, Stage_4, Stage_3, Stage_8)	5	{Stage_2, Stage_4, Stage_8, Stage_3, Stage_0}	5	{William, Noah, Jake, Olivia, Isabella}	5	15.0	3.750000	4.0	5.0	2.0	145800.0	1.299038
10.0	(Stage_0, Stage_2, Stage_5, Stage_7)	4	{Stage_2, Stage_0, Stage_7, Stage_5}	4	{Mason, Oliver, Emma, Harry}	4	16.0	5.333333	5.0	8.0	3.0	364800.0	2.054805
100.0	(Stage_2, Stage_3, Stage_8)	3	{Stage_3, Stage_2, Stage_8}	3	{Mason, Liam, Isabella}	3	17.0	8.500000	8.5	9.0	8.0	21600.0	0.500000
101.0	(Stage_0, Stage_2, Stage_5, Stage_3, Stage_7, ...	7	{Stage_2, Stage_9, Stage_7, Stage_8, Stage_3, ...	7	{Liam, Barbara, Mason, Harry, Margaret, Mary, ...	7	26.0	4.333333	4.0	9.0	1.0	854400.0	3.144660

\n", "

" ], "text/plain": [ " trace trace_length \\\n", "0.0 (Stage_0, Stage_3, Stage_5, Stage_2, Stage_7) 5 \n", "1.0 (Stage_0, Stage_2, Stage_4, Stage_3, Stage_8) 5 \n", "10.0 (Stage_0, Stage_2, Stage_5, Stage_7) 4 \n", "100.0 (Stage_2, Stage_3, Stage_8) 3 \n", "101.0 (Stage_0, Stage_2, Stage_5, Stage_3, Stage_7, ... 7 \n", "\n", " unique_activities \\\n", "0.0 {Stage_2, Stage_7, Stage_3, Stage_5, Stage_0} \n", "1.0 {Stage_2, Stage_4, Stage_8, Stage_3, Stage_0} \n", "10.0 {Stage_2, Stage_0, Stage_7, Stage_5} \n", "100.0 {Stage_3, Stage_2, Stage_8} \n", "101.0 {Stage_2, Stage_9, Stage_7, Stage_8, Stage_3, ... \n", "\n", " unique_activities_num loop_percent \\\n", "0.0 5 0.0 \n", "1.0 5 0.0 \n", "10.0 4 0.0 \n", "100.0 3 0.0 \n", "101.0 7 0.0 \n", "\n", " unique_users unique_users_num \\\n", "0.0 {Callum, Liam, Patricia, Emma, Oliver} 5 \n", "1.0 {William, Noah, Jake, Olivia, Isabella} 5 \n", "10.0 {Mason, Oliver, Emma, Harry} 4 \n", "100.0 {Mason, Liam, Isabella} 3 \n", "101.0 {Liam, Barbara, Mason, Harry, Margaret, Mary, ... 7 \n", "\n", " total_duration mean_duration median_duration max_duration \\\n", "0.0 16.0 4.000000 4.5 6.0 \n", "1.0 15.0 3.750000 4.0 5.0 \n", "10.0 16.0 5.333333 5.0 8.0 \n", "100.0 17.0 8.500000 8.5 9.0 \n", "101.0 26.0 4.333333 4.0 9.0 \n", "\n", " min_duration variance_duration std_duration \n", "0.0 1.0 388800.0 2.121320 \n", "1.0 2.0 145800.0 1.299038 \n", "10.0 3.0 364800.0 2.054805 \n", "100.0 8.0 21600.0 0.500000 \n", "101.0 1.0 854400.0 3.144660 " ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "id_metric = IdMetric(data_holder, time_unit='day')\n", "id_metric.apply().head()" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "### 4. TraceMetric" ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "pycharm": { "is_executing": true, "name": "#%%\n" }, "scrolled": false }, "outputs": [ { "data": { "text/html": [ "

	count	ids	trace_length	unique_activities_num	unique_users	unique_users_num	total_duration	mean_duration	median_duration	max_duration	min_duration	variance_duration	std_duration
(Stage_0, Stage_2, Stage_3, Stage_8)	12	{939.0, 652.0, 240.0, 276.0, 434.0, 833.0, 228...	4	4	{Liam, Mason, Mary, Patricia, Noah, Jake, Amel...	10	196.0	16.333333	18.0	25.0	8.0	2.035200e+06	4.853407
(Stage_0, Stage_2, Stage_3, Stage_5, Stage_8)	12	{193.0, 20.0, 124.0, 585.0, 520.0, 211.0, 81.0...	5	5	{Liam, Margaret, Mary, Patricia, Emma, Noah, J...	11	247.0	20.583333	20.5	32.0	12.0	2.296200e+06	5.155229
(Stage_0, Stage_2, Stage_5)	11	{600.0, 792.0, 743.0, 457.0, 514.0, 73.0, 58.0...	3	3	{Mason, Margaret, Patricia, Emma, Jake, Oliver...	7	129.0	11.727273	10.0	18.0	6.0	1.006810e+06	3.413633
(Stage_0, Stage_2, Stage_3, Stage_5, Stage_7, Stage_8)	10	{738.0, 246.0, 478.0, 563.0, 349.0, 770.0, 790...	6	6	{Callum, Liam, Mason, Harry, Margaret, Mary, P...	14	252.0	25.200000	26.0	32.0	16.0	1.931904e+06	4.728636
(Stage_0, Stage_2, Stage_3, Stage_5)	10	{174.0, 564.0, 924.0, 267.0, 78.0, 231.0, 834....	4	4	{Liam, Mason, Margaret, Patricia, Emma, Noah, ...	9	161.0	16.100000	14.5	25.0	11.0	1.649376e+06	4.369210

\n", "

" ], "text/plain": [ " count \\\n", "(Stage_0, Stage_2, Stage_3, Stage_8) 12 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_8) 12 \n", "(Stage_0, Stage_2, Stage_5) 11 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_7, S... 10 \n", "(Stage_0, Stage_2, Stage_3, Stage_5) 10 \n", "\n", " ids \\\n", "(Stage_0, Stage_2, Stage_3, Stage_8) {939.0, 652.0, 240.0, 276.0, 434.0, 833.0, 228... \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_8) {193.0, 20.0, 124.0, 585.0, 520.0, 211.0, 81.0... \n", "(Stage_0, Stage_2, Stage_5) {600.0, 792.0, 743.0, 457.0, 514.0, 73.0, 58.0... \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_7, S... {738.0, 246.0, 478.0, 563.0, 349.0, 770.0, 790... \n", "(Stage_0, Stage_2, Stage_3, Stage_5) {174.0, 564.0, 924.0, 267.0, 78.0, 231.0, 834.... \n", "\n", " trace_length \\\n", "(Stage_0, Stage_2, Stage_3, Stage_8) 4 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_8) 5 \n", "(Stage_0, Stage_2, Stage_5) 3 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_7, S... 6 \n", "(Stage_0, Stage_2, Stage_3, Stage_5) 4 \n", "\n", " unique_activities_num \\\n", "(Stage_0, Stage_2, Stage_3, Stage_8) 4 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_8) 5 \n", "(Stage_0, Stage_2, Stage_5) 3 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_7, S... 6 \n", "(Stage_0, Stage_2, Stage_3, Stage_5) 4 \n", "\n", " loop_percent \\\n", "(Stage_0, Stage_2, Stage_3, Stage_8) 0.0 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_8) 0.0 \n", "(Stage_0, Stage_2, Stage_5) 0.0 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_7, S... 0.0 \n", "(Stage_0, Stage_2, Stage_3, Stage_5) 0.0 \n", "\n", " unique_users \\\n", "(Stage_0, Stage_2, Stage_3, Stage_8) {Liam, Mason, Mary, Patricia, Noah, Jake, Amel... \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_8) {Liam, Margaret, Mary, Patricia, Emma, Noah, J... \n", "(Stage_0, Stage_2, Stage_5) {Mason, Margaret, Patricia, Emma, Jake, Oliver... \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_7, S... {Callum, Liam, Mason, Harry, Margaret, Mary, P... \n", "(Stage_0, Stage_2, Stage_3, Stage_5) {Liam, Mason, Margaret, Patricia, Emma, Noah, ... \n", "\n", " unique_users_num \\\n", "(Stage_0, Stage_2, Stage_3, Stage_8) 10 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_8) 11 \n", "(Stage_0, Stage_2, Stage_5) 7 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_7, S... 14 \n", "(Stage_0, Stage_2, Stage_3, Stage_5) 9 \n", "\n", " total_duration \\\n", "(Stage_0, Stage_2, Stage_3, Stage_8) 196.0 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_8) 247.0 \n", "(Stage_0, Stage_2, Stage_5) 129.0 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_7, S... 252.0 \n", "(Stage_0, Stage_2, Stage_3, Stage_5) 161.0 \n", "\n", " mean_duration \\\n", "(Stage_0, Stage_2, Stage_3, Stage_8) 16.333333 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_8) 20.583333 \n", "(Stage_0, Stage_2, Stage_5) 11.727273 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_7, S... 25.200000 \n", "(Stage_0, Stage_2, Stage_3, Stage_5) 16.100000 \n", "\n", " median_duration \\\n", "(Stage_0, Stage_2, Stage_3, Stage_8) 18.0 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_8) 20.5 \n", "(Stage_0, Stage_2, Stage_5) 10.0 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_7, S... 26.0 \n", "(Stage_0, Stage_2, Stage_3, Stage_5) 14.5 \n", "\n", " max_duration \\\n", "(Stage_0, Stage_2, Stage_3, Stage_8) 25.0 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_8) 32.0 \n", "(Stage_0, Stage_2, Stage_5) 18.0 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_7, S... 32.0 \n", "(Stage_0, Stage_2, Stage_3, Stage_5) 25.0 \n", "\n", " min_duration \\\n", "(Stage_0, Stage_2, Stage_3, Stage_8) 8.0 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_8) 12.0 \n", "(Stage_0, Stage_2, Stage_5) 6.0 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_7, S... 16.0 \n", "(Stage_0, Stage_2, Stage_3, Stage_5) 11.0 \n", "\n", " variance_duration \\\n", "(Stage_0, Stage_2, Stage_3, Stage_8) 2.035200e+06 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_8) 2.296200e+06 \n", "(Stage_0, Stage_2, Stage_5) 1.006810e+06 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_7, S... 1.931904e+06 \n", "(Stage_0, Stage_2, Stage_3, Stage_5) 1.649376e+06 \n", "\n", " std_duration \n", "(Stage_0, Stage_2, Stage_3, Stage_8) 4.853407 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_8) 5.155229 \n", "(Stage_0, Stage_2, Stage_5) 3.413633 \n", "(Stage_0, Stage_2, Stage_3, Stage_5, Stage_7, S... 4.728636 \n", "(Stage_0, Stage_2, Stage_3, Stage_5) 4.369210 " ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "trace_metric = TraceMetric(data_holder, time_unit='day')\n", "trace_metric.apply().head()" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "### 5. UserMetric" ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "pycharm": { "is_executing": true, "name": "#%%\n" }, "scrolled": true }, "outputs": [ { "data": { "text/html": [ "

	count	unique_activities	unique_activities_num	unique_ids	unique_ids_num	throughput	workload	total_duration	mean_duration	median_duration	max_duration	min_duration	variance_duration	std_duration
Oliver	493	{Stage_0}	1	{640.0, 490.0, 575.0, 584.0, 46.0, 369.0, 906....	493	0.195093	0.110513	2527.0	5.199588	5.0	9.0	1.0	572380.429813	2.573864
Isabella	427	{Stage_8}	1	{640.0, 490.0, 584.0, 46.0, 900.0, 832.0, 293....	427	0.610000	0.095718	700.0	5.303030	6.0	9.0	1.0	544502.479339	2.510401
Emma	415	{Stage_5}	1	{640.0, 362.0, 136.0, 968.0, 46.0, 608.0, 58.0...	415	0.242974	0.093028	1708.0	4.936416	5.0	9.0	1.0	596459.941862	2.627446
Harry	328	{Stage_7}	1	{136.0, 46.0, 5.0, 776.0, 38.0, 293.0, 629.0, ...	328	0.303142	0.073526	1082.0	4.852018	5.0	9.0	1.0	617630.823061	2.673669
Jake	265	{Stage_0}	1	{67.0, 307.0, 249.0, 136.0, 697.0, 733.0, 968....	265	0.209984	0.059404	1262.0	4.853846	5.0	9.0	1.0	555766.721893	2.536234

\n", "

" ], "text/plain": [ " count unique_activities unique_activities_num \\\n", "Oliver 493 {Stage_0} 1 \n", "Isabella 427 {Stage_8} 1 \n", "Emma 415 {Stage_5} 1 \n", "Harry 328 {Stage_7} 1 \n", "Jake 265 {Stage_0} 1 \n", "\n", " unique_ids unique_ids_num \\\n", "Oliver {640.0, 490.0, 575.0, 584.0, 46.0, 369.0, 906.... 493 \n", "Isabella {640.0, 490.0, 584.0, 46.0, 900.0, 832.0, 293.... 427 \n", "Emma {640.0, 362.0, 136.0, 968.0, 46.0, 608.0, 58.0... 415 \n", "Harry {136.0, 46.0, 5.0, 776.0, 38.0, 293.0, 629.0, ... 328 \n", "Jake {67.0, 307.0, 249.0, 136.0, 697.0, 733.0, 968.... 265 \n", "\n", " throughput workload total_duration mean_duration \\\n", "Oliver 0.195093 0.110513 2527.0 5.199588 \n", "Isabella 0.610000 0.095718 700.0 5.303030 \n", "Emma 0.242974 0.093028 1708.0 4.936416 \n", "Harry 0.303142 0.073526 1082.0 4.852018 \n", "Jake 0.209984 0.059404 1262.0 4.853846 \n", "\n", " median_duration max_duration min_duration variance_duration \\\n", "Oliver 5.0 9.0 1.0 572380.429813 \n", "Isabella 6.0 9.0 1.0 544502.479339 \n", "Emma 5.0 9.0 1.0 596459.941862 \n", "Harry 5.0 9.0 1.0 617630.823061 \n", "Jake 5.0 9.0 1.0 555766.721893 \n", "\n", " std_duration \n", "Oliver 2.573864 \n", "Isabella 2.510401 \n", "Emma 2.627446 \n", "Harry 2.673669 \n", "Jake 2.536234 " ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "user_metric = UserMetric(data_holder, time_unit='day')\n", "user_metric.apply().head()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "### TokenReplay" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`TokenReplay` allows to compute *fitness* that indicates to what extent a Petri net captures a business process (1 for good, 0 for bad). Fitness is calculated for each event trace (id) by replaying it in the given Petri net as follows: \n", "$$ Fitness = \\frac{1}{2}\\big(1-\\frac{missed}{consumed}\\big) + \\frac{1}{2}\\big(1-\\frac{remaining}{produced}\\big) $$\n", "- $produced$ – the number of produced tokens\n", "- $consumed$ – the number of consumed tokens\n", "- $remaining$ – the number of remaining tokens\n", "- $missing$ – the number of missing tokens\n", "\n", "The following metrics can be obtained:\n", "- values of the four components and fitness for each event traces\n", "- mean fitness of all traces (__mean_fitness__)\n", "- fitness of the log by substituting the values in the formula by sum of the corresponding values of each event trace (__average_fitness__)" ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "pycharm": { "is_executing": true, "name": "#%%\n" } }, "outputs": [], "source": [ "from sberpm.metrics import TokenReplay" ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "data": { "text/html": [ "

	id	stages	c	p	m	r	fitness
0	0.0	(Stage_0, Stage_3, Stage_5, Stage_2, Stage_7)	7	8	5	6	0.267857
1	1.0	(Stage_0, Stage_2, Stage_4, Stage_3, Stage_8)	6	9	4	7	0.277778
2	10.0	(Stage_0, Stage_2, Stage_5, Stage_7)	6	7	4	5	0.309524
3	100.0	(Stage_2, Stage_3, Stage_8)	4	5	2	3	0.450000
4	101.0	(Stage_0, Stage_2, Stage_5, Stage_3, Stage_7, ...	14	10	10	6	0.342857
...	...	...	...	...	...	...	...
994	995.0	(Stage_0, Stage_2, Stage_5)	4	6	2	4	0.416667
995	996.0	(Stage_3, Stage_2, Stage_5, Stage_7, Stage_8)	7	7	5	5	0.285714
996	997.0	(Stage_0, Stage_2, Stage_3, Stage_5, Stage_6, ...	10	11	8	9	0.190909
997	998.0	(Stage_0, Stage_1, Stage_2, Stage_3, Stage_9)	11	11	6	6	0.454545
998	999.0	(Stage_0, Stage_2, Stage_3, Stage_5, Stage_7)	7	8	5	6	0.267857

\n", "

999 rows × 7 columns

\n", "

" ], "text/plain": [ " id stages c p m r \\\n", "0 0.0 (Stage_0, Stage_3, Stage_5, Stage_2, Stage_7) 7 8 5 6 \n", "1 1.0 (Stage_0, Stage_2, Stage_4, Stage_3, Stage_8) 6 9 4 7 \n", "2 10.0 (Stage_0, Stage_2, Stage_5, Stage_7) 6 7 4 5 \n", "3 100.0 (Stage_2, Stage_3, Stage_8) 4 5 2 3 \n", "4 101.0 (Stage_0, Stage_2, Stage_5, Stage_3, Stage_7, ... 14 10 10 6 \n", ".. ... ... .. .. .. .. \n", "994 995.0 (Stage_0, Stage_2, Stage_5) 4 6 2 4 \n", "995 996.0 (Stage_3, Stage_2, Stage_5, Stage_7, Stage_8) 7 7 5 5 \n", "996 997.0 (Stage_0, Stage_2, Stage_3, Stage_5, Stage_6, ... 10 11 8 9 \n", "997 998.0 (Stage_0, Stage_1, Stage_2, Stage_3, Stage_9) 11 11 6 6 \n", "998 999.0 (Stage_0, Stage_2, Stage_3, Stage_5, Stage_7) 7 8 5 6 \n", "\n", " fitness \n", "0 0.267857 \n", "1 0.277778 \n", "2 0.309524 \n", "3 0.450000 \n", "4 0.342857 \n", ".. ... \n", "994 0.416667 \n", "995 0.285714 \n", "996 0.190909 \n", "997 0.454545 \n", "998 0.267857 \n", "\n", "[999 rows x 7 columns]" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "token_replay = TokenReplay(data_holder, alpha_miner.graph)\n", "token_replay.apply()\n", "token_replay.result" ] }, { "cell_type": "code", "execution_count": 29, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "mean: 0.36138350149788057\n", "average: 0.3365537369071906\n" ] } ], "source": [ "print('mean:', token_replay.mean_fitness)\n", "print('average:', token_replay.average_fitness)" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "# Metrics + Graphs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In _sberpm_ there is an opportunity to visualize metrics on the graph. In order to do it, one can use `Graph` class that has the following methods:\n", "- __add_node_metric(...)__ – add metric to the graph node\n", "- __add_edge_metric(...)__ – add metric to the graph edge" ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "pycharm": { "is_executing": true, "name": "#%%\n" } }, "outputs": [], "source": [ "# Calculation of the metrics\n", "nodes_count_metric = activity_metric.count().to_dict()\n", "edges_count_metric = transition_metric.count().to_dict()\n", "mean_time_node_metric = activity_metric.mean_duration().fillna(0).to_dict()\n", "\n", "# Getting the graph\n", "graph = causal_miner.graph\n", "\n", "# Adding the metrics to the graph\n", "graph.add_node_metric('count', nodes_count_metric)\n", "graph.add_edge_metric('count', edges_count_metric)\n", "graph.add_node_metric('mean_time', mean_time_node_metric)" ] }, { "cell_type": "code", "execution_count": 31, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "data": { "text/html": [ "\r\n", "\r\n", "\r\n", "\r\n", "\r\n" ], "text/plain": [ "" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "painter = GraphvizPainter()\n", "\n", "# Specify which metrics influence the nodes' colors and the edges' widths\n", "painter.apply(graph, node_style_metric='count', edge_style_metric='count')\n", "# or \n", "# painter.apply(graph, node_style_metric='mean_time', edge_style_metric='count')\n", "\n", "painter.write_graph(\"metric_graph.png\", format = 'png')\n", "painter.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To delete metrics from the graph, the following methods should be used:\n", "- __clear_node_metrics(...)__ – delete all metrics from nodes\n", "- __clear_edge_metrics__ – delete all metrics from edges" ] }, { "cell_type": "code", "execution_count": 32, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [], "source": [ "graph.clear_node_metrics()\n", "graph.clear_edge_metrics()" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "# III. BPMN" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In order to save process graph in a BPMN (Business Process Model and Notation) format, one can use `BpmnExporter` from __`sberpm.bpmn`__ module. It has the following methods:\n", "- __apply_petri(...)__ – construct BPMN for a Petri net \n", "- __get_string_representation()__ – get BPMN notation of the graph\n", "- __write(...)__ – write graph in a BPMN format to file\n", "\n", "At this moment, only Petri nets can be converted to BPMN." ] }, { "cell_type": "code", "execution_count": 33, "metadata": { "pycharm": { "is_executing": true, "name": "#%%\n" } }, "outputs": [], "source": [ "from sberpm.bpmn import BpmnExporter" ] }, { "cell_type": "code", "execution_count": 34, "metadata": { "pycharm": { "is_executing": true, "name": "#%%\n" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "D:\\sber\\envs\\my_env\\lib\\site-packages\\sberpm\\bpmn\\_bpmn_graph_to_file\\_bpmn_exporter.py:63: RuntimeWarning: Impossible to create orthogonal edges, splines will be created instead.\n", " warnings.warn(\"Impossible to create orthogonal edges, splines will be created instead.\", RuntimeWarning)\n" ] }, { "data": { "text/plain": [ "'\\n\\t\\n\\t\\t\\n\\t\\n\\t\\n\\t\\n\\t\\n\\t\\n\\tNo insight__\n", " - __Need for optimization__ (negative insights)\n", " \n", "The mean duration and the return rate for both optimal insights and the ones demanding optimization are displayed in the corner of the image. \n" ] }, { "cell_type": "code", "execution_count": 58, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "data": { "text/html": [ "\r\n", "\r\n", "\r\n", "\r\n", "\r\n" ], "text/plain": [ "" ] }, "execution_count": 58, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Transition duration\n", "auto_i.apply(miner=simple_miner, mode='time')\n", "graph = auto_i.get_graph()\n", "\n", "painter = GraphvizPainter()\n", "painter.apply_insights(graph)\n", "painter.show()" ] }, { "cell_type": "code", "execution_count": 59, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "data": { "text/html": [ "\r\n", "\r\n", "\r\n", "\r\n", "\r\n" ], "text/plain": [ "" ] }, "execution_count": 59, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Loops\n", "auto_i.apply(miner=simple_miner, mode='cycles')\n", "graph = auto_i.get_graph()\n", "\n", "painter = GraphvizPainter()\n", "painter.apply_insights(graph)\n", "painter.show()" ] }, { "cell_type": "code", "execution_count": 60, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "data": { "text/html": [ "\r\n", "\r\n", "\r\n", "\r\n", "\r\n" ], "text/plain": [ "" ] }, "execution_count": 60, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Hybrid mode\n", "auto_i.apply(miner=simple_miner, mode='overall')\n", "graph = auto_i.get_graph()\n", "\n", "painter = GraphvizPainter()\n", "painter.apply_insights(graph)\n", "painter.show()" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "### Description of the Results\n", "\n", "The results are set out in a table format. For each element insight and metric indicators are given: \n", "- __1__ – negative insight\n", "- __0__ – not insight\n", "- __-1__ – positive insight" ] }, { "cell_type": "code", "execution_count": 61, "metadata": { "pycharm": { "is_executing": true, "name": "#%%\n" } }, "outputs": [ { "data": { "text/html": [ "

	activities	count	mean_duration	throughput	user_value	insights
0	Stage_0	1	0	-1	1	1
1	Stage_1	0	-1	0	-1	-1
2	Stage_2	1	0	0	0	1
3	Stage_3	0	0	0	0	0
4	Stage_4	0	0	0	0	0

\n", "

" ], "text/plain": [ " activities count mean_duration loop_percent aver_count_in_trace \\\n", "0 Stage_0 1 0 0 0 \n", "1 Stage_1 0 -1 0 0 \n", "2 Stage_2 1 0 0 0 \n", "3 Stage_3 0 0 0 0 \n", "4 Stage_4 0 0 0 0 \n", "\n", " throughput unique_users_num user_value insights \n", "0 -1 0 1 1 \n", "1 0 0 -1 -1 \n", "2 0 0 0 1 \n", "3 0 0 0 0 \n", "4 0 0 0 0 " ] }, "execution_count": 61, "metadata": {}, "output_type": "execute_result" } ], "source": [ "auto_i.describe_nodes().head()" ] }, { "cell_type": "code", "execution_count": 62, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "data": { "text/html": [ "

	transitions	count	throughput	insights
0	(Stage_0, Stage_1)	1	0	1
1	(Stage_0, Stage_2)	1	0	1
2	(Stage_0, Stage_3)	1	0	1
3	(Stage_0, Stage_4)	0	1	1
4	(Stage_0, Stage_5)	0	0	0

\n", "

" ], "text/plain": [ " transitions count mean_duration loop_percent \\\n", "0 (Stage_0, Stage_1) 1 0 0 \n", "1 (Stage_0, Stage_2) 1 0 0 \n", "2 (Stage_0, Stage_3) 1 0 0 \n", "3 (Stage_0, Stage_4) 0 0 0 \n", "4 (Stage_0, Stage_5) 0 0 0 \n", "\n", " aver_count_in_trace throughput unique_users_num user_value insights \n", "0 0 0 0 0 1 \n", "1 0 0 0 0 1 \n", "2 0 0 0 0 1 \n", "3 0 1 0 0 1 \n", "4 0 0 0 0 0 " ] }, "execution_count": 62, "metadata": {}, "output_type": "execute_result" } ], "source": [ "auto_i.describe_edges().head()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "my_env", "language": "python", "name": "my_env" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.4" } }, "nbformat": 4, "nbformat_minor": 2 }