google-research

1
{
2
  "nbformat": 4,
3
  "nbformat_minor": 0,
4
  "metadata": {
5
    "colab": {
6
      "name": "[PRIME, ICLR 2022] Colab Parsing Dataset",
7
      "provenance": []
8
    },
9
    "kernelspec": {
10
      "name": "python3",
11
      "display_name": "Python 3"
12
    },
13
    "language_info": {
14
      "name": "python"
15
    }
16
  },
17
  "cells": [
18
    {
19
      "cell_type": "markdown",
20
      "source": [
21
        "\n",
22
        "Copyright 2022 Google LLC.\n",
23
        "\n",
24
        "Licensed under the Apache License, Version 2.0 (the \"License\");"
25
      ],
26
      "metadata": {
27
        "id": "4rSCDmvABaV4"
28
      }
29
    },
30
    {
31
      "cell_type": "code",
32
      "execution_count": 1,
33
      "metadata": {
34
        "cellView": "form",
35
        "id": "N6ZcTf1hBQHO"
36
      },
37
      "outputs": [],
38
      "source": [
39
        "#@title License\n",
40
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
41
        "# you may not use this file except in compliance with the License.\n",
42
        "# You may obtain a copy of the License at\n",
43
        "#\n",
44
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
45
        "#\n",
46
        "# Unless required by applicable law or agreed to in writing, software\n",
47
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
48
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
49
        "# See the License for the specific language governing permissions and\n",
50
        "# limitations under the License."
51
      ]
52
    },
53
    {
54
      "cell_type": "markdown",
55
      "source": [
56
        "#  Instructions to use PRIME dataset parser\n",
57
        "\n",
58
        "This colab provides the details to parse the\n",
59
        "[PRIME](https://arxiv.org/abs/2110.11346) dataset available on Google Cloud Storage: [gs://gresearch/prime](https://console.cloud.google.com/storage/browser/gresearch/prime)\n",
60
        "\n",
61
        "You may download the dataset either by using the Google Cloud Storage web interface or using gsutil:\n",
62
        "\n",
63
        "```\n",
64
        "gsutil cp -r gs://gresearch/prime /tmp/prime/\n",
65
        "```\n",
66
        "\n",
67
        "The PRIME dataset contains **10** microarchitectural parameters, including number of compute units, on-chip memory size, and number of SIMD units, along with the Runtime (millisecond) and Chip Area Usage (mm$^2$) for all the nine applications that we evaluated in our paper.\n",
68
        "\n",
69
        "The data in this dataset are collected using an industry-grade cycle-accurate simulator. \n",
70
        "\n",
71
        "This dataset contains both infeasible and feasible data points as described in\n",
72
        "[PRIME](https://arxiv.org/abs/2110.11346). The descriptors of the collected\n",
73
        "data are presented in the table below (Table 1).\n",
74
        "\n",
75
        "|                  | # of Infeasible | # of Feasible | Max Runtime (ms) | Min Runtime (ms) | Average Runtime (ms) |\n",
76
        "|------------------|-----------------|---------------|------------------|------------------|----------------------|\n",
77
        "| **MobileNetEdgeTPU** |          384355 |        115711 |         16352.26 |           252.22 |               529.13 |\n",
78
        "| **MobilenetV2**      |          744718 |        255414 |          7398.13 |           191.35 |               375.05 |\n",
79
        "| **MobilenetV3**      |          797460 |        202672 |          7001.46 |           405.19 |               993.75 |\n",
80
        "| **M4**               |          791984 |        208148 |         35881.35 |           335.59 |               794.33 |\n",
81
        "| **M5**               |          698618 |        301514 |         35363.55 |           202.55 |               440.52 |\n",
82
        "| **M6**               |          756468 |        243664 |          4236.90 |           127.79 |               301.74 |\n",
83
        "| **UNet**             |          449578 |         51128 |        124987.51 |           610.96 |              3681.75 |\n",
84
        "| **T-RNN Dec**        |          405607 |         94459 |          4447.74 |           128.05 |               662.44 |\n",
85
        "| **T-RNN Enc**        |          410933 |         88880 |          5112.82 |           127.97 |               731.20 |"
86
      ],
87
      "metadata": {
88
        "id": "0cvfAbQ3B5bc"
89
      }
90
    },
91
    {
92
      "cell_type": "code",
93
      "source": [
94
        "#@title Listing the data for the studied application\n",
95
        "!gsutil ls gs://gresearch/prime"
96
      ],
97
      "metadata": {
98
        "colab": {
99
          "base_uri": "https://localhost:8080/"
100
        },
101
        "cellView": "form",
102
        "id": "1_BsXdRIKbq6",
103
        "outputId": "a11a3083-a571-434e-ecbf-11565a091cfc"
104
      },
105
      "execution_count": 2,
106
      "outputs": [
107
        {
108
          "output_type": "stream",
109
          "name": "stdout",
110
          "text": [
111
            "gs://gresearch/prime/MobileNetEdgeTPU/\n",
112
            "gs://gresearch/prime/MobilenetEdgeTPU/\n",
113
            "gs://gresearch/prime/MobilenetV2/\n",
114
            "gs://gresearch/prime/MobilenetV3/\n",
115
            "gs://gresearch/prime/m4/\n",
116
            "gs://gresearch/prime/m5/\n",
117
            "gs://gresearch/prime/m6/\n",
118
            "gs://gresearch/prime/t_rnn_dec/\n",
119
            "gs://gresearch/prime/t_rnn_enc/\n",
120
            "gs://gresearch/prime/u-net/\n"
121
          ]
122
        }
123
      ]
124
    },
125
    {
126
      "cell_type": "code",
127
      "source": [
128
        "#@title Importing the necessary libraries\n",
129
        "import tensorflow as tf\n",
130
        "import numpy as np"
131
      ],
132
      "metadata": {
133
        "id": "NsrEON0DKvIL"
134
      },
135
      "execution_count": 12,
136
      "outputs": []
137
    },
138
    {
139
      "cell_type": "code",
140
      "source": [
141
        "#@title APIs for parsing PRIME datasets\n",
142
        "def parse_prime_tfrecords(proto):\n",
143
        "  prime_feature_description = {  \n",
144
        "    'param_1': tf.io.FixedLenFeature([], tf.float32),\n",
145
        "    'param_2': tf.io.FixedLenFeature([], tf.float32),\n",
146
        "    'param_3': tf.io.FixedLenFeature([], tf.float32),\n",
147
        "    'param_4': tf.io.FixedLenFeature([], tf.float32),\n",
148
        "    'param_5': tf.io.FixedLenFeature([], tf.float32),\n",
149
        "    'param_6': tf.io.FixedLenFeature([], tf.float32),\n",
150
        "    'param_7': tf.io.FixedLenFeature([], tf.float32),\n",
151
        "    'param_8': tf.io.FixedLenFeature([], tf.float32),\n",
152
        "    'param_9': tf.io.FixedLenFeature([], tf.float32),\n",
153
        "    'param_10': tf.io.FixedLenFeature([], tf.float32),\n",
154
        "    'runtime': tf.io.FixedLenFeature([], tf.float32),\n",
155
        "    'area': tf.io.FixedLenFeature([], tf.float32),\n",
156
        "    'infeasible':tf.io.FixedLenFeature([], tf.int64),\n",
157
        "  }\n",
158
        "  return tf.io.parse_single_example(proto, prime_feature_description)"
159
      ],
160
      "metadata": {
161
        "cellView": "form",
162
        "id": "uMWHVyBHKpja"
163
      },
164
      "execution_count": 6,
165
      "outputs": []
166
    },
167
    {
168
      "cell_type": "code",
169
      "source": [
170
        "#@title Parsing the dataset for the studied application\n",
171
        "model_name = 'm4' #@param [\"MobilenetEdgeTPU\", \"MobilenetV2\", \"MobilenetV3\", \"m4\", \"m5\", \"m6\", \"t_rnn_dec\", \"t_rnn_enc\", \"u-net\"]\n",
172
        "filenames = tf.io.gfile.glob(f'gs://gresearch/prime/{model_name}/*.tfrecord')\n",
173
        "raw_dataset = tf.data.TFRecordDataset(filenames, num_parallel_reads=64)\n",
174
        "parsed_dataset = raw_dataset.map(parse_prime_tfrecords)"
175
      ],
176
      "metadata": {
177
        "cellView": "form",
178
        "id": "WAZFGWMgKuFR"
179
      },
180
      "execution_count": 10,
181
      "outputs": []
182
    },
183
    {
184
      "cell_type": "code",
185
      "source": [
186
        "#@title Reproducing the data in the Table 1\n",
187
        "number_of_infeasibles = 0\n",
188
        "number_of_feasibles = 0\n",
189
        "latency = []\n",
190
        "# Parsing the whole dataset for an application may take 15-20 minutes.\n",
191
        "for p in parsed_dataset:\n",
192
        "  if p['infeasible'] == 0:\n",
193
        "    number_of_feasibles+=1\n",
194
        "    latency.append(p['runtime'])\n",
195
        "  else:\n",
196
        "    number_of_infeasibles+=1\n",
197
        "print(f'{number_of_infeasibles},{number_of_feasibles},{max(latency)},{min(latency)},{np.average(latency)}')"
198
      ],
199
      "metadata": {
200
        "colab": {
201
          "base_uri": "https://localhost:8080/"
202
        },
203
        "id": "rAYVi6wqLVGb",
204
        "outputId": "60033c83-59a1-4512-8fc5-aef46628fa7d"
205
      },
206
      "execution_count": 13,
207
      "outputs": [
208
        {
209
          "output_type": "stream",
210
          "name": "stdout",
211
          "text": [
212
            "791984,208148,35881.3515625,335.59100341796875,794.33349609375\n"
213
          ]
214
        }
215
      ]
216
    }
217
  ]
218
}