tqdm

Форк
0
/
DEMO.ipynb 
890 строк · 28.4 Кб
1
{
2
 "cells": [
3
  {
4
   "cell_type": "markdown",
5
   "metadata": {},
6
   "source": [
7
    "<h1 align=\"center\">tqdm</h1>\n",
8
    "<img src=\"https://tqdm.github.io/img/logo.gif\" align=\"left\" />\n",
9
    "\n",
10
    "[![Py-Versions](https://img.shields.io/pypi/pyversions/tqdm.svg?logo=python&logoColor=white)](https://pypi.org/project/tqdm)|[![Versions](https://img.shields.io/pypi/v/tqdm.svg)](https://tqdm.github.io/releases)|[![Conda-Forge-Status](https://img.shields.io/conda/v/conda-forge/tqdm.svg?label=conda-forge&logo=conda-forge)](https://anaconda.org/conda-forge/tqdm)|[![Docker](https://img.shields.io/badge/docker-pull-blue.svg?logo=docker&logoColor=white)](https://hub.docker.com/r/tqdm/tqdm)|[![Snapcraft](https://img.shields.io/badge/snap-install-82BEA0.svg?logo=snapcraft)](https://snapcraft.io/tqdm)\n",
11
    "-|-|-|-|-\n",
12
    "\n",
13
    "[![Build-Status](https://img.shields.io/github/actions/workflow/status/tqdm/tqdm/test.yml?branch=master&label=tqdm&logo=GitHub)](https://github.com/tqdm/tqdm/actions/workflows/test.yml)|[![Coverage-Status](https://img.shields.io/coveralls/github/tqdm/tqdm/master?logo=coveralls)](https://coveralls.io/github/tqdm/tqdm)|[![Branch-Coverage-Status](https://codecov.io/gh/tqdm/tqdm/branch/master/graph/badge.svg)](https://codecov.io/gh/tqdm/tqdm)|[![Codacy-Grade](https://app.codacy.com/project/badge/Grade/3f965571598f44549c7818f29cdcf177)](https://www.codacy.com/gh/tqdm/tqdm/dashboard)|[![Libraries-Rank](https://img.shields.io/librariesio/sourcerank/pypi/tqdm.svg?logo=koding&logoColor=white)](https://libraries.io/pypi/tqdm)|[![PyPI-Downloads](https://img.shields.io/pypi/dm/tqdm.svg?label=pypi%20downloads&logo=PyPI&logoColor=white)](https://pepy.tech/project/tqdm)\n",
14
    "-|-|-|-|-|-\n",
15
    "\n",
16
    "[![DOI](https://img.shields.io/badge/DOI-10.5281/zenodo.595120-blue.svg)](https://doi.org/10.5281/zenodo.595120)|[![LICENCE](https://img.shields.io/pypi/l/tqdm.svg)](https://raw.githubusercontent.com/tqdm/tqdm/master/LICENCE)|[![OpenHub-Status](https://www.openhub.net/p/tqdm/widgets/project_thin_badge?format=gif)](https://www.openhub.net/p/tqdm?ref=Thin+badge)|[![binder-demo](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tqdm/tqdm/master?filepath=DEMO.ipynb)|[![awesome-python](https://awesome.re/mentioned-badge.svg)](https://github.com/vinta/awesome-python)\n",
17
    "-|-|-|-|-\n",
18
    "\n",
19
    "`tqdm` derives from the Arabic word *taqaddum* (تقدّم) which can mean\n",
20
    "\"progress,\" and is an abbreviation for \"I love you so much\" in Spanish\n",
21
    "(*te quiero demasiado*).\n",
22
    "\n",
23
    "Instantly make your loops show a smart progress meter - just wrap any\n",
24
    "iterable with `tqdm(iterable)`, and you're done!"
25
   ]
26
  },
27
  {
28
   "cell_type": "code",
29
   "execution_count": null,
30
   "metadata": {},
31
   "outputs": [],
32
   "source": [
33
    "from tqdm import tqdm\n",
34
    "for i in tqdm(range(10000)):\n",
35
    "    pass"
36
   ]
37
  },
38
  {
39
   "cell_type": "markdown",
40
   "metadata": {},
41
   "source": [
42
    "`trange(N)` can be also used as a convenient shortcut for\n",
43
    "`tqdm(range(N))`."
44
   ]
45
  },
46
  {
47
   "cell_type": "code",
48
   "execution_count": null,
49
   "metadata": {},
50
   "outputs": [],
51
   "source": [
52
    "from tqdm import trange\n",
53
    "for i in trange(10000, unit_scale=True, desc=\"hello\", unit=\"epoch\"):\n",
54
    "    pass"
55
   ]
56
  },
57
  {
58
   "cell_type": "markdown",
59
   "metadata": {},
60
   "source": [
61
    "![Screenshot](https://tqdm.github.io/img/tqdm.gif)|[![Video](https://tqdm.github.io/img/video.jpg)](https://tqdm.github.io/video) [![Slides](https://tqdm.github.io/img/slides.jpg)](https://tqdm.github.io/PyData2019/slides.html) [![Merch](https://tqdm.github.io/img/merch.jpg)](https://tqdm.github.io/merch)\n",
62
    "-|-\n",
63
    "\n",
64
    "It can also be executed as a module with pipes:"
65
   ]
66
  },
67
  {
68
   "cell_type": "code",
69
   "execution_count": null,
70
   "metadata": {},
71
   "outputs": [],
72
   "source": [
73
    "! seq 9999999 | tqdm --bytes | wc -l"
74
   ]
75
  },
76
  {
77
   "cell_type": "markdown",
78
   "metadata": {},
79
   "source": [
80
    "```sh\n",
81
    "tar -zcf - docs/ | tqdm --bytes --total `du -sb docs/ | cut -f1` > backup.tgz\n",
82
    " 44%|██████████████▊                   | 153M/352M [00:14<00:18, 11.0MB/s]\n",
83
    "```"
84
   ]
85
  },
86
  {
87
   "cell_type": "markdown",
88
   "metadata": {},
89
   "source": [
90
    "Overhead is low -- about 60ns per iteration (80ns with `tqdm.gui`), and\n",
91
    "is unit tested against performance regression. By comparison, the\n",
92
    "well-established\n",
93
    "[ProgressBar](https://github.com/niltonvolpato/python-progressbar) has\n",
94
    "an 800ns/iter overhead.\n",
95
    "\n",
96
    "In addition to its low overhead, `tqdm` uses smart algorithms to predict\n",
97
    "the remaining time and to skip unnecessary iteration displays, which\n",
98
    "allows for a negligible overhead in most cases.\n",
99
    "\n",
100
    "`tqdm` works on any platform (Linux, Windows, Mac, FreeBSD, NetBSD,\n",
101
    "Solaris/SunOS), in any console or in a GUI, and is also friendly with\n",
102
    "IPython/Jupyter notebooks.\n",
103
    "\n",
104
    "`tqdm` does not require any dependencies (not even `curses`!), just\n",
105
    "Python and an environment supporting `carriage return \\r` and\n",
106
    "`line feed \\n` control characters.\n",
107
    "\n",
108
    "---\n",
109
    "\n",
110
    "## Usage\n",
111
    "\n",
112
    "`tqdm` is very versatile and can be used in a number of ways.\n",
113
    "The three main ones are given below.\n",
114
    "\n",
115
    "### Iterable-based\n",
116
    "\n",
117
    "Wrap `tqdm()` around any iterable:"
118
   ]
119
  },
120
  {
121
   "cell_type": "code",
122
   "execution_count": null,
123
   "metadata": {},
124
   "outputs": [],
125
   "source": [
126
    "from tqdm import tqdm\n",
127
    "from time import sleep"
128
   ]
129
  },
130
  {
131
   "cell_type": "code",
132
   "execution_count": null,
133
   "metadata": {},
134
   "outputs": [],
135
   "source": [
136
    "text = \"\"\n",
137
    "for char in tqdm([\"a\", \"b\", \"c\", \"d\"]):\n",
138
    "    sleep(0.25)\n",
139
    "    text = text + char"
140
   ]
141
  },
142
  {
143
   "cell_type": "markdown",
144
   "metadata": {},
145
   "source": [
146
    "`trange(i)` is a special optimised instance of `tqdm(range(i))`:"
147
   ]
148
  },
149
  {
150
   "cell_type": "code",
151
   "execution_count": null,
152
   "metadata": {},
153
   "outputs": [],
154
   "source": [
155
    "from tqdm import trange\n",
156
    "\n",
157
    "for i in trange(100):\n",
158
    "    sleep(0.01)"
159
   ]
160
  },
161
  {
162
   "cell_type": "markdown",
163
   "metadata": {},
164
   "source": [
165
    "Instantiation outside of the loop allows for manual control over `tqdm()`:"
166
   ]
167
  },
168
  {
169
   "cell_type": "code",
170
   "execution_count": null,
171
   "metadata": {},
172
   "outputs": [],
173
   "source": [
174
    "pbar = tqdm([\"a\", \"b\", \"c\", \"d\"])\n",
175
    "for char in pbar:\n",
176
    "    sleep(0.25)\n",
177
    "    pbar.set_description(\"Processing %s\" % char)"
178
   ]
179
  },
180
  {
181
   "cell_type": "markdown",
182
   "metadata": {},
183
   "source": [
184
    "### Manual\n",
185
    "\n",
186
    "Manual control of `tqdm()` updates using a `with` statement:"
187
   ]
188
  },
189
  {
190
   "cell_type": "code",
191
   "execution_count": null,
192
   "metadata": {},
193
   "outputs": [],
194
   "source": [
195
    "with tqdm(total=100) as pbar:\n",
196
    "    for i in range(10):\n",
197
    "        sleep(0.1)\n",
198
    "        pbar.update(10)"
199
   ]
200
  },
201
  {
202
   "cell_type": "markdown",
203
   "metadata": {},
204
   "source": [
205
    "If the optional variable `total` (or an iterable with `len()`) is\n",
206
    "provided, predictive stats are displayed.\n",
207
    "\n",
208
    "`with` is also optional (you can just assign `tqdm()` to a variable,\n",
209
    "but in this case don't forget to `del` or `close()` at the end:"
210
   ]
211
  },
212
  {
213
   "cell_type": "code",
214
   "execution_count": null,
215
   "metadata": {},
216
   "outputs": [],
217
   "source": [
218
    "pbar = tqdm(total=100)\n",
219
    "for i in range(10):\n",
220
    "    sleep(0.1)\n",
221
    "    pbar.update(10)\n",
222
    "pbar.close()"
223
   ]
224
  },
225
  {
226
   "cell_type": "markdown",
227
   "metadata": {},
228
   "source": [
229
    "### Module"
230
   ]
231
  },
232
  {
233
   "cell_type": "markdown",
234
   "metadata": {},
235
   "source": [
236
    "Perhaps the most wonderful use of `tqdm` is in a script or on the\n",
237
    "command line. Simply inserting `tqdm` (or `python -m tqdm`) between\n",
238
    "pipes will pass through all `stdin` to `stdout` while printing progress\n",
239
    "to `stderr`.\n",
240
    "\n",
241
    "The example below demonstrated counting the number of lines in all\n",
242
    "Python files in the current directory, with timing information included."
243
   ]
244
  },
245
  {
246
   "cell_type": "code",
247
   "execution_count": null,
248
   "metadata": {},
249
   "outputs": [],
250
   "source": [
251
    "! time find . -name '*.py' -type f -exec cat \\{} \\; | wc -l"
252
   ]
253
  },
254
  {
255
   "cell_type": "code",
256
   "execution_count": null,
257
   "metadata": {},
258
   "outputs": [],
259
   "source": [
260
    "! time find . -name '*.py' -type f -exec cat \\{} \\; | tqdm | wc -l"
261
   ]
262
  },
263
  {
264
   "cell_type": "markdown",
265
   "metadata": {},
266
   "source": [
267
    "Note that the usual arguments for `tqdm` can also be specified."
268
   ]
269
  },
270
  {
271
   "cell_type": "code",
272
   "execution_count": null,
273
   "metadata": {},
274
   "outputs": [],
275
   "source": [
276
    "! find . -name '*.py' -type f -exec cat \\{} \\; | tqdm --unit loc --unit-scale --total 4104300 --null"
277
   ]
278
  },
279
  {
280
   "cell_type": "markdown",
281
   "metadata": {},
282
   "source": [
283
    "Backing up a large directory?\n",
284
    "\n",
285
    "```sh\n",
286
    "tar -zcf - docs/ | tqdm --bytes --total `du -sb docs/ | cut -f1` > backup.tgz\n",
287
    " 44%|██████████████▊                   | 153M/352M [00:14<00:18, 11.0MB/s]\n",
288
    "```\n",
289
    "\n",
290
    "This can be beautified further:\n",
291
    "\n",
292
    "```sh\n",
293
    "BYTES=\"$(du -sb docs/ | cut -f1)\"\n",
294
    "tar -cf - docs/ \\\n",
295
    "  | tqdm --bytes --total \"$BYTES\" --desc Processing | gzip \\\n",
296
    "  | tqdm --bytes --total \"$BYTES\" --desc Compressed --position 1 \\\n",
297
    "  > ~/backup.tgz\n",
298
    "Processing: 100%|██████████████████████| 352M/352M [00:14<00:00, 30.2MB/s]\n",
299
    "Compressed:  42%|█████████▎            | 148M/352M [00:14<00:19, 10.9MB/s]\n",
300
    "```\n",
301
    "\n",
302
    "Or done on a file level using 7-zip:\n",
303
    "\n",
304
    "```sh\n",
305
    "7z a -bd -r backup.7z docs/ | grep Compressing \\\n",
306
    "  | tqdm --total $(find docs/ -type f | wc -l) --unit files \\\n",
307
    "  | grep -v Compressing\n",
308
    "100%|██████████████████████████▉| 15327/15327 [01:00<00:00, 712.96files/s]\n",
309
    "```"
310
   ]
311
  },
312
  {
313
   "cell_type": "markdown",
314
   "metadata": {},
315
   "source": [
316
    "## Documentation"
317
   ]
318
  },
319
  {
320
   "cell_type": "code",
321
   "execution_count": null,
322
   "metadata": {},
323
   "outputs": [],
324
   "source": [
325
    "tqdm?"
326
   ]
327
  },
328
  {
329
   "cell_type": "code",
330
   "execution_count": null,
331
   "metadata": {},
332
   "outputs": [],
333
   "source": [
334
    "! tqdm --help"
335
   ]
336
  },
337
  {
338
   "cell_type": "markdown",
339
   "metadata": {},
340
   "source": [
341
    "## Examples and Advance Usage\n",
342
    "\n",
343
    "- See the [examples](https://github.com/tqdm/tqdm/tree/master/examples)\n",
344
    "  folder;\n",
345
    "- import the module and run `help()`;\n",
346
    "- consult the [wiki](https://github.com/tqdm/tqdm/wiki)\n",
347
    "    - this has an\n",
348
    "      [excellent article](https://github.com/tqdm/tqdm/wiki/How-to-make-a-great-Progress-Bar)\n",
349
    "      on how to make a **great** progressbar;\n",
350
    "- check out the [slides from PyData London](https://tqdm.github.io/PyData2019/slides.html), or\n",
351
    "- run this file!\n",
352
    "\n",
353
    "### Description and additional stats\n",
354
    "\n",
355
    "Custom information can be displayed and updated dynamically on `tqdm` bars\n",
356
    "with the `desc` and `postfix` arguments:"
357
   ]
358
  },
359
  {
360
   "cell_type": "code",
361
   "execution_count": null,
362
   "metadata": {},
363
   "outputs": [],
364
   "source": [
365
    "from tqdm import tqdm, trange\n",
366
    "from random import random, randint\n",
367
    "from time import sleep\n",
368
    "\n",
369
    "with trange(10) as t:\n",
370
    "    for i in t:\n",
371
    "        # Description will be displayed on the left\n",
372
    "        t.set_description('GEN %i' % i)\n",
373
    "        # Postfix will be displayed on the right,\n",
374
    "        # formatted automatically based on argument's datatype\n",
375
    "        t.set_postfix(loss=random(), gen=randint(1,999), str='h',\n",
376
    "                      lst=[1, 2])\n",
377
    "        sleep(0.1)\n",
378
    "\n",
379
    "with tqdm(total=10, bar_format=\"{postfix[0]} {postfix[1][value]:>8.2g}\",\n",
380
    "          postfix=[\"Batch\", dict(value=0)]) as t:\n",
381
    "    for i in range(10):\n",
382
    "        sleep(0.1)\n",
383
    "        t.postfix[1][\"value\"] = i / 2\n",
384
    "        t.update()"
385
   ]
386
  },
387
  {
388
   "cell_type": "markdown",
389
   "metadata": {},
390
   "source": [
391
    "Points to remember when using `{postfix[...]}` in the `bar_format` string:\n",
392
    "\n",
393
    "- `postfix` also needs to be passed as an initial argument in a\n",
394
    "  compatible format, and\n",
395
    "- `postfix` will be auto-converted to a string if it is a `dict`-like\n",
396
    "  object. To prevent this behaviour, insert an extra item into the\n",
397
    "  dictionary where the key is not a string.\n",
398
    "\n",
399
    "Additional `bar_format` parameters may also be defined by overriding\n",
400
    "`format_dict`, and the bar itself may be modified using `ascii`:"
401
   ]
402
  },
403
  {
404
   "cell_type": "code",
405
   "execution_count": null,
406
   "metadata": {},
407
   "outputs": [],
408
   "source": [
409
    "from tqdm import tqdm\n",
410
    "from time import sleep\n",
411
    "\n",
412
    "class TqdmExtraFormat(tqdm):\n",
413
    "    \"\"\"Provides a `total_time` format parameter\"\"\"\n",
414
    "    @property\n",
415
    "    def format_dict(self):\n",
416
    "        d = super().format_dict\n",
417
    "        total_time = d[\"elapsed\"] * (d[\"total\"] or 0) / max(d[\"n\"], 1)\n",
418
    "        d.update(total_time=self.format_interval(total_time) + \" in total\")\n",
419
    "        return d\n",
420
    "\n",
421
    "for i in TqdmExtraFormat(\n",
422
    "      range(9), ascii=\" .oO0\",\n",
423
    "      bar_format=\"{total_time}: {percentage:.0f}%|{bar}{r_bar}\"):\n",
424
    "    if i == 4:\n",
425
    "        break"
426
   ]
427
  },
428
  {
429
   "cell_type": "markdown",
430
   "metadata": {},
431
   "source": [
432
    "Note that `{bar}` also supports a format specifier `[width][type]`.\n",
433
    "\n",
434
    "- `width`\n",
435
    "    + unspecified (default): automatic to fill `ncols`\n",
436
    "    + `int >= 0`: fixed width overriding `ncols` logic\n",
437
    "    + `int < 0`: subtract from the automatic default\n",
438
    "- `type`\n",
439
    "    + `a`: ascii (`ascii=True` override)\n",
440
    "    + `u`: unicode (`ascii=False` override)\n",
441
    "    + `b`: blank (`ascii=\"  \"` override)\n",
442
    "\n",
443
    "This means a fixed bar with right-justified text may be created by\n",
444
    "using: `bar_format=\"{l_bar}{bar:10}|{bar:-10b}right-justified\"`"
445
   ]
446
  },
447
  {
448
   "cell_type": "markdown",
449
   "metadata": {},
450
   "source": [
451
    "### Nested progress bars\n",
452
    "\n",
453
    "`tqdm` supports nested progress bars. Here's an example:"
454
   ]
455
  },
456
  {
457
   "cell_type": "code",
458
   "execution_count": null,
459
   "metadata": {},
460
   "outputs": [],
461
   "source": [
462
    "from tqdm.auto import trange\n",
463
    "from time import sleep\n",
464
    "\n",
465
    "for i in trange(4, desc='1st loop'):\n",
466
    "    for j in trange(5, desc='2nd loop'):\n",
467
    "        for k in trange(50, desc='3rd loop', leave=False):\n",
468
    "            sleep(0.01)"
469
   ]
470
  },
471
  {
472
   "cell_type": "markdown",
473
   "metadata": {},
474
   "source": [
475
    "For manual control over positioning (e.g. for multi-processing use),\n",
476
    "you may specify `position=n` where `n=0` for the outermost bar, `n=1`\n",
477
    "for the next, and so on. However, it's best to check if tqdm can work\n",
478
    "without manual position first.\n",
479
    "\n",
480
    "```python\n",
481
    "from time import sleep\n",
482
    "from tqdm import trange, tqdm\n",
483
    "from multiprocessing import Pool, RLock, freeze_support\n",
484
    "\n",
485
    "L = list(range(9))\n",
486
    "\n",
487
    "def progresser(n):\n",
488
    "    interval = 0.001 / (n + 2)\n",
489
    "    total = 5000\n",
490
    "    text = \"#{}, est. {:<04.2}s\".format(n, interval * total)\n",
491
    "    for _ in trange(total, desc=text, position=n):\n",
492
    "        sleep(interval)\n",
493
    "\n",
494
    "if __name__ == '__main__':\n",
495
    "    freeze_support()  # for Windows support\n",
496
    "    tqdm.set_lock(RLock())  # for managing output contention\n",
497
    "    p = Pool(initializer=tqdm.set_lock, initargs=(tqdm.get_lock(),))\n",
498
    "    p.map(progresser, L)\n",
499
    "```\n",
500
    "\n",
501
    "Note that in Python 3, `tqdm.write` is thread-safe:\n",
502
    "\n",
503
    "```python\n",
504
    "from time import sleep\n",
505
    "from tqdm import tqdm, trange\n",
506
    "from concurrent.futures import ThreadPoolExecutor\n",
507
    "\n",
508
    "L = list(range(9))\n",
509
    "\n",
510
    "def progresser(n):\n",
511
    "    interval = 0.001 / (n + 2)\n",
512
    "    total = 5000\n",
513
    "    text = \"#{}, est. {:<04.2}s\".format(n, interval * total)\n",
514
    "    for _ in trange(total, desc=text):\n",
515
    "        sleep(interval).auto\n",
516
    "    if n == 6:\n",
517
    "        tqdm.write(\"n == 6 completed.\")\n",
518
    "        tqdm.write(\"`tqdm.write()` is thread-safe in py3!\")\n",
519
    "\n",
520
    "if __name__ == '__main__':\n",
521
    "    with ThreadPoolExecutor() as p:\n",
522
    "        p.map(progresser, L)\n",
523
    "```"
524
   ]
525
  },
526
  {
527
   "cell_type": "markdown",
528
   "metadata": {},
529
   "source": [
530
    "### Hooks and callbacks\n",
531
    "\n",
532
    "`tqdm` can easily support callbacks/hooks and manual updates.\n",
533
    "Here's an example with `urllib`:\n",
534
    "\n",
535
    "**`urllib.urlretrieve` documentation**\n",
536
    "\n",
537
    "> [...]\n",
538
    "> If present, the hook function will be called once\n",
539
    "> on establishment of the network connection and once after each block read\n",
540
    "> thereafter. The hook will be passed three arguments; a count of blocks\n",
541
    "> transferred so far, a block size in bytes, and the total size of the file.\n",
542
    "> [...]"
543
   ]
544
  },
545
  {
546
   "cell_type": "code",
547
   "execution_count": null,
548
   "metadata": {},
549
   "outputs": [],
550
   "source": [
551
    "import urllib, os\n",
552
    "from tqdm import tqdm\n",
553
    "urllib = getattr(urllib, 'request', urllib)\n",
554
    "\n",
555
    "class TqdmUpTo(tqdm):\n",
556
    "    \"\"\"Provides `update_to(n)` which uses `tqdm.update(delta_n)`.\"\"\"\n",
557
    "    def update_to(self, b=1, bsize=1, tsize=None):\n",
558
    "        \"\"\"\n",
559
    "        b  : int, optional\n",
560
    "            Number of blocks transferred so far [default: 1].\n",
561
    "        bsize  : int, optional\n",
562
    "            Size of each block (in tqdm units) [default: 1].\n",
563
    "        tsize  : int, optional\n",
564
    "            Total size (in tqdm units). If [default: None] remains unchanged.\n",
565
    "        \"\"\"\n",
566
    "        if tsize is not None:\n",
567
    "            self.total = tsize\n",
568
    "        return self.update(b * bsize - self.n)  # also sets self.n = b * bsize\n",
569
    "\n",
570
    "eg_link = \"https://caspersci.uk.to/matryoshka.zip\"\n",
571
    "with TqdmUpTo(unit='B', unit_scale=True, unit_divisor=1024, miniters=1,\n",
572
    "              desc=eg_link.split('/')[-1]) as t:  # all optional kwargs\n",
573
    "    urllib.urlretrieve(eg_link, filename=os.devnull,\n",
574
    "                       reporthook=t.update_to, data=None)\n",
575
    "    t.total = t.n"
576
   ]
577
  },
578
  {
579
   "cell_type": "markdown",
580
   "metadata": {},
581
   "source": [
582
    "Inspired by [twine#242](https://github.com/pypa/twine/pull/242).\n",
583
    "Functional alternative in\n",
584
    "[examples/tqdm_wget.py](https://github.com/tqdm/tqdm/blob/master/examples/tqdm_wget.py).\n",
585
    "\n",
586
    "It is recommend to use `miniters=1` whenever there is potentially large\n",
587
    "differences in iteration speed (e.g. downloading a file over a patchy\n",
588
    "connection).\n",
589
    "\n",
590
    "**Wrapping read/write methods**\n",
591
    "\n",
592
    "To measure throughput through a file-like object's `read` or `write`\n",
593
    "methods, use `CallbackIOWrapper`:\n",
594
    "\n",
595
    "```python\n",
596
    "from tqdm import tqdm\n",
597
    "from tqdm.utils import CallbackIOWrapper\n",
598
    "\n",
599
    "with tqdm(total=file_obj.size,\n",
600
    "          unit='B', unit_scale=True, unit_divisor=1024) as t:\n",
601
    "    fobj = CallbackIOWrapper(t.update, file_obj, \"read\")\n",
602
    "    while True:\n",
603
    "        chunk = fobj.read(chunk_size)\n",
604
    "        if not chunk:\n",
605
    "            break\n",
606
    "    t.reset()\n",
607
    "    # ... continue to use `t` for something else\n",
608
    "```\n",
609
    "\n",
610
    "Alternatively, use the even simpler `wrapattr` convenience function,\n",
611
    "which would condense both the `urllib` and `CallbackIOWrapper` examples\n",
612
    "down to:"
613
   ]
614
  },
615
  {
616
   "cell_type": "code",
617
   "execution_count": null,
618
   "metadata": {},
619
   "outputs": [],
620
   "source": [
621
    "import urllib, os\n",
622
    "from tqdm import tqdm\n",
623
    "\n",
624
    "eg_link = \"https://caspersci.uk.to/matryoshka.zip\"\n",
625
    "response = getattr(urllib, 'request', urllib).urlopen(eg_link)\n",
626
    "with tqdm.wrapattr(open(os.devnull, \"wb\"), \"write\",\n",
627
    "                   miniters=1, desc=eg_link.split('/')[-1],\n",
628
    "                   total=getattr(response, 'length', None)) as fout:\n",
629
    "    for chunk in response:\n",
630
    "        fout.write(chunk)"
631
   ]
632
  },
633
  {
634
   "cell_type": "markdown",
635
   "metadata": {},
636
   "source": [
637
    "The `requests` equivalent is nearly identical:"
638
   ]
639
  },
640
  {
641
   "cell_type": "code",
642
   "execution_count": null,
643
   "metadata": {},
644
   "outputs": [],
645
   "source": [
646
    "import requests, os\n",
647
    "from tqdm import tqdm\n",
648
    "\n",
649
    "eg_link = \"https://caspersci.uk.to/matryoshka.zip\"\n",
650
    "response = requests.get(eg_link, stream=True)\n",
651
    "with tqdm.wrapattr(open(os.devnull, \"wb\"), \"write\",\n",
652
    "                   miniters=1, desc=eg_link.split('/')[-1],\n",
653
    "                   total=int(response.headers.get('content-length', 0))) as fout:\n",
654
    "    for chunk in response.iter_content(chunk_size=4096):\n",
655
    "        fout.write(chunk)"
656
   ]
657
  },
658
  {
659
   "cell_type": "markdown",
660
   "metadata": {},
661
   "source": [
662
    "### Pandas Integration\n",
663
    "\n",
664
    "Due to popular demand we've added support for `pandas` -- here's an example\n",
665
    "for `DataFrame.progress_apply` and `DataFrameGroupBy.progress_apply`:"
666
   ]
667
  },
668
  {
669
   "cell_type": "code",
670
   "execution_count": null,
671
   "metadata": {},
672
   "outputs": [],
673
   "source": [
674
    "import pandas as pd\n",
675
    "import numpy as np\n",
676
    "from tqdm import tqdm\n",
677
    "\n",
678
    "df = pd.DataFrame(np.random.randint(0, 100, (100000, 6)))\n",
679
    "\n",
680
    "# Register `pandas.progress_apply` and `pandas.Series.map_apply` with `tqdm`\n",
681
    "# (can use `tqdm.gui.tqdm`, `tqdm.notebook.tqdm`, optional kwargs, etc.)\n",
682
    "tqdm.pandas(desc=\"my bar!\")\n",
683
    "\n",
684
    "# Now you can use `progress_apply` instead of `apply`\n",
685
    "# and `progress_map` instead of `map`\n",
686
    "df.progress_apply(lambda x: x**2)\n",
687
    "# can also groupby:\n",
688
    "# df.groupby(0).progress_apply(lambda x: x**2)"
689
   ]
690
  },
691
  {
692
   "cell_type": "markdown",
693
   "metadata": {},
694
   "source": [
695
    "In case you're interested in how this works (and how to modify it for\n",
696
    "your own callbacks), see the\n",
697
    "[examples](https://github.com/tqdm/tqdm/tree/master/examples) folder or\n",
698
    "import the module and run `help()`.\n",
699
    "\n",
700
    "### Keras Integration\n",
701
    "\n",
702
    "A `keras` callback is also available:\n",
703
    "\n",
704
    "```python\n",
705
    "from tqdm.keras import TqdmCallback\n",
706
    "\n",
707
    "...\n",
708
    "\n",
709
    "model.fit(..., verbose=0, callbacks=[TqdmCallback()])\n",
710
    "```\n",
711
    "\n",
712
    "### IPython/Jupyter Integration\n",
713
    "\n",
714
    "IPython/Jupyter is supported via the `tqdm.notebook` submodule:"
715
   ]
716
  },
717
  {
718
   "cell_type": "code",
719
   "execution_count": null,
720
   "metadata": {},
721
   "outputs": [],
722
   "source": [
723
    "from tqdm.notebook import trange, tqdm\n",
724
    "from time import sleep\n",
725
    "\n",
726
    "for i in trange(3, desc='1st loop'):\n",
727
    "    for j in tqdm(range(100), desc='2nd loop'):\n",
728
    "        sleep(0.01)"
729
   ]
730
  },
731
  {
732
   "cell_type": "markdown",
733
   "metadata": {},
734
   "source": [
735
    "In addition to `tqdm` features, the submodule provides a native Jupyter\n",
736
    "widget (compatible with IPython v1-v4 and Jupyter), fully working nested\n",
737
    "bars and colour hints (blue: normal, green: completed, red:\n",
738
    "error/interrupt, light blue: no ETA); as demonstrated below.\n",
739
    "\n",
740
    "![Screenshot-Jupyter3](https://tqdm.github.io/img/jupyter-3.gif)\n",
741
    "\n",
742
    "The `notebook` version supports percentage or pixels for overall width\n",
743
    "(e.g.: `ncols='100%'` or `ncols='480px'`).\n",
744
    "\n",
745
    "It is also possible to let `tqdm` automatically choose between console\n",
746
    "or notebook versions by using the `autonotebook` submodule:"
747
   ]
748
  },
749
  {
750
   "cell_type": "code",
751
   "execution_count": null,
752
   "metadata": {},
753
   "outputs": [],
754
   "source": [
755
    "from tqdm.autonotebook import tqdm\n",
756
    "tqdm.pandas()"
757
   ]
758
  },
759
  {
760
   "cell_type": "markdown",
761
   "metadata": {},
762
   "source": [
763
    "Note that this will issue a `TqdmExperimentalWarning` if run in a\n",
764
    "notebook since it is not meant to be possible to distinguish between\n",
765
    "`jupyter notebook` and `jupyter console`. Use `auto` instead of\n",
766
    "`autonotebook` to suppress this warning.\n",
767
    "\n",
768
    "Note that notebooks will display the bar in the cell where it was\n",
769
    "created. This may be a different cell from the one where it is used. If\n",
770
    "this is not desired, the creation of the bar must be delayed/moved to\n",
771
    "the cell where it is desired to be displayed.\n",
772
    "\n",
773
    "Another possibility is to have a single bar (near the top of the\n",
774
    "notebook) which is constantly re-used (using `reset()` rather than\n",
775
    "`close()`). For this reason, the notebook version (unlike the CLI\n",
776
    "version) does not automatically call `close()` upon `Exception`."
777
   ]
778
  },
779
  {
780
   "cell_type": "code",
781
   "execution_count": null,
782
   "metadata": {},
783
   "outputs": [],
784
   "source": [
785
    "from tqdm.notebook import tqdm\n",
786
    "pbar = tqdm()"
787
   ]
788
  },
789
  {
790
   "cell_type": "code",
791
   "execution_count": null,
792
   "metadata": {},
793
   "outputs": [],
794
   "source": [
795
    "# different cell\n",
796
    "iterable = range(100)\n",
797
    "pbar.reset(total=len(iterable))  # initialise with new `total`\n",
798
    "for i in iterable:\n",
799
    "    pbar.update()\n",
800
    "pbar.refresh()  # force print final status but don't `close()`"
801
   ]
802
  },
803
  {
804
   "cell_type": "markdown",
805
   "metadata": {},
806
   "source": [
807
    "### Writing messages\n",
808
    "\n",
809
    "This is a work in progress (see\n",
810
    "[#737](https://github.com/tqdm/tqdm/issues/737)).\n",
811
    "\n",
812
    "Since `tqdm` uses a simple printing mechanism to display progress bars,\n",
813
    "you should not write any message in the terminal using `print()` while a\n",
814
    "progressbar is open.\n",
815
    "\n",
816
    "To write messages in the terminal without any collision with `tqdm` bar\n",
817
    "display, a `.write()` method is provided:"
818
   ]
819
  },
820
  {
821
   "cell_type": "code",
822
   "execution_count": null,
823
   "metadata": {},
824
   "outputs": [],
825
   "source": [
826
    "from tqdm.auto import tqdm, trange\n",
827
    "from time import sleep\n",
828
    "\n",
829
    "bar = trange(10)\n",
830
    "for i in bar:\n",
831
    "    # Print using tqdm class method .write()\n",
832
    "    sleep(0.1)\n",
833
    "    if not (i % 3):\n",
834
    "        tqdm.write(\"Done task %i\" % i)\n",
835
    "    # Can also use bar.write()"
836
   ]
837
  },
838
  {
839
   "cell_type": "markdown",
840
   "metadata": {},
841
   "source": [
842
    "By default, this will print to standard output `sys.stdout`. but you can\n",
843
    "specify any file-like object using the `file` argument. For example,\n",
844
    "this can be used to redirect the messages writing to a log file or class.\n",
845
    "\n",
846
    "[![README-Hits](https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&style=social&r=https://github.com/tqdm/tqdm&l=https://tqdm.github.io/img/favicon.png&f=https://tqdm.github.io/img/logo.gif)](https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&a=plot&r=https://github.com/tqdm/tqdm&l=https://tqdm.github.io/img/favicon.png&f=https://tqdm.github.io/img/logo.gif&style=social)|(Since 19 May 2016)\n",
847
    "-|-"
848
   ]
849
  },
850
  {
851
   "cell_type": "markdown",
852
   "metadata": {},
853
   "source": [
854
    "## Do your own experiments here 👇\n",
855
    "\n",
856
    "Try `tqdm` youself by adding your code below and running your own experiments."
857
   ]
858
  },
859
  {
860
   "cell_type": "code",
861
   "execution_count": null,
862
   "metadata": {},
863
   "outputs": [],
864
   "source": [
865
    "import tqdm\n",
866
    "\n",
867
    "# your code here\n",
868
    "tqdm."
869
   ]
870
  }
871
 ],
872
 "metadata": {
873
  "kernelspec": {
874
   "display_name": "Python 3",
875
   "language": "python",
876
   "name": "python3"
877
  },
878
  "language_info": {
879
   "codemirror_mode": {
880
    "name": "ipython"
881
   },
882
   "file_extension": ".py",
883
   "mimetype": "text/x-python",
884
   "name": "python",
885
   "nbconvert_exporter": "python"
886
  }
887
 },
888
 "nbformat": 4,
889
 "nbformat_minor": 2
890
}
891

Использование cookies

Мы используем файлы cookie в соответствии с Политикой конфиденциальности и Политикой использования cookies.

Нажимая кнопку «Принимаю», Вы даете АО «СберТех» согласие на обработку Ваших персональных данных в целях совершенствования нашего веб-сайта и Сервиса GitVerse, а также повышения удобства их использования.

Запретить использование cookies Вы можете самостоятельно в настройках Вашего браузера.