tqdm
/
DEMO.ipynb
890 строк · 28.4 Кб
1{
2"cells": [
3{
4"cell_type": "markdown",
5"metadata": {},
6"source": [
7"<h1 align=\"center\">tqdm</h1>\n",
8"<img src=\"https://tqdm.github.io/img/logo.gif\" align=\"left\" />\n",
9"\n",
10"[![Py-Versions](https://img.shields.io/pypi/pyversions/tqdm.svg?logo=python&logoColor=white)](https://pypi.org/project/tqdm)|[![Versions](https://img.shields.io/pypi/v/tqdm.svg)](https://tqdm.github.io/releases)|[![Conda-Forge-Status](https://img.shields.io/conda/v/conda-forge/tqdm.svg?label=conda-forge&logo=conda-forge)](https://anaconda.org/conda-forge/tqdm)|[![Docker](https://img.shields.io/badge/docker-pull-blue.svg?logo=docker&logoColor=white)](https://hub.docker.com/r/tqdm/tqdm)|[![Snapcraft](https://img.shields.io/badge/snap-install-82BEA0.svg?logo=snapcraft)](https://snapcraft.io/tqdm)\n",
11"-|-|-|-|-\n",
12"\n",
13"[![Build-Status](https://img.shields.io/github/actions/workflow/status/tqdm/tqdm/test.yml?branch=master&label=tqdm&logo=GitHub)](https://github.com/tqdm/tqdm/actions/workflows/test.yml)|[![Coverage-Status](https://img.shields.io/coveralls/github/tqdm/tqdm/master?logo=coveralls)](https://coveralls.io/github/tqdm/tqdm)|[![Branch-Coverage-Status](https://codecov.io/gh/tqdm/tqdm/branch/master/graph/badge.svg)](https://codecov.io/gh/tqdm/tqdm)|[![Codacy-Grade](https://app.codacy.com/project/badge/Grade/3f965571598f44549c7818f29cdcf177)](https://www.codacy.com/gh/tqdm/tqdm/dashboard)|[![Libraries-Rank](https://img.shields.io/librariesio/sourcerank/pypi/tqdm.svg?logo=koding&logoColor=white)](https://libraries.io/pypi/tqdm)|[![PyPI-Downloads](https://img.shields.io/pypi/dm/tqdm.svg?label=pypi%20downloads&logo=PyPI&logoColor=white)](https://pepy.tech/project/tqdm)\n",
14"-|-|-|-|-|-\n",
15"\n",
16"[![DOI](https://img.shields.io/badge/DOI-10.5281/zenodo.595120-blue.svg)](https://doi.org/10.5281/zenodo.595120)|[![LICENCE](https://img.shields.io/pypi/l/tqdm.svg)](https://raw.githubusercontent.com/tqdm/tqdm/master/LICENCE)|[![OpenHub-Status](https://www.openhub.net/p/tqdm/widgets/project_thin_badge?format=gif)](https://www.openhub.net/p/tqdm?ref=Thin+badge)|[![binder-demo](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tqdm/tqdm/master?filepath=DEMO.ipynb)|[![awesome-python](https://awesome.re/mentioned-badge.svg)](https://github.com/vinta/awesome-python)\n",
17"-|-|-|-|-\n",
18"\n",
19"`tqdm` derives from the Arabic word *taqaddum* (تقدّم) which can mean\n",
20"\"progress,\" and is an abbreviation for \"I love you so much\" in Spanish\n",
21"(*te quiero demasiado*).\n",
22"\n",
23"Instantly make your loops show a smart progress meter - just wrap any\n",
24"iterable with `tqdm(iterable)`, and you're done!"
25]
26},
27{
28"cell_type": "code",
29"execution_count": null,
30"metadata": {},
31"outputs": [],
32"source": [
33"from tqdm import tqdm\n",
34"for i in tqdm(range(10000)):\n",
35" pass"
36]
37},
38{
39"cell_type": "markdown",
40"metadata": {},
41"source": [
42"`trange(N)` can be also used as a convenient shortcut for\n",
43"`tqdm(range(N))`."
44]
45},
46{
47"cell_type": "code",
48"execution_count": null,
49"metadata": {},
50"outputs": [],
51"source": [
52"from tqdm import trange\n",
53"for i in trange(10000, unit_scale=True, desc=\"hello\", unit=\"epoch\"):\n",
54" pass"
55]
56},
57{
58"cell_type": "markdown",
59"metadata": {},
60"source": [
61"![Screenshot](https://tqdm.github.io/img/tqdm.gif)|[![Video](https://tqdm.github.io/img/video.jpg)](https://tqdm.github.io/video) [![Slides](https://tqdm.github.io/img/slides.jpg)](https://tqdm.github.io/PyData2019/slides.html) [![Merch](https://tqdm.github.io/img/merch.jpg)](https://tqdm.github.io/merch)\n",
62"-|-\n",
63"\n",
64"It can also be executed as a module with pipes:"
65]
66},
67{
68"cell_type": "code",
69"execution_count": null,
70"metadata": {},
71"outputs": [],
72"source": [
73"! seq 9999999 | tqdm --bytes | wc -l"
74]
75},
76{
77"cell_type": "markdown",
78"metadata": {},
79"source": [
80"```sh\n",
81"tar -zcf - docs/ | tqdm --bytes --total `du -sb docs/ | cut -f1` > backup.tgz\n",
82" 44%|██████████████▊ | 153M/352M [00:14<00:18, 11.0MB/s]\n",
83"```"
84]
85},
86{
87"cell_type": "markdown",
88"metadata": {},
89"source": [
90"Overhead is low -- about 60ns per iteration (80ns with `tqdm.gui`), and\n",
91"is unit tested against performance regression. By comparison, the\n",
92"well-established\n",
93"[ProgressBar](https://github.com/niltonvolpato/python-progressbar) has\n",
94"an 800ns/iter overhead.\n",
95"\n",
96"In addition to its low overhead, `tqdm` uses smart algorithms to predict\n",
97"the remaining time and to skip unnecessary iteration displays, which\n",
98"allows for a negligible overhead in most cases.\n",
99"\n",
100"`tqdm` works on any platform (Linux, Windows, Mac, FreeBSD, NetBSD,\n",
101"Solaris/SunOS), in any console or in a GUI, and is also friendly with\n",
102"IPython/Jupyter notebooks.\n",
103"\n",
104"`tqdm` does not require any dependencies (not even `curses`!), just\n",
105"Python and an environment supporting `carriage return \\r` and\n",
106"`line feed \\n` control characters.\n",
107"\n",
108"---\n",
109"\n",
110"## Usage\n",
111"\n",
112"`tqdm` is very versatile and can be used in a number of ways.\n",
113"The three main ones are given below.\n",
114"\n",
115"### Iterable-based\n",
116"\n",
117"Wrap `tqdm()` around any iterable:"
118]
119},
120{
121"cell_type": "code",
122"execution_count": null,
123"metadata": {},
124"outputs": [],
125"source": [
126"from tqdm import tqdm\n",
127"from time import sleep"
128]
129},
130{
131"cell_type": "code",
132"execution_count": null,
133"metadata": {},
134"outputs": [],
135"source": [
136"text = \"\"\n",
137"for char in tqdm([\"a\", \"b\", \"c\", \"d\"]):\n",
138" sleep(0.25)\n",
139" text = text + char"
140]
141},
142{
143"cell_type": "markdown",
144"metadata": {},
145"source": [
146"`trange(i)` is a special optimised instance of `tqdm(range(i))`:"
147]
148},
149{
150"cell_type": "code",
151"execution_count": null,
152"metadata": {},
153"outputs": [],
154"source": [
155"from tqdm import trange\n",
156"\n",
157"for i in trange(100):\n",
158" sleep(0.01)"
159]
160},
161{
162"cell_type": "markdown",
163"metadata": {},
164"source": [
165"Instantiation outside of the loop allows for manual control over `tqdm()`:"
166]
167},
168{
169"cell_type": "code",
170"execution_count": null,
171"metadata": {},
172"outputs": [],
173"source": [
174"pbar = tqdm([\"a\", \"b\", \"c\", \"d\"])\n",
175"for char in pbar:\n",
176" sleep(0.25)\n",
177" pbar.set_description(\"Processing %s\" % char)"
178]
179},
180{
181"cell_type": "markdown",
182"metadata": {},
183"source": [
184"### Manual\n",
185"\n",
186"Manual control of `tqdm()` updates using a `with` statement:"
187]
188},
189{
190"cell_type": "code",
191"execution_count": null,
192"metadata": {},
193"outputs": [],
194"source": [
195"with tqdm(total=100) as pbar:\n",
196" for i in range(10):\n",
197" sleep(0.1)\n",
198" pbar.update(10)"
199]
200},
201{
202"cell_type": "markdown",
203"metadata": {},
204"source": [
205"If the optional variable `total` (or an iterable with `len()`) is\n",
206"provided, predictive stats are displayed.\n",
207"\n",
208"`with` is also optional (you can just assign `tqdm()` to a variable,\n",
209"but in this case don't forget to `del` or `close()` at the end:"
210]
211},
212{
213"cell_type": "code",
214"execution_count": null,
215"metadata": {},
216"outputs": [],
217"source": [
218"pbar = tqdm(total=100)\n",
219"for i in range(10):\n",
220" sleep(0.1)\n",
221" pbar.update(10)\n",
222"pbar.close()"
223]
224},
225{
226"cell_type": "markdown",
227"metadata": {},
228"source": [
229"### Module"
230]
231},
232{
233"cell_type": "markdown",
234"metadata": {},
235"source": [
236"Perhaps the most wonderful use of `tqdm` is in a script or on the\n",
237"command line. Simply inserting `tqdm` (or `python -m tqdm`) between\n",
238"pipes will pass through all `stdin` to `stdout` while printing progress\n",
239"to `stderr`.\n",
240"\n",
241"The example below demonstrated counting the number of lines in all\n",
242"Python files in the current directory, with timing information included."
243]
244},
245{
246"cell_type": "code",
247"execution_count": null,
248"metadata": {},
249"outputs": [],
250"source": [
251"! time find . -name '*.py' -type f -exec cat \\{} \\; | wc -l"
252]
253},
254{
255"cell_type": "code",
256"execution_count": null,
257"metadata": {},
258"outputs": [],
259"source": [
260"! time find . -name '*.py' -type f -exec cat \\{} \\; | tqdm | wc -l"
261]
262},
263{
264"cell_type": "markdown",
265"metadata": {},
266"source": [
267"Note that the usual arguments for `tqdm` can also be specified."
268]
269},
270{
271"cell_type": "code",
272"execution_count": null,
273"metadata": {},
274"outputs": [],
275"source": [
276"! find . -name '*.py' -type f -exec cat \\{} \\; | tqdm --unit loc --unit-scale --total 4104300 --null"
277]
278},
279{
280"cell_type": "markdown",
281"metadata": {},
282"source": [
283"Backing up a large directory?\n",
284"\n",
285"```sh\n",
286"tar -zcf - docs/ | tqdm --bytes --total `du -sb docs/ | cut -f1` > backup.tgz\n",
287" 44%|██████████████▊ | 153M/352M [00:14<00:18, 11.0MB/s]\n",
288"```\n",
289"\n",
290"This can be beautified further:\n",
291"\n",
292"```sh\n",
293"BYTES=\"$(du -sb docs/ | cut -f1)\"\n",
294"tar -cf - docs/ \\\n",
295" | tqdm --bytes --total \"$BYTES\" --desc Processing | gzip \\\n",
296" | tqdm --bytes --total \"$BYTES\" --desc Compressed --position 1 \\\n",
297" > ~/backup.tgz\n",
298"Processing: 100%|██████████████████████| 352M/352M [00:14<00:00, 30.2MB/s]\n",
299"Compressed: 42%|█████████▎ | 148M/352M [00:14<00:19, 10.9MB/s]\n",
300"```\n",
301"\n",
302"Or done on a file level using 7-zip:\n",
303"\n",
304"```sh\n",
305"7z a -bd -r backup.7z docs/ | grep Compressing \\\n",
306" | tqdm --total $(find docs/ -type f | wc -l) --unit files \\\n",
307" | grep -v Compressing\n",
308"100%|██████████████████████████▉| 15327/15327 [01:00<00:00, 712.96files/s]\n",
309"```"
310]
311},
312{
313"cell_type": "markdown",
314"metadata": {},
315"source": [
316"## Documentation"
317]
318},
319{
320"cell_type": "code",
321"execution_count": null,
322"metadata": {},
323"outputs": [],
324"source": [
325"tqdm?"
326]
327},
328{
329"cell_type": "code",
330"execution_count": null,
331"metadata": {},
332"outputs": [],
333"source": [
334"! tqdm --help"
335]
336},
337{
338"cell_type": "markdown",
339"metadata": {},
340"source": [
341"## Examples and Advance Usage\n",
342"\n",
343"- See the [examples](https://github.com/tqdm/tqdm/tree/master/examples)\n",
344" folder;\n",
345"- import the module and run `help()`;\n",
346"- consult the [wiki](https://github.com/tqdm/tqdm/wiki)\n",
347" - this has an\n",
348" [excellent article](https://github.com/tqdm/tqdm/wiki/How-to-make-a-great-Progress-Bar)\n",
349" on how to make a **great** progressbar;\n",
350"- check out the [slides from PyData London](https://tqdm.github.io/PyData2019/slides.html), or\n",
351"- run this file!\n",
352"\n",
353"### Description and additional stats\n",
354"\n",
355"Custom information can be displayed and updated dynamically on `tqdm` bars\n",
356"with the `desc` and `postfix` arguments:"
357]
358},
359{
360"cell_type": "code",
361"execution_count": null,
362"metadata": {},
363"outputs": [],
364"source": [
365"from tqdm import tqdm, trange\n",
366"from random import random, randint\n",
367"from time import sleep\n",
368"\n",
369"with trange(10) as t:\n",
370" for i in t:\n",
371" # Description will be displayed on the left\n",
372" t.set_description('GEN %i' % i)\n",
373" # Postfix will be displayed on the right,\n",
374" # formatted automatically based on argument's datatype\n",
375" t.set_postfix(loss=random(), gen=randint(1,999), str='h',\n",
376" lst=[1, 2])\n",
377" sleep(0.1)\n",
378"\n",
379"with tqdm(total=10, bar_format=\"{postfix[0]} {postfix[1][value]:>8.2g}\",\n",
380" postfix=[\"Batch\", dict(value=0)]) as t:\n",
381" for i in range(10):\n",
382" sleep(0.1)\n",
383" t.postfix[1][\"value\"] = i / 2\n",
384" t.update()"
385]
386},
387{
388"cell_type": "markdown",
389"metadata": {},
390"source": [
391"Points to remember when using `{postfix[...]}` in the `bar_format` string:\n",
392"\n",
393"- `postfix` also needs to be passed as an initial argument in a\n",
394" compatible format, and\n",
395"- `postfix` will be auto-converted to a string if it is a `dict`-like\n",
396" object. To prevent this behaviour, insert an extra item into the\n",
397" dictionary where the key is not a string.\n",
398"\n",
399"Additional `bar_format` parameters may also be defined by overriding\n",
400"`format_dict`, and the bar itself may be modified using `ascii`:"
401]
402},
403{
404"cell_type": "code",
405"execution_count": null,
406"metadata": {},
407"outputs": [],
408"source": [
409"from tqdm import tqdm\n",
410"from time import sleep\n",
411"\n",
412"class TqdmExtraFormat(tqdm):\n",
413" \"\"\"Provides a `total_time` format parameter\"\"\"\n",
414" @property\n",
415" def format_dict(self):\n",
416" d = super().format_dict\n",
417" total_time = d[\"elapsed\"] * (d[\"total\"] or 0) / max(d[\"n\"], 1)\n",
418" d.update(total_time=self.format_interval(total_time) + \" in total\")\n",
419" return d\n",
420"\n",
421"for i in TqdmExtraFormat(\n",
422" range(9), ascii=\" .oO0\",\n",
423" bar_format=\"{total_time}: {percentage:.0f}%|{bar}{r_bar}\"):\n",
424" if i == 4:\n",
425" break"
426]
427},
428{
429"cell_type": "markdown",
430"metadata": {},
431"source": [
432"Note that `{bar}` also supports a format specifier `[width][type]`.\n",
433"\n",
434"- `width`\n",
435" + unspecified (default): automatic to fill `ncols`\n",
436" + `int >= 0`: fixed width overriding `ncols` logic\n",
437" + `int < 0`: subtract from the automatic default\n",
438"- `type`\n",
439" + `a`: ascii (`ascii=True` override)\n",
440" + `u`: unicode (`ascii=False` override)\n",
441" + `b`: blank (`ascii=\" \"` override)\n",
442"\n",
443"This means a fixed bar with right-justified text may be created by\n",
444"using: `bar_format=\"{l_bar}{bar:10}|{bar:-10b}right-justified\"`"
445]
446},
447{
448"cell_type": "markdown",
449"metadata": {},
450"source": [
451"### Nested progress bars\n",
452"\n",
453"`tqdm` supports nested progress bars. Here's an example:"
454]
455},
456{
457"cell_type": "code",
458"execution_count": null,
459"metadata": {},
460"outputs": [],
461"source": [
462"from tqdm.auto import trange\n",
463"from time import sleep\n",
464"\n",
465"for i in trange(4, desc='1st loop'):\n",
466" for j in trange(5, desc='2nd loop'):\n",
467" for k in trange(50, desc='3rd loop', leave=False):\n",
468" sleep(0.01)"
469]
470},
471{
472"cell_type": "markdown",
473"metadata": {},
474"source": [
475"For manual control over positioning (e.g. for multi-processing use),\n",
476"you may specify `position=n` where `n=0` for the outermost bar, `n=1`\n",
477"for the next, and so on. However, it's best to check if tqdm can work\n",
478"without manual position first.\n",
479"\n",
480"```python\n",
481"from time import sleep\n",
482"from tqdm import trange, tqdm\n",
483"from multiprocessing import Pool, RLock, freeze_support\n",
484"\n",
485"L = list(range(9))\n",
486"\n",
487"def progresser(n):\n",
488" interval = 0.001 / (n + 2)\n",
489" total = 5000\n",
490" text = \"#{}, est. {:<04.2}s\".format(n, interval * total)\n",
491" for _ in trange(total, desc=text, position=n):\n",
492" sleep(interval)\n",
493"\n",
494"if __name__ == '__main__':\n",
495" freeze_support() # for Windows support\n",
496" tqdm.set_lock(RLock()) # for managing output contention\n",
497" p = Pool(initializer=tqdm.set_lock, initargs=(tqdm.get_lock(),))\n",
498" p.map(progresser, L)\n",
499"```\n",
500"\n",
501"Note that in Python 3, `tqdm.write` is thread-safe:\n",
502"\n",
503"```python\n",
504"from time import sleep\n",
505"from tqdm import tqdm, trange\n",
506"from concurrent.futures import ThreadPoolExecutor\n",
507"\n",
508"L = list(range(9))\n",
509"\n",
510"def progresser(n):\n",
511" interval = 0.001 / (n + 2)\n",
512" total = 5000\n",
513" text = \"#{}, est. {:<04.2}s\".format(n, interval * total)\n",
514" for _ in trange(total, desc=text):\n",
515" sleep(interval).auto\n",
516" if n == 6:\n",
517" tqdm.write(\"n == 6 completed.\")\n",
518" tqdm.write(\"`tqdm.write()` is thread-safe in py3!\")\n",
519"\n",
520"if __name__ == '__main__':\n",
521" with ThreadPoolExecutor() as p:\n",
522" p.map(progresser, L)\n",
523"```"
524]
525},
526{
527"cell_type": "markdown",
528"metadata": {},
529"source": [
530"### Hooks and callbacks\n",
531"\n",
532"`tqdm` can easily support callbacks/hooks and manual updates.\n",
533"Here's an example with `urllib`:\n",
534"\n",
535"**`urllib.urlretrieve` documentation**\n",
536"\n",
537"> [...]\n",
538"> If present, the hook function will be called once\n",
539"> on establishment of the network connection and once after each block read\n",
540"> thereafter. The hook will be passed three arguments; a count of blocks\n",
541"> transferred so far, a block size in bytes, and the total size of the file.\n",
542"> [...]"
543]
544},
545{
546"cell_type": "code",
547"execution_count": null,
548"metadata": {},
549"outputs": [],
550"source": [
551"import urllib, os\n",
552"from tqdm import tqdm\n",
553"urllib = getattr(urllib, 'request', urllib)\n",
554"\n",
555"class TqdmUpTo(tqdm):\n",
556" \"\"\"Provides `update_to(n)` which uses `tqdm.update(delta_n)`.\"\"\"\n",
557" def update_to(self, b=1, bsize=1, tsize=None):\n",
558" \"\"\"\n",
559" b : int, optional\n",
560" Number of blocks transferred so far [default: 1].\n",
561" bsize : int, optional\n",
562" Size of each block (in tqdm units) [default: 1].\n",
563" tsize : int, optional\n",
564" Total size (in tqdm units). If [default: None] remains unchanged.\n",
565" \"\"\"\n",
566" if tsize is not None:\n",
567" self.total = tsize\n",
568" return self.update(b * bsize - self.n) # also sets self.n = b * bsize\n",
569"\n",
570"eg_link = \"https://caspersci.uk.to/matryoshka.zip\"\n",
571"with TqdmUpTo(unit='B', unit_scale=True, unit_divisor=1024, miniters=1,\n",
572" desc=eg_link.split('/')[-1]) as t: # all optional kwargs\n",
573" urllib.urlretrieve(eg_link, filename=os.devnull,\n",
574" reporthook=t.update_to, data=None)\n",
575" t.total = t.n"
576]
577},
578{
579"cell_type": "markdown",
580"metadata": {},
581"source": [
582"Inspired by [twine#242](https://github.com/pypa/twine/pull/242).\n",
583"Functional alternative in\n",
584"[examples/tqdm_wget.py](https://github.com/tqdm/tqdm/blob/master/examples/tqdm_wget.py).\n",
585"\n",
586"It is recommend to use `miniters=1` whenever there is potentially large\n",
587"differences in iteration speed (e.g. downloading a file over a patchy\n",
588"connection).\n",
589"\n",
590"**Wrapping read/write methods**\n",
591"\n",
592"To measure throughput through a file-like object's `read` or `write`\n",
593"methods, use `CallbackIOWrapper`:\n",
594"\n",
595"```python\n",
596"from tqdm import tqdm\n",
597"from tqdm.utils import CallbackIOWrapper\n",
598"\n",
599"with tqdm(total=file_obj.size,\n",
600" unit='B', unit_scale=True, unit_divisor=1024) as t:\n",
601" fobj = CallbackIOWrapper(t.update, file_obj, \"read\")\n",
602" while True:\n",
603" chunk = fobj.read(chunk_size)\n",
604" if not chunk:\n",
605" break\n",
606" t.reset()\n",
607" # ... continue to use `t` for something else\n",
608"```\n",
609"\n",
610"Alternatively, use the even simpler `wrapattr` convenience function,\n",
611"which would condense both the `urllib` and `CallbackIOWrapper` examples\n",
612"down to:"
613]
614},
615{
616"cell_type": "code",
617"execution_count": null,
618"metadata": {},
619"outputs": [],
620"source": [
621"import urllib, os\n",
622"from tqdm import tqdm\n",
623"\n",
624"eg_link = \"https://caspersci.uk.to/matryoshka.zip\"\n",
625"response = getattr(urllib, 'request', urllib).urlopen(eg_link)\n",
626"with tqdm.wrapattr(open(os.devnull, \"wb\"), \"write\",\n",
627" miniters=1, desc=eg_link.split('/')[-1],\n",
628" total=getattr(response, 'length', None)) as fout:\n",
629" for chunk in response:\n",
630" fout.write(chunk)"
631]
632},
633{
634"cell_type": "markdown",
635"metadata": {},
636"source": [
637"The `requests` equivalent is nearly identical:"
638]
639},
640{
641"cell_type": "code",
642"execution_count": null,
643"metadata": {},
644"outputs": [],
645"source": [
646"import requests, os\n",
647"from tqdm import tqdm\n",
648"\n",
649"eg_link = \"https://caspersci.uk.to/matryoshka.zip\"\n",
650"response = requests.get(eg_link, stream=True)\n",
651"with tqdm.wrapattr(open(os.devnull, \"wb\"), \"write\",\n",
652" miniters=1, desc=eg_link.split('/')[-1],\n",
653" total=int(response.headers.get('content-length', 0))) as fout:\n",
654" for chunk in response.iter_content(chunk_size=4096):\n",
655" fout.write(chunk)"
656]
657},
658{
659"cell_type": "markdown",
660"metadata": {},
661"source": [
662"### Pandas Integration\n",
663"\n",
664"Due to popular demand we've added support for `pandas` -- here's an example\n",
665"for `DataFrame.progress_apply` and `DataFrameGroupBy.progress_apply`:"
666]
667},
668{
669"cell_type": "code",
670"execution_count": null,
671"metadata": {},
672"outputs": [],
673"source": [
674"import pandas as pd\n",
675"import numpy as np\n",
676"from tqdm import tqdm\n",
677"\n",
678"df = pd.DataFrame(np.random.randint(0, 100, (100000, 6)))\n",
679"\n",
680"# Register `pandas.progress_apply` and `pandas.Series.map_apply` with `tqdm`\n",
681"# (can use `tqdm.gui.tqdm`, `tqdm.notebook.tqdm`, optional kwargs, etc.)\n",
682"tqdm.pandas(desc=\"my bar!\")\n",
683"\n",
684"# Now you can use `progress_apply` instead of `apply`\n",
685"# and `progress_map` instead of `map`\n",
686"df.progress_apply(lambda x: x**2)\n",
687"# can also groupby:\n",
688"# df.groupby(0).progress_apply(lambda x: x**2)"
689]
690},
691{
692"cell_type": "markdown",
693"metadata": {},
694"source": [
695"In case you're interested in how this works (and how to modify it for\n",
696"your own callbacks), see the\n",
697"[examples](https://github.com/tqdm/tqdm/tree/master/examples) folder or\n",
698"import the module and run `help()`.\n",
699"\n",
700"### Keras Integration\n",
701"\n",
702"A `keras` callback is also available:\n",
703"\n",
704"```python\n",
705"from tqdm.keras import TqdmCallback\n",
706"\n",
707"...\n",
708"\n",
709"model.fit(..., verbose=0, callbacks=[TqdmCallback()])\n",
710"```\n",
711"\n",
712"### IPython/Jupyter Integration\n",
713"\n",
714"IPython/Jupyter is supported via the `tqdm.notebook` submodule:"
715]
716},
717{
718"cell_type": "code",
719"execution_count": null,
720"metadata": {},
721"outputs": [],
722"source": [
723"from tqdm.notebook import trange, tqdm\n",
724"from time import sleep\n",
725"\n",
726"for i in trange(3, desc='1st loop'):\n",
727" for j in tqdm(range(100), desc='2nd loop'):\n",
728" sleep(0.01)"
729]
730},
731{
732"cell_type": "markdown",
733"metadata": {},
734"source": [
735"In addition to `tqdm` features, the submodule provides a native Jupyter\n",
736"widget (compatible with IPython v1-v4 and Jupyter), fully working nested\n",
737"bars and colour hints (blue: normal, green: completed, red:\n",
738"error/interrupt, light blue: no ETA); as demonstrated below.\n",
739"\n",
740"![Screenshot-Jupyter3](https://tqdm.github.io/img/jupyter-3.gif)\n",
741"\n",
742"The `notebook` version supports percentage or pixels for overall width\n",
743"(e.g.: `ncols='100%'` or `ncols='480px'`).\n",
744"\n",
745"It is also possible to let `tqdm` automatically choose between console\n",
746"or notebook versions by using the `autonotebook` submodule:"
747]
748},
749{
750"cell_type": "code",
751"execution_count": null,
752"metadata": {},
753"outputs": [],
754"source": [
755"from tqdm.autonotebook import tqdm\n",
756"tqdm.pandas()"
757]
758},
759{
760"cell_type": "markdown",
761"metadata": {},
762"source": [
763"Note that this will issue a `TqdmExperimentalWarning` if run in a\n",
764"notebook since it is not meant to be possible to distinguish between\n",
765"`jupyter notebook` and `jupyter console`. Use `auto` instead of\n",
766"`autonotebook` to suppress this warning.\n",
767"\n",
768"Note that notebooks will display the bar in the cell where it was\n",
769"created. This may be a different cell from the one where it is used. If\n",
770"this is not desired, the creation of the bar must be delayed/moved to\n",
771"the cell where it is desired to be displayed.\n",
772"\n",
773"Another possibility is to have a single bar (near the top of the\n",
774"notebook) which is constantly re-used (using `reset()` rather than\n",
775"`close()`). For this reason, the notebook version (unlike the CLI\n",
776"version) does not automatically call `close()` upon `Exception`."
777]
778},
779{
780"cell_type": "code",
781"execution_count": null,
782"metadata": {},
783"outputs": [],
784"source": [
785"from tqdm.notebook import tqdm\n",
786"pbar = tqdm()"
787]
788},
789{
790"cell_type": "code",
791"execution_count": null,
792"metadata": {},
793"outputs": [],
794"source": [
795"# different cell\n",
796"iterable = range(100)\n",
797"pbar.reset(total=len(iterable)) # initialise with new `total`\n",
798"for i in iterable:\n",
799" pbar.update()\n",
800"pbar.refresh() # force print final status but don't `close()`"
801]
802},
803{
804"cell_type": "markdown",
805"metadata": {},
806"source": [
807"### Writing messages\n",
808"\n",
809"This is a work in progress (see\n",
810"[#737](https://github.com/tqdm/tqdm/issues/737)).\n",
811"\n",
812"Since `tqdm` uses a simple printing mechanism to display progress bars,\n",
813"you should not write any message in the terminal using `print()` while a\n",
814"progressbar is open.\n",
815"\n",
816"To write messages in the terminal without any collision with `tqdm` bar\n",
817"display, a `.write()` method is provided:"
818]
819},
820{
821"cell_type": "code",
822"execution_count": null,
823"metadata": {},
824"outputs": [],
825"source": [
826"from tqdm.auto import tqdm, trange\n",
827"from time import sleep\n",
828"\n",
829"bar = trange(10)\n",
830"for i in bar:\n",
831" # Print using tqdm class method .write()\n",
832" sleep(0.1)\n",
833" if not (i % 3):\n",
834" tqdm.write(\"Done task %i\" % i)\n",
835" # Can also use bar.write()"
836]
837},
838{
839"cell_type": "markdown",
840"metadata": {},
841"source": [
842"By default, this will print to standard output `sys.stdout`. but you can\n",
843"specify any file-like object using the `file` argument. For example,\n",
844"this can be used to redirect the messages writing to a log file or class.\n",
845"\n",
846"[![README-Hits](https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&style=social&r=https://github.com/tqdm/tqdm&l=https://tqdm.github.io/img/favicon.png&f=https://tqdm.github.io/img/logo.gif)](https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&a=plot&r=https://github.com/tqdm/tqdm&l=https://tqdm.github.io/img/favicon.png&f=https://tqdm.github.io/img/logo.gif&style=social)|(Since 19 May 2016)\n",
847"-|-"
848]
849},
850{
851"cell_type": "markdown",
852"metadata": {},
853"source": [
854"## Do your own experiments here 👇\n",
855"\n",
856"Try `tqdm` youself by adding your code below and running your own experiments."
857]
858},
859{
860"cell_type": "code",
861"execution_count": null,
862"metadata": {},
863"outputs": [],
864"source": [
865"import tqdm\n",
866"\n",
867"# your code here\n",
868"tqdm."
869]
870}
871],
872"metadata": {
873"kernelspec": {
874"display_name": "Python 3",
875"language": "python",
876"name": "python3"
877},
878"language_info": {
879"codemirror_mode": {
880"name": "ipython"
881},
882"file_extension": ".py",
883"mimetype": "text/x-python",
884"name": "python",
885"nbconvert_exporter": "python"
886}
887},
888"nbformat": 4,
889"nbformat_minor": 2
890}
891