NEW! Новый релиз 12.0.0 уже доступен! Подробности в Центре заботы

tqdm

0
README.rst

|Logo|

tqdm

|Py-Versions| |Versions| |Conda-Forge-Status| |Docker| |Snapcraft|

|Build-Status| |Coverage-Status| |Branch-Coverage-Status| |Codacy-Grade| |Libraries-Rank| |PyPI-Downloads|

|LICENCE| |OpenHub-Status| |binder-demo| |awesome-python|

tqdm
derives from the Arabic word taqaddum (تقدّم) which can mean "progress," and is an abbreviation for "I love you so much" in Spanish (te quiero demasiado).

Instantly make your loops show a smart progress meter - just wrap any iterable with

tqdm(iterable)
, and you're done!

.. code:: python

from tqdm import tqdm for i in tqdm(range(10000)): ...

76%|████████████████████████        | 7568/10000 [00:33<00:10, 229.00it/s]

trange(N)
can be also used as a convenient shortcut for
tqdm(range(N))
.

|Screenshot| |Video| |Slides| |Merch|

It can also be executed as a module with pipes:

.. code:: sh

$ seq 9999999 | tqdm --bytes | wc -l 75.2MB [00:00, 217MB/s] 9999999 $ tar -zcf - docs/ | tqdm --bytes --total `du -sb docs/ | cut -f1` \ > backup.tgz 32%|██████████▍ | 8.89G/27.9G [00:42<01:31, 223MB/s]

Overhead is low -- about 60ns per iteration (80ns with

tqdm.gui
), and is unit tested against performance regression. By comparison, the well-established
ProgressBar <https://github.com/niltonvolpato/python-progressbar>
__ has an 800ns/iter overhead.

In addition to its low overhead,

tqdm
uses smart algorithms to predict the remaining time and to skip unnecessary iteration displays, which allows for a negligible overhead in most cases.

tqdm
works on any platform (Linux, Windows, Mac, FreeBSD, NetBSD, Solaris/SunOS), in any console or in a GUI, and is also friendly with IPython/Jupyter notebooks.

tqdm
does not require any dependencies (not even
curses
!), just Python and an environment supporting
carriage return \r
and
line feed \n
control characters.


.. contents:: Table of contents :backlinks: top :local:

Installation

Latest PyPI stable release

|Versions| |PyPI-Downloads| |Libraries-Dependents| .. code:: sh pip install tqdm Latest development release on GitHub

|GitHub-Status| |GitHub-Stars| |GitHub-Commits| |GitHub-Forks| |GitHub-Updated|

Pull and install pre-release

devel
branch:

.. code:: sh

pip install "git+https://github.com/tqdm/tqdm.git@devel#egg=tqdm"

Latest Conda release

|Conda-Forge-Status| .. code:: sh conda install -c conda-forge tqdm Latest Snapcraft release

|Snapcraft|

There are 3 channels to choose from:

.. code:: sh

snap install tqdm # implies --stable, i.e. latest tagged release snap install tqdm --candidate # master branch snap install tqdm --edge # devel branch

Note that

snap
binaries are purely for CLI use (not
import
-able), and automatically set up
bash
tab-completion.

Latest Docker release

|Docker| .. code:: sh docker pull tqdm/tqdm docker run -i --rm tqdm/tqdm --help Other ~~~~~ There are other (unofficial) places where ``tqdm`` may be downloaded, particularly for CLI use: |Repology| .. |Repology| image:: https://repology.org/badge/tiny-repos/python:tqdm.svg :target: https://repology.org/project/python:tqdm/versions Changelog --------- The list of all changes is available either on GitHub's Releases: |GitHub-Status|, on the `wiki <https://github.com/tqdm/tqdm/wiki/Releases>`__, or on the `website <https://tqdm.github.io/releases>`__. Usage ----- ``tqdm`` is very versatile and can be used in a number of ways. The three main ones are given below. Iterable-based ~~~~~~~~~~~~~~ Wrap ``tqdm()`` around any iterable: .. code:: python from tqdm import tqdm from time import sleep text = "" for char in tqdm(["a", "b", "c", "d"]): sleep(0.25) text = text + char ``trange(i)`` is a special optimised instance of ``tqdm(range(i))``: .. code:: python from tqdm import trange for i in trange(100): sleep(0.01) Instantiation outside of the loop allows for manual control over ``tqdm()``: .. code:: python pbar = tqdm(["a", "b", "c", "d"]) for char in pbar: sleep(0.25) pbar.set_description("Processing %s" % char) Manual ~~~~~~ Manual control of ``tqdm()`` updates using a ``with`` statement: .. code:: python with tqdm(total=100) as pbar: for i in range(10): sleep(0.1) pbar.update(10) If the optional variable ``total`` (or an iterable with ``len()``) is provided, predictive stats are displayed. ``with`` is also optional (you can just assign ``tqdm()`` to a variable, but in this case don't forget to ``del`` or ``close()`` at the end: .. code:: python pbar = tqdm(total=100) for i in range(10): sleep(0.1) pbar.update(10) pbar.close() Module ~~~~~~ Perhaps the most wonderful use of ``tqdm`` is in a script or on the command line. Simply inserting ``tqdm`` (or ``python -m tqdm``) between pipes will pass through all ``stdin`` to ``stdout`` while printing progress to ``stderr``. The example below demonstrate counting the number of lines in all Python files in the current directory, with timing information included. .. code:: sh $ time find . -name '*.py' -type f -exec cat \{} \; | wc -l 857365 real 0m3.458s user 0m0.274s sys 0m3.325s $ time find . -name '*.py' -type f -exec cat \{} \; | tqdm | wc -l 857366it [00:03, 246471.31it/s] 857365 real 0m3.585s user 0m0.862s sys 0m3.358s Note that the usual arguments for ``tqdm`` can also be specified. .. code:: sh $ find . -name '*.py' -type f -exec cat \{} \; | tqdm --unit loc --unit_scale --total 857366 >> /dev/null 100%|█████████████████████████████████| 857K/857K [00:04<00:00, 246Kloc/s] Backing up a large directory? .. code:: sh $ tar -zcf - docs/ | tqdm --bytes --total `du -sb docs/ | cut -f1` \ > backup.tgz 44%|██████████████▊ | 153M/352M [00:14<00:18, 11.0MB/s] This can be beautified further: .. code:: sh $ BYTES=$(du -sb docs/ | cut -f1) $ tar -cf - docs/ \ | tqdm --bytes --total "$BYTES" --desc Processing | gzip \ | tqdm --bytes --total "$BYTES" --desc Compressed --position 1 \ > ~/backup.tgz Processing: 100%|██████████████████████| 352M/352M [00:14<00:00, 30.2MB/s] Compressed: 42%|█████████▎ | 148M/352M [00:14<00:19, 10.9MB/s] Or done on a file level using 7-zip: .. code:: sh $ 7z a -bd -r backup.7z docs/ | grep Compressing \ | tqdm --total $(find docs/ -type f | wc -l) --unit files \ | grep -v Compressing 100%|██████████████████████████▉| 15327/15327 [01:00<00:00, 712.96files/s] Pre-existing CLI programs already outputting basic progress information will benefit from ``tqdm``'s ``--update`` and ``--update_to`` flags: .. code:: sh $ seq 3 0.1 5 | tqdm --total 5 --update_to --null 100%|████████████████████████████████████| 5.0/5 [00:00<00:00, 9673.21it/s] $ seq 10 | tqdm --update --null # 1 + 2 + ... + 10 = 55 iterations 55it [00:00, 90006.52it/s] FAQ and Known Issues -------------------- |GitHub-Issues| The most common issues relate to excessive output on multiple lines, instead of a neat one-line progress bar. - Consoles in general: require support for carriage return (``CR``, ``\r``). * Some cloud logging consoles which don't support ``\r`` properly (`cloudwatch <https://github.com/tqdm/tqdm/issues/966>`__, `K8s <https://github.com/tqdm/tqdm/issues/1319>`__) may benefit from ``export TQDM_POSITION=-1``. - Nested progress bars: * Consoles in general: require support for moving cursors up to the previous line. For example, `IDLE <https://github.com/tqdm/tqdm/issues/191#issuecomment-230168030>`__, `ConEmu <https://github.com/tqdm/tqdm/issues/254>`__ and `PyCharm <https://github.com/tqdm/tqdm/issues/203>`__ (also `here <https://github.com/tqdm/tqdm/issues/208>`__, `here <https://github.com/tqdm/tqdm/issues/307>`__, and `here <https://github.com/tqdm/tqdm/issues/454#issuecomment-335416815>`__) lack full support. * Windows: additionally may require the Python module ``colorama`` to ensure nested bars stay within their respective lines. - Unicode: * Environments which report that they support unicode will have solid smooth progressbars. The fallback is an ``ascii``-only bar. * Windows consoles often only partially support unicode and thus `often require explicit ascii=True <https://github.com/tqdm/tqdm/issues/454#issuecomment-335416815>`__ (also `here <https://github.com/tqdm/tqdm/issues/499>`__). This is due to either normal-width unicode characters being incorrectly displayed as "wide", or some unicode characters not rendering. - Wrapping generators: * Generator wrapper functions tend to hide the length of iterables. ``tqdm`` does not. * Replace ``tqdm(enumerate(...))`` with ``enumerate(tqdm(...))`` or ``tqdm(enumerate(x), total=len(x), ...)``. The same applies to ``numpy.ndenumerate``. * Replace ``tqdm(zip(a, b))`` with ``zip(tqdm(a), b)`` or even ``zip(tqdm(a), tqdm(b))``. * The same applies to ``itertools``. * Some useful convenience functions can be found under ``tqdm.contrib``. - `No intermediate output in docker-compose <https://github.com/tqdm/tqdm/issues/771>`__: use ``docker-compose run`` instead of ``docker-compose up`` and ``tty: true``. - Overriding defaults via environment variables: e.g. in CI/cloud jobs, ``export TQDM_MININTERVAL=5`` to avoid log spam. This override logic is handled by the ``tqdm.utils.envwrap`` decorator (useful independent of ``tqdm``). If you come across any other difficulties, browse and file |GitHub-Issues|. Documentation ------------- |Py-Versions| |README-Hits| (Since 19 May 2016) .. code:: python class tqdm(): """ Decorate an iterable object, returning an iterator which acts exactly like the original iterable, but prints a dynamically updating progressbar every time a value is requested. """ @envwrap("TQDM_") # override defaults via env vars def __init__(self, iterable=None, desc=None, total=None, leave=True, file=None, ncols=None, mininterval=0.1, maxinterval=10.0, miniters=None, ascii=None, disable=False, unit='it', unit_scale=False, dynamic_ncols=False, smoothing=0.3, bar_format=None, initial=0, position=None, postfix=None, unit_divisor=1000, write_bytes=False, lock_args=None, nrows=None, colour=None, delay=0): Parameters ~~~~~~~~~~ * iterable : iterable, optional Iterable to decorate with a progressbar. Leave blank to manually manage the updates. * desc : str, optional Prefix for the progressbar. * total : int or float, optional The number of expected iterations. If unspecified, len(iterable) is used if possible. If float("inf") or as a last resort, only basic progress statistics are displayed (no ETA, no progressbar). If ``gui`` is True and this parameter needs subsequent updating, specify an initial arbitrary large positive number, e.g. 9e9. * leave : bool, optional If [default: True], keeps all traces of the progressbar upon termination of iteration. If ``None``, will leave only if ``position`` is ``0``. * file : ``io.TextIOWrapper`` or ``io.StringIO``, optional Specifies where to output the progress messages (default: sys.stderr). Uses ``file.write(str)`` and ``file.flush()`` methods. For encoding, see ``write_bytes``. * ncols : int, optional The width of the entire output message. If specified, dynamically resizes the progressbar to stay within this bound. If unspecified, attempts to use environment width. The fallback is a meter width of 10 and no limit for the counter and statistics. If 0, will not print any meter (only stats). * mininterval : float, optional Minimum progress display update interval [default: 0.1] seconds. * maxinterval : float, optional Maximum progress display update interval [default: 10] seconds. Automatically adjusts ``miniters`` to correspond to ``mininterval`` after long display update lag. Only works if ``dynamic_miniters`` or monitor thread is enabled. * miniters : int or float, optional Minimum progress display update interval, in iterations. If 0 and ``dynamic_miniters``, will automatically adjust to equal ``mininterval`` (more CPU efficient, good for tight loops). If > 0, will skip display of specified number of iterations. Tweak this and ``mininterval`` to get very efficient loops. If your progress is erratic with both fast and slow iterations (network, skipping items, etc) you should set miniters=1. * ascii : bool or str, optional If unspecified or False, use unicode (smooth blocks) to fill the meter. The fallback is to use ASCII characters " 123456789#". * disable : bool, optional Whether to disable the entire progressbar wrapper [default: False]. If set to None, disable on non-TTY. * unit : str, optional String that will be used to define the unit of each iteration [default: it]. * unit_scale : bool or int or float, optional If 1 or True, the number of iterations will be reduced/scaled automatically and a metric prefix following the International System of Units standard will be added (kilo, mega, etc.) [default: False]. If any other non-zero number, will scale ``total`` and ``n``. * dynamic_ncols : bool, optional If set, constantly alters ``ncols`` and ``nrows`` to the environment (allowing for window resizes) [default: False]. * smoothing : float, optional Exponential moving average smoothing factor for speed estimates (ignored in GUI mode). Ranges from 0 (average speed) to 1 (current/instantaneous speed) [default: 0.3]. * bar_format : str, optional Specify a custom bar string formatting. May impact performance. [default: '{l_bar}{bar}{r_bar}'], where l_bar='{desc}: {percentage:3.0f}%|' and r_bar='| {n_fmt}/{total_fmt} [{elapsed}<{remaining}, ' '{rate_fmt}{postfix}]' Possible vars: l_bar, bar, r_bar, n, n_fmt, total, total_fmt, percentage, elapsed, elapsed_s, ncols, nrows, desc, unit, rate, rate_fmt, rate_noinv, rate_noinv_fmt, rate_inv, rate_inv_fmt, postfix, unit_divisor, remaining, remaining_s, eta. Note that a trailing ": " is automatically removed after {desc} if the latter is empty. * initial : int or float, optional The initial counter value. Useful when restarting a progress bar [default: 0]. If using float, consider specifying ``{n:.3f}`` or similar in ``bar_format``, or specifying ``unit_scale``. * position : int, optional Specify the line offset to print this bar (starting from 0) Automatic if unspecified. Useful to manage multiple bars at once (eg, from threads). * postfix : dict or ``*``, optional Specify additional stats to display at the end of the bar. Calls ``set_postfix(**postfix)`` if possible (dict). * unit_divisor : float, optional [default: 1000], ignored unless ``unit_scale`` is True. * write_bytes : bool, optional Whether to write bytes. If (default: False) will write unicode. * lock_args : tuple, optional Passed to ``refresh`` for intermediate output (initialisation, iterating, and updating). * nrows : int, optional The screen height. If specified, hides nested bars outside this bound. If unspecified, attempts to use environment height. The fallback is 20. * colour : str, optional Bar colour (e.g. 'green', '#00ff00'). * delay : float, optional Don't display until [default: 0] seconds have elapsed. Extra CLI Options ~~~~~~~~~~~~~~~~~ * delim : chr, optional Delimiting character [default: '\n']. Use '\0' for null. N.B.: on Windows systems, Python converts '\n' to '\r\n'. * buf_size : int, optional String buffer size in bytes [default: 256] used when ``delim`` is specified. * bytes : bool, optional If true, will count bytes, ignore ``delim``, and default ``unit_scale`` to True, ``unit_divisor`` to 1024, and ``unit`` to 'B'. * tee : bool, optional If true, passes ``stdin`` to both ``stderr`` and ``stdout``. * update : bool, optional If true, will treat input as newly elapsed iterations, i.e. numbers to pass to ``update()``. Note that this is slow (~2e5 it/s) since every input must be decoded as a number. * update_to : bool, optional If true, will treat input as total elapsed iterations, i.e. numbers to assign to ``self.n``. Note that this is slow (~2e5 it/s) since every input must be decoded as a number. * null : bool, optional If true, will discard input (no stdout). * manpath : str, optional Directory in which to install tqdm man pages. * comppath : str, optional Directory in which to place tqdm completion. * log : str, optional CRITICAL|FATAL|ERROR|WARN(ING)|[default: 'INFO']|DEBUG|NOTSET. Returns ~~~~~~~ * out : decorated iterator. .. code:: python class tqdm(): def update(self, n=1): """ Manually update the progress bar, useful for streams such as reading files. E.g.: >>> t = tqdm(total=filesize) # Initialise >>> for current_buffer in stream: ... ... ... t.update(len(current_buffer)) >>> t.close() The last line is highly recommended, but possibly not necessary if ``t.update()`` will be called in such a way that ``filesize`` will be exactly reached and printed. Parameters ---------- n : int or float, optional Increment to add to the internal counter of iterations [default: 1]. If using float, consider specifying ``{n:.3f}`` or similar in ``bar_format``, or specifying ``unit_scale``. Returns ------- out : bool or None True if a ``display()`` was triggered. """ def close(self): """Cleanup and (if leave=False) close the progressbar.""" def clear(self, nomove=False): """Clear current bar display.""" def refresh(self): """ Force refresh the display of this bar. Parameters ---------- nolock : bool, optional If ``True``, does not lock. If [default: ``False``]: calls ``acquire()`` on internal lock. lock_args : tuple, optional Passed to internal lock's ``acquire()``. If specified, will only ``display()`` if ``acquire()`` returns ``True``. """ def unpause(self): """Restart tqdm timer from last print time.""" def reset(self, total=None): """ Resets to 0 iterations for repeated use. Consider combining with ``leave=True``. Parameters ---------- total : int or float, optional. Total to use for the new bar. """ def set_description(self, desc=None, refresh=True): """ Set/modify description of the progress bar. Parameters ---------- desc : str, optional refresh : bool, optional Forces refresh [default: True]. """ def set_postfix(self, ordered_dict=None, refresh=True, **tqdm_kwargs): """ Set/modify postfix (additional stats) with automatic formatting based on datatype. Parameters ---------- ordered_dict : dict or OrderedDict, optional refresh : bool, optional Forces refresh [default: True]. kwargs : dict, optional """ @classmethod def write(cls, s, file=sys.stdout, end="\n"): """Print a message via tqdm (without overlap with bars).""" @property def format_dict(self): """Public API for read-only member access.""" def display(self, msg=None, pos=None): """ Use ``self.sp`` to display ``msg`` in the specified ``pos``. Consider overloading this function when inheriting to use e.g.: ``self.some_frontend(**self.format_dict)`` instead of ``self.sp``. Parameters ---------- msg : str, optional. What to display (default: ``repr(self)``). pos : int, optional. Position to ``moveto`` (default: ``abs(self.pos)``). """ @classmethod @contextmanager def wrapattr(cls, stream, method, total=None, bytes=True, **tqdm_kwargs): """ stream : file-like object. method : str, "read" or "write". The result of ``read()`` and the first argument of ``write()`` should have a ``len()``. >>> with tqdm.wrapattr(file_obj, "read", total=file_obj.size) as fobj: ... while True: ... chunk = fobj.read(chunk_size) ... if not chunk: ... break """ @classmethod def pandas(cls, *targs, **tqdm_kwargs): """Registers the current `tqdm` class with `pandas`.""" def trange(*args, **tqdm_kwargs): """Shortcut for `tqdm(range(*args), **tqdm_kwargs)`.""" Convenience Functions

.. code:: python

def tqdm.contrib.tenumerate(iterable, start=0, total=None, tqdm_class=tqdm.auto.tqdm, **tqdm_kwargs): """Equivalent of `numpy.ndenumerate` or builtin `enumerate`.""" def tqdm.contrib.tzip(iter1, *iter2plus, **tqdm_kwargs): """Equivalent of builtin `zip`.""" def tqdm.contrib.tmap(function, *sequences, **tqdm_kwargs): """Equivalent of builtin `map`."""

Submodules

.. code:: python class tqdm.notebook.tqdm(tqdm.tqdm): """IPython/Jupyter Notebook widget.""" class tqdm.auto.tqdm(tqdm.tqdm): """Automatically chooses beween `tqdm.notebook` and `tqdm.tqdm`.""" class tqdm.asyncio.tqdm(tqdm.tqdm): """Asynchronous version.""" @classmethod def as_completed(cls, fs, *, loop=None, timeout=None, total=None, **tqdm_kwargs): """Wrapper for `asyncio.as_completed`.""" class tqdm.gui.tqdm(tqdm.tqdm): """Matplotlib GUI version.""" class tqdm.tk.tqdm(tqdm.tqdm): """Tkinter GUI version.""" class tqdm.rich.tqdm(tqdm.tqdm): """`rich.progress` version.""" class tqdm.keras.TqdmCallback(keras.callbacks.Callback): """Keras callback for epoch and batch progress.""" class tqdm.dask.TqdmCallback(dask.callbacks.Callback): """Dask callback for task progress.""" ``contrib`` +++++++++++ The ``tqdm.contrib`` package also contains experimental modules: - ``tqdm.contrib.itertools``: Thin wrappers around ``itertools`` - ``tqdm.contrib.concurrent``: Thin wrappers around ``concurrent.futures`` - ``tqdm.contrib.slack``: Posts to `Slack <https://slack.com>`__ bots - ``tqdm.contrib.discord``: Posts to `Discord <https://discord.com>`__ bots - ``tqdm.contrib.telegram``: Posts to `Telegram <https://telegram.org>`__ bots - ``tqdm.contrib.bells``: Automagically enables all optional features * ``auto``, ``pandas``, ``slack``, ``discord``, ``telegram`` Examples and Advanced Usage --------------------------- - See the `examples <https://github.com/tqdm/tqdm/tree/master/examples>`__ folder; - import the module and run ``help()``; - consult the `wiki <https://github.com/tqdm/tqdm/wiki>`__; * this has an `excellent article <https://github.com/tqdm/tqdm/wiki/How-to-make-a-great-Progress-Bar>`__ on how to make a **great** progressbar; - check out the `slides from PyData London <https://tqdm.github.io/PyData2019/slides.html>`__, or - run the |binder-demo|. Description and additional stats

Custom information can be displayed and updated dynamically on

tqdm
bars with the
desc
and
postfix
arguments:

.. code:: python

from tqdm import tqdm, trange from random import random, randint from time import sleep with trange(10) as t: for i in t: # Description will be displayed on the left t.set_description('GEN %i' % i) # Postfix will be displayed on the right, # formatted automatically based on argument's datatype t.set_postfix(loss=random(), gen=randint(1,999), str='h', lst=[1, 2]) sleep(0.1) with tqdm(total=10, bar_format="{postfix[0]} {postfix[1][value]:>8.2g}", postfix=["Batch", {"value": 0}]) as t: for i in range(10): sleep(0.1) t.postfix[1]["value"] = i / 2 t.update()

Points to remember when using

{postfix[...]}
in the
bar_format
string:

  • postfix
    also needs to be passed as an initial argument in a compatible format, and
  • postfix
    will be auto-converted to a string if it is a
    dict
    -like object. To prevent this behaviour, insert an extra item into the dictionary where the key is not a string.

Additional

bar_format
parameters may also be defined by overriding
format_dict
, and the bar itself may be modified using
ascii
:

.. code:: python

from tqdm import tqdm class TqdmExtraFormat(tqdm): """Provides a `total_time` format parameter""" @property def format_dict(self): d = super().format_dict total_time = d["elapsed"] * (d["total"] or 0) / max(d["n"], 1) d.update(total_time=self.format_interval(total_time) + " in total") return d for i in TqdmExtraFormat( range(9), ascii=" .oO0", bar_format="{total_time}: {percentage:.0f}%|{bar}{r_bar}"): if i == 4: break

.. code::

00:00 in total: 44%|0000. | 4/9 [00:00<00:00, 962.93it/s]

Note that

{bar}
also supports a format specifier
[width][type]
.

  • width

    • unspecified (default): automatic to fill
      ncols
    • int >= 0
      : fixed width overriding
      ncols
      logic
    • int < 0
      : subtract from the automatic default
  • type

    • a
      : ascii (
      ascii=True
      override)
    • u
      : unicode (
      ascii=False
      override)
    • b
      : blank (
      ascii=" "
      override)

This means a fixed bar with right-justified text may be created by using:

bar_format="{l_bar}{bar:10}|{bar:-10b}right-justified"

Nested progress bars

``tqdm`` supports nested progress bars. Here's an example: .. code:: python from tqdm.auto import trange from time import sleep for i in trange(4, desc='1st loop'): for j in trange(5, desc='2nd loop'): for k in trange(50, desc='3rd loop', leave=False): sleep(0.01) For manual control over positioning (e.g. for multi-processing use), you may specify ``position=n`` where ``n=0`` for the outermost bar, ``n=1`` for the next, and so on. However, it's best to check if ``tqdm`` can work without manual ``position`` first. .. code:: python from time import sleep from tqdm import trange, tqdm from multiprocessing import Pool, RLock, freeze_support L = list(range(9)) def progresser(n): interval = 0.001 / (n + 2) total = 5000 text = f"#{n}, est. {interval * total:<04.2}s" for _ in trange(total, desc=text, position=n): sleep(interval) if __name__ == '__main__': freeze_support() # for Windows support tqdm.set_lock(RLock()) # for managing output contention p = Pool(initializer=tqdm.set_lock, initargs=(tqdm.get_lock(),)) p.map(progresser, L) Note that in Python 3, ``tqdm.write`` is thread-safe: .. code:: python from time import sleep from tqdm import tqdm, trange from concurrent.futures import ThreadPoolExecutor L = list(range(9)) def progresser(n): interval = 0.001 / (n + 2) total = 5000 text = f"#{n}, est. {interval * total:<04.2}s" for _ in trange(total, desc=text): sleep(interval) if n == 6: tqdm.write("n == 6 completed.") tqdm.write("`tqdm.write()` is thread-safe in py3!") if __name__ == '__main__': with ThreadPoolExecutor() as p: p.map(progresser, L) Hooks and callbacks ~~~~~~~~~~~~~~~~~~~ ``tqdm`` can easily support callbacks/hooks and manual updates. Here's an example with ``urllib``: **``urllib.urlretrieve`` documentation** | [...] | If present, the hook function will be called once | on establishment of the network connection and once after each block read | thereafter. The hook will be passed three arguments; a count of blocks | transferred so far, a block size in bytes, and the total size of the file. | [...] .. code:: python import urllib, os from tqdm import tqdm urllib = getattr(urllib, 'request', urllib) class TqdmUpTo(tqdm): """Provides `update_to(n)` which uses `tqdm.update(delta_n)`.""" def update_to(self, b=1, bsize=1, tsize=None): """ b : int, optional Number of blocks transferred so far [default: 1]. bsize : int, optional Size of each block (in tqdm units) [default: 1]. tsize : int, optional Total size (in tqdm units). If [default: None] remains unchanged. """ if tsize is not None: self.total = tsize return self.update(b * bsize - self.n) # also sets self.n = b * bsize eg_link = "https://caspersci.uk.to/matryoshka.zip" with TqdmUpTo(unit='B', unit_scale=True, unit_divisor=1024, miniters=1, desc=eg_link.split('/')[-1]) as t: # all optional kwargs urllib.urlretrieve(eg_link, filename=os.devnull, reporthook=t.update_to, data=None) t.total = t.n Inspired by `twine#242 <https://github.com/pypa/twine/pull/242>`__. Functional alternative in `examples/tqdm_wget.py <https://github.com/tqdm/tqdm/blob/master/examples/tqdm_wget.py>`__. It is recommend to use ``miniters=1`` whenever there is potentially large differences in iteration speed (e.g. downloading a file over a patchy connection). **Wrapping read/write methods** To measure throughput through a file-like object's ``read`` or ``write`` methods, use ``CallbackIOWrapper``: .. code:: python from tqdm.auto import tqdm from tqdm.utils import CallbackIOWrapper with tqdm(total=file_obj.size, unit='B', unit_scale=True, unit_divisor=1024) as t: fobj = CallbackIOWrapper(t.update, file_obj, "read") while True: chunk = fobj.read(chunk_size) if not chunk: break t.reset() # ... continue to use `t` for something else Alternatively, use the even simpler ``wrapattr`` convenience function, which would condense both the ``urllib`` and ``CallbackIOWrapper`` examples down to: .. code:: python import urllib, os from tqdm import tqdm eg_link = "https://caspersci.uk.to/matryoshka.zip" response = getattr(urllib, 'request', urllib).urlopen(eg_link) with tqdm.wrapattr(open(os.devnull, "wb"), "write", miniters=1, desc=eg_link.split('/')[-1], total=getattr(response, 'length', None)) as fout: for chunk in response: fout.write(chunk) The ``requests`` equivalent is nearly identical: .. code:: python import requests, os from tqdm import tqdm eg_link = "https://caspersci.uk.to/matryoshka.zip" response = requests.get(eg_link, stream=True) with tqdm.wrapattr(open(os.devnull, "wb"), "write", miniters=1, desc=eg_link.split('/')[-1], total=int(response.headers.get('content-length', 0))) as fout: for chunk in response.iter_content(chunk_size=4096): fout.write(chunk) **Custom callback** ``tqdm`` is known for intelligently skipping unnecessary displays. To make a custom callback take advantage of this, simply use the return value of ``update()``. This is set to ``True`` if a ``display()`` was triggered. .. code:: python from tqdm.auto import tqdm as std_tqdm def external_callback(*args, **kwargs): ... class TqdmExt(std_tqdm): def update(self, n=1): displayed = super().update(n) if displayed: external_callback(**self.format_dict) return displayed ``asyncio`` ~~~~~~~~~~~ Note that ``break`` isn't currently caught by asynchronous iterators. This means that ``tqdm`` cannot clean up after itself in this case: .. code:: python from tqdm.asyncio import tqdm async for i in tqdm(range(9)): if i == 2: break Instead, either call ``pbar.close()`` manually or use the context manager syntax: .. code:: python from tqdm.asyncio import tqdm with tqdm(range(9)) as pbar: async for i in pbar: if i == 2: break Pandas Integration ~~~~~~~~~~~~~~~~~~ Due to popular demand we've added support for ``pandas`` -- here's an example for ``DataFrame.progress_apply`` and ``DataFrameGroupBy.progress_apply``: .. code:: python import pandas as pd import numpy as np from tqdm import tqdm df = pd.DataFrame(np.random.randint(0, 100, (100000, 6))) # Register `pandas.progress_apply` and `pandas.Series.map_apply` with `tqdm` # (can use `tqdm.gui.tqdm`, `tqdm.notebook.tqdm`, optional kwargs, etc.) tqdm.pandas(desc="my bar!") # Now you can use `progress_apply` instead of `apply` # and `progress_map` instead of `map` df.progress_apply(lambda x: x**2) # can also groupby: # df.groupby(0).progress_apply(lambda x: x**2) In case you're interested in how this works (and how to modify it for your own callbacks), see the `examples <https://github.com/tqdm/tqdm/tree/master/examples>`__ folder or import the module and run ``help()``. Keras Integration ~~~~~~~~~~~~~~~~~ A ``keras`` callback is also available: .. code:: python from tqdm.keras import TqdmCallback ... model.fit(..., verbose=0, callbacks=[TqdmCallback()]) Dask Integration ~~~~~~~~~~~~~~~~ A ``dask`` callback is also available: .. code:: python from tqdm.dask import TqdmCallback with TqdmCallback(desc="compute"): ... arr.compute() # or use callback globally cb = TqdmCallback(desc="global") cb.register() arr.compute() IPython/Jupyter Integration

IPython/Jupyter is supported via the

tqdm.notebook
submodule:

.. code:: python

from tqdm.notebook import trange, tqdm from time import sleep for i in trange(3, desc='1st loop'): for j in tqdm(range(100), desc='2nd loop'): sleep(0.01)

In addition to

tqdm
features, the submodule provides a native Jupyter widget (compatible with IPython v1-v4 and Jupyter), fully working nested bars and colour hints (blue: normal, green: completed, red: error/interrupt, light blue: no ETA); as demonstrated below.

|Screenshot-Jupyter1| |Screenshot-Jupyter2| |Screenshot-Jupyter3|

The

notebook
version supports percentage or pixels for overall width (e.g.:
ncols='100%'
or
ncols='480px'
).

It is also possible to let

tqdm
automatically choose between console or notebook versions by using the
autonotebook
submodule:

.. code:: python

from tqdm.autonotebook import tqdm tqdm.pandas()

Note that this will issue a

TqdmExperimentalWarning
if run in a notebook since it is not meant to be possible to distinguish between
jupyter notebook
and
jupyter console
. Use
auto
instead of
autonotebook
to suppress this warning.

Note that notebooks will display the bar in the cell where it was created. This may be a different cell from the one where it is used. If this is not desired, either

  • delay the creation of the bar to the cell where it must be displayed, or
  • create the bar with
    display=False
    , and in a later cell call
    display(bar.container)
    :

.. code:: python

from tqdm.notebook import tqdm pbar = tqdm(..., display=False)

.. code:: python

# different cell display(pbar.container)

The

keras
callback has a
display()
method which can be used likewise:

.. code:: python

from tqdm.keras import TqdmCallback cbk = TqdmCallback(display=False)

.. code:: python

# different cell cbk.display() model.fit(..., verbose=0, callbacks=[cbk])

Another possibility is to have a single bar (near the top of the notebook) which is constantly re-used (using

reset()
rather than
close()
). For this reason, the notebook version (unlike the CLI version) does not automatically call
close()
upon
Exception
.

.. code:: python

from tqdm.notebook import tqdm pbar = tqdm()

.. code:: python

# different cell iterable = range(100) pbar.reset(total=len(iterable)) # initialise with new `total` for i in iterable: pbar.update() pbar.refresh() # force print final status but don't `close()`

Custom Integration

To change the default arguments (such as making ``dynamic_ncols=True``), simply use built-in Python magic: .. code:: python from functools import partial from tqdm import tqdm as std_tqdm tqdm = partial(std_tqdm, dynamic_ncols=True) For further customisation, ``tqdm`` may be inherited from to create custom callbacks (as with the ``TqdmUpTo`` example `above <#hooks-and-callbacks>`__) or for custom frontends (e.g. GUIs such as notebook or plotting packages). In the latter case: 1. ``def __init__()`` to call ``super().__init__(..., gui=True)`` to disable terminal ``status_printer`` creation. 2. Redefine: ``close()``, ``clear()``, ``display()``. Consider overloading ``display()`` to use e.g. ``self.frontend(**self.format_dict)`` instead of ``self.sp(repr(self))``. Some submodule examples of inheritance: - `tqdm/notebook.py <https://github.com/tqdm/tqdm/blob/master/tqdm/notebook.py>`__ - `tqdm/gui.py <https://github.com/tqdm/tqdm/blob/master/tqdm/gui.py>`__ - `tqdm/tk.py <https://github.com/tqdm/tqdm/blob/master/tqdm/tk.py>`__ - `tqdm/contrib/slack.py <https://github.com/tqdm/tqdm/blob/master/tqdm/contrib/slack.py>`__ - `tqdm/contrib/discord.py <https://github.com/tqdm/tqdm/blob/master/tqdm/contrib/discord.py>`__ - `tqdm/contrib/telegram.py <https://github.com/tqdm/tqdm/blob/master/tqdm/contrib/telegram.py>`__ Dynamic Monitor/Meter

You can use a

tqdm
as a meter which is not monotonically increasing. This could be because
n
decreases (e.g. a CPU usage monitor) or
total
changes.

One example would be recursively searching for files. The

total
is the number of objects found so far, while
n
is the number of those objects which are files (rather than folders):

.. code:: python

from tqdm import tqdm import os.path def find_files_recursively(path, show_progress=True): files = [] # total=1 assumes `path` is a file t = tqdm(total=1, unit="file", disable=not show_progress) if not os.path.exists(path): raise IOError("Cannot find:" + path) def append_found_file(f): files.append(f) t.update() def list_found_dir(path): """returns os.listdir(path) assuming os.path.isdir(path)""" listing = os.listdir(path) # subtract 1 since a "file" we found was actually this directory t.total += len(listing) - 1 # fancy way to give info without forcing a refresh t.set_postfix(dir=path[-10:], refresh=False) t.update(0) # may trigger a refresh return listing def recursively_search(path): if os.path.isdir(path): for f in list_found_dir(path): recursively_search(os.path.join(path, f)) else: append_found_file(path) recursively_search(path) t.set_postfix(dir=path) t.close() return files

Using

update(0)
is a handy way to let
tqdm
decide when to trigger a display refresh to avoid console spamming.

Writing messages

This is a work in progress (see `#737 <https://github.com/tqdm/tqdm/issues/737>`__). Since ``tqdm`` uses a simple printing mechanism to display progress bars, you should not write any message in the terminal using ``print()`` while a progressbar is open. To write messages in the terminal without any collision with ``tqdm`` bar display, a ``.write()`` method is provided: .. code:: python from tqdm.auto import tqdm, trange from time import sleep bar = trange(10) for i in bar: # Print using tqdm class method .write() sleep(0.1) if not (i % 3): tqdm.write("Done task %i" % i) # Can also use bar.write() By default, this will print to standard output ``sys.stdout``. but you can specify any file-like object using the ``file`` argument. For example, this can be used to redirect the messages writing to a log file or class. Redirecting writing

If using a library that can print messages to the console, editing the library by replacing

print()
with
tqdm.write()
may not be desirable. In that case, redirecting
sys.stdout
to
tqdm.write()
is an option.

To redirect

sys.stdout
, create a file-like class that will write any input string to
tqdm.write()
, and supply the arguments
file=sys.stdout, dynamic_ncols=True
.

A reusable canonical example is given below:

.. code:: python

from time import sleep import contextlib import sys from tqdm import tqdm from tqdm.contrib import DummyTqdmFile @contextlib.contextmanager def std_out_err_redirect_tqdm(): orig_out_err = sys.stdout, sys.stderr try: sys.stdout, sys.stderr = map(DummyTqdmFile, orig_out_err) yield orig_out_err[0] # Relay exceptions except Exception as exc: raise exc # Always restore sys.stdout/err if necessary finally: sys.stdout, sys.stderr = orig_out_err def some_fun(i): print("Fee, fi, fo,".split()[i]) # Redirect stdout to tqdm.write() (don't forget the `as save_stdout`) with std_out_err_redirect_tqdm() as orig_stdout: # tqdm needs the original stdout # and dynamic_ncols=True to autodetect console width for i in tqdm(range(3), file=orig_stdout, dynamic_ncols=True): sleep(.5) some_fun(i) # After the `with`, printing is restored print("Done!")

Redirecting

logging

Similar to ``sys.stdout``/``sys.stderr`` as detailed above, console ``logging`` may also be redirected to ``tqdm.write()``. Warning: if also redirecting ``sys.stdout``/``sys.stderr``, make sure to redirect ``logging`` first if needed. Helper methods are available in ``tqdm.contrib.logging``. For example: .. code:: python import logging from tqdm import trange from tqdm.contrib.logging import logging_redirect_tqdm LOG = logging.getLogger(__name__) if __name__ == '__main__': logging.basicConfig(level=logging.INFO) with logging_redirect_tqdm(): for i in trange(9): if i == 4: LOG.info("console logging redirected to `tqdm.write()`") # logging restored Monitoring thread, intervals and miniters

tqdm
implements a few tricks to increase efficiency and reduce overhead.

  • Avoid unnecessary frequent bar refreshing:
    mininterval
    defines how long to wait between each refresh.
    tqdm
    always gets updated in the background, but it will display only every
    mininterval
    .
  • Reduce number of calls to check system clock/time.
  • mininterval
    is more intuitive to configure than
    miniters
    . A clever adjustment system
    dynamic_miniters
    will automatically adjust
    miniters
    to the amount of iterations that fit into time
    mininterval
    . Essentially,
    tqdm
    will check if it's time to print without actually checking time. This behaviour can be still be bypassed by manually setting
    miniters
    .

However, consider a case with a combination of fast and slow iterations. After a few fast iterations,

dynamic_miniters
will set
miniters
to a large number. When iteration rate subsequently slows,
miniters
will remain large and thus reduce display update frequency. To address this:

  • maxinterval
    defines the maximum time between display refreshes. A concurrent monitoring thread checks for overdue updates and forces one where necessary.

The monitoring thread should not have a noticeable overhead, and guarantees updates at least every 10 seconds by default. This value can be directly changed by setting the

monitor_interval
of any
tqdm
instance (i.e.
t = tqdm.tqdm(...); t.monitor_interval = 2
). The monitor thread may be disabled application-wide by setting
tqdm.tqdm.monitor_interval = 0
before instantiation of any
tqdm
bar.

Merch

You can buy

tqdm branded merch <https://tqdm.github.io/merch>
__ now!

Contributions

|GitHub-Commits| |GitHub-Issues| |GitHub-PRs| |OpenHub-Status| |GitHub-Contributions| |CII Best Practices|

All source code is hosted on

GitHub <https://github.com/tqdm/tqdm>
__. Contributions are welcome.

See the

CONTRIBUTING <https://github.com/tqdm/tqdm/blob/master/CONTRIBUTING.md>
__ file for more information.

Developers who have made significant contributions, ranked by SLoC (surviving lines of code,

git fame <https://github.com/casperdcl/git-fame>
__
-wMC --excl '\.(png|gif|jpg)$'
), are:

==================== ======================================================== ==== ================================ Name ID SLoC Notes ==================== ======================================================== ==== ================================ Casper da Costa-Luis

casperdcl <https://github.com/casperdcl>
__ ~80% primary maintainer |Gift-Casper| Stephen Larroque
lrq3000 <https://github.com/lrq3000>
__ ~9% team member Martin Zugnoni
martinzugnoni <https://github.com/martinzugnoni>
__ ~3% Daniel Ecer
de-code <https://github.com/de-code>
__ ~2% Richard Sheridan
richardsheridan <https://github.com/richardsheridan>
__ ~1% Guangshuo Chen
chengs <https://github.com/chengs>
__ ~1% Helio Machado
0x2b3bfa0 <https://github.com/0x2b3bfa0>
__ ~1% Kyle Altendorf
altendky <https://github.com/altendky>
__ <1% Noam Yorav-Raphael
noamraph <https://github.com/noamraph>
__ <1% original author Matthew Stevens
mjstevens777 <https://github.com/mjstevens777>
__ <1% Hadrien Mary
hadim <https://github.com/hadim>
__ <1% team member Mikhail Korobov
kmike <https://github.com/kmike>
__ <1% team member ==================== ======================================================== ==== ================================

Ports to Other Languages

A list is available on `this wiki page <https://github.com/tqdm/tqdm/wiki/tqdm-ports>`__. LICENCE ------- Open Source (OSI approved): |LICENCE| Citation information: |DOI| |README-Hits| (Since 19 May 2016) .. |Logo| image:: https://tqdm.github.io/img/logo.gif .. |Screenshot| image:: https://tqdm.github.io/img/tqdm.gif .. |Video| image:: https://tqdm.github.io/img/video.jpg :target: https://tqdm.github.io/video .. |Slides| image:: https://tqdm.github.io/img/slides.jpg :target: https://tqdm.github.io/PyData2019/slides.html .. |Merch| image:: https://tqdm.github.io/img/merch.jpg :target: https://tqdm.github.io/merch .. |Build-Status| image:: https://img.shields.io/github/actions/workflow/status/tqdm/tqdm/test.yml?branch=master&label=tqdm&logo=GitHub :target: https://github.com/tqdm/tqdm/actions/workflows/test.yml .. |Coverage-Status| image:: https://img.shields.io/coveralls/github/tqdm/tqdm/master?logo=coveralls :target: https://coveralls.io/github/tqdm/tqdm .. |Branch-Coverage-Status| image:: https://codecov.io/gh/tqdm/tqdm/branch/master/graph/badge.svg :target: https://codecov.io/gh/tqdm/tqdm .. |Codacy-Grade| image:: https://app.codacy.com/project/badge/Grade/3f965571598f44549c7818f29cdcf177 :target: https://www.codacy.com/gh/tqdm/tqdm/dashboard .. |CII Best Practices| image:: https://bestpractices.coreinfrastructure.org/projects/3264/badge :target: https://bestpractices.coreinfrastructure.org/projects/3264 .. |GitHub-Status| image:: https://img.shields.io/github/tag/tqdm/tqdm.svg?maxAge=86400&logo=github&logoColor=white :target: https://github.com/tqdm/tqdm/releases .. |GitHub-Forks| image:: https://img.shields.io/github/forks/tqdm/tqdm.svg?logo=github&logoColor=white :target: https://github.com/tqdm/tqdm/network .. |GitHub-Stars| image:: https://img.shields.io/github/stars/tqdm/tqdm.svg?logo=github&logoColor=white :target: https://github.com/tqdm/tqdm/stargazers .. |GitHub-Commits| image:: https://img.shields.io/github/commit-activity/y/tqdm/tqdm.svg?logo=git&logoColor=white :target: https://github.com/tqdm/tqdm/graphs/commit-activity .. |GitHub-Issues| image:: https://img.shields.io/github/issues-closed/tqdm/tqdm.svg?logo=github&logoColor=white :target: https://github.com/tqdm/tqdm/issues?q= .. |GitHub-PRs| image:: https://img.shields.io/github/issues-pr-closed/tqdm/tqdm.svg?logo=github&logoColor=white :target: https://github.com/tqdm/tqdm/pulls .. |GitHub-Contributions| image:: https://img.shields.io/github/contributors/tqdm/tqdm.svg?logo=github&logoColor=white :target: https://github.com/tqdm/tqdm/graphs/contributors .. |GitHub-Updated| image:: https://img.shields.io/github/last-commit/tqdm/tqdm/master.svg?logo=github&logoColor=white&label=pushed :target: https://github.com/tqdm/tqdm/pulse .. |Gift-Casper| image:: https://img.shields.io/badge/dynamic/json.svg?color=ff69b4&label=gifts%20received&prefix=%C2%A3&query=%24..sum&url=https%3A%2F%2Fcaspersci.uk.to%2Fgifts.json :target: https://cdcl.ml/sponsor .. |Versions| image:: https://img.shields.io/pypi/v/tqdm.svg :target: https://tqdm.github.io/releases .. |PyPI-Downloads| image:: https://img.shields.io/pypi/dm/tqdm.svg?label=pypi%20downloads&logo=PyPI&logoColor=white :target: https://pepy.tech/project/tqdm .. |Py-Versions| image:: https://img.shields.io/pypi/pyversions/tqdm.svg?logo=python&logoColor=white :target: https://pypi.org/project/tqdm .. |Conda-Forge-Status| image:: https://img.shields.io/conda/v/conda-forge/tqdm.svg?label=conda-forge&logo=conda-forge :target: https://anaconda.org/conda-forge/tqdm .. |Snapcraft| image:: https://img.shields.io/badge/snap-install-82BEA0.svg?logo=snapcraft :target: https://snapcraft.io/tqdm .. |Docker| image:: https://img.shields.io/badge/docker-pull-blue.svg?logo=docker&logoColor=white :target: https://hub.docker.com/r/tqdm/tqdm .. |Libraries-Rank| image:: https://img.shields.io/librariesio/sourcerank/pypi/tqdm.svg?logo=koding&logoColor=white :target: https://libraries.io/pypi/tqdm .. |Libraries-Dependents| image:: https://img.shields.io/librariesio/dependent-repos/pypi/tqdm.svg?logo=koding&logoColor=white :target: https://github.com/tqdm/tqdm/network/dependents .. |OpenHub-Status| image:: https://www.openhub.net/p/tqdm/widgets/project_thin_badge?format=gif :target: https://www.openhub.net/p/tqdm?ref=Thin+badge .. |awesome-python| image:: https://awesome.re/mentioned-badge.svg :target: https://github.com/vinta/awesome-python .. |LICENCE| image:: https://img.shields.io/pypi/l/tqdm.svg :target: https://raw.githubusercontent.com/tqdm/tqdm/master/LICENCE .. |DOI| image:: https://img.shields.io/badge/DOI-10.5281/zenodo.595120-blue.svg :target: https://doi.org/10.5281/zenodo.595120 .. |binder-demo| image:: https://mybinder.org/badge_logo.svg :target: https://mybinder.org/v2/gh/tqdm/tqdm/master?filepath=DEMO.ipynb .. |Screenshot-Jupyter1| image:: https://tqdm.github.io/img/jupyter-1.gif .. |Screenshot-Jupyter2| image:: https://tqdm.github.io/img/jupyter-2.gif .. |Screenshot-Jupyter3| image:: https://tqdm.github.io/img/jupyter-3.gif .. |README-Hits| image:: https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&style=social&r=https://github.com/tqdm/tqdm&l=https://tqdm.github.io/img/favicon.png&f=https://tqdm.github.io/img/logo.gif :target: https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&a=plot&r=https://github.com/tqdm/tqdm&l=https://tqdm.github.io/img/favicon.png&f=https://tqdm.github.io/img/logo.gif&style=social

Использование cookies

Мы используем файлы cookie в соответствии с Политикой конфиденциальности и Политикой использования cookies.

Нажимая кнопку «Принимаю», Вы даете АО «СберТех» согласие на обработку Ваших персональных данных в целях совершенствования нашего веб-сайта и Сервиса GitVerse, а также повышения удобства их использования.

Запретить использование cookies Вы можете самостоятельно в настройках Вашего браузера.