|Logo|

tqdm

|Py-Versions| |Versions| |Conda-Forge-Status| |Docker| |Snapcraft|

|Build-Status| |Coverage-Status| |Branch-Coverage-Status| |Codacy-Grade| |Libraries-Rank| |PyPI-Downloads|

|LICENCE| |OpenHub-Status| |binder-demo| |awesome-python|

Instantly make your loops show a smart progress meter - just wrap any iterable with

.. code:: python

from tqdm import tqdm
for i in tqdm(range(10000)):
    ...

|Screenshot| |Video| |Slides| |Merch|

It can also be executed as a module with pipes:

.. code:: sh

$ seq 9999999 | tqdm --bytes | wc -l
75.2MB [00:00, 217MB/s]
9999999

$ tar -zcf - docs/ | tqdm --bytes --total `du -sb docs/ | cut -f1` \
    > backup.tgz
 32%|██████████▍                      | 8.89G/27.9G [00:42<01:31, 223MB/s]

Overhead is low -- about 60ns per iteration (80ns with

In addition to its low overhead,

.. contents:: Table of contents :backlinks: top :local:

Installation

Latest PyPI stable release

|Versions| |PyPI-Downloads| |Libraries-Dependents|

.. code:: sh

    pip install tqdm

Latest development release on GitHub

|GitHub-Status| |GitHub-Stars| |GitHub-Commits| |GitHub-Forks| |GitHub-Updated|

Pull and install pre-release

.. code:: sh

pip install "git+https://github.com/tqdm/tqdm.git@devel#egg=tqdm"

Latest Conda release

|Conda-Forge-Status|

.. code:: sh

    conda install -c conda-forge tqdm

Latest Snapcraft release

|Snapcraft|

There are 3 channels to choose from:

.. code:: sh

snap install tqdm  # implies --stable, i.e. latest tagged release
snap install tqdm  --candidate  # master branch
snap install tqdm  --edge  # devel branch

Note that

Latest Docker release

|Docker|

.. code:: sh

    docker pull tqdm/tqdm
    docker run -i --rm tqdm/tqdm --help

Other
~~~~~

There are other (unofficial) places where ``tqdm`` may be downloaded, particularly for CLI use:

|Repology|

.. |Repology| image:: https://repology.org/badge/tiny-repos/python:tqdm.svg
   :target: https://repology.org/project/python:tqdm/versions

Changelog
---------

The list of all changes is available either on GitHub's Releases:
|GitHub-Status|, on the
`wiki <https://github.com/tqdm/tqdm/wiki/Releases>`__, or on the
`website <https://tqdm.github.io/releases>`__.


Usage
-----

``tqdm`` is very versatile and can be used in a number of ways.
The three main ones are given below.

Iterable-based
~~~~~~~~~~~~~~

Wrap ``tqdm()`` around any iterable:

.. code:: python

    from tqdm import tqdm
    from time import sleep

    text = ""
    for char in tqdm(["a", "b", "c", "d"]):
        sleep(0.25)
        text = text + char

``trange(i)`` is a special optimised instance of ``tqdm(range(i))``:

.. code:: python

    from tqdm import trange

    for i in trange(100):
        sleep(0.01)

Instantiation outside of the loop allows for manual control over ``tqdm()``:

.. code:: python

    pbar = tqdm(["a", "b", "c", "d"])
    for char in pbar:
        sleep(0.25)
        pbar.set_description("Processing %s" % char)

Manual
~~~~~~

Manual control of ``tqdm()`` updates using a ``with`` statement:

.. code:: python

    with tqdm(total=100) as pbar:
        for i in range(10):
            sleep(0.1)
            pbar.update(10)

If the optional variable ``total`` (or an iterable with ``len()``) is
provided, predictive stats are displayed.

``with`` is also optional (you can just assign ``tqdm()`` to a variable,
but in this case don't forget to ``del`` or ``close()`` at the end:

.. code:: python

    pbar = tqdm(total=100)
    for i in range(10):
        sleep(0.1)
        pbar.update(10)
    pbar.close()

Module
~~~~~~

Perhaps the most wonderful use of ``tqdm`` is in a script or on the command
line. Simply inserting ``tqdm`` (or ``python -m tqdm``) between pipes will pass
through all ``stdin`` to ``stdout`` while printing progress to ``stderr``.

The example below demonstrate counting the number of lines in all Python files
in the current directory, with timing information included.

.. code:: sh

    $ time find . -name '*.py' -type f -exec cat \{} \; | wc -l
    857365

    real    0m3.458s
    user    0m0.274s
    sys     0m3.325s

    $ time find . -name '*.py' -type f -exec cat \{} \; | tqdm | wc -l
    857366it [00:03, 246471.31it/s]
    857365

    real    0m3.585s
    user    0m0.862s
    sys     0m3.358s

Note that the usual arguments for ``tqdm`` can also be specified.

.. code:: sh

    $ find . -name '*.py' -type f -exec cat \{} \; |
        tqdm --unit loc --unit_scale --total 857366 >> /dev/null
    100%|█████████████████████████████████| 857K/857K [00:04<00:00, 246Kloc/s]

Backing up a large directory?

.. code:: sh

    $ tar -zcf - docs/ | tqdm --bytes --total `du -sb docs/ | cut -f1` \
      > backup.tgz
     44%|██████████████▊                   | 153M/352M [00:14<00:18, 11.0MB/s]

This can be beautified further:

.. code:: sh

    $ BYTES=$(du -sb docs/ | cut -f1)
    $ tar -cf - docs/ \
      | tqdm --bytes --total "$BYTES" --desc Processing | gzip \
      | tqdm --bytes --total "$BYTES" --desc Compressed --position 1 \
      > ~/backup.tgz
    Processing: 100%|██████████████████████| 352M/352M [00:14<00:00, 30.2MB/s]
    Compressed:  42%|█████████▎            | 148M/352M [00:14<00:19, 10.9MB/s]

Or done on a file level using 7-zip:

.. code:: sh

    $ 7z a -bd -r backup.7z docs/ | grep Compressing \
      | tqdm --total $(find docs/ -type f | wc -l) --unit files \
      | grep -v Compressing
    100%|██████████████████████████▉| 15327/15327 [01:00<00:00, 712.96files/s]

Pre-existing CLI programs already outputting basic progress information will
benefit from ``tqdm``'s ``--update`` and ``--update_to`` flags:

.. code:: sh

    $ seq 3 0.1 5 | tqdm --total 5 --update_to --null
    100%|████████████████████████████████████| 5.0/5 [00:00<00:00, 9673.21it/s]
    $ seq 10 | tqdm --update --null  # 1 + 2 + ... + 10 = 55 iterations
    55it [00:00, 90006.52it/s]

FAQ and Known Issues
--------------------

|GitHub-Issues|

The most common issues relate to excessive output on multiple lines, instead
of a neat one-line progress bar.

- Consoles in general: require support for carriage return (``CR``, ``\r``).

  * Some cloud logging consoles which don't support ``\r`` properly
    (`cloudwatch <https://github.com/tqdm/tqdm/issues/966>`__,
    `K8s <https://github.com/tqdm/tqdm/issues/1319>`__) may benefit from
    ``export TQDM_POSITION=-1``.

- Nested progress bars:

  * Consoles in general: require support for moving cursors up to the
    previous line. For example,
    `IDLE <https://github.com/tqdm/tqdm/issues/191#issuecomment-230168030>`__,
    `ConEmu <https://github.com/tqdm/tqdm/issues/254>`__ and
    `PyCharm <https://github.com/tqdm/tqdm/issues/203>`__ (also
    `here <https://github.com/tqdm/tqdm/issues/208>`__,
    `here <https://github.com/tqdm/tqdm/issues/307>`__, and
    `here <https://github.com/tqdm/tqdm/issues/454#issuecomment-335416815>`__)
    lack full support.
  * Windows: additionally may require the Python module ``colorama``
    to ensure nested bars stay within their respective lines.

- Unicode:

  * Environments which report that they support unicode will have solid smooth
    progressbars. The fallback is an ``ascii``-only bar.
  * Windows consoles often only partially support unicode and thus
    `often require explicit ascii=True <https://github.com/tqdm/tqdm/issues/454#issuecomment-335416815>`__
    (also `here <https://github.com/tqdm/tqdm/issues/499>`__). This is due to
    either normal-width unicode characters being incorrectly displayed as
    "wide", or some unicode characters not rendering.

- Wrapping generators:

  * Generator wrapper functions tend to hide the length of iterables.
    ``tqdm`` does not.
  * Replace ``tqdm(enumerate(...))`` with ``enumerate(tqdm(...))`` or
    ``tqdm(enumerate(x), total=len(x), ...)``.
    The same applies to ``numpy.ndenumerate``.
  * Replace ``tqdm(zip(a, b))`` with ``zip(tqdm(a), b)`` or even
    ``zip(tqdm(a), tqdm(b))``.
  * The same applies to ``itertools``.
  * Some useful convenience functions can be found under ``tqdm.contrib``.

- `No intermediate output in docker-compose <https://github.com/tqdm/tqdm/issues/771>`__:
  use ``docker-compose run`` instead of ``docker-compose up`` and ``tty: true``.

- Overriding defaults via environment variables:
  e.g. in CI/cloud jobs, ``export TQDM_MININTERVAL=5`` to avoid log spam.
  This override logic is handled by the ``tqdm.utils.envwrap`` decorator
  (useful independent of ``tqdm``).

If you come across any other difficulties, browse and file |GitHub-Issues|.

Documentation
-------------

|Py-Versions| |README-Hits| (Since 19 May 2016)

.. code:: python

    class tqdm():
      """
      Decorate an iterable object, returning an iterator which acts exactly
      like the original iterable, but prints a dynamically updating
      progressbar every time a value is requested.
      """

      @envwrap("TQDM_")  # override defaults via env vars
      def __init__(self, iterable=None, desc=None, total=None, leave=True,
                   file=None, ncols=None, mininterval=0.1,
                   maxinterval=10.0, miniters=None, ascii=None, disable=False,
                   unit='it', unit_scale=False, dynamic_ncols=False,
                   smoothing=0.3, bar_format=None, initial=0, position=None,
                   postfix=None, unit_divisor=1000, write_bytes=False,
                   lock_args=None, nrows=None, colour=None, delay=0):

Parameters
~~~~~~~~~~

* iterable  : iterable, optional  
    Iterable to decorate with a progressbar.
    Leave blank to manually manage the updates.
* desc  : str, optional  
    Prefix for the progressbar.
* total  : int or float, optional  
    The number of expected iterations. If unspecified,
    len(iterable) is used if possible. If float("inf") or as a last
    resort, only basic progress statistics are displayed
    (no ETA, no progressbar).
    If ``gui`` is True and this parameter needs subsequent updating,
    specify an initial arbitrary large positive number,
    e.g. 9e9.
* leave  : bool, optional  
    If [default: True], keeps all traces of the progressbar
    upon termination of iteration.
    If ``None``, will leave only if ``position`` is ``0``.
* file  : ``io.TextIOWrapper`` or ``io.StringIO``, optional  
    Specifies where to output the progress messages
    (default: sys.stderr). Uses ``file.write(str)`` and ``file.flush()``
    methods.  For encoding, see ``write_bytes``.
* ncols  : int, optional  
    The width of the entire output message. If specified,
    dynamically resizes the progressbar to stay within this bound.
    If unspecified, attempts to use environment width. The
    fallback is a meter width of 10 and no limit for the counter and
    statistics. If 0, will not print any meter (only stats).
* mininterval  : float, optional  
    Minimum progress display update interval [default: 0.1] seconds.
* maxinterval  : float, optional  
    Maximum progress display update interval [default: 10] seconds.
    Automatically adjusts ``miniters`` to correspond to ``mininterval``
    after long display update lag. Only works if ``dynamic_miniters``
    or monitor thread is enabled.
* miniters  : int or float, optional  
    Minimum progress display update interval, in iterations.
    If 0 and ``dynamic_miniters``, will automatically adjust to equal
    ``mininterval`` (more CPU efficient, good for tight loops).
    If > 0, will skip display of specified number of iterations.
    Tweak this and ``mininterval`` to get very efficient loops.
    If your progress is erratic with both fast and slow iterations
    (network, skipping items, etc) you should set miniters=1.
* ascii  : bool or str, optional  
    If unspecified or False, use unicode (smooth blocks) to fill
    the meter. The fallback is to use ASCII characters " 123456789#".
* disable  : bool, optional  
    Whether to disable the entire progressbar wrapper
    [default: False]. If set to None, disable on non-TTY.
* unit  : str, optional  
    String that will be used to define the unit of each iteration
    [default: it].
* unit_scale  : bool or int or float, optional  
    If 1 or True, the number of iterations will be reduced/scaled
    automatically and a metric prefix following the
    International System of Units standard will be added
    (kilo, mega, etc.) [default: False]. If any other non-zero
    number, will scale ``total`` and ``n``.
* dynamic_ncols  : bool, optional  
    If set, constantly alters ``ncols`` and ``nrows`` to the
    environment (allowing for window resizes) [default: False].
* smoothing  : float, optional  
    Exponential moving average smoothing factor for speed estimates
    (ignored in GUI mode). Ranges from 0 (average speed) to 1
    (current/instantaneous speed) [default: 0.3].
* bar_format  : str, optional  
    Specify a custom bar string formatting. May impact performance.
    [default: '{l_bar}{bar}{r_bar}'], where
    l_bar='{desc}: {percentage:3.0f}%|' and
    r_bar='| {n_fmt}/{total_fmt} [{elapsed}<{remaining}, '
    '{rate_fmt}{postfix}]'
    Possible vars: l_bar, bar, r_bar, n, n_fmt, total, total_fmt,
    percentage, elapsed, elapsed_s, ncols, nrows, desc, unit,
    rate, rate_fmt, rate_noinv, rate_noinv_fmt,
    rate_inv, rate_inv_fmt, postfix, unit_divisor,
    remaining, remaining_s, eta.
    Note that a trailing ": " is automatically removed after {desc}
    if the latter is empty.
* initial  : int or float, optional  
    The initial counter value. Useful when restarting a progress
    bar [default: 0]. If using float, consider specifying ``{n:.3f}``
    or similar in ``bar_format``, or specifying ``unit_scale``.
* position  : int, optional  
    Specify the line offset to print this bar (starting from 0)
    Automatic if unspecified.
    Useful to manage multiple bars at once (eg, from threads).
* postfix  : dict or ``*``, optional  
    Specify additional stats to display at the end of the bar.
    Calls ``set_postfix(**postfix)`` if possible (dict).
* unit_divisor  : float, optional  
    [default: 1000], ignored unless ``unit_scale`` is True.
* write_bytes  : bool, optional  
    Whether to write bytes. If (default: False) will write unicode.
* lock_args  : tuple, optional  
    Passed to ``refresh`` for intermediate output
    (initialisation, iterating, and updating).
* nrows  : int, optional  
    The screen height. If specified, hides nested bars outside this
    bound. If unspecified, attempts to use environment height.
    The fallback is 20.
* colour  : str, optional  
    Bar colour (e.g. 'green', '#00ff00').
* delay  : float, optional  
    Don't display until [default: 0] seconds have elapsed.

Extra CLI Options
~~~~~~~~~~~~~~~~~

* delim  : chr, optional  
    Delimiting character [default: '\n']. Use '\0' for null.
    N.B.: on Windows systems, Python converts '\n' to '\r\n'.
* buf_size  : int, optional  
    String buffer size in bytes [default: 256]
    used when ``delim`` is specified.
* bytes  : bool, optional  
    If true, will count bytes, ignore ``delim``, and default
    ``unit_scale`` to True, ``unit_divisor`` to 1024, and ``unit`` to 'B'.
* tee  : bool, optional  
    If true, passes ``stdin`` to both ``stderr`` and ``stdout``.
* update  : bool, optional  
    If true, will treat input as newly elapsed iterations,
    i.e. numbers to pass to ``update()``. Note that this is slow
    (~2e5 it/s) since every input must be decoded as a number.
* update_to  : bool, optional  
    If true, will treat input as total elapsed iterations,
    i.e. numbers to assign to ``self.n``. Note that this is slow
    (~2e5 it/s) since every input must be decoded as a number.
* null  : bool, optional  
    If true, will discard input (no stdout).
* manpath  : str, optional  
    Directory in which to install tqdm man pages.
* comppath  : str, optional  
    Directory in which to place tqdm completion.
* log  : str, optional  
    CRITICAL|FATAL|ERROR|WARN(ING)|[default: 'INFO']|DEBUG|NOTSET.

Returns
~~~~~~~

* out  : decorated iterator.  

.. code:: python

    class tqdm():
      def update(self, n=1):
          """
          Manually update the progress bar, useful for streams
          such as reading files.
          E.g.:
          >>> t = tqdm(total=filesize) # Initialise
          >>> for current_buffer in stream:
          ...    ...
          ...    t.update(len(current_buffer))
          >>> t.close()
          The last line is highly recommended, but possibly not necessary if
          ``t.update()`` will be called in such a way that ``filesize`` will be
          exactly reached and printed.

          Parameters
          ----------
          n  : int or float, optional
              Increment to add to the internal counter of iterations
              [default: 1]. If using float, consider specifying ``{n:.3f}``
              or similar in ``bar_format``, or specifying ``unit_scale``.

          Returns
          -------
          out  : bool or None
              True if a ``display()`` was triggered.
          """

      def close(self):
          """Cleanup and (if leave=False) close the progressbar."""

      def clear(self, nomove=False):
          """Clear current bar display."""

      def refresh(self):
          """
          Force refresh the display of this bar.

          Parameters
          ----------
          nolock  : bool, optional
              If ``True``, does not lock.
              If [default: ``False``]: calls ``acquire()`` on internal lock.
          lock_args  : tuple, optional
              Passed to internal lock's ``acquire()``.
              If specified, will only ``display()`` if ``acquire()`` returns ``True``.
          """

      def unpause(self):
          """Restart tqdm timer from last print time."""

      def reset(self, total=None):
          """
          Resets to 0 iterations for repeated use.

          Consider combining with ``leave=True``.

          Parameters
          ----------
          total  : int or float, optional. Total to use for the new bar.
          """

      def set_description(self, desc=None, refresh=True):
          """
          Set/modify description of the progress bar.

          Parameters
          ----------
          desc  : str, optional
          refresh  : bool, optional
              Forces refresh [default: True].
          """

      def set_postfix(self, ordered_dict=None, refresh=True, **tqdm_kwargs):
          """
          Set/modify postfix (additional stats)
          with automatic formatting based on datatype.

          Parameters
          ----------
          ordered_dict  : dict or OrderedDict, optional
          refresh  : bool, optional
              Forces refresh [default: True].
          kwargs  : dict, optional
          """

      @classmethod
      def write(cls, s, file=sys.stdout, end="\n"):
          """Print a message via tqdm (without overlap with bars)."""

      @property
      def format_dict(self):
          """Public API for read-only member access."""

      def display(self, msg=None, pos=None):
          """
          Use ``self.sp`` to display ``msg`` in the specified ``pos``.

          Consider overloading this function when inheriting to use e.g.:
          ``self.some_frontend(**self.format_dict)`` instead of ``self.sp``.

          Parameters
          ----------
          msg  : str, optional. What to display (default: ``repr(self)``).
          pos  : int, optional. Position to ``moveto``
            (default: ``abs(self.pos)``).
          """

      @classmethod
      @contextmanager
      def wrapattr(cls, stream, method, total=None, bytes=True, **tqdm_kwargs):
          """
          stream  : file-like object.
          method  : str, "read" or "write". The result of ``read()`` and
              the first argument of ``write()`` should have a ``len()``.

          >>> with tqdm.wrapattr(file_obj, "read", total=file_obj.size) as fobj:
          ...     while True:
          ...         chunk = fobj.read(chunk_size)
          ...         if not chunk:
          ...             break
          """

      @classmethod
      def pandas(cls, *targs, **tqdm_kwargs):
          """Registers the current `tqdm` class with `pandas`."""

    def trange(*args, **tqdm_kwargs):
        """Shortcut for `tqdm(range(*args), **tqdm_kwargs)`."""

Convenience Functions

.. code:: python

def tqdm.contrib.tenumerate(iterable, start=0, total=None,
                            tqdm_class=tqdm.auto.tqdm, **tqdm_kwargs):
    """Equivalent of `numpy.ndenumerate` or builtin `enumerate`."""

def tqdm.contrib.tzip(iter1, *iter2plus, **tqdm_kwargs):
    """Equivalent of builtin `zip`."""

def tqdm.contrib.tmap(function, *sequences, **tqdm_kwargs):
    """Equivalent of builtin `map`."""

Submodules

.. code:: python

    class tqdm.notebook.tqdm(tqdm.tqdm):
        """IPython/Jupyter Notebook widget."""

    class tqdm.auto.tqdm(tqdm.tqdm):
        """Automatically chooses beween `tqdm.notebook` and `tqdm.tqdm`."""

    class tqdm.asyncio.tqdm(tqdm.tqdm):
      """Asynchronous version."""
      @classmethod
      def as_completed(cls, fs, *, loop=None, timeout=None, total=None,
                       **tqdm_kwargs):
          """Wrapper for `asyncio.as_completed`."""

    class tqdm.gui.tqdm(tqdm.tqdm):
        """Matplotlib GUI version."""

    class tqdm.tk.tqdm(tqdm.tqdm):
        """Tkinter GUI version."""

    class tqdm.rich.tqdm(tqdm.tqdm):
        """`rich.progress` version."""

    class tqdm.keras.TqdmCallback(keras.callbacks.Callback):
        """Keras callback for epoch and batch progress."""

    class tqdm.dask.TqdmCallback(dask.callbacks.Callback):
        """Dask callback for task progress."""


``contrib``
+++++++++++

The ``tqdm.contrib`` package also contains experimental modules:

- ``tqdm.contrib.itertools``: Thin wrappers around ``itertools``
- ``tqdm.contrib.concurrent``: Thin wrappers around ``concurrent.futures``
- ``tqdm.contrib.slack``: Posts to `Slack <https://slack.com>`__ bots
- ``tqdm.contrib.discord``: Posts to `Discord <https://discord.com>`__ bots
- ``tqdm.contrib.telegram``: Posts to `Telegram <https://telegram.org>`__ bots
- ``tqdm.contrib.bells``: Automagically enables all optional features

  * ``auto``, ``pandas``, ``slack``, ``discord``, ``telegram``

Examples and Advanced Usage
---------------------------

- See the `examples <https://github.com/tqdm/tqdm/tree/master/examples>`__
  folder;
- import the module and run ``help()``;
- consult the `wiki <https://github.com/tqdm/tqdm/wiki>`__;

  * this has an
    `excellent article <https://github.com/tqdm/tqdm/wiki/How-to-make-a-great-Progress-Bar>`__
    on how to make a **great** progressbar;

- check out the `slides from PyData London <https://tqdm.github.io/PyData2019/slides.html>`__, or
- run the |binder-demo|.

Description and additional stats

Custom information can be displayed and updated dynamically on

.. code:: python

from tqdm import tqdm, trange
from random import random, randint
from time import sleep

with trange(10) as t:
    for i in t:
        # Description will be displayed on the left
        t.set_description('GEN %i' % i)
        # Postfix will be displayed on the right,
        # formatted automatically based on argument's datatype
        t.set_postfix(loss=random(), gen=randint(1,999), str='h',
                      lst=[1, 2])
        sleep(0.1)

with tqdm(total=10, bar_format="{postfix[0]} {postfix[1][value]:>8.2g}",
          postfix=["Batch", {"value": 0}]) as t:
    for i in range(10):
        sleep(0.1)
        t.postfix[1]["value"] = i / 2
        t.update()

Points to remember when using

• postfix
  also needs to be passed as an initial argument in a compatible format, and
• postfix
  will be auto-converted to a string if it is a
  dict
  -like object. To prevent this behaviour, insert an extra item into the dictionary where the key is not a string.

Additional

.. code:: python

from tqdm import tqdm
class TqdmExtraFormat(tqdm):
    """Provides a `total_time` format parameter"""
    @property
    def format_dict(self):
        d = super().format_dict
        total_time = d["elapsed"] * (d["total"] or 0) / max(d["n"], 1)
        d.update(total_time=self.format_interval(total_time) + " in total")
        return d

for i in TqdmExtraFormat(
      range(9), ascii=" .oO0",
      bar_format="{total_time}: {percentage:.0f}%|{bar}{r_bar}"):
    if i == 4:
        break

.. code::

00:00 in total: 44%|0000.     | 4/9 [00:00<00:00, 962.93it/s]

Note that

• width
  • unspecified (default): automatic to fill
    ncols
  • int >= 0
    : fixed width overriding
    ncols
    logic
  • int < 0
    : subtract from the automatic default

• type
  • a
    : ascii (
    ascii=True
    override)
  • u
    : unicode (
    ascii=False
    override)
  • b
    : blank (
    ascii=" "
    override)

This means a fixed bar with right-justified text may be created by using:

Nested progress bars

``tqdm`` supports nested progress bars. Here's an example:

.. code:: python

    from tqdm.auto import trange
    from time import sleep

    for i in trange(4, desc='1st loop'):
        for j in trange(5, desc='2nd loop'):
            for k in trange(50, desc='3rd loop', leave=False):
                sleep(0.01)

For manual control over positioning (e.g. for multi-processing use),
you may specify ``position=n`` where ``n=0`` for the outermost bar,
``n=1`` for the next, and so on.
However, it's best to check if ``tqdm`` can work without manual ``position``
first.

.. code:: python

    from time import sleep
    from tqdm import trange, tqdm
    from multiprocessing import Pool, RLock, freeze_support

    L = list(range(9))

    def progresser(n):
        interval = 0.001 / (n + 2)
        total = 5000
        text = f"#{n}, est. {interval * total:<04.2}s"
        for _ in trange(total, desc=text, position=n):
            sleep(interval)

    if __name__ == '__main__':
        freeze_support()  # for Windows support
        tqdm.set_lock(RLock())  # for managing output contention
        p = Pool(initializer=tqdm.set_lock, initargs=(tqdm.get_lock(),))
        p.map(progresser, L)

Note that in Python 3, ``tqdm.write`` is thread-safe:

.. code:: python

    from time import sleep
    from tqdm import tqdm, trange
    from concurrent.futures import ThreadPoolExecutor

    L = list(range(9))

    def progresser(n):
        interval = 0.001 / (n + 2)
        total = 5000
        text = f"#{n}, est. {interval * total:<04.2}s"
        for _ in trange(total, desc=text):
            sleep(interval)
        if n == 6:
            tqdm.write("n == 6 completed.")
            tqdm.write("`tqdm.write()` is thread-safe in py3!")

    if __name__ == '__main__':
        with ThreadPoolExecutor() as p:
            p.map(progresser, L)

Hooks and callbacks
~~~~~~~~~~~~~~~~~~~

``tqdm`` can easily support callbacks/hooks and manual updates.
Here's an example with ``urllib``:

**``urllib.urlretrieve`` documentation**

    | [...]
    | If present, the hook function will be called once
    | on establishment of the network connection and once after each block read
    | thereafter. The hook will be passed three arguments; a count of blocks
    | transferred so far, a block size in bytes, and the total size of the file.
    | [...]

.. code:: python

    import urllib, os
    from tqdm import tqdm
    urllib = getattr(urllib, 'request', urllib)

    class TqdmUpTo(tqdm):
        """Provides `update_to(n)` which uses `tqdm.update(delta_n)`."""
        def update_to(self, b=1, bsize=1, tsize=None):
            """
            b  : int, optional
                Number of blocks transferred so far [default: 1].
            bsize  : int, optional
                Size of each block (in tqdm units) [default: 1].
            tsize  : int, optional
                Total size (in tqdm units). If [default: None] remains unchanged.
            """
            if tsize is not None:
                self.total = tsize
            return self.update(b * bsize - self.n)  # also sets self.n = b * bsize

    eg_link = "https://caspersci.uk.to/matryoshka.zip"
    with TqdmUpTo(unit='B', unit_scale=True, unit_divisor=1024, miniters=1,
                  desc=eg_link.split('/')[-1]) as t:  # all optional kwargs
        urllib.urlretrieve(eg_link, filename=os.devnull,
                           reporthook=t.update_to, data=None)
        t.total = t.n

Inspired by `twine#242 <https://github.com/pypa/twine/pull/242>`__.
Functional alternative in
`examples/tqdm_wget.py <https://github.com/tqdm/tqdm/blob/master/examples/tqdm_wget.py>`__.

It is recommend to use ``miniters=1`` whenever there is potentially
large differences in iteration speed (e.g. downloading a file over
a patchy connection).

**Wrapping read/write methods**

To measure throughput through a file-like object's ``read`` or ``write``
methods, use ``CallbackIOWrapper``:

.. code:: python

    from tqdm.auto import tqdm
    from tqdm.utils import CallbackIOWrapper

    with tqdm(total=file_obj.size,
              unit='B', unit_scale=True, unit_divisor=1024) as t:
        fobj = CallbackIOWrapper(t.update, file_obj, "read")
        while True:
            chunk = fobj.read(chunk_size)
            if not chunk:
                break
        t.reset()
        # ... continue to use `t` for something else

Alternatively, use the even simpler ``wrapattr`` convenience function,
which would condense both the ``urllib`` and ``CallbackIOWrapper`` examples
down to:

.. code:: python

    import urllib, os
    from tqdm import tqdm

    eg_link = "https://caspersci.uk.to/matryoshka.zip"
    response = getattr(urllib, 'request', urllib).urlopen(eg_link)
    with tqdm.wrapattr(open(os.devnull, "wb"), "write",
                       miniters=1, desc=eg_link.split('/')[-1],
                       total=getattr(response, 'length', None)) as fout:
        for chunk in response:
            fout.write(chunk)

The ``requests`` equivalent is nearly identical:

.. code:: python

    import requests, os
    from tqdm import tqdm

    eg_link = "https://caspersci.uk.to/matryoshka.zip"
    response = requests.get(eg_link, stream=True)
    with tqdm.wrapattr(open(os.devnull, "wb"), "write",
                       miniters=1, desc=eg_link.split('/')[-1],
                       total=int(response.headers.get('content-length', 0))) as fout:
        for chunk in response.iter_content(chunk_size=4096):
            fout.write(chunk)

**Custom callback**

``tqdm`` is known for intelligently skipping unnecessary displays. To make a
custom callback take advantage of this, simply use the return value of
``update()``. This is set to ``True`` if a ``display()`` was triggered.

.. code:: python

    from tqdm.auto import tqdm as std_tqdm

    def external_callback(*args, **kwargs):
        ...

    class TqdmExt(std_tqdm):
        def update(self, n=1):
            displayed = super().update(n)
            if displayed:
                external_callback(**self.format_dict)
            return displayed

``asyncio``
~~~~~~~~~~~

Note that ``break`` isn't currently caught by asynchronous iterators.
This means that ``tqdm`` cannot clean up after itself in this case:

.. code:: python

    from tqdm.asyncio import tqdm

    async for i in tqdm(range(9)):
        if i == 2:
            break

Instead, either call ``pbar.close()`` manually or use the context manager syntax:

.. code:: python

    from tqdm.asyncio import tqdm

    with tqdm(range(9)) as pbar:
        async for i in pbar:
            if i == 2:
                break

Pandas Integration
~~~~~~~~~~~~~~~~~~

Due to popular demand we've added support for ``pandas`` -- here's an example
for ``DataFrame.progress_apply`` and ``DataFrameGroupBy.progress_apply``:

.. code:: python

    import pandas as pd
    import numpy as np
    from tqdm import tqdm

    df = pd.DataFrame(np.random.randint(0, 100, (100000, 6)))

    # Register `pandas.progress_apply` and `pandas.Series.map_apply` with `tqdm`
    # (can use `tqdm.gui.tqdm`, `tqdm.notebook.tqdm`, optional kwargs, etc.)
    tqdm.pandas(desc="my bar!")

    # Now you can use `progress_apply` instead of `apply`
    # and `progress_map` instead of `map`
    df.progress_apply(lambda x: x**2)
    # can also groupby:
    # df.groupby(0).progress_apply(lambda x: x**2)

In case you're interested in how this works (and how to modify it for your
own callbacks), see the
`examples <https://github.com/tqdm/tqdm/tree/master/examples>`__
folder or import the module and run ``help()``.

Keras Integration
~~~~~~~~~~~~~~~~~

A ``keras`` callback is also available:

.. code:: python

    from tqdm.keras import TqdmCallback

    ...

    model.fit(..., verbose=0, callbacks=[TqdmCallback()])

Dask Integration
~~~~~~~~~~~~~~~~

A ``dask`` callback is also available:

.. code:: python

    from tqdm.dask import TqdmCallback

    with TqdmCallback(desc="compute"):
        ...
        arr.compute()

    # or use callback globally
    cb = TqdmCallback(desc="global")
    cb.register()
    arr.compute()

IPython/Jupyter Integration

IPython/Jupyter is supported via the

.. code:: python

from tqdm.notebook import trange, tqdm
from time import sleep

for i in trange(3, desc='1st loop'):
    for j in tqdm(range(100), desc='2nd loop'):
        sleep(0.01)

In addition to

|Screenshot-Jupyter1| |Screenshot-Jupyter2| |Screenshot-Jupyter3|

The

It is also possible to let

.. code:: python

from tqdm.autonotebook import tqdm
tqdm.pandas()

Note that this will issue a

Note that notebooks will display the bar in the cell where it was created. This may be a different cell from the one where it is used. If this is not desired, either

• delay the creation of the bar to the cell where it must be displayed, or
• create the bar with
  display=False
  , and in a later cell call
  display(bar.container)
  :

.. code:: python

from tqdm.notebook import tqdm
pbar = tqdm(..., display=False)

.. code:: python

# different cell
display(pbar.container)

The

.. code:: python

from tqdm.keras import TqdmCallback
cbk = TqdmCallback(display=False)

.. code:: python

# different cell
cbk.display()
model.fit(..., verbose=0, callbacks=[cbk])

Another possibility is to have a single bar (near the top of the notebook) which is constantly re-used (using

.. code:: python

from tqdm.notebook import tqdm
pbar = tqdm()

.. code:: python

# different cell
iterable = range(100)
pbar.reset(total=len(iterable))  # initialise with new `total`
for i in iterable:
    pbar.update()
pbar.refresh()  # force print final status but don't `close()`

Custom Integration

To change the default arguments (such as making ``dynamic_ncols=True``),
simply use built-in Python magic:

.. code:: python

    from functools import partial
    from tqdm import tqdm as std_tqdm
    tqdm = partial(std_tqdm, dynamic_ncols=True)

For further customisation,
``tqdm`` may be inherited from to create custom callbacks (as with the
``TqdmUpTo`` example `above <#hooks-and-callbacks>`__) or for custom frontends
(e.g. GUIs such as notebook or plotting packages). In the latter case:

1. ``def __init__()`` to call ``super().__init__(..., gui=True)`` to disable
   terminal ``status_printer`` creation.
2. Redefine: ``close()``, ``clear()``, ``display()``.

Consider overloading ``display()`` to use e.g.
``self.frontend(**self.format_dict)`` instead of ``self.sp(repr(self))``.

Some submodule examples of inheritance:

- `tqdm/notebook.py <https://github.com/tqdm/tqdm/blob/master/tqdm/notebook.py>`__
- `tqdm/gui.py <https://github.com/tqdm/tqdm/blob/master/tqdm/gui.py>`__
- `tqdm/tk.py <https://github.com/tqdm/tqdm/blob/master/tqdm/tk.py>`__
- `tqdm/contrib/slack.py <https://github.com/tqdm/tqdm/blob/master/tqdm/contrib/slack.py>`__
- `tqdm/contrib/discord.py <https://github.com/tqdm/tqdm/blob/master/tqdm/contrib/discord.py>`__
- `tqdm/contrib/telegram.py <https://github.com/tqdm/tqdm/blob/master/tqdm/contrib/telegram.py>`__

Dynamic Monitor/Meter

You can use a

One example would be recursively searching for files. The

.. code:: python

from tqdm import tqdm
import os.path

def find_files_recursively(path, show_progress=True):
    files = []
    # total=1 assumes `path` is a file
    t = tqdm(total=1, unit="file", disable=not show_progress)
    if not os.path.exists(path):
        raise IOError("Cannot find:" + path)

    def append_found_file(f):
        files.append(f)
        t.update()

    def list_found_dir(path):
        """returns os.listdir(path) assuming os.path.isdir(path)"""
        listing = os.listdir(path)
        # subtract 1 since a "file" we found was actually this directory
        t.total += len(listing) - 1
        # fancy way to give info without forcing a refresh
        t.set_postfix(dir=path[-10:], refresh=False)
        t.update(0)  # may trigger a refresh
        return listing

    def recursively_search(path):
        if os.path.isdir(path):
            for f in list_found_dir(path):
                recursively_search(os.path.join(path, f))
        else:
            append_found_file(path)

    recursively_search(path)
    t.set_postfix(dir=path)
    t.close()
    return files

Using

Writing messages

This is a work in progress (see
`#737 <https://github.com/tqdm/tqdm/issues/737>`__).

Since ``tqdm`` uses a simple printing mechanism to display progress bars,
you should not write any message in the terminal using ``print()`` while
a progressbar is open.

To write messages in the terminal without any collision with ``tqdm`` bar
display, a ``.write()`` method is provided:

.. code:: python

    from tqdm.auto import tqdm, trange
    from time import sleep

    bar = trange(10)
    for i in bar:
        # Print using tqdm class method .write()
        sleep(0.1)
        if not (i % 3):
            tqdm.write("Done task %i" % i)
        # Can also use bar.write()

By default, this will print to standard output ``sys.stdout``. but you can
specify any file-like object using the ``file`` argument. For example, this
can be used to redirect the messages writing to a log file or class.

Redirecting writing

If using a library that can print messages to the console, editing the library by replacing

To redirect

A reusable canonical example is given below:

.. code:: python

from time import sleep
import contextlib
import sys
from tqdm import tqdm
from tqdm.contrib import DummyTqdmFile


@contextlib.contextmanager
def std_out_err_redirect_tqdm():
    orig_out_err = sys.stdout, sys.stderr
    try:
        sys.stdout, sys.stderr = map(DummyTqdmFile, orig_out_err)
        yield orig_out_err[0]
    # Relay exceptions
    except Exception as exc:
        raise exc
    # Always restore sys.stdout/err if necessary
    finally:
        sys.stdout, sys.stderr = orig_out_err

def some_fun(i):
    print("Fee, fi, fo,".split()[i])

# Redirect stdout to tqdm.write() (don't forget the `as save_stdout`)
with std_out_err_redirect_tqdm() as orig_stdout:
    # tqdm needs the original stdout
    # and dynamic_ncols=True to autodetect console width
    for i in tqdm(range(3), file=orig_stdout, dynamic_ncols=True):
        sleep(.5)
        some_fun(i)

# After the `with`, printing is restored
print("Done!")

Redirecting

Similar to ``sys.stdout``/``sys.stderr`` as detailed above, console ``logging``
may also be redirected to ``tqdm.write()``.

Warning: if also redirecting ``sys.stdout``/``sys.stderr``, make sure to
redirect ``logging`` first if needed.

Helper methods are available in ``tqdm.contrib.logging``. For example:

.. code:: python

    import logging
    from tqdm import trange
    from tqdm.contrib.logging import logging_redirect_tqdm

    LOG = logging.getLogger(__name__)

    if __name__ == '__main__':
        logging.basicConfig(level=logging.INFO)
        with logging_redirect_tqdm():
            for i in trange(9):
                if i == 4:
                    LOG.info("console logging redirected to `tqdm.write()`")
        # logging restored

Monitoring thread, intervals and miniters

• Avoid unnecessary frequent bar refreshing:
  mininterval
  defines how long to wait between each refresh.
  tqdm
  always gets updated in the background, but it will display only every
  mininterval
  .
• Reduce number of calls to check system clock/time.
• mininterval
  is more intuitive to configure than
  miniters
  . A clever adjustment system
  dynamic_miniters
  will automatically adjust
  miniters
  to the amount of iterations that fit into time
  mininterval
  . Essentially,
  tqdm
  will check if it's time to print without actually checking time. This behaviour can be still be bypassed by manually setting
  miniters
  .

However, consider a case with a combination of fast and slow iterations. After a few fast iterations,

• maxinterval
  defines the maximum time between display refreshes. A concurrent monitoring thread checks for overdue updates and forces one where necessary.

The monitoring thread should not have a noticeable overhead, and guarantees updates at least every 10 seconds by default. This value can be directly changed by setting the

Merch

You can buy

Contributions

|GitHub-Commits| |GitHub-Issues| |GitHub-PRs| |OpenHub-Status| |GitHub-Contributions| |CII Best Practices|

All source code is hosted on

See the

Developers who have made significant contributions, ranked by SLoC (surviving lines of code,

==================== ======================================================== ==== ================================ Name ID SLoC Notes ==================== ======================================================== ==== ================================ Casper da Costa-Luis

Ports to Other Languages

A list is available on
`this wiki page <https://github.com/tqdm/tqdm/wiki/tqdm-ports>`__.


LICENCE
-------

Open Source (OSI approved): |LICENCE|

Citation information: |DOI|

|README-Hits| (Since 19 May 2016)

.. |Logo| image:: https://tqdm.github.io/img/logo.gif
.. |Screenshot| image:: https://tqdm.github.io/img/tqdm.gif
.. |Video| image:: https://tqdm.github.io/img/video.jpg
   :target: https://tqdm.github.io/video
.. |Slides| image:: https://tqdm.github.io/img/slides.jpg
   :target: https://tqdm.github.io/PyData2019/slides.html
.. |Merch| image:: https://tqdm.github.io/img/merch.jpg
   :target: https://tqdm.github.io/merch
.. |Build-Status| image:: https://img.shields.io/github/actions/workflow/status/tqdm/tqdm/test.yml?branch=master&label=tqdm&logo=GitHub
   :target: https://github.com/tqdm/tqdm/actions/workflows/test.yml
.. |Coverage-Status| image:: https://img.shields.io/coveralls/github/tqdm/tqdm/master?logo=coveralls
   :target: https://coveralls.io/github/tqdm/tqdm
.. |Branch-Coverage-Status| image:: https://codecov.io/gh/tqdm/tqdm/branch/master/graph/badge.svg
   :target: https://codecov.io/gh/tqdm/tqdm
.. |Codacy-Grade| image:: https://app.codacy.com/project/badge/Grade/3f965571598f44549c7818f29cdcf177
   :target: https://www.codacy.com/gh/tqdm/tqdm/dashboard
.. |CII Best Practices| image:: https://bestpractices.coreinfrastructure.org/projects/3264/badge
   :target: https://bestpractices.coreinfrastructure.org/projects/3264
.. |GitHub-Status| image:: https://img.shields.io/github/tag/tqdm/tqdm.svg?maxAge=86400&logo=github&logoColor=white
   :target: https://github.com/tqdm/tqdm/releases
.. |GitHub-Forks| image:: https://img.shields.io/github/forks/tqdm/tqdm.svg?logo=github&logoColor=white
   :target: https://github.com/tqdm/tqdm/network
.. |GitHub-Stars| image:: https://img.shields.io/github/stars/tqdm/tqdm.svg?logo=github&logoColor=white
   :target: https://github.com/tqdm/tqdm/stargazers
.. |GitHub-Commits| image:: https://img.shields.io/github/commit-activity/y/tqdm/tqdm.svg?logo=git&logoColor=white
   :target: https://github.com/tqdm/tqdm/graphs/commit-activity
.. |GitHub-Issues| image:: https://img.shields.io/github/issues-closed/tqdm/tqdm.svg?logo=github&logoColor=white
   :target: https://github.com/tqdm/tqdm/issues?q=
.. |GitHub-PRs| image:: https://img.shields.io/github/issues-pr-closed/tqdm/tqdm.svg?logo=github&logoColor=white
   :target: https://github.com/tqdm/tqdm/pulls
.. |GitHub-Contributions| image:: https://img.shields.io/github/contributors/tqdm/tqdm.svg?logo=github&logoColor=white
   :target: https://github.com/tqdm/tqdm/graphs/contributors
.. |GitHub-Updated| image:: https://img.shields.io/github/last-commit/tqdm/tqdm/master.svg?logo=github&logoColor=white&label=pushed
   :target: https://github.com/tqdm/tqdm/pulse
.. |Gift-Casper| image:: https://img.shields.io/badge/dynamic/json.svg?color=ff69b4&label=gifts%20received&prefix=%C2%A3&query=%24..sum&url=https%3A%2F%2Fcaspersci.uk.to%2Fgifts.json
   :target: https://cdcl.ml/sponsor
.. |Versions| image:: https://img.shields.io/pypi/v/tqdm.svg
   :target: https://tqdm.github.io/releases
.. |PyPI-Downloads| image:: https://img.shields.io/pypi/dm/tqdm.svg?label=pypi%20downloads&logo=PyPI&logoColor=white
   :target: https://pepy.tech/project/tqdm
.. |Py-Versions| image:: https://img.shields.io/pypi/pyversions/tqdm.svg?logo=python&logoColor=white
   :target: https://pypi.org/project/tqdm
.. |Conda-Forge-Status| image:: https://img.shields.io/conda/v/conda-forge/tqdm.svg?label=conda-forge&logo=conda-forge
   :target: https://anaconda.org/conda-forge/tqdm
.. |Snapcraft| image:: https://img.shields.io/badge/snap-install-82BEA0.svg?logo=snapcraft
   :target: https://snapcraft.io/tqdm
.. |Docker| image:: https://img.shields.io/badge/docker-pull-blue.svg?logo=docker&logoColor=white
   :target: https://hub.docker.com/r/tqdm/tqdm
.. |Libraries-Rank| image:: https://img.shields.io/librariesio/sourcerank/pypi/tqdm.svg?logo=koding&logoColor=white
   :target: https://libraries.io/pypi/tqdm
.. |Libraries-Dependents| image:: https://img.shields.io/librariesio/dependent-repos/pypi/tqdm.svg?logo=koding&logoColor=white
    :target: https://github.com/tqdm/tqdm/network/dependents
.. |OpenHub-Status| image:: https://www.openhub.net/p/tqdm/widgets/project_thin_badge?format=gif
   :target: https://www.openhub.net/p/tqdm?ref=Thin+badge
.. |awesome-python| image:: https://awesome.re/mentioned-badge.svg
   :target: https://github.com/vinta/awesome-python
.. |LICENCE| image:: https://img.shields.io/pypi/l/tqdm.svg
   :target: https://raw.githubusercontent.com/tqdm/tqdm/master/LICENCE
.. |DOI| image:: https://img.shields.io/badge/DOI-10.5281/zenodo.595120-blue.svg
   :target: https://doi.org/10.5281/zenodo.595120
.. |binder-demo| image:: https://mybinder.org/badge_logo.svg
   :target: https://mybinder.org/v2/gh/tqdm/tqdm/master?filepath=DEMO.ipynb
.. |Screenshot-Jupyter1| image:: https://tqdm.github.io/img/jupyter-1.gif
.. |Screenshot-Jupyter2| image:: https://tqdm.github.io/img/jupyter-2.gif
.. |Screenshot-Jupyter3| image:: https://tqdm.github.io/img/jupyter-3.gif
.. |README-Hits| image:: https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&style=social&r=https://github.com/tqdm/tqdm&l=https://tqdm.github.io/img/favicon.png&f=https://tqdm.github.io/img/logo.gif
   :target: https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&a=plot&r=https://github.com/tqdm/tqdm&l=https://tqdm.github.io/img/favicon.png&f=https://tqdm.github.io/img/logo.gif&style=social