llama.cpp/tools/llama-bench

Table of contents

Syntax

Examples

Text generation with different models

Prompt processing with different batch sizes

Different numbers of threads

Different numbers of layers offloaded to the GPU

Different prefilled context

Output formats

Markdown

CSV

JSON

JSONL

SQL