Model Conversion Example

Overview

Setup

Causal Language Model Conversion

Download the original model

Set the MODELPATH

Inspecting the original tensors

Running the original model

Model conversion

Inspecting the converted model

Running the converted model

Model logits verification

Quantizing the model

Quantizing QAT (Quantization Aware Training) models

Embedding Language Model Conversion

Download the original model

Running the original model

Using SentenceTransformer with numbered layers

Model conversion

Run the converted model

Model logits verification

llama-server verification

Quantizing the model

Quantizing QAT (Quantization Aware Training) models

Perplexity Evaluation

Simple perplexity evaluation

Full perplexity evaluation

HuggingFace utilities

Create a new Hugging Face Model (model repository)

Upload a GGUF model to model repository

Create a new Collection

Add model to a Collection