LLM-FineTuning-Large-Language-Models

1
{
2
 "cells": [
3
  {
4
   "cell_type": "markdown",
5
   "metadata": {},
6
   "source": [
7
    "## Nous-Hermes-2-Yi-34B-GGUF in Kaggle free GPU with llama.cpp"
8
   ]
9
  },
10
  {
11
   "cell_type": "markdown",
12
   "metadata": {},
13
   "source": [
14
    "### Checkout my [Twitter(@rohanpaul_ai)](https://twitter.com/rohanpaul_ai) for daily LLM bits"
15
   ]
16
  },
17
  {
18
   "cell_type": "code",
19
   "execution_count": null,
20
   "metadata": {},
21
   "outputs": [],
22
   "source": [
23
    "!pip install --upgrade trl peft accelerate bitsandbytes datasets -q\n",
24
    "\n",
25
    "!pip3 install huggingface-hub -q\n",
26
    "\n",
27
    "!huggingface-cli download \\\n",
28
    "TheBloke/Nous-Hermes-2-Yi-34B-GGUF nous-hermes-2-yi-34b.Q2_K.gguf \\\n",
29
    "--local-dir . --local-dir-use-symlinks False\n",
30
    "\n",
31
    "!git clone https://github.com/ggerganov/llama.cpp.git\n",
32
    "\n",
33
    "cd ./llama.cpp\n",
34
    "\n",
35
    "!make\n",
36
    "\n",
37
    "!git clone https://github.com/ggerganov/llama.cpp.git\n",
38
    "\n",
39
    "cd ./llama.cpp\n",
40
    "\n",
41
    "!make\n",
42
    "\n",
43
    "\n",
44
    "!./main -ngl 35 \\\n",
45
    "-m /kaggle/working/nous-hermes-2-yi-34b.Q2_K.gguf \\\n",
46
    "--color -c 4096 \\\n",
47
    "--temp 0.7 \\\n",
48
    "--repeat_penalty 1.1 \\\n",
49
    "-n -1 \\\n",
50
    "-p \"system\\n{\\\"You are a friendly AI\\\"}\\nuser\\n{\\\"Tell me if you know python coding\\\"}\\nassistant\"\n"
51
   ]
52
  },
53
  {
54
   "cell_type": "markdown",
55
   "metadata": {},
56
   "source": [
57
    "Run Nous-Hermes-2-Yi-34B-GGUF in Kaggle's free GPU with llama.cpp\n",
58
    "\n",
59
    "You can also run it in colab just by changing the path of downloaded Model file"
60
   ]
61
  }
62
 ],
63
 "metadata": {
64
  "language_info": {
65
   "name": "python"
66
  }
67
 },
68
 "nbformat": 4,
69
 "nbformat_minor": 2
70
}
71