promptflow

1
{
2
 "cells": [
3
  {
4
   "cell_type": "markdown",
5
   "metadata": {},
6
   "source": [
7
    "# Prompty output format"
8
   ]
9
  },
10
  {
11
   "cell_type": "markdown",
12
   "metadata": {},
13
   "source": [
14
    "**Learning Objectives** - Upon completing this tutorial, you should be able to:\n",
15
    "\n",
16
    "- Understand how to handle output format of prompty like: text, json_object.\n",
17
    "- Understand how to consume stream output of prompty\n",
18
    "\n",
19
    "## 0. Install dependent packages"
20
   ]
21
  },
22
  {
23
   "cell_type": "code",
24
   "execution_count": null,
25
   "metadata": {},
26
   "outputs": [],
27
   "source": [
28
    "%%capture --no-stderr\n",
29
    "%pip install promptflow-devkit"
30
   ]
31
  },
32
  {
33
   "cell_type": "markdown",
34
   "metadata": {},
35
   "source": [
36
    "## 1. Create necessary connections\n",
37
    "Connection helps securely store and manage secret keys or other sensitive credentials required for interacting with LLM and other external tools for example Azure Content Safety.\n",
38
    "\n",
39
    "Above prompty uses connection `open_ai_connection` inside, we need to set up the connection if we haven't added it before. After created, it's stored in local db and can be used in any flow.\n",
40
    "\n",
41
    "Prepare your Azure Open AI resource follow this [instruction](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal) and get your `api_key` if you don't have one.\n",
42
    "\n",
43
    "Note: you need the new [gpt-35-turbo (0125) version](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models#gpt-35-models) to use the json_object response_format feature."
44
   ]
45
  },
46
  {
47
   "cell_type": "code",
48
   "execution_count": null,
49
   "metadata": {},
50
   "outputs": [],
51
   "source": [
52
    "from promptflow.client import PFClient\n",
53
    "from promptflow.connections import AzureOpenAIConnection, OpenAIConnection\n",
54
    "\n",
55
    "# client can help manage your runs and connections.\n",
56
    "pf = PFClient()\n",
57
    "try:\n",
58
    "    conn_name = \"open_ai_connection\"\n",
59
    "    conn = pf.connections.get(name=conn_name)\n",
60
    "    print(\"using existing connection\")\n",
61
    "except:\n",
62
    "    # Follow https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/create-resource?pivots=web-portal to create an Azure Open AI resource.\n",
63
    "    connection = AzureOpenAIConnection(\n",
64
    "        name=conn_name,\n",
65
    "        api_key=\"<your_AOAI_key>\",\n",
66
    "        api_base=\"<your_AOAI_endpoint>\",\n",
67
    "        api_type=\"azure\",\n",
68
    "    )\n",
69
    "\n",
70
    "    # use this if you have an existing OpenAI account\n",
71
    "    # connection = OpenAIConnection(\n",
72
    "    #     name=conn_name,\n",
73
    "    #     api_key=\"<user-input>\",\n",
74
    "    # )\n",
75
    "\n",
76
    "    conn = pf.connections.create_or_update(connection)\n",
77
    "    print(\"successfully created connection\")\n",
78
    "\n",
79
    "print(conn)"
80
   ]
81
  },
82
  {
83
   "cell_type": "markdown",
84
   "metadata": {},
85
   "source": [
86
    "## 2. Format prompty output"
87
   ]
88
  },
89
  {
90
   "cell_type": "markdown",
91
   "metadata": {},
92
   "source": [
93
    "### Text output\n",
94
    "By default the prompty returns the message of first choices."
95
   ]
96
  },
97
  {
98
   "cell_type": "code",
99
   "execution_count": null,
100
   "metadata": {},
101
   "outputs": [],
102
   "source": [
103
    "with open(\"text_format.prompty\") as fin:\n",
104
    "    print(fin.read())"
105
   ]
106
  },
107
  {
108
   "cell_type": "code",
109
   "execution_count": null,
110
   "metadata": {},
111
   "outputs": [],
112
   "source": [
113
    "from promptflow.core import Prompty\n",
114
    "\n",
115
    "# load prompty as a flow\n",
116
    "f = Prompty.load(\"text_format.prompty\")\n",
117
    "# execute the flow as function\n",
118
    "question = \"What is the capital of France?\"\n",
119
    "result = f(first_name=\"John\", last_name=\"Doe\", question=question)\n",
120
    "\n",
121
    "# note: the result is a string\n",
122
    "result"
123
   ]
124
  },
125
  {
126
   "cell_type": "markdown",
127
   "metadata": {},
128
   "source": [
129
    "### Json object output\n",
130
    "\n",
131
    "When the user meets the following conditions, prompty returns content of first choices as a dict.\n",
132
    "- Define `response_format` to `type: json_object` in parameters \n",
133
    "- Specify the return json format in template.\n",
134
    "\n",
135
    "Note: response_format is compatible with GPT-4 Turbo and all GPT-3.5 Turbo models newer than gpt-3.5-turbo-1106. For more details, refer to this [document](https://platform.openai.com/docs/api-reference/chat/create#chat-create-response_format)."
136
   ]
137
  },
138
  {
139
   "cell_type": "code",
140
   "execution_count": null,
141
   "metadata": {},
142
   "outputs": [],
143
   "source": [
144
    "with open(\"json_format.prompty\") as fin:\n",
145
    "    print(fin.read())"
146
   ]
147
  },
148
  {
149
   "cell_type": "code",
150
   "execution_count": null,
151
   "metadata": {},
152
   "outputs": [],
153
   "source": [
154
    "from promptflow.core import Prompty\n",
155
    "\n",
156
    "# load prompty as a flow\n",
157
    "f = Prompty.load(\"json_format.prompty\")\n",
158
    "# execute the flow as function\n",
159
    "question = \"What is the capital of France?\"\n",
160
    "result = f(first_name=\"John\", last_name=\"Doe\", question=question)\n",
161
    "\n",
162
    "# note: the result is a dict\n",
163
    "result"
164
   ]
165
  },
166
  {
167
   "cell_type": "markdown",
168
   "metadata": {},
169
   "source": [
170
    "### All choices\n",
171
    "\n",
172
    "When the user configures response as `all`, prompty will return the raw LLM response which has all the choices.\n"
173
   ]
174
  },
175
  {
176
   "cell_type": "code",
177
   "execution_count": null,
178
   "metadata": {},
179
   "outputs": [],
180
   "source": [
181
    "with open(\"all_response.prompty\") as fin:\n",
182
    "    print(fin.read())"
183
   ]
184
  },
185
  {
186
   "cell_type": "code",
187
   "execution_count": null,
188
   "metadata": {},
189
   "outputs": [],
190
   "source": [
191
    "from promptflow.core import Prompty\n",
192
    "\n",
193
    "# load prompty as a flow\n",
194
    "f = Prompty.load(\"all_response.prompty\")\n",
195
    "# execute the flow as function\n",
196
    "question = \"What is the capital of France?\"\n",
197
    "result = f(first_name=\"John\", last_name=\"Doe\", question=question)\n",
198
    "\n",
199
    "# note: the result is a ChatCompletion object\n",
200
    "print(result.choices[0])"
201
   ]
202
  },
203
  {
204
   "cell_type": "markdown",
205
   "metadata": {},
206
   "source": [
207
    "### Streaming output"
208
   ]
209
  },
210
  {
211
   "cell_type": "markdown",
212
   "metadata": {},
213
   "source": [
214
    "When `stream=true` is configured in the parameters of a prompt whose output format is text, promptflow sdk will return a generator type, which item is the content of each chunk."
215
   ]
216
  },
217
  {
218
   "cell_type": "code",
219
   "execution_count": null,
220
   "metadata": {},
221
   "outputs": [],
222
   "source": [
223
    "with open(\"stream_output.prompty\") as fin:\n",
224
    "    print(fin.read())"
225
   ]
226
  },
227
  {
228
   "cell_type": "code",
229
   "execution_count": null,
230
   "metadata": {},
231
   "outputs": [],
232
   "source": [
233
    "from promptflow.core import Prompty\n",
234
    "\n",
235
    "# load prompty as a flow\n",
236
    "f = Prompty.load(\"stream_output.prompty\")\n",
237
    "# execute the flow as function\n",
238
    "question = \"What's the steps to get rich?\"\n",
239
    "result = f(question=question)\n",
240
    "for item in result:\n",
241
    "    print(item, end=\"\")"
242
   ]
243
  },
244
  {
245
   "cell_type": "markdown",
246
   "metadata": {},
247
   "source": [
248
    "Notes: When `stream=True`, if the response format is `json_object` or response is `all`, LLM response will be returned directly. For more details about handle stream response, refer to this [document](https://platform.openai.com/docs/api-reference/chat/create#chat-create-stream).\n"
249
   ]
250
  },
251
  {
252
   "cell_type": "markdown",
253
   "metadata": {},
254
   "source": [
255
    "### Batch run with text output"
256
   ]
257
  },
258
  {
259
   "cell_type": "code",
260
   "execution_count": null,
261
   "metadata": {},
262
   "outputs": [],
263
   "source": [
264
    "from promptflow.client import PFClient\n",
265
    "\n",
266
    "data = \"./data.jsonl\"  # path to the data file\n",
267
    "\n",
268
    "# create run with the flow and data\n",
269
    "pf = PFClient()\n",
270
    "base_run = pf.run(\n",
271
    "    flow=\"text_format.prompty\",\n",
272
    "    data=data,\n",
273
    "    column_mapping={\n",
274
    "        \"question\": \"${data.question}\",\n",
275
    "    },\n",
276
    "    stream=True,\n",
277
    ")"
278
   ]
279
  },
280
  {
281
   "cell_type": "code",
282
   "execution_count": null,
283
   "metadata": {},
284
   "outputs": [],
285
   "source": [
286
    "details = pf.get_details(base_run)\n",
287
    "details.head(10)"
288
   ]
289
  },
290
  {
291
   "cell_type": "markdown",
292
   "metadata": {},
293
   "source": [
294
    "### Batch run with stream output"
295
   ]
296
  },
297
  {
298
   "cell_type": "code",
299
   "execution_count": null,
300
   "metadata": {},
301
   "outputs": [],
302
   "source": [
303
    "from promptflow.client import PFClient\n",
304
    "\n",
305
    "data = \"./data.jsonl\"  # path to the data file\n",
306
    "\n",
307
    "# create run with the flow and data\n",
308
    "pf = PFClient()\n",
309
    "base_run = pf.run(\n",
310
    "    flow=\"stream_output.prompty\",\n",
311
    "    data=data,\n",
312
    "    column_mapping={\n",
313
    "        \"question\": \"${data.question}\",\n",
314
    "    },\n",
315
    "    stream=True,\n",
316
    ")"
317
   ]
318
  },
319
  {
320
   "cell_type": "code",
321
   "execution_count": null,
322
   "metadata": {},
323
   "outputs": [],
324
   "source": [
325
    "details = pf.get_details(base_run)\n",
326
    "details.head(10)"
327
   ]
328
  }
329
 ],
330
 "metadata": {
331
  "build_doc": {
332
   "author": [
333
    "lalala123123@github.com",
334
    "wangchao1230@github.com"
335
   ],
336
   "category": "local",
337
   "section": "Prompty",
338
   "weight": 30
339
  },
340
  "kernelspec": {
341
   "display_name": "prompt",
342
   "language": "python",
343
   "name": "python3"
344
  },
345
  "language_info": {
346
   "codemirror_mode": {
347
    "name": "ipython",
348
    "version": 3
349
   },
350
   "file_extension": ".py",
351
   "mimetype": "text/x-python",
352
   "name": "python",
353
   "nbconvert_exporter": "python",
354
   "pygments_lexer": "ipython3",
355
   "version": "3.9.18"
356
  }
357
 },
358
 "nbformat": 4,
359
 "nbformat_minor": 2
360
}
361