promptflow
Chat stream async
A chat flow defined using async class entry that return output in stream mode. It demonstrates how to create a chatbot that can remember previous interactions and use the conversation history to generate next message.
Prerequisites
Install promptflow sdk and other dependencies in this folder:
pip install -r requirements.txt
What you will learn
In this flow, you will learn:
- how to compose a chat flow that return output in stream mode.
- prompt template format of LLM tool chat api. Message delimiter is a separate line containing role name and colon: "system:", "user:", "assistant:".
See OpenAI Chat for more about message role.
system:You are a chatbot having a conversation with a human.user:{{question}}
- how to consume chat history in prompt.
{% for item in chat_history %}{{item.role}}:{{item.content}}{% endfor %}
Run flow
-
Prepare your Azure Open AI resource follow this instruction and get your
api_key
if you don't have one. -
Setup connection
Go to "Prompt flow" "Connections" tab. Click on "Create" button, select one of LLM tool supported connection types and fill in the configurations.
Or use CLI to create connection:
# Override keys with --set to avoid yaml file changespf connection create --file ../../connections/azure_openai.yml --set api_key=<your_api_key> api_base=<your_api_base> --name open_ai_connection
Note in flow.flex.yaml we are using connection named open_ai_connection
.
# show registered connectionpf connection show --name open_ai_connection
- Run as normal Python file
python flow.py
- Test flow
pf flow test --flow flow:ChatFlow --init init.json
- Test flow with yaml
You'll need to write flow entry
flow.flex.yaml
to test with prompt flow.
# run chat flow with default question in flow.flex.yamlpf flow test --flow . --init init.json
# run chat flow with new questionpf flow test --flow . --init init.json --inputs question="What's Azure Machine Learning?"
pf flow test --flow . --init init.json --inputs question="What is ChatGPT? Please explain with consise statement."
- Create run with multiple lines data
pf run create --flow . --init init.json --data ./data.jsonl --column-mapping question='${data.question}' --stream
You can also skip providing column-mapping
if provided data has same column name as the flow.
Reference here for default behavior when column-mapping
not provided in CLI.
- List and show run meta
# list created runpf run list
# get a sample run name
name=$(pf run list -r 10 | jq '.[] | select(.name | contains("chat_async_stream_")) | .name'| head -n 1 | tr -d '"')# show specific run detailpf run show --name $name
# show outputpf run show-details --name $name
# visualize run in browserpf run visualize --name $name
Run flow in cloud
- Assume we already have a connection named
open_ai_connection
in workspace.
# set default workspaceaz account set -s <your_subscription_id>az configure --defaults group=<your_resource_group_name> workspace=<your_workspace_name>
- Create run
# run with environment variable reference connection in azureml workspacepfazure run create --flow . --init ./init.json --data ./data.jsonl --column-mapping question='${data.question}' --stream# run using yaml filepfazure run create --file run.yml --stream