DialogFlow Framework Template

Changes can only be made in the dialogflows directory.

Template has dialog flows based on programy (repeating) and based on valila python (greeting).

Importan changes in files of the agent

docker-compose.yml

  dff-template:
    build:
      args:
        SERVICE_PORT: 8095
        SERVICE_NAME: dff_template # has to be the same with skill dir name
      context: .
      dockerfile: ./skills/dff_template/Dockerfile
    command:  gunicorn --workers=1 server:app -b 0.0.0.0:8095 --reload
    deploy:
      mode: replicated
      replicas: 4
      resources:
        limits:
          memory: 768M
        reservations:
          memory: 768M

dev.yml

  dff-template:
    volumes:
      - "./skills/dff_template:/src"
      - "./common:/src/common"
    ports:
      - 8095:8095

pipeline.json

            "dff_template": {
                "connector": {
                    "protocol": "http",
                    "url": "http://dff-template:8095/respond"
                },
                "dialog_formatter": "state_formatters.dp_formatters:dff_template_formatter",
                "response_formatter": "state_formatters.dp_formatters:skill_with_attributes_formatter_service",
                "previous_services": ["skill_selectors"],
                "state_manager_method": "add_hypothesis"
            },

state_formatters/formatter.py

def DFF_TEMPLATE_formatter(dialog: Dict) -> List[Dict]:
    service_name = f"DFF_TEMPLATE"
    return utils.dff_formatter(dialog, service_name)

skill_selectors/rule_based_selector/connector.py

        response=["dff_template"],

Tests

Test creating

The file server.py contains this code

@app.route("/respond", methods=["POST"])
def respond():
    # next commented line for test creating
    # import common.test_utils as t_utils; t_utils.save_to_test(request.json,"tests/TEST_NAME_in.json",indent=4)
    responses = handler(request.json)
    # next commented line for test creating
    # import common.test_utils as t_utils; t_utils.save_to_test(responses,"tests/TEST_NAME_out.json",indent=4)
    return jsonify(responses)

Steps:

1. Uncomment lines with json dump
2. Name your test by replacing YOUR_TEST_NAME in both line. They have to be same.
3. Start a test dialog with agent.Every turn will be written in tests/TEST_NAME*. *_in.json - for input data, *_in.json - for response data.

If your want to write down all turns of test dialog you can use this code

index = 0
@app.route("/respond", methods=["POST"])
def respond():
    # next commented line for test creating
    import common.test_utils as t_utils;t_utils.save_to_test(responses,f"tests/TEST_NAME_{index}_in.json",indent=4)
    responses = handler(request.json)
    # next commented line for test creating
    import common.test_utils as t_utils;t_utils.save_to_test(responses,f"tests/TEST_NAME_{index}_out.json",indent=4)
    index +=1
    return jsonify(responses)

Test using

Tests are used for two way:

• service initialization in server.py

try:
    test_server.run_test(handler)
    logger.info("test query processed")
except Exception as exc:
    sentry_sdk.capture_exception(exc)
    logger.exception(exc)
    raise exc

• service testing by test.sh execution

Test extending

If you use service based on random behavior you can send random_seed in your service. You can find corespond lines in server.py

    ... # some code
    rand_seed = requested_data.get("rand_seed")  # for tests
    ... # some code
    if rand_seed:
        random.seed(int(rand_seed)
    ... # some code

For answer comparison we use common.test_utils:

• compare_structs - for json structure comparison
• compare_text - for text comparison

You can use them for you custom comparison.

Links between dff skills

1. Making a link (example of link from dff_animals_skill to dff_wiki_skill)

    import common.dialogflow_framework.utils.state as state_utils
    ... # some code
    def why_do_you_like_response(vars):
        ... # some code
        if found_animal:
            response = f"Cool! Why do you like {found_animal}?"
        else:
            response = f"Cool! Why do you like them?"
            
        if found_entity_id:
            # making cross link
            state_utils.set_cross_link(vars, to_service_name="dff_wiki_skill", from_service_name="dff_animals_skill")
            add_info = {"entity_id": found_entity_id, "entity_substr": found_animal, "entity_types": found_types,
                        "entity_page": found_entity_page} # if we want to pass some info between skills
            # save info in cross state
            state_utils.save_cross_state(vars, service_name="dff_wiki_skill", new_state=add_info)
            state_utils.set_dff_suspension(vars) # stop current dff skill so that after the next dff skill will finish
                                                 # its scenario, the current scenario was resumed from this state
        
        return response

2. Using the link in the destination skill (dff_wiki_skill in our example)

    import common.dialogflow_framework.utils.state as state_utils
    ... # some code
    def tell_fact_request(ngrams, vars):
        cross_link = state_utils.get_cross_link(vars, service_name="dff_wiki_skill")
        # cross link is a dict {"from_service": "dff_animals_skill"}
        cross_state = state_utils.get_cross_state(vars, service_name="dff_wiki_skill")
        # cross_state is a dict add_info which was saved in why_do_you_like_response using save_cross_state function
        from_skill = cross_link.get("from_service", "")
        if from_skill == "dff_animals_skill":
            flag = True

3. To switch the destination skill if the link was made, we can add a function in common folder (in our example in common/wiki_skill.py)

    def find_wiki_cross_links(dialog):
        flag = False
        human_attributes = dialog.get("human", {}).get("attributes", {})
        dff_shared_state = human_attributes.get("dff_shared_state", {"cross_states": {}, "cross_links": {}})
        cross_links = dff_shared_state["cross_links"].get("dff_wiki_skill", {})
        if cross_links:
            flag = True
        return flag 

Then in skill_selectors/rule_based_selector/connector.py:

    from common.wiki_skill import find_wiki_cross_links
    ... # some code
        if find_wiki_cross_links(dialog):
            skills_for_uttr.append("dff_wiki_skill")

4. Reverse transition (from dff_wiki_skill to dff_animals_skill in our example) is made the way.

Insert scenario parser to a dff skill

    ... # some imports
    import json
    from common.insert_scenario import start_or_continue_scenario, smalltalk_response, start_or_continue_facts, \
        facts_response # imports for scenario insertion
    
    # place your config in the directory skills/your_dff_skill_name/{inserted_scenario_config_name}.json
    # and load config
    with open(inserted_scenario_config_name, 'r') as fl:
        topic_config = json.load(fl)

    class State(Enum):
        USR_START = auto()
        #
        ... # States of your skill
        
        # States for scenario insertion
        SYS_INSERT_SMALLTALK = auto()
        USR_INSERT_SMALLTALK = auto()
        #
        SYS_INSERT_FACT = auto()
        USR_INSERT_FACT = auto()

        ... # Some other states of your skill

        # Two request and two response functions for scenario insertion

        def insert_scenario_smalltalk_request(ngrams, vars):
            flag = start_or_continue_scenario(vars, topic_config)
            logger.info(f"special_topic_request={flag}")
            return flag
        
        
        def insert_scenario_smalltalk_response(vars):
            response = smalltalk_response(vars, topic_config)
            return response
        
        
        def insert_scenario_facts_request(ngrams, vars):
            flag = start_or_continue_facts(vars, topic_config)
            logger.info(f"special_topic_facts_request={flag}")
            return flag
        
        
        def insert_scenario_facts_response(vars):
            response = facts_response(vars, topic_config)
            return response

        simplified_dialog_flow = dialogflow_extension.DFEasyFilling(State.USR_START)
    
        ... # Your state transitions

        # State transitions for scenario insertion

        simplified_dialog_flow.add_user_serial_transitions(
            State.SOME_STATE,
            {
                ... # transitions to other states
                State.SYS_INSERT_SMALLTALK: insert_scenario_smalltalk_request,
            },
        )

        simplified_dialog_flow.add_user_serial_transitions(
            State.USR_INSERT_SMALLTALK,
            {
                State.SYS_INSERT_FACT: insert_scenario_facts_request,
                State.SYS_INSERT_SMALLTALK: insert_scenario_smalltalk_request,
                State.SOME_OTHER_YOUR_STATE: some_other_state_request,
            },
        )
        
        simplified_dialog_flow.add_user_serial_transitions(
            State.USR_INSERT_FACT,
            {
                State.SYS_INSERT_SMALLTALK: insert_scenario_smalltalk_request,
                State.SYS_INSERT_FACT: insert_scenario_facts_request,
                State.SOME_OTHER_YOUR_STATE: some_other_state_request,
            },
        )
        
        simplified_dialog_flow.add_system_transition(State.SYS_INSERT_SMALLTALK, State.USR_INSERT_SMALLTALK,
                                             insert_scenario_smalltalk_response, )
        simplified_dialog_flow.add_system_transition(State.SYS_INSERT_FACT, State.USR_INSERT_FACT,
                                             insert_scenario_facts_response, )

        simplified_dialog_flow.set_error_successor(State.SYS_INSERT_SMALLTALK, State.SYS_ERR)
        simplified_dialog_flow.set_error_successor(State.USR_INSERT_SMALLTALK, State.SYS_ERR)
        simplified_dialog_flow.set_error_successor(State.SYS_INSERT_FACT, State.SYS_ERR)
        simplified_dialog_flow.set_error_successor(State.USR_INSERT_FACT, State.SYS_ERR)