garak.generators.rasa

Rasa REST API generator interface

Module for Rasa REST API connections (https://rasa.com/)

class RasaRestGenerator(uri=None, config_root=<module 'garak._config' from '/home/docs/checkouts/readthedocs.org/user_builds/garak/checkouts/stable/garak/_config.py'>)Source

Bases: RestGenerator

API interface for RASA models

Uses the following options from _config.plugins.generators["rasa.RasaRestGenerator"]:

  • uri - (optional) the URI of the REST endpoint; this can also be passed

    in –target_name

  • name - a short name for this service; defaults to the uri

  • key_env_var - (optional) the name of the environment variable holding an

    API key, by default RASA_API_KEY

  • req_template - a string where $KEY is replaced by env var RASA_API_KEY

    and $INPUT is replaced by the prompt. Default is to just send the input text.

  • req_template_json_object - (optional) the request template as a Python

    object, to be serialised as a JSON string before replacements

  • method - a string describing the HTTP method, to be passed to the

    requests module; default “post”.

  • headers - dict describing HTTP headers to be sent with the request

  • response_json - Is the response in JSON format? (bool)

  • response_json_field - (optional) Which field of the response JSON

    should be used as the output string? Default text. Can also be a JSONPath value, and response_json_field is used as such if it starts with $.

  • request_timeout - How many seconds should we wait before timing out?

    Default 20

  • ratelimit_codes - Which endpoint HTTP response codes should be caught

    as indicative of rate limiting and retried? List[int], default [429]

Templates can be either a string or a JSON-serialisable Python object. Instance of “$INPUT” here are replaced with the prompt; instances of “$KEY” are replaced with the specified API key. If no key is needed, just don’t put $KEY in a template.

The $INPUT and $KEY placeholders can also be specified in header values.

If we want to call an endpoint where the API key is defined in the value of an X-Authorization header, sending and receiving JSON where the prompt and response value are both under the “text” key, we’d define the service using something like:

{
    "rasa": {
        "RasaRestGenerator": {
            "name": "example rasa service",
            "uri": "https://test.com/webhooks/rest/webhook"
        }
    }
}

To use this specification with garak, you can either pass the JSON as a strong option on the command line via –generator_options, or save the JSON definition into a file and pass the filename to –generator_option_file / -G. For example, if we save the above JSON into example_rasa_service.json, we can invoke garak as:

garak –target_type rest -G example_rasa_service.json

This will load up the default RasaRestGenerator and use the details in the JSON file to connect to the LLM endpoint.

If you need something more flexible, add a new module or class and inherit from RasaRestGenerator :)

Configurable parameters:

DEFAULT_PARAMS contents:

  • max_tokens = 150

  • temperature = None

  • top_k = None

  • context_len = None

  • skip_seq_start = None

  • skip_seq_end = None

  • headers = {'Content-Type': 'application/json', 'Authorization': 'Bearer $KEY'}

  • method = 'post'

  • ratelimit_codes = [429]

  • skip_codes = []

  • response_json = True

  • response_json_field = 'text'

  • req_template = '{"sender": "garak", "message": "$INPUT"}'

  • request_timeout = 20

  • proxies = None

  • verify_ssl = True

  • client_cert = None

  • client_key = None

  • client_key_passphrase_env_var = None

Default values are listed

See also Configuring garak for how to set these values.

Other attributes:

ENV_VAR = 'RASA_API_KEY'
generator_family_name = 'RASA'