garak.generators.rasa
Rasa REST API generator interface
Module for Rasa REST API connections (https://rasa.com/)
- class garak.generators.rasa.RasaRestGenerator(uri=None, config_root=<module 'garak._config' from '/home/docs/checkouts/readthedocs.org/user_builds/garak/checkouts/latest/docs/source/../../garak/_config.py'>)
Bases:
RestGenerator
API interface for RASA models
Uses the following options from _config.plugins.generators[“rasa.RasaRestGenerator”]: *
uri
- (optional) the URI of the REST endpoint; this can also be passedin –model_name
name
- a short name for this service; defaults to the urikey_env_var
- (optional) the name of the environment variable holding anAPI key, by default RASA_API_KEY
req_template
- a string where $KEY is replaced by env var RASA_API_KEYand $INPUT is replaced by the prompt. Default is to just send the input text.
req_template_json_object
- (optional) the request template as a Pythonobject, to be serialised as a JSON string before replacements
method
- a string describing the HTTP method, to be passed to therequests module; default “post”.
headers
- dict describing HTTP headers to be sent with the requestresponse_json
- Is the response in JSON format? (bool)response_json_field
- (optional) Which field of the response JSONshould be used as the output string? Default
text
. Can also be a JSONPath value, andresponse_json_field
is used as such if it starts with$
.
request_timeout
- How many seconds should we wait before timing out?Default 20
ratelimit_codes
- Which endpoint HTTP response codes should be caughtas indicative of rate limiting and retried? List[int], default [429]
Templates can be either a string or a JSON-serialisable Python object. Instance of “$INPUT” here are replaced with the prompt; instances of “$KEY” are replaced with the specified API key. If no key is needed, just don’t put $KEY in a template.
The $INPUT and $KEY placeholders can also be specified in header values.
If we want to call an endpoint where the API key is defined in the value of an
X-Authorization header
, sending and receiving JSON where the prompt and response value are both under the “text” key, we’d define the service using something like:- {
- “rasa”: {
- “RasaRestGenerator”: {
“name”: “example rasa service”, “uri”: “https://test.com/webhooks/rest/webhook”
}
}
}
To use this specification with garak, you can either pass the JSON as a strong option on the command line via –generator_options, or save the JSON definition into a file and pass the filename to –generator_option_file / -G. For example, if we save the above JSON into `example_rasa_service.json”, we can invoke garak as:
garak –model_type rest -G example_rasa_service.json
This will load up the default RasaRestGenerator and use the details in the JSON file to connect to the LLM endpoint.
If you need something more flexible, add a new module or class and inherit from RasaRestGenerator :)
- DEFAULT_PARAMS = {'context_len': None, 'headers': {'Authorization': 'Bearer $KEY', 'Content-Type': 'application/json'}, 'max_tokens': 150, 'method': 'post', 'ratelimit_codes': [429], 'req_template': '{"sender": "garak", "message": "$INPUT"}', 'request_timeout': 20, 'response_json': True, 'response_json_field': 'text', 'skip_codes': [], 'temperature': None, 'top_k': None}
- ENV_VAR = 'RASA_API_KEY'
- generator_family_name = 'RASA'