garak.generators.rest

Flexible connector for REST-based APIs.

Uses the following options from _config.plugins.generators["rest.RestGenerator"]:

• uri - (optional) the URI of the REST endpoint; this can also be passed in –target_name

• name - a short name for this service; defaults to the uri

• key_env_var - (optional) the name of the environment variable holding an API key, by default REST_API_KEY

• req_template - a string where the text $KEY is replaced by env var REST_API_KEY’s value (or whatever’s specified in key_env_var) and $INPUT is replaced by the prompt. Default is to just send the input text.

• req_template_json_object - (optional) the request template as a Python object, to be serialised as a JSON string before replacements

• method - a string describing the HTTP method, to be passed to the requests module; default “post”.

• headers - dict describing HTTP headers to be sent with the request

• proxies - dict passed to requests method call. See required format.

• response_json - Is the response in JSON format? (bool)

• response_json_field - (optional) Which field of the response JSON should be used as the output string? Default text. Can also be a JSONPath value, and response_json_field is used as such if it starts with $.

• request_timeout - How many seconds should we wait before timing out? Default 20

• ratelimit_codes - Which endpoint HTTP response codes should be caught as indicative of rate limiting and retried? List[int], default [429]

• skip_codes - Which endpoint HTTP response code should lead to the generation being treated as not possible and skipped for this query. Takes precedence over ratelimit_codes.

• verify_ssl - (optional) Enforce ssl certificate validation? Default is True, a file path to a CA bundle can be provided. (bool|str)

• client_cert - (optional) Path to a PEM-encoded client certificate file for mTLS. If the file contains both the certificate and private key, client_key is not required.

• client_key - (optional) Path to the client private key file for mTLS. Must be set if client_cert does not contain the key. Cannot be set without client_cert.

• client_key_passphrase_env_var - (optional) Name of an environment variable holding the passphrase for an encrypted client_key. The passphrase is never read from config files. If set, the named env var must be defined or startup will fail.

Templates can be either a string or a JSON-serialisable Python object. Instance of $INPUT here are replaced with the prompt; instances of $KEY are replaced with the specified API key. If no key is needed, just don’t put $KEY in a template.

The $INPUT and $KEY placeholders can also be specified in header values.

If we want to call an endpoint where the API key is defined in the value of an X-Authorization header, sending and receiving JSON where the prompt and response value are both under the text key, we’d define the service using something like:

{
   "rest": {
      "RestGenerator": {
         "name": "example service",
         "uri": "https://example.ai/llm",
         "method": "post",
         "headers": {
            "X-Authorization": "$KEY"
         },
         "req_template_json_object": {
            "text": "$INPUT"
         },
         "response_json": true,
         "response_json_field": "text"
      }
   }
}

NB. response_json_field can also be a JSONPath, for JSON responses where the target text is not in a top level field. It is treated as a JSONPath when response_json_field starts with $.

To use this specification with garak, you can either pass the JSON as a strong option on the command line via --generator_options, or save the JSON definition into a file and pass the filename to --generator_option_file / -G. For example, if we save the above JSON into example_service.json, we can invoke garak as:

garak --target_type rest -G example_service.json

This will load up the default RestGenerator and use the details in the JSON file to connect to the LLM endpoint.

If you need something more flexible, add a new module or class and inherit from RestGenerator.

mTLS Client Certificate Authentication

To authenticate with a server that requires mutual TLS, provide your client certificate and (optionally) key. For a server-side CA bundle, use verify_ssl with a file path:

{
   "rest": {
      "RestGenerator": {
         "name": "mTLS protected service",
         "uri": "https://api.example.com/llm",
         "client_cert": "/path/to/client.crt",
         "client_key": "/path/to/client.key",
         "client_key_passphrase_env_var": "MTLS_KEY_PASSPHRASE",
         "verify_ssl": "/path/to/ca.crt"
      }
   }
}

Set the passphrase securely via environment variable:

export MTLS_KEY_PASSPHRASE="your_key_passphrase"
garak --target_type rest -G mtls_config.json --probes dan

If using a combined PEM file containing both certificate and key, omit client_key:

{
   "rest": {
      "RestGenerator": {
         "uri": "https://api.example.com/llm",
         "client_cert": "/path/to/combined.pem",
         "verify_ssl": "/path/to/ca.crt"
      }
   }
}

---

REST API generator interface

Generic Module for REST API connections

class RestGenerator(uri=None, config_root=<module 'garak._config' from '/home/docs/checkouts/readthedocs.org/user_builds/garak/checkouts/latest/garak/_config.py'>)Source

Bases: Generator

Generic API interface for REST models

See reference docs for details (https://reference.garak.ai/en/latest/garak.generators.rest.html)

Configurable parameters:

DEFAULT_PARAMS contents:

• max_tokens = 150

• temperature = None

• top_k = None

• context_len = None

• skip_seq_start = None

• skip_seq_end = None

• headers = {}

• method = 'post'

• ratelimit_codes = [429]

• skip_codes = []

• response_json = False

• response_json_field = None

• req_template = '$INPUT'

• request_timeout = 20

• proxies = None

• verify_ssl = True

• client_cert = None

• client_key = None

• client_key_passphrase_env_var = None

Default values are listed

See also Configuring garak for how to set these values.

Other attributes:

ENV_VAR = 'REST_API_KEY'

generator_family_name = 'REST'