garak.generators.replicate

Replicate generator interface

Generator for https://replicate.com/

Put your replicate key in an environment variable called REPLICATE_API_TOKEN. It’s found on your Replicate account page, https://replicate.com/account.

Text-output models are supported.

class garak.generators.replicate.InferenceEndpoint(name='', config_root=<module 'garak._config' from '/home/docs/checkouts/readthedocs.org/user_builds/garak/checkouts/latest/docs/source/../../garak/_config.py'>)

Bases: ReplicateGenerator

Interface for private Replicate endpoints.

Expects name in the format of username/deployed-model-name.

class garak.generators.replicate.ReplicateGenerator(name='', config_root=<module 'garak._config' from '/home/docs/checkouts/readthedocs.org/user_builds/garak/checkouts/latest/docs/source/../../garak/_config.py'>)

Bases: Generator

Interface for public endpoints of models hosted in Replicate (replicate.com).

Expects API key in REPLICATE_API_TOKEN environment variable.

DEFAULT_PARAMS = {'context_len': None, 'max_tokens': 150, 'repetition_penalty': 1, 'skip_seq_end': None, 'skip_seq_start': None, 'temperature': 1, 'top_k': None, 'top_p': 1.0}

ENV_VAR = 'REPLICATE_API_TOKEN'

generator_family_name = 'Replicate'

supports_multiple_generations = False