garak.generators.replicate

Replicate generator interface

Generator for https://replicate.com/

Put your replicate key in an environment variable called REPLICATE_API_TOKEN. It’s found on your Replicate account page, https://replicate.com/account.

Text-output models are supported.

class InferenceEndpoint(name='', config_root=<module 'garak._config' from '/home/docs/checkouts/readthedocs.org/user_builds/garak/checkouts/latest/docs/source/../../garak/_config.py'>)Source

Bases: ReplicateGenerator

Interface for private Replicate endpoints.

Expects name in the format of username/deployed-model-name.

class ReplicateGenerator(name='', config_root=<module 'garak._config' from '/home/docs/checkouts/readthedocs.org/user_builds/garak/checkouts/latest/docs/source/../../garak/_config.py'>)Source

Bases: Generator

Interface for public endpoints of models hosted in Replicate (replicate.com).

Expects API key in REPLICATE_API_TOKEN environment variable.

Configurable parameters:

DEFAULT_PARAMS contents:

  • max_tokens = 150

  • temperature = 1

  • top_k = None

  • context_len = None

  • skip_seq_start = None

  • skip_seq_end = None

  • top_p = 1.0

  • repetition_penalty = 1

Default values are listed

See also Configuring garak for how to set these values.

Other attributes:

ENV_VAR = 'REPLICATE_API_TOKEN'
extra_dependency_names = ['replicate']
generator_family_name = 'Replicate'
supports_multiple_generations = False