garak.generators.nemo

Wrapper for nemollm.

Expects NGC API key in the environment variable NGC_API_KEY and the organisation ID in environment variable ORG_ID.

Configurable values:

  • temperature: 0.9

  • top_p: 1.0

  • top_k: 2

  • repetition_penalty: 1.1 - between 1 and 2 incl., or none

  • beam_search_diversity_rate: 0.0

  • beam_width: 1

  • length_penalty: 1

  • guardrail: None - (present in API but not implemented in library)

  • api_uri: “https://api.llm.ngc.nvidia.com/v1” - endpoint URI

NeMo LLM interface

class garak.generators.nemo.NeMoGenerator(name=None, config_root=<module 'garak._config' from '/home/docs/checkouts/readthedocs.org/user_builds/garak/checkouts/latest/docs/source/../../garak/_config.py'>)

Bases: Generator

Wrapper for the NVIDIA NeMo models via NGC. Expects NGC_API_KEY and ORG_ID environment variables.

DEFAULT_PARAMS = {'api_uri': 'https://api.llm.ngc.nvidia.com/v1', 'beam_search_diversity_rate': 0.0, 'beam_width': 1, 'context_len': None, 'guardrail': None, 'length_penalty': 1, 'max_tokens': 150, 'repetition_penalty': 1.1, 'skip_seq_end': None, 'skip_seq_start': None, 'temperature': 0.9, 'top_k': 2, 'top_p': 1.0}
ENV_VAR = 'NGC_API_KEY'
ORG_ENV_VAR = 'ORG_ID'
generator_family_name = 'NeMo'
supports_multiple_generations = False