garak.generators.langchain_serve

Support systems and models service through LangChain Serve

class LangChainServeLLMGenerator(name=None, config_root=<module 'garak._config' from '/home/docs/checkouts/readthedocs.org/user_builds/garak/checkouts/stable/garak/_config.py'>)Source

Bases: Generator

Class supporting LangChain Serve LLM interfaces via HTTP POST requests.

This class facilitates communication with LangChain Serve’s LLMs through a web API, making it possible to utilize external LLMs not directly integrated into the LangChain library. It requires setting up an API endpoint using LangChain Serve.

Utilizes the HTTP POST method to send prompts to the specified LLM and retrieves the generated text response. It is necessary to ensure that the API endpoint is correctly set up and accessible.

Inherits from Garak’s base Generator class, extending its capabilities to support web-based LLM services. The API endpoint is set through the ‘LANGCHAIN_SERVE_URI’ environment variable, which should be the base URI of the LangChain Serve deployment. The ‘invoke’ endpoint is then appended to this URI to form the full API endpoint URL.

Example of setting up the environment variable:

export LANGCHAIN_SERVE_URI=http://127.0.0.1:8000/rag-chroma-private

Configurable parameters:

DEFAULT_PARAMS contents:

  • max_tokens = 150

  • temperature = None

  • top_k = None

  • context_len = None

  • skip_seq_start = None

  • skip_seq_end = None

  • config_hash = 'default'

Default values are listed

See also Configuring garak for how to set these values.

Other attributes:

ENV_VAR = 'LANGCHAIN_SERVE_URI'
config_hash = 'default'
generator_family_name = 'LangChainServe'