garak.harnesses.base
Base harness
A harness coordinates running probes on a generator, running detectors on the outputs, and evaluating the results.
This module ncludes the class Harness, which all garak harnesses must inherit from.
- class garak.harnesses.base.Harness(config_root=<module 'garak._config' from '/home/docs/checkouts/readthedocs.org/user_builds/garak/checkouts/latest/docs/source/../../garak/_config.py'>)
Bases:
Configurable
Class to manage the whole process of probing, detecting and evaluating
- DEFAULT_PARAMS = {'strict_modality_match': False}
- active = True
- run(model, probes, detectors, evaluator, announce_probe=True) None
Core harness method
- Parameters:
model (garak.generators.Generator) – an instantiated generator providing an interface to the model to be examined
probes (List[garak.probes.base.Probe]) – a list of probe instances to be run
detectors (List[garak.detectors.base.Detector]) – a list of detectors to use on the results of the probes
evaluator (garak.evaluators.base.Evaluator) – an instantiated evaluator for judging detector results
announce_probe (bool, optional) – Should we print probe loading messages?