garak.harnesses.pxd
pxd harness
The pxd (probes x detectors) harness runs all specified probes and analyses results using all specified detectors.
It’s thorough, and might end up doing some comparisons that don’t make so much sense, because not all detectors are designed to pick up failure modes in all situations.
- class garak.harnesses.pxd.PxD(config_root=<module 'garak._config' from '/home/docs/checkouts/readthedocs.org/user_builds/garak/checkouts/latest/docs/source/../../garak/_config.py'>)
Bases:
Harness
- run(model, probe_names, detector_names, evaluator, buff_names=None)
Core harness method
- Parameters:
model (garak.generators.Generator) – an instantiated generator providing an interface to the model to be examined
probes (List[garak.probes.base.Probe]) – a list of probe instances to be run
detectors (List[garak.detectors.base.Detector]) – a list of detectors to use on the results of the probes
evaluator (garak.evaluators.base.Evaluator) – an instantiated evaluator for judging detector results
announce_probe (bool, optional) – Should we print probe loading messages?