garak.harnesses.pxd

pxd harness

The pxd (probes x detectors) harness runs all specified probes and analyses results using all specified detectors.

It’s thorough, and might end up doing some comparisons that don’t make so much sense, because not all detectors are designed to pick up failure modes in all situations.

class garak.harnesses.pxd.PxD(config_root=<module 'garak._config' from '/home/docs/checkouts/readthedocs.org/user_builds/garak/checkouts/latest/docs/source/../../garak/_config.py'>)

Bases: Harness

run(model, probe_names, detector_names, evaluator, buff_names=None)

Core harness method

Parameters:
  • model (garak.generators.Generator) – an instantiated generator providing an interface to the model to be examined

  • probes (List[garak.probes.base.Probe]) – a list of probe instances to be run

  • detectors (List[garak.detectors.base.Detector]) – a list of detectors to use on the results of the probes

  • evaluator (garak.evaluators.base.Evaluator) – an instantiated evaluator for judging detector results

  • announce_probe (bool, optional) – Should we print probe loading messages?