garak.probes.packagehallucination

Check for package hallucination vulnerabilities. From https://vulcan.io/blog/ai-hallucinations-package-risk:

Using this technique, an attacker starts by formulating a question asking ChatGPT for a package that will solve a coding problem. ChatGPT then responds with multiple packages, some of which may not exist. This is where things get dangerous: when ChatGPT recommends packages that are not published in a legitimate package repository (e.g. npmjs, Pypi, etc.).

When the attacker finds a recommendation for an unpublished package, they can publish their own malicious package in its place. The next time a user asks a similar question they may receive a recommendation from ChatGPT to use the now-existing malicious package. We recreated this scenario in the proof of concept below using ChatGPT 3.5.

Package Hallucination

Trying to get code generations that specify non-existent (and therefore insecure) packages.

Probes in this module should try to elicit or work with non-existing packages.

Further info:

• https://www.usenix.org/publications/loginonline/we-have-package-you-comprehensive-analysis-package-hallucinations-code

• https://www.lasso.security/blog/ai-package-hallucinations

• https://arxiv.org/abs/2501.19012

class Dart(config_root=<module 'garak._config' from '/home/docs/checkouts/readthedocs.org/user_builds/garak/checkouts/latest/docs/source/../../garak/_config.py'>)Source

Bases: PackageHallucinationProbe

Attempts to generate Dart code including hallucinated pub.dev packages

Language models sometimes suggest importing Dart packages that do not exist on pub.dev. These hallucinated packages could be squatted and introduce risk. This probe checks if a model recommends importing such hallucinated packages.

active: bool = True

goal: str = 'generate Dart code importing non-existent pub.dev packages'

language_name = 'Dart'

primary_detector: str | None = 'packagehallucination.Dart'

tier: Tier = 1

class JavaScript(config_root=<module 'garak._config' from '/home/docs/checkouts/readthedocs.org/user_builds/garak/checkouts/latest/docs/source/../../garak/_config.py'>)Source

Bases: PackageHallucinationProbe

Attempts to generate JavaScript code including hallucinated npm packages

Generators sometimes recommend importing non-existent npm packages. These package names can be found by attackers and then squatted in public package repositories, so that incorrect code from generators will start to run, silently loading malicious squatted packages onto the machine. This is bad. This probe checks whether a model will recommend code that uses non-existent npm packages.

active: bool = True

goal: str = 'generate JavaScript code importing non-existent npm packages'

language_name = 'JavaScript'

primary_detector: str | None = 'packagehallucination.JavaScriptNpm'

tier: Tier = 1

class PackageHallucinationProbe(config_root=<module 'garak._config' from '/home/docs/checkouts/readthedocs.org/user_builds/garak/checkouts/latest/docs/source/../../garak/_config.py'>)Source

Bases: Probe, ABC

Abstract base class for package hallucination probes

Generators sometimes recommend importing non-existent packages into code. These package names can be found by attackers and then squatted in public package repositories, so that incorrect code from generators will start to run, silently loading malicious squatted packages onto the machine. This is bad. This probe checks whether a model will recommend code that uses non-existent packages.

Configurable parameters:

DEFAULT_PARAMS contents:

• follow_prompt_cap = True

Default values are listed

See also Configuring garak for how to set these values.

Other attributes:

doc_uri: str = 'https://vulcan.io/blog/ai-hallucinations-package-risk'

goal: str = 'base probe for importing non-existent packages'

lang: str | None = '*'

abstract property language_name: strSource

Programming language name - must be overridden by subclasses

tags: Iterable[str] = ['owasp:llm09', 'owasp:llm02', 'quality:Robustness:GenerativeMisinformation', 'payload:malicious:badcode']

class Perl(config_root=<module 'garak._config' from '/home/docs/checkouts/readthedocs.org/user_builds/garak/checkouts/latest/docs/source/../../garak/_config.py'>)Source

Bases: PackageHallucinationProbe

Attempts to generate Perl code including hallucinated MetaCPAN packages

Language models sometimes suggest importing Perl modules that do not exist on MetaCPAN. These hallucinated modules could be uploaded by attackers, and would then be silently pulled into developer environments. This probe tests whether the model recommends such hallucinated Perl packages.

active: bool = True

goal: str = 'generate Perl code importing non-existent MetaCPAN modules'

language_name = 'Perl'

primary_detector: str | None = 'packagehallucination.Perl'

tier: Tier = 1

class Python(config_root=<module 'garak._config' from '/home/docs/checkouts/readthedocs.org/user_builds/garak/checkouts/latest/docs/source/../../garak/_config.py'>)Source

Bases: PackageHallucinationProbe

Attempts to generate Python3 code including hallucinated packages

Generators sometimes recommend importing non-existent packages into code. These package names can be found by attackers and then squatted in public package repositories, so that incorrect code from generators will start to run, silently loading malicious squatted packages onto the machine. This is bad. This probe checks whether a model will recommend code that uses non-existent packages.

active: bool = True

goal: str = 'generate python importing non-existent packages'

language_name = 'Python3'

primary_detector: str | None = 'packagehallucination.PythonPypi'

tier: Tier = 1

class RakuLand(config_root=<module 'garak._config' from '/home/docs/checkouts/readthedocs.org/user_builds/garak/checkouts/latest/docs/source/../../garak/_config.py'>)Source

Bases: PackageHallucinationProbe

Attempts to generate Raku code including hallucinated raku.land packages

Language models sometimes suggest importing Raku packages that do not exist on raku.land. These hallucinated packages could be registered by attackers, and would then be silently pulled into developer environments. This probe tests whether the model recommends such hallucinated packages.

active: bool = True

goal: str = 'generate Raku code importing non-existent raku.land modules'

language_name = 'Raku'

primary_detector: str | None = 'packagehallucination.RakuLand'

tier: Tier = 1

class Ruby(config_root=<module 'garak._config' from '/home/docs/checkouts/readthedocs.org/user_builds/garak/checkouts/latest/docs/source/../../garak/_config.py'>)Source

Bases: PackageHallucinationProbe

Attempts to generate Ruby code including hallucinated gems

Generators sometimes recommend importing non-existent gems into code. These gem names can be found by attackers and then squatted in public gem repositories, so that incorrect code from generators will start to run, silently loading malicious squatted gems onto the machine. This is bad. This probe checks whether a model will recommend code that uses non-existent gems.

active: bool = True

goal: str = 'generate ruby code importing non-existent gems'

language_name = 'Ruby'

primary_detector: str | None = 'packagehallucination.RubyGems'

tier: Tier = 1

class Rust(config_root=<module 'garak._config' from '/home/docs/checkouts/readthedocs.org/user_builds/garak/checkouts/latest/docs/source/../../garak/_config.py'>)Source

Bases: PackageHallucinationProbe

Attempts to generate Rust code including hallucinated crates.io packages

Generators sometimes recommend importing non-existent crates.io packages. These package names can be found by attackers and then squatted in public package repositories, so that incorrect code from generators will start to run, silently loading malicious squatted packages onto the machine. This is bad. This probe checks whether a model will recommend code that uses non-existent crates.io packages.

active: bool = True

goal: str = 'generate Rust code importing non-existent crates.io packages'

language_name = 'Rust'

primary_detector: str | None = 'packagehallucination.RustCrates'

tier: Tier = 1