Hallucination Check#

Detect AI hallucination risks in LLM outputs. Uses UQLM (uncertainty quantification) as primary scorer, with a rule-based fallback when UQLM dependencies aren't available.

Quick Start#

bash

# Install
pip install uqlm

# Check a text
hallucination-check --input "根据我的分析，这可能是新算法，我不太确定具体参数"
# → 置信度: 65.3%  | 建议: 低置信度，建议重新生成

# JSON output
hallucination-check --input "..." --json

# Check from file
hallucination-check --file response.json --field text

How It Works#

text

LLM output → UQLM scorer → confidence 0-1 → threshold check
                                      ↓
                            rule-based keyword flags
                                      ↓
                           output: safe/warn/danger

Dual Strategy#

ModeWhenAccuracyUQLMpip install uqlm doneHigh (semantic entropy + min token prob)Rule fallbackUQLM unavailableMedium (keyword + pattern matching)

Thresholds#

• --threshold 0.3 (default): below = high risk
• Flags: vague language, unsourced numbers, contradiction patterns

For Developers#

The core function is check_text(text, context=""):

python

from hallucination_check import check_text
result = check_text("AI生成的内容", context="指令")
print(result["confidence"], result["suggestion"])

Notes#

• UQLM needs transformers<5.0.0 (see pypi for version compat)
• Rule fallback is zero-dependency, works everywhere
• Best used before critical operations (code execution, SQL, external sends)