SEEM Skill#

Structured Episodic & Entity Memory for multi-turn conversations.

Quick Start#

python

from seem_skill import SEEMSkill, SEEMConfig, RecallMode

config = SEEMConfig()
skill = SEEMSkill(config)

# Store conversation
memory_id = skill.store({
    "text": "Lena asked about Scottish Terriers",
    "speaker": "Alice"
})

# Recall (default: LITE mode — facts + episodic memory, no raw chunks)
result = skill.recall({"text": "What did Lena ask?"}, top_k=3)
# result = {"memories": [...], "facts": [...]}

# Recall with raw chunks
result = skill.recall({"text": "What did Lena ask?"}, mode=RecallMode.PRO)

# Recall with backfill
result = skill.recall({"text": "What did Lena ask?"}, mode=RecallMode.MAX)

Recall Modes#

ModeFactsEpisodic MemoryRaw ChunksBackfillLite (default)✅✅ (summary + events)❌❌Pro✅✅✅ (top_k)❌Max✅✅✅ (top_k + backfill ≤ 2×top_k)✅

• Lite: Lightest context. Facts + structured memory only. Best for LLM agents that want concise context.
• Pro: Includes raw observation text for the top_k retrieved chunks.
• Max: Full context with backfill from associated memories (up to 2×top_k chunks).

Retrieval Strategies#

StrategyMethodBest ForDPRDense vector similaritySimple keyword-matching queriesHybrid RRFDense + BM25 sparse fusionMixed keyword + semantic queriesPPRPersonalized PageRank over knowledge graphMulti-hop, entity-rich queries

Default strategy is configured in config.py (currently ppr).

Configuration#

Environment Variables (Recommended)#

bash

export LLM_API_KEY="sk-xxx"
export LLM_BASE_URL="https://api.deepseek.com"
export LLM_MODEL="deepseek-chat"

export MM_ENCODER_API_KEY="sk-xxx"
export MM_ENCODER_BASE_URL="https://api.siliconflow.cn/v1"
export MM_ENCODER_MODEL="Qwen/Qwen3-Embedding-8B"

Unified Configuration File#

All default settings are centralized in seem_skill/config.py:

python

LLM_CONFIG = {
    "base_url": "https://api.deepseek.com",
    "model": "deepseek-chat",
}

EMBEDDING_CONFIG = {
    "base_url": "https://api.siliconflow.cn/v1",
    "model": "Qwen/Qwen3-Embedding-8B",
}

Custom Configuration#

Override defaults programmatically:

python

config = SEEMConfig(
    llm_api_key="your-key",
    llm_model="custom-model",
    retrieve_strategy=RetrieveStrategy.PPR,
    top_k_facts=10,
    ppr_damping=0.6,
)

Key Parameters#

ParameterDefaultDescriptionretrieve_strategy``hybrid_rrfDPR / Hybrid RRF / PPRtop_k_chunks3Number of chunks to retrievetop_k_facts5Number of fact triples to retrievetop_k_candidates3Integration candidate countrrf_rank_constant30RRF smoothing constantppr_damping0.5PPR teleport probabilitybackfill_chunks5Max additional chunks per backfillenable_fact_graphTrueBuild fact graph on storeentity_similarity_threshold0.9Entity linking thresholdenable_integrationTrueDynamic memory integrationintegration_window3Batch size for deferred integration

Operations#

Store#

bash

python scripts/cli_memory.py store --text "Your message" --speaker user
python scripts/cli_memory.py store --dialogue-id "D1:1" --speaker "Alice" --text "Message"

Recall#

bash

python scripts/cli_memory.py recall --query "Your query" --mode lite
python scripts/cli_memory.py recall --query "Your query" --mode pro --strategy ppr --top-k 5
python scripts/cli_memory.py recall --query "Your query" --mode max --top-k-facts 10

Facts (Knowledge Graph)#

bash

python scripts/cli_memory.py facts               # Show all fact triples
python scripts/cli_memory.py facts --entity 小米   # Filter by entity

Display (Detailed)#

bash

python scripts/cli_memory.py display
python scripts/cli_memory.py display --dialogue-id "D1:1"

View (Compact 5W1H)#

bash

python scripts/cli_memory.py view

Stats#

bash

python scripts/cli_memory.py stats

Clear#

bash

python scripts/cli_memory.py clear --yes

Features#

• Episodic Memory Extraction: LLM extracts structured summary + events (5W1H) from each turn
• Fact Graph Construction: Extracts subject-predicate-object triples, builds NetworkX knowledge graph
• Fact Deduplication: Two-stage dedup — normalized exact match (O(1)) + embedding similarity (threshold 0.93)
• PPR Retrieval: Personalized PageRank over entity-fact-chunk graph for graph-aware retrieval
• Three Recall Modes: Lite/Pro/Max controlling context granularity
• Dynamic Integration: Auto-merges related memories (MODERATE or STRONG coherence)
• Hybrid Retrieval: Dense (vector) + Sparse (BM25) with RRF fusion
• Entity Linking: Embedding-based entity normalization (threshold 0.9)
• Multimodal Support: Images participate in embedding and retrieval
• LRU Cache: Reduces repeated embedding computation
• NetworkX Graph: Full graph algorithms available (PPR, connected components, etc.)

Architecture#

Store Pipeline:

1. Chunk storage (raw observation)
2. Episodic extraction (LLM) → summary + events
3. Fact extraction from events → subject-predicate-object triples
4. Fact deduplication (exact match + embedding similarity)
5. Entity node creation and fact graph construction (NetworkX)
6. Multimodal embedding
7. Candidate retrieval (dense similarity)
8. Integration judgment (LLM, MODERATE or STRONG → integrate)
9. Memory merge/insert

Recall Pipeline:

1. Query encoding
2. Strategy routing (DPR / Hybrid RRF / PPR)
3. Chunk retrieval (strategy-specific, returns top_k chunks with scores)
4. Fact retrieval (vector similarity, returns top_k facts)
5. Result assembly (mode-dependent):
   • LITE: structured memory (summary + events) + facts
   • PRO: + raw chunks (top_k)
   • MAX: + backfill chunks (up to 2×top_k)

Graph Structure (NetworkX DiGraph):

• Node types: entity, chunk
• Edge types: entity_chunk (entity → chunk), fact (entity ↔ entity), synonymy (entity ↔ entity)
• Fact deduplication: normalized exact match + embedding similarity (threshold 0.93)

File Structure#

text

SEEM/
├── SKILL.md              # This file
├── README.md             # Quick reference
├── config.py             # Unified configuration (LLM + Embedding)
├── requirements.txt      # Python dependencies
├── __init__.py           # Package entry point
├── core/
│   ├── __init__.py
│   ├── seem_skill.py     # Core implementation (SEEMSkill class)
│   ├── schema.py         # Data structures (SEEMConfig, RecallMode, etc.)
│   ├── prompts.py        # LLM prompts
│   └── utils.py          # LLM client, embedding, BM25, cache
├── scripts/
│   └── cli_memory.py     # CLI: store, recall, facts, display, view, stats, clear
├── data/                 # Persistent storage (auto-created)
└── tests/

Dependencies#

• openai>=1.0.0 — LLM and embedding API client
• numpy>=1.21.0 — Vector operations
• networkx>=3.0 — Knowledge graph, PPR, connected components
• scipy>=1.0 — Required by nx.pagerank()
• rank-bm25>=0.2.2 — BM25 sparse retrieval
• nltk>=3.8.0 — Tokenization

When to Use SEEM#

• Multi-turn conversations need structured context preservation
• Complex event relationships exist across dialogue turns
• Need entity-centric retrieval (fact graph + PPR)
• Want control over context granularity (Lite/Pro/Max modes)
• Dynamic memory integration is valuable

Troubleshooting#

API Key Errors#

text

Error: Missing API keys

Set environment variables or update config.py:

bash

export LLM_API_KEY="sk-xxx"
export MM_ENCODER_API_KEY="sk-xxx"

PPR Requires scipy#

text

ModuleNotFoundError: No module named 'scipy'

bash

pip install scipy networkx