SUBSYSTEM 01 • CLIENT SURFACE & INGESTION
AUTHENTICATED
Client Surface & API Gateway (Web / Mobile / Slack / REST)
TLS termination, rate limiting, and user session authorization
↓ [QUERY REWRITE & POLICY CHECK]
SUBSYSTEM 02 • ORCHESTRATION & GUARDRAILS
CORE PIPELINE
LangGraph / AutoGen Core
Deterministic graph execution, policy guardrails & context assembling
↓ [HYBRID VECTOR RETRIEVAL] ↓ [EPISODIC AGENT MEMORY]
SUBSYSTEM 03 • VECTOR RETRIEVAL
Qdrant / Pinecone Index
HNSW dense vector index, sparse BM25 & semantic reranking
SUBSYSTEM 04 • AGENT MEMORY
Mem0 / Redis Cache
Long-term episodic facts, user entity profiles & semantic cache
↓ [AUGMENTED PROMPT & CONTINUOUS BATCHING]
SUBSYSTEM 05 • INFERENCE CLUSTER
LLM BACKEND
vLLM Engine + DeepSeek-R1 / Claude 3.7 API (RunPod GPU Cloud)
High-throughput tensor parallel serving & low-latency streaming