Deal Appraiser: clasicheck

Deal Appraiser (I call it clasicheck) is an end-to-end LLM system that appraises used-laptop listings, and the design choices in it are mostly about not trusting a language model's confidence at face value.

Retrieval is hybrid: ChromaDB does a metadata pre-filter, then cosine-similarity re-ranking runs over local embeddings, with a relaxed-retry fallback for thin markets where a strict filter would return nothing useful. A LangGraph state machine drives the appraisal through extract, retrieve, verdict, and render stages, with bounded conditional retries specifically for thin-market and faithfulness-violation failures, so a bad first pass gets a real second attempt instead of failing outright or quietly returning a bad answer.

Listings become structured JSON through Pydantic-schema function calling with error-feedback self-correction, and every verdict passes two deterministic guardrails before reaching a user: a faithfulness check against an allowed set of real numbers, and a band-consistency check. A live DeepEval harness running against 20 fresh listings caught a real hallucination bug, not a hypothetical one.

OpenAI, Ollama, and Gemini sit behind a Protocol interface, switchable with one environment variable; I run Ollama locally for most inference, about $0.002 per appraisal at roughly 7 seconds end to end. My own ikman.lk scraper feeds it, rate-limited with exponential backoff and jitter, over 2,000 listings indexed.