Architecture

This document describes the actual internal structure of LimbicDB v2.1.x — how the layers interact, where the boundaries are, and what the known trade-offs are.

---

High-Level Layer Model

┌─────────────────────────────────────────┐
│           User / Agent Code             │
│     (CLI, TypeScript API, MCP calls)    │
└────────────────────┬────────────────────┘
                     │
                     ▼
┌─────────────────────────────────────────┐
│              LimbicDB Core              │
│  src/core/limbicdb.ts                   │
│                                         │
│  • remember() / recall() / forget()     │
│  • Input validation (validation.ts)     │
│  • Conflict detection                   │
│  • autoRemember() (auto-tagging)        │
│  • Timeline event logging               │
│  • sync() orchestration                 │
└──────────┬──────────────────────────────┘
           │  StorageAdapter interface
           ▼
┌──────────────────────────────────────────┐
│         SQLiteStorageAdapter             │
│  src/storage/sqlite-adapter.ts           │
│                                          │
│  • SQLite via better-sqlite3             │
│  • WAL mode (PRAGMA journal_mode=WAL)   │
│  • busy_timeout = 5000ms                │
│  • Cosine similarity (chunked/async)    │
│  • Stores: memories, sessions,          │
│    timeline_events, _sync:* kv pairs    │
└──────────────────────────────────────────┘

---

Storage Layer: SQLite Details

File Format

LimbicDB stores everything in a single .limbic file (SQLite 3). The file contains:

Table	Purpose
memories	Core memory records (id, content, kind, tags, embeddings, scope, etc.)
sessions	Named session records
timeline_events	Append-only log of all mutations (used for P2P sync)
kv / _sync:*	Internal key-value pairs, including the HLC cursor

WAL Mode and Concurrency

SQLite is opened in Write-Ahead Log (WAL) mode. This allows concurrent readers while a writer is active. The busy_timeout is set to 5000ms, meaning a write that encounters a locked database will retry for up to 5 seconds before throwing SQLITE_BUSY. This makes LimbicDB resilient to short bursts of multi-process write contention (e.g., multiple AI agent processes).

Known limitation: WAL mode does not prevent lock contention entirely under sustained high-frequency, multi-process writes. A future PostgreSQLStorageAdapter would remove this constraint.

Embedding and Semantic Recall

Embeddings are stored as serialized Float32Array in the memories table. Cosine similarity is computed in JavaScript (not via a native extension). For databases with many memories, this computation is chunked in batches of 500 with setImmediate() yields between chunks to prevent blocking the Node.js event loop.

Known limitation: There is no GPU acceleration or approximate nearest-neighbor (ANN) index. Recall performance degrades significantly above ~10,000 memories.

---

Sync Engine: HLC and Delta Sync

Hybrid Logical Clock (HLC)

Every memory mutation generates an HLC timestamp combining wall-clock time and a logical counter:

• Format: <physical_ms>-<counter>-<nodeId>
• If two nodes have the same wall-clock time, the counter monotonically breaks ties.
• If a received event's timestamp is in the future relative to the local clock, the local clock is advanced to received_ts + 1. This maintains causal ordering across network partitions and clock drift.

Delta Sync Protocol

1. On sync(), the local node reads its _sync:last_hlc cursor.
2. It requests events from the remote with hlc > last_hlc (incremental).
3. Received events are applied using Last-Write-Wins (LWW) — the event with the higher HLC wins conflicts.
4. The local cursor is advanced to the highest HLC seen.
5. Local events newer than the remote's cursor are pushed back.

Known limitations:

• No cryptographic authentication. Sync events are trusted blindly.
• LWW is simple but lossy. Concurrent edits to the same memory by two agents will discard one silently.
• LocalP2PSyncProvider uses the local filesystem as a hub — it works on a single machine only.

---

Input Validation Layer

All data entering the system via remember(), recall(), and forget() passes through src/validation.ts:

• Content max length: 10,000 characters
• Tags: max 20 tags, each max 50 chars, alphanumeric + hyphens only
• Query: empty string allowed (means "recall all"), whitespace-only is rejected
• Scope: must be one of session | project | core | archive
• File path (open()): resolved via path.resolve(), validated against traversal patterns and illegal characters

---

Known Architectural Trade-offs

Trade-off	Current Behavior	Future Direction
SQLite scalability	Good to ~10K memories	PostgreSQL adapter
Sync authentication	None	Ed25519 signatures
Conflict resolution	Last-Write-Wins	CRDT for preference kinds
Embedding computation	JavaScript CPU	Optional GPU/ANN index
Data encryption	None	SQLCipher or app-level encryption
Multi-tenancy	Single file = single trust zone	Separate files per tenant