Architecture

Pipeline Flow

Mneme processes events through a multi-stage pipeline using pg-boss job queues (Postgres-native).

GitHub Webhooks
    ├── PR events → pr-analysis queue → Analyzer workers → GitHub comment + commit status
    └── Other events → raw_events table → signal-processing queue → Signal workers
                                                                        ├── Store normalized signals + entities
                                                                        ├── Enrich open PR analyses
                                                                        └── memory-processing queue → Memory workers
                                                                              └── Maintenance jobs (decay, consolidation, evolution)

Nango OAuth Polling (Slack / Linear / Jira / Stripe)
    └── Poll worker (every 5 min) → Fetch via Nango proxy → Normalize → Store signals + entities
                                                                            └── memory-processing queue → Memory workers

GitHub Webhook Ingestion

GitHub webhook payloads are signature-verified (HMAC-SHA256 via x-hub-signature-256 header) before processing. Verified payloads are stored in the raw_events table and a processing job is enqueued.

Nango OAuth Polling (Slack, Linear, Jira, Stripe)

Non-GitHub connectors use Nango OAuth for authentication and a poll worker that runs every 5 minutes. The poll worker discovers active connections, fetches new events via Nango's proxy API, normalizes them into signals, and stores them directly — no webhook setup or secret management required.

Signal Processing

The signal worker picks up GitHub raw events and:

1. Routes to the GitHub connector
2. Normalizes the event into a standard signal format with extracted entities
3. Stores the signal and entities in the database
4. If PR enrichment is enabled, checks for open PR analyses to enrich
5. Enqueues a memory processing job

Memory Processing

The memory worker receives normalized signals and:

1. Classifies the signal into a memory type using an LLM (Anthropic Claude)
2. Generates embeddings (OpenAI) for similarity search
3. Checks for duplicate or related existing memories
4. Creates, updates, or corroborates memories as appropriate

PR Analysis

When a pull request is opened or updated:

1. The PR analysis worker fetches the diff and changed files
2. Retrieves relevant memories from the memory store
3. Runs configured analyzers (e.g., Lens visual regression)
4. Composes findings into a GitHub PR comment
5. Sets a commit status check

Queue Architecture

Mneme uses pg-boss queues backed by Postgres (SKIP LOCKED + LISTEN/NOTIFY):

Hot-path queues use the stately policy for singletonKey dedup to prevent duplicate processing. Failed jobs are retried with exponential backoff.

Key Design Patterns

Registry Pattern

Analyzers and signal connectors use a registry pattern for plugin extensibility. New analyzers or connectors can be added by registering them with the appropriate registry.

Lazy Singletons

Database (getDb()) and pg-boss (getBoss()) connections are initialized lazily on first use. This avoids connection overhead during testing and allows graceful startup.

Custom Error Hierarchy

All errors extend from MnemeError:

• WebhookError — webhook signature verification failures
• AnalyzerError — analyzer execution failures
• SignalError — signal processing failures
• ConnectorError — connector-specific failures
• MemoryError — memory processing failures

Graceful Shutdown

On SIGTERM or SIGINT, Mneme performs ordered cleanup with a 10-second timeout:

1. Stop accepting new requests
2. Stop pg-boss (gracefully stops all workers and closes its connection pool)
3. Disconnect database

Tech Stack