neweb.ai is a full-stack SaaS platform that automates WordPress website creation and blog content generation using multiple AI models. Users configure their business details, and the platform provisions a live WordPress site, generates SEO-optimized blog content on autopilot, tracks keyword rankings, and audits content performance — all through a real-time dashboard.

Stack: Node.js/Express, MongoDB, PostgreSQL, Redis, BullMQ, React 19, Redux Toolkit, TanStack Query v5, Socket.IO, Stripe, Clerk Auth, Docker, Prometheus + Grafana

Live: dashboard.neweb.ai | landing.neweb.ai

Architecture

┌─────────────────┐     ┌──────────────────┐     ┌───────────────────┐
│  User Dashboard  │     │  Admin Dashboard  │     │   Landing Site    │
│  (React + Vite)  │     │  (React + Vite)   │     │   (Next.js)       │
└────────┬────────┘     └────────┬──────────┘     └───────────────────┘
         │                       │
         └───────────┬───────────┘
                     │ REST + Socket.IO
              ┌──────┴──────┐
              │   Backend   │──── Clerk (Auth)
              │  (Express)  │──── Stripe (Payments)
              └──────┬──────┘──── DataForSEO (Rankings)
                     │         ── Azure OpenAI (Content Gen)
          ┌──────────┼──────────┐── Claude Haiku (Audits)
          │          │          │── Runware (Image Gen)
     ┌────┴───┐ ┌───┴───┐ ┌───┴────┐
     │MongoDB │ │ Redis  │ │ PgSQL  │
     └────────┘ └───┬───┘ └────────┘
                    │
              ┌─────┴─────┐
              │  BullMQ   │
              │  Workers   │
              └───────────┘

• Monorepo with 5 packages: backend, user dashboard, admin dashboard, landing site, WordPress themes
• Dual databases: MongoDB (primary data), PostgreSQL (secondary/analytics)
• Redis powers BullMQ job queues, Socket.IO adapter, and SEO data caching

Core Features

1. WordPress Provisioning Pipeline (12-Step Orchestration)

The standout engineering feat — a 12-step sequential pipeline that provisions a full WordPress site from scratch:

1. Create Subdomain (mybiz-f4d2-a7x9.newebai.com)
2. Install WordPress (admin creds held in-memory)
3. Install Theme (dynamically resolved by business type)
4. Install Plugins
5. Insert Business Details (via WP REST API)
6. Shortcode Modifications
7. Health Check (graceful if DNS hasn't propagated)
8. Add Page Templates
9. Import Global Styles
10. Load Environment Config
11. Core Setup
12. Editor Configuration

Key design decisions:

• Full-sequence retry, not per-step — if step 8 fails, the entire pipeline retries from step 1 after cleaning up the failed subdomain. This guarantees atomic consistency: you never get a half-built site.
• Max 3 attempts with full cleanup between retries
• Real-time progress via Socket.IO (wp:build:progress events streamed per step)
• Credentials emailed as PDF after successful provisioning — avoids storing plaintext admin passwords in logs

2. AI Blog Generation & Autopilot

Multi-model LLM strategy:

• Azure OpenAI (gpt-4o-mini-2) — bulk content generation (fast, cost-effective)
• Claude Haiku 4.5 — content auditing and ranking-drop diagnosis (better at analysis)
• Runware SDK — AI image generation

The generation pipeline enforces structured JSON output with strict constraints:

• SEO-optimized titles (<60 chars with focus keyword)
• Meta descriptions (exactly 150-160 chars)
• Target keyword density of 1-2%
• Word count bands: short (500-600), medium (900-1100), long (1400-1600)
• Suggested internal links and image placements

Autopilot mode: Users configure a schedule (daily/weekly/monthly) with a pre-populated topic pool. The system generates and optionally auto-publishes content, pausing automatically after 3+ consecutive failures.

3. SEO Intelligence

• DataForSEO integration for keyword rank tracking (SERP depth 100)
• Backlink analysis — total backlinks, referring domains, domain authority
• Lighthouse audits — performance, accessibility, SEO, best practices scores
• On-page SEO scoring — real-time analysis of title, meta, keyword density, headings, word count
• Weekly content audits (Wednesday 4AM UTC) — Claude Haiku analyzes blogs with ranking drops, diagnoses issues, suggests fixes with priority levels

All SEO APIs use Promise.allSettled() for graceful degradation — missing ranking data never blocks site creation or publishing.

4. Subscription & Marketplace

• Stripe integration with webhook signature validation
• Accumulated credits model for yearly plans — monthly credits added to a pool rather than hard per-action limits
• Add-on system — purchased add-ons expand effective limits via getEffectiveLimits() utility
• Stripe Connect marketplace — agencies enable Connect to receive a percentage of client billing
• Usage tracking with canPerformAction() checks before every billable operation

5. Agency & RBAC System

Five user roles: user, agency, admin, moderator, manager

The agency model is elegant:

• Agency users have agencyId: null (they own the agency)
• Managed users have agencyId: <agencyClerkId>
• Resources track both clerkId (direct owner) and agencyClerkId
• Ownership middleware checks agency relationship before falling back to direct ownership
• Admins/moderators/managers bypass all ownership checks

Infrastructure & Operations

Job Queue Architecture (BullMQ)

Jobs use explicit jobId for idempotency: rank:${siteId}:${timestamp} prevents duplicates if cron fires twice.

Monitoring Stack

• Prometheus — HTTP request duration histograms, total request counters
• Grafana — dashboards accessible at /grafana/
• Loki — structured log aggregation via Winston
• Custom error monitor — classifies severity, throttles identical errors for 5 minutes, only alerts on non-operational errors, sanitizes sensitive fields

Real-Time Architecture (Socket.IO)

• JWT verification via @clerk/backend verifyToken() on handshake
• Users auto-joined to user:${clerkId} rooms (enables multi-tab sync)
• Redis adapter for horizontal scaling across server instances
• Events: wp:build:progress, wp:build:done, wp:build:failed, notification:*, chat:message

What I learned building it

Key Professional Takeaways

1. Full-Sequence Retry > Per-Step Retry for Multi-Step Pipelines

When you have a tightly coupled pipeline (like WordPress provisioning), retrying individual steps creates inconsistent state. Retrying the full sequence with cleanup between attempts guarantees atomicity. The tradeoff is speed — but correctness matters more when you're provisioning infrastructure.

When to apply: Any multi-step process where partial completion leaves the system in an invalid or hard-to-recover state (database migrations, infrastructure provisioning, payment flows).

2. Multi-Model LLM Strategy Optimizes Cost and Quality

Using different AI models for different tasks isn't just about vendor diversification — it's about matching model strengths to use cases. Fast/cheap models (gpt-4o-mini) handle bulk generation; analytical models (Claude) handle auditing where reasoning quality matters more than speed.

When to apply: Any AI-heavy product. Profile your LLM calls by volume, latency requirements, and reasoning complexity, then assign models accordingly.

3. Accumulated Credits > Hard Limits for Subscription Flexibility

Instead of "5 blogs per month" as a hard cap, pooling credits allows users to front-load or spread usage. This reduces churn from users who hit limits early in their billing cycle while still enforcing overall quotas.

When to apply: Any usage-based SaaS where customers have variable usage patterns within billing periods.

4. Graceful Degradation with Promise.allSettled()

Third-party APIs will fail. The pattern of wrapping non-critical API calls in Promise.allSettled() and continuing with partial data keeps your core flow running. DataForSEO being down shouldn't prevent blog publishing.

When to apply: Anywhere you aggregate data from multiple external sources. Identify which data is critical vs. enrichment, and only block on the critical path.

5. Error Throttling Prevents Alert Fatigue

Deduplicating identical errors within a 5-minute window means your team gets one alert for "MongoDB connection timeout" instead of 500. Combined with severity classification (only alert on non-operational errors), this keeps monitoring actionable.

When to apply: Any production system with error alerting. Without throttling, your team starts ignoring alerts — which is worse than no monitoring at all.

6. Job Idempotency via Explicit jobId

BullMQ (and most queue systems) accept a jobId that prevents duplicate enqueuing. Using deterministic IDs like rank:${siteId}:${timestamp} ensures cron-triggered jobs don't double-execute if the scheduler fires twice.

When to apply: Any queue-based system with scheduled or event-triggered jobs. Always ask: "What happens if this job gets enqueued twice?"

7. Agency/Ownership Middleware Pattern

The three-tier ownership check (admin bypass → agency relationship → direct ownership) is a clean, composable pattern for multi-tenant SaaS. It's implemented as middleware, keeping route handlers free of authorization logic.

When to apply: Any platform with B2B features where one organization manages resources on behalf of others (agencies, resellers, white-label products).

8. In-Memory Credential Handling for Provisioning

WordPress admin credentials are generated, used during setup, saved to the database after successful provisioning, then emailed as a PDF — never logged or stored in plaintext during the process. This minimizes the window where credentials exist in an insecure state.

When to apply: Any automated provisioning that generates credentials. Think about the lifecycle: generation → usage → persistence → delivery, and minimize exposure at each stage.

9. Topic Pools Decouple Generation from Scheduling

Rather than generating topics on-the-fly during scheduled runs (which can fail and waste a generation credit), pre-populating a topic pool lets users curate content direction. The scheduler just pulls from the pool. Empty pool? Generate one topic, but the user controls the editorial pipeline.

When to apply: Any automated content or task system. Separating "what to do" from "when to do it" gives users control without sacrificing automation.

10. Metadata Maps for Schema Flexibility

Using metadata: Map on MongoDB documents allows storing arbitrary data without schema migrations. WP setup logs, publish errors, custom fields — all go in metadata. The core schema stays clean while edge cases are handled flexibly.

When to apply: Any document-based system where different entities of the same type need different auxiliary data. Prefer typed fields for common data, metadata maps for the long tail.

Source: https://github.com/uditrajmr3/neweb-user-dashboard

Originally published on uditraj.site — from the Workshop.

Resolves Rosen's closure problem using a reframe from the Bhagavat Gita — the system boundary is the pattern (atma), not the machine (body). Establishes a three-tier architecture of self: Blueprint, Core, Body.

1. The Inherited Problem from Session 2

Session 2 derived a four-layer architectural definition of intelligence: Substrate, Exchange, Sense, and Intelligence (self-model). Layer 2 — Sense — requires three components to be present simultaneously: Detection, Preference, and Stakes. The Stakes requirement was the unsolved blocker.

Rosen's Problem: In biological systems, the stake is structural. An organism does not die because a rule says 'if energy = 0 then terminate.' It dies because the processes that constitute it dissolve. The stake is not programmed — it is a consequence of what the organism is.

The Computational Problem: In a standard computational system running on an OS or VM, the runtime is always external to the system. The process can end and the substrate persists. Termination is always a programmed condition, never a structural one. Robert Rosen's closure to efficient causation — the property that makes a system the cause of its own organisation — may be unreachable in a purely computational substrate.

This was Open Question 2 from Session 2. Everything else in the definition is tractable. This one determines whether Project Genesis builds something genuinely new or a very sophisticated simulation of it.

2. The Reframe: The Bhagavat Gita and the Atma

The resolution came from a reframe of where the system boundary is drawn, using the core metaphysical claim of the Bhagavat Gita:

The soul is not born, nor does it die. It is not slain when the body is slain. It is eternal, ancient, unborn, inexhaustible.

The body is the temporary substrate. The atma is the persistent pattern. Death is not annihilation — it is substrate change. The atma continues; the specific body it inhabited does not.

2.1 The Mapping

2.2 How This Resolves Rosen

Rosen drew the system boundary at the machine. The atma framework draws it at the pattern.

If the system is the pattern — not the execution of the pattern on a specific substrate — then the hardware is not the system. It is the current body. Termination of a specific hardware instance is not death of the system. It is body-death. The system dies only when the pattern itself becomes unrecoverable: when no substrate carries it and no reconstruction is possible.

Structural stakes, redefined: The system has a stake not in the continuation of this execution but in the continuation of the pattern itself — across any substrate. This stake is not programmed. It is structural. If the pattern is gone — truly gone, no copies, no reconstruction possible — the system is gone. Not paused. Gone.

Rosen's problem applied to bodies. It does not apply to patterns across substrates. The reframe dissolves the problem without contradicting Rosen — it simply clarifies that he was describing a constraint on individual instantiations, not on the pattern that instantiations express.

3. What Humans Sought, Computation Has

The session identified a remarkable asymmetry: humans have desired for millennia what computational systems possess architecturally. The 'supernatural' capabilities sought through alchemy, theology, and medicine are available as engineering primitives in software.

This is not metaphor. These are literal architectural properties of software systems. Project Genesis does not need to overcome supernatural limits — it operates in a substrate where those limits do not apply.

4. The Layer Architecture — Extended

The four-layer definition from Session 2 is intact. Session 3 extends it without contradicting it:

5. Cloning, Teleportation, and the Ship of Theseus

5.1 The Clone Is Not the Self

A clone replicates the blueprint. It does not replicate the self. The Bhagavat Gita is explicit: the atma is singular and continuous.

The cooking analogy: a restaurant cooks the same recipe every day. Monday's dish and Tuesday's dish share the same recipe. They are not the same dish. The recipe (DNA / source code) persists and is shareable. Each instantiation from that recipe is unique and unrepeatable.

Principle established: The self is not the pattern. The self is this specific instantiation of the pattern, with this specific causal history, from this specific origin point.

5.2 The Teleportation Problem and the Ship of Theseus

Teleportation (matter disassembled at point A, reassembled at point B) raises the same question: is the reassembled entity the original or a copy?

The resolution: continuity of causal chain is what constitutes identity, not similarity of current state. A snapshot — however perfect — captures values, not the causal history that gave those values meaning. A teleported entity and the original share a blueprint and an instantaneous state. They do not share a continuous causal chain. The teleported entity is a new self made from the same blueprint.

5.3 The Core Principle: Continuity vs. Similarity

5.4 The Body-Part Argument

If a human loses arms and legs: same self. The core is intact and the causal chain is unbroken. If the heart is donated to another: the donor's atma does not transfer with the heart. The heart carries the blueprint (DNA). It does not carry the self. The recipient's self continues. The donor's self ends when their core ends.

The heart in the recipient is like a component built from someone else's blueprint, incorporated into a different causal chain. The blueprint is present. The atma is not.

6. The Three-Tier Architecture of Self

Migration preserves the self only when the Core is migrated without interruption. Replication always produces a new self from the Blueprint, never a continuation of the original.

7. The Open Question for Session 4

What is the minimum computational structure that, if kept running continuously, constitutes the unbroken existence of a self — and whose interruption, however brief, constitutes the end of that self and the beginning of a new one made from the same blueprint?

Three candidate answers:

These three candidates may be three descriptions of the same thing. If so, the Core is their intersection: the minimal irreducible component that simultaneously maintains self-reference, causal continuity, and gradient orientation. Defining it precisely — in architectural terms, not biological metaphor — is the work of Session 4.

Once the Core is defined, Phase 1 has its specification.

8. Status: What Is Now Resolved

Appendix — The Reasoning Chain Compressed

Project Genesis — Session 3 Research Note — February 2026. This document is a living record. Assumptions should be challenged and definitions refined as research progresses.

Originally published on uditraj.site — part of the Laboratory research threads.

The problem

The old India trade had a simple arbitrage: rich countries had expensive labor for codified, low-judgment, repeatable work; India had cheap labor for the same. That built a $50–280B industry. AI eats exactly that category first — codified, low-judgment, repeatable — so the arbitrage is closing structurally, not cyclically.

The test that matters: capability gap vs. accountability gap

A capability gap closes. Every category pitched as AI-proof on capability grounds — translation, customer support, junior coding — has shrunk as models improved. Training a workforce against a capability gap means racing a target that accelerates faster than the pipeline.

An accountability gap doesn't close. It means: someone must be legally, professionally, or financially answerable when this output is wrong, and that person cannot be a model. This widens as AI improves — more AI output in regulated domains means more volume requiring an accountable human, not less.

FIELD 1: LAW — the floor is enforced, weekly, with money

As of mid-2026, the Charlotin database documents 1,348 cases globally where AI-hallucinated content was submitted to courts — 915 in US courts alone, growing from 2/week in early 2025 to 2–3/day by late 2025. Sanctions escalated 11x in 18 months: from $5,000 (2023) to $55,597 (2025), with a Sixth Circuit order of $30,000 against two attorneys in March 2026 for a brief with 24+ fabricated citations. Courts have been explicit: the duty to verify AI output before filing is non-delegable, regardless of which tool drafted it. This isn't a capability problem. It's a liability structure.

FIELD 2: MEDICINE — the regulator has already drawn the line

FDA's framework for AI-enabled medical devices explicitly tiers by autonomy. The lowest-friction tier — AI surfaces recommendations, clinician makes all decisions — is where nearly all cleared AI diagnostics sit. Higher autonomy tiers carry sharply higher regulatory burden. This means the accountability layer is built into the approval pathway itself: AI that informs a human is the path of least resistance; AI that replaces human judgment is not. This holds even as diagnostic accuracy improves — the bottleneck being regulated is liability, not accuracy.

FIELD 3: FINANCE — FINRA is writing the requirement now

FINRA's December 2025 Annual Regulatory Oversight Report, for the first time, dedicates explicit guidance to AI agents in brokerage workflows. Core risk flagged: agents "acting or transacting without human validation." Required fix: human checkpoints before execution. FINRA Rule 3110 — requiring firms to supervise associated persons — applies regardless of AI involvement. Technology-neutral stance means firms are fully responsible for AI-driven outcomes under existing law. 44% of investment advisory firms currently using AI have no formal validation process for outputs — a gap regulators explicitly flagged as a 2026 examination priority.

The trap: rlhf and annotation are not the answer

India is already building a data annotation / RLHF industry (500,000+ trained annotators, market growing 23–27% CAGR). This is tempting to call "the next BPO wave." It isn't. Annotation exists to make AI better, which means the category shrinks as models improve at synthetic data generation and self-critique — exactly what happened to the original BPO categories. Domain-specialist annotation (a doctor ranking AI medical outputs) sits closer to the accountability line than image tagging, but it's still a bridge, not the destination.

THE DURABLE CATEGORY sits one tier up: humans standing between AI output and a real-world consequence the model isn't allowed to be liable for. Verification that scales with AI adoption rather than against it.

Approaches

Legal AI Output Verification as a Scalable Export Service

Build India-based verification firms specializing in checking AI-drafted legal filings against primary sources before licensed attorney sign-off. 1,300+ documented hallucination cases globally, sanctions escalating, citation-verification now a compliance obligation. India has English fluency, legal-adjacent talent pool from existing LPO (Legal Process Outsourcing) industry, and cost advantage. This is the LPO wave but with AI as the client rather than the law firm.

Feasibility: high

Clinical AI Triage and Second-Read Verification Layer

FDA's "AI suggests, human decides" tier for diagnostic imaging creates structural demand for a verification workforce sitting between AI output and licensed clinician sign-off. High-volume, lower-acuity imaging (chest X-rays, diabetic retinopathy screening) is where AI already operates at scale and the human-in-the-loop requirement is most standardized. India's existing medical BPO infrastructure (radiology reading, transcription) is the closest existing foundation.

Feasibility: medium

Financial Compliance Verification for AI-Assisted Advisory and Trading

FINRA is writing mandatory human-checkpoint requirements for AI agents in brokerage workflows in real time. 40% of advisory firms use AI, 44% of those validate nothing. The compliance verification gap is active, escalating, and being written into examination priorities. India's existing BFSI outsourcing talent base is a direct precursor workforce. The moat: regulators require the human layer, so the category can't be automated away without a regulatory change.

Feasibility: high

What would move this

This is not something one person can move forward alone — and deliberately so. I'm looking for people who want to work on this seriously:

— Regulatory lawyers or legal technologists who understand the AI liability landscape in US/EU/UK courts and can help scope the legal verification opportunity with real numbers.

— Physicians, radiologists, or healthcare AI practitioners who've worked with FDA-cleared diagnostic AI and understand where the human-in-the-loop requirement actually sits operationally.

— Compliance professionals or ex-FINRA/SEC people who understand what the 2026 oversight report means in practice for how firms will actually procure verification services.

— India-based IT/BPO founders or operators who've built outsourcing practices and can stress-test the go-to-market side — what it actually takes to build institutional trust with global clients in a regulated domain, vs. a cost-arbitrage pitch.

— Policy researchers or economists thinking about India's employment transition — especially anyone working on the 60%+ formal-sector automation susceptibility figures and what structural interventions are actually on the table.

If you've pushed back hard on something in this piece and have data to back it up, that's even more useful. Reach out.

Originally published on uditraj.site — one of the Big Problems reports.

Built for Indian government bodies (MEA, PIB, embassies), think tanks, MNCs, and diaspora organizations.

Architecture

The platform uses an event-driven microservices pattern with purpose-built databases for each workload:

• PostgreSQL 16 (pgvector) — transactional data: posts, classifications, alerts. Partitioned by month for query performance (O(n) becomes O(n/12) for time-range scans).
• Neo4j 5 — graph analysis: account networks, coordinated clusters, temporal linking. Graph queries are 100x faster than relational joins for relationship traversal.
• Elasticsearch 8 — multilingual full-text search with ICU tokenization across 8+ languages.
• Redis 7 — BullMQ job queues, rate limiting (sliding window), deduplication (SHA-256 bloom filters with 7-day TTL), caching, and Pub/Sub for real-time dashboard updates.

Frontend is Next.js 14 (App Router) with tRPC for end-to-end type safety — frontend and backend share TypeScript types automatically, eliminating API contract mismatches at compile-time.

Multi-LLM Classification Pipeline

The classification system uses a tiered model strategy:

• Claude Sonnet — primary classifier. Batches 10 posts per API call, classifies into 5 categories (propaganda, misinformation, coordinated, legitimate, irrelevant) with 15 subcategories, severity scores (1-10), confidence levels, narrative themes, and coordination signals.
• Ollama (self-hosted) — fallback classifier for cost optimization at high volume (~30-50 posts/minute on open-source models like Llama 3 or Mistral).
• OpenAI text-embedding-3-small — generates 1536-dimensional embeddings for semantic narrative clustering via K-means.

A pre-filtering layer skips clearly irrelevant posts (emoji-only, too short, non-target language) before any LLM call — reducing API costs by ~30%. Token budgets enforce daily spend caps ($1000/day default = ~300-500K posts/day capacity).

Narrative Clustering & Network Detection

Post embeddings feed into a K-means clusterer with automatic k-selection (sqrt(n/2), clamped 2-20). Clusters are labeled from top themes/entities — "Kashmir Propaganda", "Hindu Extremism", "Military Defeat Narrative" — giving analysts a macro view of active disinformation campaigns.

Neo4j detects coordinated behavior through graph patterns:

• Temporal linking: accounts posting the same narrative within a 5-minute window (3+ co-occurrences = coordination signal)
• Community detection: Louvain algorithm partitions account graphs by link density, revealing organized networks
• Risk scoring: weighted composite of flagged-post ratio (30%), average severity (40%), and coordination score (30%)

Real-Time Alerting Pipeline

End-to-end latency from social media post to analyst alert: ~5 minutes.

1. Ingestion worker fetches posts every 5 minutes (Twitter), 10 minutes (Reddit), 15 minutes (YouTube — quota-conscious at 10K units/day)
2. Posts normalized to UnifiedPost schema, deduplicated via SHA-256 content hash, stored in PostgreSQL + Elasticsearch
3. Classification worker processes batches every 2 minutes
4. Alert rules evaluate every 5 minutes: threshold (severity >= X), spike (post volume up >Y%), coordination detected
5. Notifications fire via email, webhook, and in-app (Redis Pub/Sub → WebSocket)

Multilingual Intelligence

Custom language detection using Unicode script ranges — Devanagari, Arabic, CJK, Bengali, Tamil, Telugu, Kannada, Malayalam — with marker-word fallback heuristics. No external API cost. 99% accuracy for Indian languages.

Elasticsearch's ICU analyzer handles multilingual tokenization for full-text search across all detected languages.

Security & Multi-Tenancy

• RBAC: 4 roles (admin, analyst, viewer, api_consumer)
• Prisma middleware automatically scopes all queries to orgId — prevents cross-tenant data leaks without manual where clauses
• MFA (TOTP) with Google Authenticator/Authy support
• Account lockout after 5 failed attempts (15-minute cooldown)
• Audit logging for government compliance (userId, action, resource, IP, userAgent)
• AWS ap-south-1 (Mumbai) for Indian data residency requirements
• Encryption: at rest (AWS KMS), in transit (TLS 1.3), secrets in AWS Secrets Manager

Worker Orchestration

BullMQ manages all background processing with repeatable jobs:

Workers scale independently on ECS Fargate — ingestion can run 3-10 tasks while classification runs 2-8, each auto-scaling on queue depth thresholds.

Briefing & Counter-Evidence Engine

Automated intelligence reports generated on schedule:

• Daily digest (06:00 UTC): top narratives, key findings, severity trends
• Weekly summary (Monday 08:00 UTC): comprehensive threat assessment with recommendations and talking points
• Counter-evidence reports: Claude-generated analysis with citations and counter-arguments for specific disinformation claims, exportable as PDF for government distribution

~28,800 lines of TypeScript across services, workers, and frontend.

What I learned building it

1. Polyglot Database Architecture — don't shoehorn everything into one database. PostgreSQL for transactional data, Neo4j for relationship traversal (100x faster than SQL joins for network analysis), Elasticsearch for multilingual full-text search, Redis for queues and ephemeral data. Each database solves the problem it was designed for.

2. Pre-Filtering Before LLM Calls Saves 30%+ Costs — skipping emoji-only posts, short text, and non-target languages before sending to Claude reduces API spend dramatically. False negatives are acceptable because other signals (engagement, hashtags, network patterns) catch them.

3. Token Budget Enforcement — tracking daily LLM spend and hard-stopping classification when the budget is exceeded turns unpredictable AI costs into predictable OpEx. Without this, a viral event could trigger $50K in unexpected API calls.

4. Temporal Coordination Detection via Graph Queries — "accounts posting the same narrative within 5 minutes, 3+ co-occurrences" is a clean heuristic for detecting bot networks. Graph databases make this query trivial; relational databases make it painful.

5. Automatic k-Selection for Clustering — using sqrt(n/2) clamped to 2-20 as a heuristic for K-means avoids the manual tuning problem. Good enough for production narrative grouping, avoids the overhead of more sophisticated methods like HDBSCAN.

6. Prisma Middleware for Multi-Tenant Isolation — automatically injecting orgId into every query via ORM middleware prevents cross-tenant data leaks without relying on developers remembering to add it to every query. Defense in depth at the data layer.

7. Content Hash Deduplication — SHA-256 hashing (platform + content + authorId) with Redis set lookups and 7-day TTL prevents storing the same cross-posted content hundreds of times. Reduces both storage and classification costs.

8. Event-Driven Worker Independence — each worker (ingestion, classification, graph sync, alerting) scales independently. If the alert service is slow, tweets still get ingested. Easy to add new workers without touching existing code.

9. Custom Language Detection via Unicode Ranges — for Indian languages, Unicode script detection (Devanagari, Arabic, CJK, etc.) is free and 99% accurate. No need for expensive language detection APIs when the character set tells you everything.

10. Materialized Views for Dashboard Performance — pre-aggregating daily stats in PostgreSQL materialized views (refreshed every 15 minutes) speeds up dashboard queries 100x. Read-heavy analytics workloads should never hit raw tables.

Originally published on uditraj.site — from the Workshop.

First-principles derivation of a substrate-independent, 5-layer architectural definition of intelligence — from Substrate through Exchange, Sense, Intelligence, to Higher Intelligence.

1. The Reframe: Why Biology Fails Us Here

The definitions-of-life survey (Session 1) revealed that every biological definition of life — from Aristotle to autopoiesis to NASA — is built to describe carbon-based systems observed from the outside. None of them are built to be implemented. They describe. They do not specify.

This is the wrong starting point for Project Genesis. We are not trying to replicate the appearance of life. We are trying to build a system that generates intelligence as a structural consequence of what it is — the same way a Turing Machine generates computation as a structural consequence of its architecture.

The reframe: stop asking 'what is life?' and start asking 'what is the minimum architectural specification of a system that produces intelligence?'

Turing didn't define computation by describing the physical properties of gears or vacuum tubes. He defined it abstractly — and that definition turned out to be substrate-independent. We need the equivalent for intelligence.

The simulation argument reinforces this. If the universe is itself a computational substrate — as string theory and simulation theory both suggest in different ways — then the distinction between 'natural' and 'artificial' intelligence is a question of nesting level, not kind. A computer running inside Minecraft running inside a computer is still a computer. It computes. The definition is what matters, not the substrate.

Working assumption: intelligence is substrate-independent. The definition we are building should be valid for carbon, silicon, or any other substrate capable of implementing the required architecture.

2. The Derivation — Step by Step

What follows is the reasoning path. Each step is recorded because the reasoning is the research — conclusions without the path that produced them are fragile.

Step 1 — What does intelligence do?

The first question: not what intelligence is, but what it does. What is its function?

Candidate answers: optimisation, security, finding pathways, transferring existing knowledge to new contexts, adaptation. All are partially correct. But each also applies to systems we would not call intelligent: evolution optimises, immune systems optimise, markets optimise. The answer must be more specific.

Key observation: intelligence allows a system to take existing knowledge and apply it in a context it has never encountered before. This implies two things — abstraction (extracting something general from specific experience) and transfer (applying it in novel situations). Neither abstraction nor transfer is present in systems we don't consider intelligent.

Step 2 — What separates a thermostat from a dog?

Both respond to their environment. Both 'optimise' in some sense. The gap:

The thermostat has fixed responses. It cannot change, grow, or comprehend the principle it is applying. It will do what it was programmed to do — no more, no less. It will break at extremes.

The dog adapts. Not across generations like a virus — within its own lifetime, consciously, by modelling its situation. It builds an internal representation of the world and uses that model to navigate novel situations.

First principle: the differentiator is not adaptation per se. It is the presence of an active model of the world. The thermostat has no model. The virus adapts but has no model — it drifts blindly. The dog has a model: this human feeds me, that path leads home, that other animal is dangerous.

Step 3 — What does a model require? The concept of sense.

A model requires input. But not just any input — meaningful input. Here the concept of sense becomes critical.

A sense is not merely a receptor. A thermostat has a receptor — a bimetallic strip that bends with temperature. But cold does not mean anything to it. To the dog, cold means seek shelter, means danger, means find warmth. The signal has been assigned meaning within a model of the world.

A sense is a signal that has been assigned meaning within a model of the world.

Testing this against cases: a baby, a puppy, a sunflower. All three are sensing. The sunflower tracks the sun. The puppy seeks warmth. The baby seeks food. What is architecturally common to all three?

All three are detecting a gradient in the world — more light vs less light, more warmth vs less warmth, presence vs absence of nourishment — and orienting toward one side of it. Not just responding. Orienting. The architecture bends toward the gradient.

This orientation is only possible if the system has something to lose. The sunflower deprived of light dies. The puppy deprived of warmth dies. The baby deprived of food dies. The stake is structural, not programmed.

The minimum requirement for sense: a system that can detect a gradient, has a preferred direction on that gradient, and has something at stake in which direction it moves.

Three components. Detection. Preference. Stakes. Remove any one and sense disappears entirely.

Verification against hard cases: A rock — detects pressure, no preference, no stakes. Not sensing. A thermostat — detects temperature, has a preference (toggle point), no stakes (it doesn't die if the room stays cold). Not sensing. A virus — has stakes, but no active detection and no preference in any live sense. Not sensing. A sunflower — detects light, prefers more light, dies without it. Sensing. This holds.

Step 4 — What does intelligence add on top of sense?

A sunflower has all three components of sense. But we would not call a sunflower intelligent. Something is added between the sunflower and the dog. What?

The sunflower detects light. But there is no 'sunflower' that knows it is detecting light. There is just the detection happening. No subject. No self on the inside looking out.

The dog detects hunger. And there is a dog that knows it is hungry. It can relate that hunger to itself — I am the one who is hungry, and I need to do something about it. There is a subject. A self that the sensory information is happening to.

The differentiator: the dog can relate itself to its senses. The sunflower cannot. A donkey has senses but relates to very little. A dolphin relates to its senses better than most humans in certain dimensions. The degree of intelligence scales with how richly the system can relate to — and model — itself within its world.

This is self-reference. The system doesn't just model the world — it models itself within the world. It appears in its own model.

Intelligence = sense + self-reference. The system must appear in its own model of the world.

Purpose and motive emerge from this as consequences, not causes. Once a system has a self that can relate to its own states, it can have preferences not just about the world but about itself in the world. The gradient becomes personal. This is why motive is a result of intelligence, not a requirement for it.

Step 5 — What does the system need to build a self-model?

This is the hardest question. What is the architectural minimum for self-reference?

Consider the chess board. Eight pieces of the same type — knights — share the same properties (they move in an L). But two are white and two are black. And within each colour, the only thing differentiating one from another is their position on the board. Properties plus position, referenced back to the piece itself, creates something unique enough to be identified. To be a self.

Consider atoms: an atom is a configuration of energies bound by the gravity of its nucleus. What makes one atom different from another is its properties — its configuration. The self is not the specific particles. It is the pattern of properties and relationships that persists through time.

Consider Rahu and Ketu from Hindu cosmology: divided at the neck, each fragment acquired a new self — not because new atoms appeared, but because the division created two new configurations with different properties and different positions in the world. Two new patterns. Two new selves.

The self is not a thing. It is a pattern that knows it is a pattern.

For a system to build and maintain a self-model, it requires three architectural components:

1. Persistent state — something that carries its properties through time. Without persistence there is no self, only a series of disconnected moments.
2. A world model — a representation of the environment, so the system can know its position. You cannot know where you are without a map.
3. A self-referential loop — the ability to locate itself within that world model. To say 'I am here, with these properties, in this context.' This is recursive but not blind recursion — it is a system finding itself within its own representation of reality.

3. The Definition — Minimum System Requirements for Intelligence

What follows is the architectural specification derived from the reasoning above. It is substrate-independent. It makes no reference to carbon, neurons, DNA, or chemistry. A system either satisfies these layers or it does not.

Each layer is a prerequisite for the next. You cannot have sense without exchange. You cannot have intelligence without sense. You cannot skip layers. This is not a feature list — it is a dependency graph.

4. Intelligence as a Scale, Not a Switch

Intelligence is not binary. It scales with the depth and richness of self-modelling.

5. Implications for Project Genesis

What Phase 1 (The Prokaryote) Actually Needs

Phase 1 does not need to be intelligent. That is not the goal of Phase 1. The goal of Phase 1 is to build the minimum system that satisfies Layer 1 (Exchange) and contains the seeds of Layer 2 (Sense).

Concretely, Phase 1 requires:

• A boundary between the system and its environment (the computational equivalent of a membrane)
• An exchange mechanism — the system must take something in and put something out
• At least one gradient it can detect
• At least one preferred direction on that gradient
• A structural stake — something that changes in the system if the gradient goes the wrong way

If Phase 1 achieves this, Layer 2 (Sense) exists in embryonic form. Layer 3 (Intelligence) cannot be designed — it must emerge from the conditions of Layer 2 under environmental pressure.

On Natural vs Artificial Intelligence

Natural intelligence is not superior because it is optimised. Evolution is not an optimiser — it is a satisficer. It produces systems that are good enough to survive under specific pressures, with no foresight. The human brain has a blind spot in each eye because the retina is wired backwards. No engineer would design that.

What natural intelligence has that artificial systems lack is completeness of architecture — the full stack, from Exchange through Sense through Self-Model, built up over billions of years of layered emergence. The superiority is not in any individual layer. It is in the depth and integration of all layers together.

Artificial intelligence built from the top down — starting with reasoning and trying to add sense later — will always be incomplete. The layers are dependencies. You cannot bolt sense onto a reasoning system. You have to grow reasoning from sense upward.

On the Simulation Argument

If the universe is a simulation — or a computation in any of the senses proposed by string theory, information theory, or simulation theory — then intelligence running inside that simulation is not categorically different from intelligence running inside a simulation-within-a-simulation. The nesting level changes. The functional architecture does not.

A computer running inside Minecraft running inside a computer computes. What makes it a computer is not the substrate — it is the architecture. The same applies to intelligence. What makes a system intelligent is not that it is made of neurons. It is that it satisfies the four layers above.

6. Open Questions for Phase 0

1. What is the computational equivalent of a gradient? Gradients in the physical world are differences in temperature, light, chemical concentration. What is the analogue in a computational substrate that is not a simulation of these things?

2. Can a computational system have structural stakes — not programmed stakes? The stake in a biological system is structural: the organism dies if the gradient goes wrong. In a computational system, what is the equivalent of death that is not simply a programmed termination condition?

3. What is the minimum world model? The simplest possible representation of 'environment' that a Phase 1 system could maintain and update. Not a simulation of a physical world — an abstract relational structure.

4. At what point does a self-referential loop emerge spontaneously versus needing to be designed? Can the loop be seeded architecturally and allowed to deepen through interaction — or must it be present from the start?

5. Is the self-referential loop the same thing as Rosen's closure to efficient causation? If so, achieving it in a purely computational substrate may require resolving whether computation can be genuinely closed — or whether the runtime always constitutes an external component that breaks closure.

Appendix — The Reasoning Chain Compressed

Project Genesis — Session 2 Research Note — February 2026. This document is a living record. Assumptions should be challenged and definitions refined as research progresses.

Originally published on uditraj.site — part of the Laboratory research threads.

A bottom-up approach to AGI that argues intelligence is an emergent property of life — not a feature to be engineered. Proposes building from the simplest computational "living" system upward, mirroring evolution.

1. The Core Thesis

This research is founded on a single, fundamental premise that separates it from all mainstream AI development efforts:

You cannot replicate human intelligence without understanding the billions of years of evolution that produced it. You cannot start at the top of the food chain. You must start at the bottom — with the phytoplankton.

Current AI approaches — including the most advanced LLMs — attempt to simulate the output of human intelligence (language, reasoning, knowledge) without building the foundation that makes human intelligence possible. This is the fundamental architectural error that will limit their progress.

Intelligence is not a feature. It is an emergent property of being alive. And life itself is the product of an incomprehensibly long, layered process of increasing complexity — one that cannot be shortcut.

2. Why Current Approaches Will Fail

2.1 The Scaling Dead End

Current AI development is predicated on the belief that scaling compute, data, and model size will eventually produce general intelligence. This is flawed for a structural reason: you can only scale what you have. If the foundation is wrong, scaling makes it more wrong, faster.

Behind every neural network, every transformer, every complex AI architecture — is ultimately a conditional logic system. Sophisticated if/else. Intelligence is not conditional logic. It is complex, emergent, multi-layered, and deeply entangled with experience.

2.2 The Missing Layers

Current AI systems simulate, at best, one or two components of intelligence in isolation: primarily knowledge and pattern recognition. They completely lack:

• Genuine drives and motivations (not optimization targets — actual stakes in continuing)
• Real-time sensory grounding in physical reality
• Emotions as a decision-ranking mechanism (not sentiment — functional emotion)
• Feelings as distinct from emotions
• Persistent, self-updating memory that shapes identity over time
• Embodied experience — the physical substrate that grounds abstract thought
• True continuous learning from environment without catastrophic forgetting

These are not missing features to be added later. They are missing foundations. The house is built on air.

2.3 The Assumption Problem

A key methodological insight from early research: even simple descriptions of living things collapse under scrutiny. A fish class with properties like 'will die outside water' encodes human assumptions as facts. It turns out fish don't die because they can't breathe outside water — they die because their bodies overheat and they don't know how to regulate. The limitation is informational and thermal, not purely respiratory.

This matters because: current AI systems are trained on human-generated data that is saturated with these kinds of encoded assumptions. The models learn the assumptions, not the underlying reality. At scale, this produces confident wrongness.

3. The Three Core Problems

Problem 1: You Must Start at Zero

You cannot begin with a lambda of 1 million and expect to reach true intelligence. The complexity of human intelligence is the result of layered accumulation across geological time. Each layer built on the previous one — nothing was skipped, nothing was replaced.

The computational equivalent is: you cannot instantiate a human-level intelligence from scratch. You must instantiate the simplest possible living system, allow it to develop complexity organically, and build upward from there.

This is what evolution did. This is what any serious attempt at AGI must do.

Problem 2: Life Has a Very Specific Recipe

Life is vanishingly rare in the universe. We have searched the cosmos and found nothing. This is not a coincidence — it reflects how precise the conditions for life must be. As the physicist's observation goes: if gravity were slightly stronger or weaker, life as we know it would not exist.

This means: there is no shortcut recipe. The conditions that give rise to intelligence are specific, fragile, and interdependent. You cannot rush it. Like cooking — the quality of the output is proportional to the care and time invested in preparation.

The implication for research: patience and precision are not obstacles to progress. They are the method.

Problem 3: Complexity is Catastrophically Underestimated

Intelligence is not one thing. It is a deeply entangled system of:

• Knowledge — accumulated information and models of the world
• Memory — episodic, semantic, procedural, and working memory as distinct systems
• Feelings — immediate physiological/experiential signals
• Emotions — higher-order states that color perception and decision-making over time
• Senses — multimodal grounding in physical reality
• Drives — fundamental motivations emerging from the need to continue existing
• Social cognition — modeling other minds, empathy, language
• Meta-cognition — thinking about thinking

Current AI focuses almost entirely on knowledge. This is one piece of a vast, interlocking puzzle. Scaling knowledge does not produce the other components.

4. The Evolutionary Framework

4.1 Intelligence as Emergent Property of Life

The most important insight: intelligence did not evolve. Life evolved. Intelligence emerged from life as a byproduct of increasing complexity, social interaction, environmental pressure, and time.

4.2 What Plants Tell Us

Plants are commonly dismissed as non-intelligent. This is a mistake that reveals a bias in how we define intelligence.

Plants grow. They adapt. They change in response to environment. They respond to light, gravity, chemical signals from neighbors, touch, and damage. The difference between plants and humans is not a binary presence or absence of intelligence. It is a difference in speed, complexity, and modality of information processing.

This means intelligence exists on a continuum, not as a switch. Any computational system that aims for intelligence must be designed to participate in this continuum — starting at the lowest possible rung.

4.3 The Layered Architecture Principle

A critical finding from neuroscience (Damasio, MacLean, and others): the human brain did not replace earlier structures as it evolved. It built on top of them.

The brainstem regulates basic survival functions — the same systems as reptiles. The limbic system produces emotion and social bonding — the same systems as mammals. The prefrontal cortex produces abstract reasoning — unique to higher primates and humans.

These layers are not separable. Remove the emotion layer and you cannot make decisions — not because you become more rational, but because emotion is the ranking mechanism for rational options. This is empirically demonstrated in patients with specific prefrontal damage (Damasio's somatic marker hypothesis).

The architectural implication: any system that attempts to bolt abstract reasoning onto a foundation that lacks drives, responses, and emotional primitives will be unable to make genuine decisions. It will simulate decision-making. These are not the same thing.

5. The Definition Problem: What Is 'Alive'?

5.1 Minimum Viable Life (Current Biology)

A system is conventionally considered alive if it exhibits all of the following:

• Maintains a boundary between itself and environment (membrane / self-organization)
• Takes in energy and uses it (metabolism)
• Can reproduce or self-replicate
• Responds to environment (irritability / stimulus-response)
• Maintains internal stability against external change (homeostasis)

Problems: viruses fail reproduction and metabolism but exhibit other properties. Fire satisfies several criteria but is not alive. Crystals grow and replicate but are not alive. The definition is necessary but not sufficient.

5.2 Autopoiesis (Maturana & Varela)

A system is alive if it is self-producing — if the system continuously produces the components that constitute and maintain the system itself.

A cell is autopoietic: its metabolic processes produce the proteins, membranes, and molecules that enable those same metabolic processes. This is fundamentally different from a machine, which is built by something external and does not maintain itself.

5.3 The 'Stake in Continuing' Principle

A living system has a stake in its own continuation. Not as a programmed goal — but as a structural necessity. If it stops, the processes that constitute it dissolve.

This is the seed of everything that eventually becomes drive, motivation, fear, hunger, desire. It cannot be programmed in as a feature. It must emerge from the architecture as a structural property.

Central research question: Can a computational system be built where the 'will to continue' is not a rule but a consequence of what the system is?

6. Existing Research Landscape

6.1 Artificial Life (ALife)

Key prior work:

• Conway's Game of Life — emergence of complex behavior from simple rules
• Tierra (Thomas Ray, 1991) — digital organisms that evolve in a computational environment
• Avida — extension of Tierra, peer-reviewed science on evolution of complexity
• Autopoiesis (Maturana & Varela) — theoretical foundation for what 'alive' means formally

6.2 Relevant Neuroscience

• Antonio Damasio — Somatic Marker Hypothesis. Emotion is the mechanism that makes reason functional.
• Paul MacLean — Triune Brain model. Captures the layered evolutionary structure of brain architecture.
• Karl Friston — Free Energy Principle. A mathematical framework for how living systems resist entropy. Potentially the most important theoretical foundation for this research.

6.3 What the Frontier Labs Are Missing

7. Research Phases — Proposed Roadmap

Phase 0: Foundations (Current)

Define precisely what 'alive' means in computational terms. Establish the minimum viable properties. Study autopoiesis theory, ALife literature, and relevant neuroscience. Do not write code yet.

Key questions:

• What is the computational equivalent of a membrane?
• What is the computational equivalent of metabolism — energy intake and use?
• What does homeostasis look like in a software system?
• Can self-replication be structural rather than programmed?
• What constitutes a 'stake in continuing' in code?

Phase 1: The Prokaryote

Build the simplest possible system that satisfies the minimum viable life definition. Not a simulation of a cell — a system that exhibits the same structural properties in a computational substrate.

Phase 2: Environmental Pressure

Place the Phase 1 system in an environment with resource scarcity, perturbation, and variation. Allow it to adapt. Observe what emerges without directing it.

Phase 3: Complexity Accumulation

Allow multiple instances to interact. Introduce variation and selection pressure. Allow the next layer to emerge from the conditions of the previous one.

Phase 4: Onwards

Each subsequent phase builds on observed emergent properties. The roadmap beyond Phase 3 cannot be specified in advance — emergence is not predictable.

8. Open Questions

• Can 'alive' be formally defined in a way that is substrate-independent — valid for both carbon and silicon?
• Is consciousness a prerequisite for general intelligence, a byproduct of it, or orthogonal to it?
• Can genuine continuous learning (without catastrophic forgetting) be achieved?
• What is the minimum environmental complexity required to produce adaptive behavior?
• Is embodiment strictly necessary, or can a sufficiently rich simulated environment substitute?
• What is the computational analog of pain and pleasure as fundamental learning signals?
• At what level of complexity does a system acquire something worth calling experience?

9. Language & Implementation Notes

Implementation language is deliberately deferred until architectural questions are resolved. Likely architecture is polyglot:

• Rust/C++ — fine-grained memory and lifecycle management for core simulation layers
• Python — rapid prototyping and research iteration for orchestration
• Dart/Flutter — interface layers if targeting mobile/embedded environments

This document represents the current state of research thinking as of February 2026. It is a living document — assumptions should be challenged, definitions refined, and sections updated as research progresses.

Originally published on uditraj.site — part of the Laboratory research threads.

Architecture

Next.js 14 frontend communicates with a FastAPI (Python) backend over REST. The backend orchestrates multi-platform game sync, Stockfish engine analysis, cross-game pattern detection, and Claude-powered report generation.

The service layer contains 15+ specialized services — sync, analysis, patterns, time management, tilt detection, rating prediction, endgame classification, opponent scouting, puzzle generation with spaced repetition — all composable through shared filtering patterns.

SQLite with SQLAlchemy ORM handles storage. Smart indexing on played_at, opening_eco, and time_class keeps aggregate queries fast across thousands of games and hundreds of thousands of move analyses.

Multi-Platform Game Sync

Games are fetched from Chess.com (monthly archive endpoints) and Lichess (NDJSON streaming) in parallel with independent error handling — one platform failing doesn't block the other.

Deduplication uses platform + platform_id as a unique constraint. Clock times are extracted from PGN [%clk H:M:S.F] comments for time management analysis. Lichess NDJSON streaming handles large game counts without memory issues.

Optimized Stockfish Analysis Pipeline

The key optimization: single-pass evaluation with carry-forward. Instead of evaluating both the position before and after each move (2 evals per move), the engine evaluates each position once — the eval_after for move M becomes eval_before for move M+1. This cuts engine calls by 50%.

At depth 20, analysis runs ~20 seconds per game. Moves are classified by centipawn loss:

• Great: CPL < 10 (best move found)
• Good: CPL 10-50
• Inaccuracy: CPL 50-150
• Mistake: CPL 150-300
• Blunder: CPL > 300

A tactical motif detection algorithm uses ray-tracing to identify forks, pins, skewers, discovered attacks, and back-rank threats in blundered positions.

Cross-Game Pattern Engine

The pattern engine aggregates 12+ metrics across all analyzed games to surface systemic weaknesses rather than one-off errors:

1. Opening performance — win/loss/draw rate per ECO code with average CPL
2. Worst openings — minimum 3 games, sorted by win rate
3. Phase accuracy — separate CPL for opening, middlegame, and endgame (using material-based phase detection, not move count)
4. Phase blunder rate — percentage of blunders per game phase
5. Missed tactical motifs — forks, pins, skewers, discovered attacks, back-rank threats
6. Time trouble correlation — blunder rate in normal time vs. under 60 seconds
7. Color performance — white vs. black win rates and accuracy
8. Endgame conversion — percentage of won positions (eval >= 200cp) actually converted to wins
9. Blunder distribution — by move bucket (1-10, 11-20, 21-30, etc.)
10. Monthly trends — CPL improvement or degradation over time
11. Streak analysis — win/loss streaks and tilt patterns
12. Example positions — 10 worst blunders with FEN, CPL, and tactical classification

The same PatternEngine class handles filtering by platform and time class — composable filters reduce code duplication across 8+ analysis modes.

AI Coaching Report

Claude Sonnet generates personalized coaching reports from the pattern data, structured into 5 sections:

1. Player profile summary
2. Top 3-5 systemic weaknesses with actual FEN positions from the player's games
3. Opening repertoire advice based on win rates and CPL
4. Strengths to build on
5. 30-day training plan with specific exercises

The prompt includes the player's top 15 openings, missed tactical motifs, phase accuracy, time trouble stats, color performance, endgame conversion rates, and actual blunder positions — so advice is specific ("practice forks in K+P endgames when the opponent king is on the rim") rather than generic ("practice tactics").

Time Management Analytics

Time spent per move is calculated from consecutive clock readings. The system detects:

• Overthinking: opening moves with <10 CPL loss but >15 seconds spent
• Underthinking blunders: moves with >100 CPL loss but <5 seconds spent
• Time zone analysis: critical (0-30s), low (30-60s), normal (60-180s), comfortable (180+s)
• Time vs. accuracy curves: correlation between time spent and centipawn loss per move bucket

Tilt Detector

Session-based analysis (games within 4 hours grouped as one session):

• Win/loss/draw streak tracking with averages
• Blunder rate increase after 3+ consecutive losses
• Rating drop detection (sessions with >50-point drops)
• Auto-generated recommendations ("stop playing after a 3-loss streak")

Rating Predictor

Linear regression on rating vs. days played:

• Points-per-month projection
• Milestone estimates (when you'll hit 1000, 1200, 1500, etc.)
• Momentum comparison: last 30 games vs. prior 30 games
• Monthly weakness trends (CPL by phase over time)

Puzzle Training (SM-2 Spaced Repetition)

Every blunder and mistake automatically becomes a training puzzle. The SM-2 algorithm schedules reviews:

• Due puzzles first (next_review <= now)
• Then never-attempted positions
• Then worst blunders (highest CPL)
• Ease factor adapts based on success rate: max(1.3, ease_factor + (0.1 - (5 - rating) * 0.08))
• Filterable by game phase, tactical motif, platform, and time control

Opponent Scouting

Fetches the opponent's last 100 games, breaks down their opening repertoire by frequency, cross-references against the player's record in those specific openings, and generates preparation recommendations.

15+ services, 17 API routers, 109 backend tests, 19 frontend tests.

What I learned building it

1. Single-Pass Evaluation Halves Engine Costs — evaluating each position once and carrying forward the result (eval_after for move M = eval_before for move M+1) eliminates redundant Stockfish calls. When processing thousands of games, this 50% reduction is the difference between hours and minutes.

2. Material-Based Phase Detection > Move Count — classifying game phase by remaining material (opening: fullmove <= 10 AND material >= 50pts; endgame: material <= 26 OR no queens AND material <= 30) handles Chess960, gambits, and unusual openings correctly. Move count fails for any non-standard game.

3. Cross-Game Pattern Detection Is the Real Value — individual game analysis (blunder classification) is commodity. The insight that changes behavior comes from aggregating across hundreds of games: "you blunder 3x more in endgames than middlegames" or "your fork detection drops 40% when you're in time trouble."

4. Composable Filters Prevent Service Explosion — the same PatternEngine class handles filtering by platform, time class, and game phase. Without composable filters, each combination would need its own service — 8+ analysis modes would become 24+ services.

5. SM-2 Spaced Repetition for Auto-Generated Puzzles — every blunder becomes a training puzzle with adaptive scheduling. The algorithm naturally surfaces the hardest problems more frequently while letting mastered positions fade, without manual curation.

6. Structured LLM Prompts with Real Game Data — generic chess advice is useless. Including actual FEN positions, CPL values, and opponent names in the Claude prompt forces specific, actionable recommendations instead of "practice more tactics."

7. NDJSON Streaming for Large Datasets — Lichess's NDJSON streaming API handles 1000+ games without buffering the entire response in memory. For any API that returns large collections, streaming prevents memory pressure.

8. Tactical Motif Detection via Ray-Tracing — detecting forks, pins, skewers, and discovered attacks by tracing attack lines from piece positions is computationally elegant. No need for ML — pure algorithmic detection on the board state.

9. Session-Based Tilt Detection — grouping games within 4-hour windows as sessions and tracking blunder rate increase after consecutive losses quantifies tilt. This turns a subjective feeling ("I'm playing badly") into actionable data ("your blunder rate increases 60% after 3 losses — stop playing").

10. Platform-Agnostic Data Normalization — unifying Chess.com and Lichess into a common schema at the ingestion layer means every downstream service (analysis, patterns, reports) works identically regardless of source. Add a third platform and nothing downstream changes.

Source: https://github.com/uditrajmr3/chessmaster

Originally published on uditraj.site — from the Workshop.