work

**What it is** — a two-sided clinical-AI system, built solo. **Gretel** ([gretel.lemomo.ai](https://gretel.lemomo.ai) · [App Store](https://apps.apple.com/us/app/gretel/id6786951507)) is a real consumer iOS + Watch app, live on the App Store — non-diagnostic: biosignal shifts → lightweight voice check-ins → structured, medication-aware, auditable episodes, with on-device AI (Apple Foundation Models, grounded). **Hansel** ([hansel.lemomo.ai](https://hansel.lemomo.ai), demo sign-in) is a hospital-facing clinician console closing the loop: episode monitoring → AI-drafted follow-up suggestions (draft-only) → clinic check-ins pushed back into the app, with threaded patient replies. Hansel is fully functional but shown as an **all-synthetic demo** — clinical software of this class is heavily regulated — so it is a personal work sample, not a deployed clinical product: no real patient data, no users claimed. **Architecture decision** — correctness-first. **Deterministic clinical executors** (schema validation + wire-contract runtime invariants, de-identification, terminology coding — SNOMED/LOINC/HPO, medication-completeness prompts, calculators) are kept strictly separate from the **LLM adviser**, which only produces drafts for a clinician and is never the system of record. A **provenance hub** labels every output real / mock / license-required / production-path. The thesis is **model-improvement-proof**: a stronger frontier model should improve coverage without ever becoming the source of truth for coding, medication or calculation. Honesty is a feature, not a disclaimer. **Hardest part** — the two halves were built in parallel sessions and integrate over a **frozen, machine-checkable contract**, so neither side could drift; 75 tests on the console side. Cleared App Store **health-data privacy review** (HealthKit + cloud-AI consent gates). **Stack** — Gretel: SwiftUI iOS + watchOS, HealthKit, on-device Apple Foundation Models. Hansel: React + TypeScript, TanStack Router/Query, AJV, Tailwind/shadcn. **Next step** — a FHIR layer is what I'd build next; it does not exist today.

**What it is** — the front page of [xiayangzhang.com](https://xiayangzhang.com) is a 7B model running 24/7 on my Mac, writing a continuous dream journal; visitors wake it. This entry is about the machinery underneath. **Architecture decision** — own the forward pass instead of calling an API: a custom **mlx-lm generate loop** with per-token continuous sampling scheduling (sampling parameters move smoothly mid-generation), a **persistent KV cache** that survives across episodes, steer injection at episode boundaries, and visitor-driven **KV-cache perturbation** — visitors literally tamper with the dream's memory. Live per-token **entropy / top-k telemetry** drives the visuals: you can watch the model think. **Hardest trade-off** — `set_context` rebuilds the whole KV cache, which kills the running dream. So full context updates land only at reconnect, and mid-dream influence is restricted to single gentle steers at episode boundaries — a real constraint that shaped the whole update model. **Boundaries (honest)** — this is inference control and observability, not training: no fine-tuning claim (a LoRA phase is planned, not done). Stack: MLX dream engine + Node daemon + Vite front-end; a cloud-model fallback lane handles guestbook replies.

**What it is** — a realtime-voice AI writing/story coach **demo** (work sample), built solo end-to-end: [tiktalk.xiayangzhang.com](https://tiktalk.xiayangzhang.com). No users or customers claimed — it exists to show I can ship streaming voice. **What's hard** — realtime voice is a latency problem wearing a product costume: streaming voice via **OpenAI Realtime**, tool calls fired mid-conversation, **latency masking** so the coach feels present while tools run, and session state that survives the whole conversation. The engineering is in the seams — keeping the voice channel alive and natural while structured work happens underneath.

**What it is** — "tcpdump for LLM gateways": an open-source Go transparent proxy ([github.com/2nd1st/api-log](https://github.com/2nd1st/api-log)) that records GPT / Claude / Gemini traffic into an append-only audit trail (JSONL + SQLite index) with replay. The foundation for eval and trace analysis over my own traffic. **Hardest part — a real production-shaped incident.** The companion analysis tool had to ingest what the gateway had recorded (the gateway has captured ~200GB of my own homelab LLM traffic in total; the analysed cluster is ~175K traces / 20GB). The gateway stores `.jsonl.gz`, and gzip has no random access — a naive ingester spun up 12 concurrent decompressions, crushed the live gateway and leaked its connection pool. The fix: **bounded transport + per-fetch watchdog + single-writer exactly-once semantics + a resumable cursor**. Verified adversarially — fail-before / pass-after. **Companion (building now)** — **api-log-analytic** (Go + SQLite + Svelte 5): three analysis lenses over the recorded traffic. It analyses inference traffic; it is not a model-serving system. Not public yet — releasing. **Boundary** — personal homelab corpus, not production multi-tenant traffic; a work sample in the observability/LLMOps space.

**What it is** — an open-source cognitive-agent framework in TypeScript ([github.com/2nd1st/aima](https://github.com/2nd1st/aima)): Postgres + pgvector, persistent and auditable agent state, memory slots, MCP integration. **Architecture decision** — "**governed, not orchestrated**": guardrails and judges are first-class citizens, and agents run inside budgeted, auditable envelopes (budget-aware memory injection, episodic memory compression) rather than a free-form orchestration graph. The bet: as agents get more capable, the scarce thing isn't orchestration — it's accountable state and enforceable limits. **Status (honest)** — a work sample / credential, not a product with adopters. Built in an intense early-2026 sprint; its design informed everything I've built since — Breadcrumbs' correctness-first executor/adviser split is its direct descendant.

**What it is** — the layer under everything else: a self-built **multi-provider LLM gateway** (GPT / Gemini / Grok / Claude behind one API) and an **IncusOS homelab** (dual-Xeon, ZFS, ~20 services — reverse proxy, Git + container registry, uptime monitoring, several long-running agent VMs). **The point** — I don't hand-write deep infra; I design the architecture and write the **agent harness that drives agents to run it**. The homelab is operated 24/7 by agents via an Agent Handbook + skills. That's the proof of the approach — harness-led ops — not a claim of cloud-scale experience. **Throughput** — ~14.4B tokens/mo across self-run harnesses (~10.7B via Claude Code, ~97% cache-hit): my own dev/agent consumption, not product traffic, and a conservative floor. **Boundaries (honest)** — single-host homelab, not production multi-tenant scale; the GPU is a weak Quadro, not a serving cluster. Gaps I'm deliberately ramping, in order: vLLM/TGI first, then managed K8s / Terraform.

**What it is** — a live **AI + AR education research platform**; I'm the primary engineer in a research collaboration with a university Research Fellow, running on real research data. The collaboration began in 2023, when the Fellow sought out my AR-analytics work at Dreemar; the current AI-platform generation was built in 2026 and is in active development. **What it does** — a WebXR platform delivering an **agentic voice tutor** with persistent cross-session memory; **assessment** via tool-call quizzes and per-lesson probes; **personalization** via per-student persona modelling; and feedback from audio emotion analysis plus **spatial replay** of student view-direction and movement. **Architecture notes** — cross-session memory is the interesting part: the tutor carries a per-student model between lessons, and assessment happens through tool calls the model makes mid-conversation rather than quizzes bolted on the side. Multi-model by design: GPT for analysis, OpenAI Realtime for voice, Gemini for dynamic storybook imagery. Next.js / React / TypeScript · Babylon.js · Supabase/Postgres · Playwright + Vitest. **Access** — a research project on a private repo; no public link. Walkthrough on request.

**What it is** — my commercial shipped record. At Dreemar (Melbourne AR studio) I was the **sole developer for the final ~1.5 years** — de-facto the whole engineering team: frontend, backend, CMS, web editor, mobile apps, plus all production **AWS** operations. **Shipped** — 6+ AR apps on the App Store / Google Play (Unity / C#), mostly children's education AR used in real Australian schools, including a **Deakin University** collaboration and Victorian Tech School apps; one consumer AR app, **HeyAR**, reached **tens of thousands of downloads**. Also **DreemXR** (React Native) — a downloadless marker-tracking AR app (iOS App Clip + Android Instant App). **AI line** — in the final stretch I built **ARI-AI**, an early-stage multi-agent AI marketing-content SaaS (pre-users), solo, deployed on AWS: Next.js/TypeScript front-end, Python **CrewAI** multi-agent backend, **LiteLLM** gateway across OpenAI + Anthropic, **Langfuse** tracing + Prometheus metrics. The value is the architecture and observability stack, not user scale. **Thread to now** — I also built the platform's **spatial analytics** (position/orientation capture → session replay, attention targets, dwell-time patterns); that work is what led to the AREAR research collaboration.

**What it is** — a small open-source Python tool ([github.com/2nd1st/claudoros](https://github.com/2nd1st/claudoros)): a Claude Code session/focus monitor that reads the local JSONL transcripts into a live dashboard — which sessions are running, what each is doing, where the tokens go. **Why it exists** — I run many agent sessions in parallel; this is the cockpit. A utility, not a product.

**What it is** — a Rust CLI/library: **baseline-delta verification for agent-authored changes in dirty repos**. An agent edits a repo that already carries lint/type debt; loomomo gates **only the findings this change introduced** (new / persisting / resolved), standing on **SARIF** for finding identity across heterogeneous tools. **Honest positioning** — baseline / new-code-only gating is not a new category (SonarQube new-code, Semgrep --baseline-commit, reviewdog, Betterer all exist). The contribution is the combination: it lives inside the agent edit-loop, auto-detects the repo's tool profile, normalises heterogeneous gates onto SARIF, and keeps durable run-state. **Status (honest)** — v0.1, not public yet, benchmarks not run — releasing. It's the verification layer under my own agent workflows.