The Chronicles
Stories behind work that shipped, and notes on what is changing in data and AI. Longer series are published on Medium and listed here with everything else.
AI
Agents and the governance that keeps them affordable and safe.
Agents for data work
Coding agents, harnesses and the guardrails that make them safe on production pipelines.
Nothing here yet. The first piece is in progress; subscribe to get it when it lands.
AI governance & FinOps
Cortex cost control, RBAC and OAuth for AI workloads, spend that stays predictable.
Nothing here yet. The first piece is in progress; subscribe to get it when it lands.
Models
The durable layer: semantic definitions, open tables and catalogs.
Semantic layer & ontology
Semantic views, metrics and the business definitions that both humans and LLMs read.
Connecting Atlan to Snowflake's ontology: tags, lineage and semantic views across the boundary
Keeping tags, lineage and semantic views consistent when governance lives in Atlan and the definitions live in Snowflake.
The semantic layer: semantic views, dbt metrics and business definitions
Where business definitions should live, and how Snowflake semantic views and dbt metrics divide the work.
Lakehouse & Iceberg
Open table formats, catalogs and the governance that keeps them honest.
Nothing here yet. The first piece is in progress; subscribe to get it when it lands.
Data
Pipelines, change data capture and the foundations under everything.
Real-time & pipelines
Change data capture, streaming into Iceberg and Snowflake, orchestration at scale.
The quietest process in Airflow: what the triggerer did for 200 parallel Dataproc jobs
How Airflow's triggerer and deferrable operators let a single scheduler supervise 200 concurrent Dataproc jobs without burning worker slots.
Data & Engineering Innovators Spotlight: Ved Prakash
An interview with Artie on real-time CDC with Project Siphon, proactive cost governance for Snowflake Cortex, and where AI will change data engineering next.
Foundations
How models, platforms and teams actually work, explained from first principles.
How a Gen AI model reaches production
From raw data to millions of conversations. Seven stages, from data collection and pre-training through alignment, evaluation, infrastructure, deployment and monitoring.