How a Gen AI model reaches production

When a team says “the model output looks right, but nobody agrees on it,” the problem is rarely the model. It is usually the pipeline and the definitions around it. This piece walks through the seven stages every production model passes through, so the conversation can start from the right place.

Stage 1: Data collection

Everything starts with data. Trillions of tokens from the web, books and code are collected, cleaned and tokenized. Frontier training sets run to roughly 15 trillion tokens.

1. Web crawl and scraping. Books, code repositories, websites.
2. Cleaning and filtering. Remove duplicates, personal information and toxic content.
3. Tokenization. Text is split into sub-word tokens, for example with byte-pair encoding.
4. Dataset mixing. Balance across languages, domains and code types.

Stage 2: Pre-training

The model learns language by predicting the next token across trillions of examples on large GPU clusters.

1. Objective. Next-token prediction: given these words, what comes next?
2. Architecture. A transformer with an attention mechanism.
3. Scale. Thousands of GPUs, weeks of compute, billions of parameters.
4. Output. A base model that can complete text.

Stage 3: Alignment

Base models are capable but unrefined. Alignment teaches them to follow instructions, be honest and avoid harm.

1. Supervised fine-tuning. Train on curated instruction and response pairs.
2. RLHF. Reinforcement learning from human feedback to improve outputs.
3. Constitutional AI. Self-critique guided by written principles.
4. Red teaming. Adversarial testing to find weaknesses.

Stage 4: Evaluation

Before deployment, models are measured across benchmarks, safety tests and real-world tasks.

1. Benchmark suites. MMLU, HumanEval, MATH and others.
2. Human evaluation. Blind comparisons by human raters.
3. Safety testing. Bias, toxicity and refusal behaviour.
4. Domain tests. Accuracy on code, medical and legal tasks.

Stage 5: Infrastructure

Serving a model at scale needs optimised infrastructure, from accelerators to load balancers.

1. Accelerator clusters. GPUs such as H100 and A100, or custom TPUs.
2. Model optimisation. Quantization, distillation, KV-cache tuning.
3. Serving frameworks. vLLM, TensorRT, Triton.
4. Auto-scaling. Absorb traffic spikes without degrading latency.

Stage 6: Deployment

The model is wrapped in APIs, safety filters and monitoring before it reaches users.

1. API gateway. Rate limiting, authentication, versioning.
2. Safety filters. Input and output moderation.
3. A/B testing. Compare model versions in production.
4. Rollout. Canary, then staged, then full.

Stage 7: Monitoring

After launch, continuous monitoring protects quality, catches drift and feeds the next round of improvements.

1. Latency and throughput. Response times and capacity.
2. Quality. User satisfaction and task completion.
3. Drift detection. Watch for output degradation over time.
4. Feedback loops. User reports flow back into alignment.

What this means for data teams

Six of these seven stages are data engineering problems: collection, cleaning, evaluation datasets, serving infrastructure, telemetry and feedback capture. The teams that ship reliable AI are the ones that treat the model as one component in a pipeline they already know how to run.