A hands-on guide to AI field engineering: learn how inference platforms work, then run nineteen labs from your Mac — seven against Fireworks for a few dollars, nine locally for nothing, three on how models are trained — and eighteen narrated videos that explain every concept underneath them.
Field engineers build proofs of concept, benchmark systems, debug production issues, and explain tradeoffs to customers. Each skill below maps to a lab you can run.
| Field engineering task | What to learn | Where to practise |
|---|---|---|
| Build end-to-end POCs inside customer codebases | Can you ship working code under someone else's constraints, fast? | Capstone |
| Architect inference foundations and size deployments | Memory maths, concurrency, GPU shapes, cost per million tokens | L3 · L5 · F1 |
| Run load tests; set latency, throughput and cost baselines | Do you benchmark properly — traffic profile, p95, goodput? | L3 · F1 |
| Deploy new model families on vLLM and SGLang; pick shapes, quantization, serving patterns | Hands-on-keyboard with the open serving stack | L2 · L4 · L9 |
| Guide model selection, fine-tuning strategy (SFT, DPO, RFT) and evaluation | Do you know which method fits which problem — and the cost of each? | F5 · F6 · L8 · T1–T3 |
| Build and run fine-tuning pipelines with customers | Dataset format, job launch, deploy, measure, iterate | F5 · L8 |
| Design evaluation frameworks measuring production quality | Not MMLU: task accuracy, tool-call success, latency, cost | F6 |
| Lead discovery; earn trust with ML engineers and VPs in one meeting | Two registers, one story. Practise both out loud | Talk tracks |
| Turn recurring pain into product proposals | Have you got an opinion about what's missing in the platform? | Capstone memo |
Read all three before the loop. Quoting one accurately, then asking a good question about it, is the cheapest credibility you will ever buy.
The platform splits into four jobs: serve a model, train a model, evaluate it, and route between models. Underneath all of it sits their own inference engine — the reason the pitch is "specialized intelligence you own", not "cheap GPUs".
Contrast to hold in your head: a raw GPU cloud sells you the bottom row and hands you the rest. Fireworks sells everything except your application — and increasingly, the training loop too.
Per-token API, OpenAI-compatible, Standard / Priority / Fast tiers. Billed on input, cached input and output separately.
Default starting point. No provisioning, no idle cost.
Private replicas billed per GPU-second, autoscaling, scale-to-zero (1 hour idle by default — shorten it).
Needed for LoRA serving, custom shapes, speculative-decoding flags.
JSONL in, JSONL out, 50% of serverless price, caching discounts stack. 12–72h windows.
Where your eval runs and bulk jobs belong.
Committed GPU capacity, lower hourly price, sold through sales.
Bills for the whole term whether used or not — never suggest it casually.
SFT, DPO, ORPO and RFT as managed jobs via UI, firectl or REST. Priced per 1M training tokens.
The "own your weights" story, and the cheapest real fine-tune you can run.
Custom training loops in Python — custom loss, distillation, inference in the loop. Serverless (per token) or dedicated (per GPU-hour) compute.
GA in Aug 2026. This is what "Fireworks Training" means now.
Their open-source, trace-first eval framework (pip install eval-protocol). Rule-based, LLM-judge or hybrid rewards; feeds RFT.
The bridge from "we think it's better" to a reward function.
One base deployment, many adapters (default quota 100). Needs a BF16 shape with --enable-addons; FP8/FP4 shapes can't host adapters.
How per-customer fine-tunes stay affordable.
Their own kernels. V4 targets B200 with NVFP4 and claims 250+ tokens/sec.
The answer to "why not just run vLLM ourselves?"
Default model-based speculation, custom --draft-model, n-gram, or predicted outputs. Measure with perf_metrics_in_response.
Docs warn a bad drafter makes things slower — good nuance to voice.
JSON mode and JSON-schema mode (2020-12, $ref/$defs, no external refs), plus grammar mode.
Two docs rules: put the schema in the prompt too; for reasoning models omit response_format.
The engine runs inside the customer's own VPC, across many clouds and regions.
The answer to data-residency objections in regulated APAC accounts.
Task-level, cache-aware routing across open and closed models, with centralized cost controls.
Reframes "which model?" as a portfolio question. Great discovery hook.
| Serverless, $ per 1M tokens | input | cached | output |
|---|---|---|---|
| Nemotron 3.5 Lightning 30B A3B | 0.05 | 0.01 | 0.20 |
| GLM 5.3 Flash | 0.15 | 0.03 | 0.50 |
| gpt-oss 120B | 0.15 | 0.015 | 0.60 |
| DeepSeek V4.1 Flash | 0.30 | 0.006 | 1.20 |
| GLM 5.3 | 1.40 | 0.26 | 4.40 |
| Qwen 3.8 Max | 2.00 | 0.25 | 6.00 |
| Kimi K3 | 3.00 | 0.30 | 15.00 |
| Size tier <4B / 4–16B / >16B | 0.10 · 0.20 · 0.90 flat | ||
| On-demand GPUs | $ / hour |
|---|---|
| H100 80GB · H200 141GB | 8.00 |
| B200 180GB | 13.00 |
| B300 288GB | 15.00 |
| GB300 288GB | 20.00 |
| Billed per GPU-second, no start-up charge. Region-pinned (US/EU/APAC) costs 1.5×. | |
| Managed training, $ / 1M training tokens | LoRA SFT | LoRA DPO | Full SFT |
|---|---|---|---|
| ≤16B | 0.50 | 1.00 | 1.00 |
| 16–80B | 3.00 | 6.00 | 6.00 |
| 80–300B | 6.00 | 12.00 | 12.00 |
| >300B | 10.00 | 20.00 | 20.00 |
You can do every Fireworks lab here for roughly the price of lunch — as long as you understand one asymmetry: per-token endpoints cost cents; a GPU deployment costs dollars per hour, whether or not you use it.
firectl signin firectl whoami firectl quota list # see GPU + LoRA + spend quotas firectl quota update monthly-spend-usd # hard pause at 100%, warning at 80% firectl deployment list # should be empty when you finish for the day
Break-even compares the monthly serverless bill against one H100 on-demand replica at $8/hour running 24×7 ($5,840/month). Below that line, per-token is almost always the right answer — and it is the honest answer to give a customer.
A realistic first fine-tune — 2,000 examples, 600 tokens each, 2 epochs of LoRA SFT on an 8B model — is about 2.4M training tokens, which is roughly $1.20. The serving window is what you actually have to watch.
Every lab below has a folder in the field-engineer-labs repo. The page explains what to do and why; the folder holds commented code that you run in order, and every script saves its numbers to results/. The capstone turns those numbers into a customer sizing memo.
The matching 3-minute video, for the idea in plain words.
The lab card here: the point of the lab, the steps, the line to say.
cd lessons/NN-…, read its README, run the scripts. Try --target mock first, then the real server.
The numbers land in results/*.csv. Lesson 15 builds the memo from them.
cd field-engineer-labs python3 -m venv .venv && source .venv/bin/activate pip install -r requirements.txt && pip install -e . cp .env.example .env make mock & # offline fake LLM that behaves like a real one make smoke # every lesson, end to end, ~2 min make setup # then: Ollama, llama.cpp, MLX on your Mac
Every script takes the same switch, so one harness measures every backend:--target mock | ollama | llamacpp | mlx | fireworks | spark
felab/ shared toolkit (targets, timing, results, graders) mock_server.py offline server: prefill, batching, prefix cache, JSON data/ the triage dataset used by lessons 07–11 lessons/NN-*/ README + commented scripts, in course order results/ your measurements → the capstone memo scripts/ smoke_test.sh (also runs in CI) Makefile make help
| # | Lesson and folder | Lab card | Video | Cost |
|---|
Only lessons 10, 13 and (optionally) 18 create billed Fireworks deployments. Lesson 10 ends in a teardown script, and lesson 13 deletes its deployment automatically on exit. Run make fw-check at the end of every session.
Each lab is scoped to one practical field engineering skill, and ends with something you can show or say. Run them in order, after the guardrails checklist in the previous section.
Use the Mac for iteration and the Spark for the server stack. Both are unified-memory machines, so the same sizing logic applies to each — only the bandwidth changes. The Spark is compute-rich and bandwidth-poor: 128 GB of unified memory at 273 GB/s, with prefill performance far beyond its price class and decode speed that is firmly mid-range. That combination makes it an unusually good teaching machine — the bottleneck is visible in every measurement.
Pick your own Mac in the device list — the same arithmetic governs Apple silicon, which is also unified memory. For dense models, measured single-stream decode lands at roughly 60–85% of the ceiling. For mixture-of-experts models the file is much bigger than the bytes actually read per token — which is exactly why gpt-oss-120b (59 GiB on disk) decodes faster on a Spark than a dense 14B model.
Invert the formula on an MoE model and you can discover its active footprint in front of someone: gpt-oss-120b measures about 55 tokens/sec on a Spark, so 273 ÷ 55 ≈ 5 GB read per token — out of a 63 GB file. Under 10% of the model is touched per token. That single sum explains MoE better than any diagram.
| Published DGX Spark measurements | Backend | Prefill tok/s | Decode tok/s |
|---|---|---|---|
| Llama 3.1 8B · NVFP4 | TensorRT-LLM | 10,257 | 38.7 |
| Qwen3 14B · NVFP4 | TensorRT-LLM | 5,929 | 22.7 |
| gpt-oss 20B · MXFP4 | llama.cpp | 3,670 | 82.7 |
| gpt-oss 120B · MXFP4 | llama.cpp | 1,725 | 55.4 |
| Llama 3.1 8B · FP8 · batch 1 → 32 | SGLang | 7,991 | 20.5 → 368 aggregate |
| Llama 3.1 70B · FP8 · batch 1 | SGLang | 803 | 2.7 |
| Qwen3 235B · NVFP4 · two Sparks | TensorRT-LLM | 23,477 | 11.7 |
NVIDIA's own guidance: the host OS, driver and CUDA move together on a fixed cadence; get new features from NGC containers instead.
The vLLM pip path on arm64 has bitten people (torch/CUDA mismatches, --enforce-eager costing 20–30%).
Unified memory means the usual memory fields report N/A. Use free -h, htop, or the DGX Dashboard on localhost:11000.
Don't debug an OOM with the wrong instrument.
sudo sh -c 'sync; echo 3 > /proc/sys/vm/drop_caches' appears in four of NVIDIA's own playbooks.
Page-cache pressure in a unified pool shows up as mysterious slowdowns.
export BNB_CUDA_VERSION=130 before QLoRA runs.
Straight from NVIDIA's fine-tuning guide: CUDA 13.1 isn't supported yet.
-DCMAKE_CUDA_ARCHITECTURES=121a-real
GB10 is compute capability 12.1; generic builds miss it.
NVIDIA ships ~65 DGX Spark playbooks (vLLM, SGLang, TensorRT-LLM, llama.cpp, Unsloth, PyTorch fine-tune, NVFP4 quantization, two-Spark clustering).
They pin working container tags — which is most of the battle on arm64.
Everything here runs on an Apple-silicon Mac with Ollama, llama.cpp and MLX. The last lab is the Spark track over SSH, for the parts that need real server software: vLLM, SGLang, NVFP4 and multi-node. Same benchmark script throughout, pointed at different base URLs.
These pair with the training deep dive, videos 14 to 18. T1 shrinks each algorithm until you can see it working, in numpy, in seconds. T2 runs full fine-tuning, LoRA and DoRA on a real model on your Mac. T3 turns realistic failures into preference pairs and runs DPO locally, and optionally on Fireworks.
One workload, taken end to end, is worth more than fifteen disconnected labs. Pick something enterprise-shaped — support-ticket triage, contract clause extraction, a coding agent for an internal SDK — and run the whole arc.
Every concept in this lab book has a 3–4 minute video: what it is, why it matters, how you will run it, with worked examples. Videos 14–18 are a training deep dive: full fine-tuning, SFT, LoRA and its family, RLHF and DPO. Rendered with Remotion, narrated with Kokoro, subtitled, and delivered as MP4 files in this conversation. Each one opens with what you will learn and closes with a three-point recap and the lab to run next.
| # | Video | What it explains | Then run |
|---|---|---|---|
| 1 | inference-101 | Prefill vs decode, TTFT, ITL, throughput, goodput | F1 |
| 2 | kv-cache | KV maths, PagedAttention, FP8 KV cache | L5 |
| 3 | gpu-bandwidth | Roofline, decode ceiling, device comparison | L3 · L4 |
| 4 | quantization | FP16 → FP8 → 4-bit, NVFP4, KV quantization | L4 |
| 5 | batching | Continuous batching, chunked prefill, speculation | L3 |
| 6 | training | SFT, LoRA, DPO, RFT and how to choose | F5 · L8 |
| 7 | offloading | Sizing formula, offloading, MoE, TP vs PP | L7 |
| 8 | platform | Serving layers, capacity modes, maturity curve | Capstone |
| 9 | serving-stack | What an inference server does; why containers; the five flags | L2 · L9 |
| 10 | benchmarking | Traffic profiles, concurrency sweeps, percentiles | L3 |
| 11 | prefix-caching | Why agent traffic is mostly repeats, and how to exploit it | L6 · F2 |
| 12 | qlora | Frozen 4-bit base + adapter; data and evals decide it | L8 · F5 |
| 13 | scale-out | Speculative decoding, acceptance rate, what a second box buys | L9 |
| 14 | full-finetuning | The training loop; every weight moves; ≈16 bytes/param; ZeRO, FSDP, checkpointing; when it is worth it | T1 · T2 |
| 15 | sft | Data format, loss masking, how much data, the knobs, reading loss curves, what SFT can't do | T2 · F5 · L8 |
| 16 | peft | LoRA's B·A, adapter arithmetic, QLoRA, DoRA, adapters, prefix tuning, IA³, multi-LoRA economics | T2 |
| 17 | rlhf | Why preferences, reward models (Bradley-Terry), PPO, the KL leash, reward hacking, GRPO and RFT | T1 |
| 18 | dpo | The DPO insight and loss, one step with numbers, pairs from your own product, IPO/KTO/ORPO/SimPO | T3 |
The delivered MP4s were rendered on Linux with the ONNX build of Kokoro. On your Mac, mlx-audio runs the same Kokoro-82M weights on the Apple GPU — faster, and it is the path to use if you want to change the script, the voice or the pacing.
cd fireworks-explainers npm install npx remotion browser ensure # narration on Apple silicon (MLX) pip install mlx-audio soundfile numpy python scripts/tts_mlx.py --voice af_heart --speed 0.96 # all 18 python scripts/tts_mlx.py --video kv-cache # just one npm run dev # Remotion Studio, preview with audio npm run render # out/<id>.mp4, subtitles burned in
src/narration.json is the single source of truth: narration text, scene order and the visual each scene uses.
Re-run TTS, re-render. Subtitles regenerate from the same text.
Each scene gets a 0.35s lead-in and a 0.7s tail (1.1s on titles and recaps), and speed is 0.96.
Change them in scripts/tts_*.py if you want it brisker.
Sentence-level cues in a fixed 200px band below the content area, timed by character count across each scene's audio.
The band is reserved, so captions can never cover a diagram.
Add an entry to narration.json; a composition appears automatically in the Studio.
New visuals go in src/components/Visuals.tsx and are referenced by name.
| Concept | Video | Lab | The sentence that shows you've run it |
|---|---|---|---|
| Prefill vs decode | inference-101 | F1 | "TTFT is prefill plus queue; ITL is decode. Which one is the customer complaining about?" |
| KV cache sizing | kv-cache | L5 | "Weights fit. A single 128K-token request needs ~40 GB of KV on top — that's the real limit." |
| PagedAttention | kv-cache | L5 · L9 | "Fixed-size blocks and a block table, like OS paging — 2–4× more concurrent requests." |
| Memory bandwidth | gpu-bandwidth | L4 | "Decode ceiling is bandwidth over bytes per token. On a Spark that's 273 over the model size." |
| Roofline / batching | gpu-bandwidth | L3 | "Batch 1 is memory-bound; I sweep concurrency and report the curve, not one number." |
| Quantization | quantization | L4 | "FP8 is close to free on Hopper and Blackwell; 4-bit needs an eval before I'd ship it." |
| Prefix / prompt caching | prefix-caching | F2 · L6 | "Their system prompt is 6K tokens on every call — caching removes most of the prefill cost." |
| Speculative decoding | batching | F7 | "Same output distribution, fewer big-model passes — and a mismatched drafter makes it slower." |
| SFT / LoRA | training | F5 · L8 | "One base deployment, many adapters — a fine-tune per customer at base-model economics." |
| Full fine-tuning | full-finetuning | T2 | "Sixteen bytes a parameter to train against two to serve. I right-size and try LoRA before I go full." |
| PEFT family | peft | T1 · T2 | "QLoRA when memory is the limit, DoRA when there's a quality gap, multi-LoRA when there are many tenants." |
| RLHF & reward hacking | rlhf | T1 | "A reward model is a proxy; the KL leash stops the policy gaming it. Checkable tasks get a grader instead." |
| DPO | dpo | T3 | "Same pairs as RLHF, two models instead of four, no RL loop. SFT first, then DPO, and I watch accuracy." |
| DPO vs RFT | training | F6 | "Can you write the answer, compare two, or score one automatically? That picks the method." |
| MoE & active params | offloading | L7 | "Memory is sized on total parameters; speed on active ones. That's why 120B beats dense 30B here." |
| Offloading | offloading | L7 | "Offload is the last resort — you're trading a 273 GB/s pipe for a far narrower one." |
| Tensor vs pipeline parallel | offloading | L9 | "Tensor parallel syncs every layer, so it stays inside a box. Pipeline tolerates slower links." |
| Serving engine internals | serving-stack | L2 · L9 | "API, scheduler, KV manager, kernels — and containers keep the CUDA stack honest." |
| Benchmark method | benchmarking | L3 | "Traffic profile first, then a concurrency curve, then where p95 crosses the target." |
| Agent prefix reuse | prefix-caching | L6 · F2 | "Stable content first, variable last — otherwise there is nothing to cache." |
| QLoRA mechanics | qlora | L8 | "Frozen four-bit base, small adapter. The dataset and the eval set decide the result." |
| Multi-node reality | scale-out | L9 | "Two boxes add capacity, not bandwidth. 235B across two decodes at about 12 tok/s." |
| Platform economics | platform | F3 · capstone | "Serverless until utilisation justifies dedicated; batch for evals; specialise to cut unit cost." |
Days 1–10 cover serving, benchmarking and the capstone. Days 11–12 are the training track (videos 14–18, labs T1–T3), for when you want depth on how models are trained.
| Question | To a VP | To an ML engineer |
|---|---|---|
| "Why are we slow?" | "Two different problems wear the same word. The wait before the answer starts is one fix; the speed it types at is another. Yours is the first, and it's mostly repeated work we can cache." | "p95 TTFT is 2.8s with a 6K static prefix and no prefix caching. Enable it, add chunked prefill, and re-run the same traffic replay." |
| "Should we fine-tune?" | "Only if you can measure what 'better' means. Give me 100 examples you'd defend in front of a customer and I'll tell you in a week — with the cost." | "SFT for format and domain, DPO if you only have preferences, RFT if the grader is programmatic. LoRA first; full-parameter only if three tests say so." |
| "Why not self-host vLLM?" | "You can, and some teams should. The question is whether serving is a differentiator for you or a tax." | "vLLM is excellent. You'd own kernel upgrades, quantization tuning, autoscaling, multi-LoRA batching and the on-call. Here's what that cost us in hours last quarter." |
| "Can we keep data in-region?" | "Yes — the engine can run inside your own cloud account, so data never leaves your environment." | "BYOC/Virtual Cloud deployment in your VPC; region-pinned dedicated deployments are also possible at a 1.5× rate." |
| "How do we cut cost 3×?" | "Three levers, in order: stop paying for repeated context, move routine traffic to a smaller model, and specialise that model on your data." | "Prompt caching, task-level routing, then a LoRA on a 4–16B base. Measure each with the same eval set and report $/1K tasks." |
Everything here was read on 23 September 2026. Fireworks' catalogue and prices change weekly; NVIDIA's playbooks pin container tags that move. Verify anything you plan to quote.