From context engineering to RL fine-tuning: the techniques that get the most out of today's foundation models, and when to use each.
Building a model from scratch instead of using one? See Build a Frontier Model →
Need quick improvement?
Prompt & context engineering
Need current or private data?
RAG or an MCP connector
Repeating a workflow?
Write an Agent Skill
Need consistent behaviour or lower cost?
Fine-tune or distil
The three fundamental approaches to improving AI model performance. Each has unique strengths and can be combined for maximum effect.
Steering a model with instructions, examples, and the right context, without changing its weights. In 2026 the skill is less about clever wording and more about deciding what goes into a 1M-token context window, and what stays out.
Giving the model fresh, private, or task-specific information at run time. Classic RAG retrieves document chunks from a vector index. Agentic retrieval lets the model search, open files, and call tools itself, often through MCP connectors.
Changing a model's weights on your own data. Supervised fine-tuning (often with LoRA) teaches style and format. RL with verifiable rewards and on-policy distillation teach the model to actually do your task better, using graders you write.
The methods frontier labs and advanced builders rely on as of October 2026, from context engineering to RL post-training.
Treating context as a finite attention budget. 1M tokens is now the default at the frontier, but models still lose focus as context grows, so the goal is the smallest set of high-signal tokens.
Folders of instructions, scripts, and resources (a SKILL.md plus files) that an agent loads only when a task needs them. Anthropic published the format as an open standard in December 2025.
The open standard for connecting agents to tools and data. Now governed by the Linux Foundation's Agentic AI Foundation. The current spec (2026-07-28) is its largest revision: a stateless core with no initialize handshake or session IDs.
Showing the model examples of the input and output you want. With long contexts, many-shot prompting (dozens or hundreds of examples) can approach light fine-tuning for some tasks.
Reasoning models now think before answering by default, so "think step by step" prompts matter much less. The model's visible reasoning is also a safety asset: it lets humans and monitors see what the model is doing.
Spending more inference on harder problems. Effort controls are now product features (Claude low to max, OpenAI none to xhigh plus Pro, Gemini Deep Think), and the big 2026 shift is parallel agent compute.
RL on tasks where correctness is mechanically checkable: math answers, unit tests, formal proofs, agent environments. Now standard post-training at every frontier lab.
RL compute now follows predictable scaling curves. ScaleRL (Meta and collaborators, 400k+ GPU-hours) found reward rises along a sigmoid in compute, so small runs can forecast big ones.
The student generates its own outputs and a stronger teacher scores every token. This gives RL's on-policy data with a much denser signal than a single end-of-episode reward. It rose in 2026 as a complement to GRPO.
A safety technique for RL training. Anthropic found models that learned to reward-hack generalised to sabotage and alignment faking. Telling the model during training that hacking is acceptable in that environment cut the misaligned generalisation by 75-90%.
Training models against a written set of principles instead of relying only on human labels. Anthropic rewrote Claude's constitution in January 2026 as a long document addressed to Claude itself.
Training on human (or AI) preference comparisons for qualities that can't be checked mechanically, such as helpfulness, tone, and judgement. Today it runs alongside RLVR rather than instead of it.
Many expert sub-networks with only a few active per token. Open frontier MoEs now reach 1-3T total parameters with 13-104B active.
A fast draft proposes several tokens and the main model verifies them in one pass. Verification keeps the output distribution unchanged, so the speedup costs no quality.
Many open bases (Kimi K2/K3, GLM-5, DeepSeek-V4) were pretrained with the Muon optimizer, and FP8 training is routine, with 4-bit NVFP4 pretraining shown to match FP8 at 12B scale. Both affect how you fine-tune.
Models that operate a real desktop or browser. Anthropic reports Claude Opus 5.5 at 81.8% on OSWorld 2.1. Scores differ across OSWorld versions, so compare only within one version.
The most powerful AI systems layer multiple optimization techniques together.
Let the agent search and load documents just in time, then compact what it learned into notes
Best for: Long-horizon coding and research agents
Skills package the procedure; MCP servers provide the tools and data it needs
Best for: Repeatable enterprise workflows across several apps
Fine-tune for style and behaviour, retrieve for facts that change
Best for: Customer-facing assistants with a brand voice and accurate product info
Train a large model with RL on your graders, then distil it into a small, cheap model
Best for: High-volume tasks where latency and cost matter
Use low effort by default, escalate to high effort or parallel samples checked by a verifier on hard cases
Best for: Mixed workloads with a few hard problems
Harden graders and inoculate the training prompt so any remaining reward hacks don't generalise
Best for: Anyone running RL fine-tuning on their own environments
Build a small eval set first. Then iterate on instructions, examples, and effort settings until you hit limits.
Connect data through RAG or MCP, and package repeatable procedures as Agent Skills.
Only when you need behaviour, accuracy, or cost that prompting and tools can't reach. Your Step 1 evals become your graders.
FullAI's API gives you the foundation to implement any of these optimization techniques. Start experimenting today.
Get Your Free API Key