Shahzad Ali

Shahzad Ali · Learning notes

The evolution of GPT.

Selected milestones from 2018 to GPT-5.6 in 2026.

My notes on what changes between generations: scale, training, inputs, reasoning and the software around the model.

SHAHZAD ALI / MY NOTES / 05AI, in pictures

From predicting text
to working with tools.

Scale, training, modalities and the surrounding software have all evolved.

  1. 01

    GPT-1

    About 117 million parameters. Pre-training on text, followed by task-specific fine-tuning.

    Release source ↗
  2. 02

    GPT-2

    The largest model has 1.5 billion parameters. Full release followed on 5 November 2019.

    Release source ↗
  3. 03

    GPT-3

    The largest model has 175 billion parameters. Prompt examples help specify the task.

    Release source ↗
  4. 04

    GPT-4

    Text and image inputs with text output. Image access was initially limited.

    Release source ↗
  5. 05

    GPT-4o

    Text, vision and audio within an omnimodal model. Product access rolled out in stages.

    Release source ↗
  6. 06

    GPT-5

    The ChatGPT system combined fast responses, deeper reasoning and a router.

    Release source ↗
  7. 07

    GPT-5.6

    Sol, Terra and Luna. Ultra introduced a default four-agent setup for demanding tasks.

    Release source ↗
What I look for

Useful capability+Reliable results+Cost per completed job

Selected releases, 2018–2026. A benchmark result belongs to a specific model, test, tool setup and reasoning effort.

  1. Parameters and training

    Scale is one part of the story

    Early GPT releases grew from roughly 117 million to 1.5 billion to 175 billion parameters. A parameter is a number adjusted during training.

    Data, training objectives and later training also shape behaviour. Parameter counts alone give a limited picture of usefulness. Later releases do not provide a directly comparable public count.

    How pre-training works
    OpenAIOpenAIBrown and colleaguesOpenAI

  2. Capabilities

    The model and the application both matter

    A model has specific input and output capabilities. The application can connect it to search, voice, code execution, memory and other models.

    When I compare products, I want to know which model, tools and access level produced the result. A familiar product name can cover several components.

    The layers behind ChatGPT
    OpenAIOpenAIOpenAI

  3. Reasoning and agents

    Effort and delegation can work together

    Reasoning effort guides how much computation a supported model spends working through a response. Delegation assigns separate tasks to other agents.

    Some products package both. OpenAI describes GPT-5.6 Ultra as coordinating four agents by default. More work uses resources; the outcome still needs checking.

    How an agent works
    OpenAIOpenAI

  4. Reading benchmarks

    A score needs its test conditions

    A useful comparison names the model snapshot, benchmark version, tools, prompting and effort. Results from different settings can answer different questions.

    For my own work, I compare models on a representative task and check the quality of the final result, the time taken and the cost.

    How to evaluate an AI system
    OpenAIOpenAI

  5. Tokens and pricing

    Cost belongs to the whole job

    Input tokens, output tokens, cached input and tool use can have different prices. Reasoning tokens also contribute to usage. Retries and extra agents add to the total.

    For a decision today, I would read the current model documentation and measure cost per accepted result. Static price tables become outdated.

    What a token is
    OpenAIOpenAI

  6. Use and later training

    Context influences this conversation

    The current conversation supplies context while the model’s trained parameters usually stay fixed. Content can contribute to later training according to the product and data settings.

    Personal users can opt out; Temporary Chats are excluded from training. Business and API data are excluded by default. Sharing feedback can carry separate conditions.

    What context means
    Brown and colleaguesOpenAI

Sources behind the technical details

Sources for these notes.

Reviewed . Technical details are linked to their sources. My perspective on how I use AI is personal.

  1. Primary source · First released

    Improving language understanding with unsupervised learning ↗

    OpenAI. The original GPT research and training approach.

  2. Primary source · First released

    Better language models and their implications ↗

    OpenAI. Announces GPT-2 and its largest 1.5-billion-parameter model.

  3. Primary source · First released

    GPT-2: 1.5B release ↗

    OpenAI. Dates the full-model release separately from the initial announcement.

  4. Primary research · First released

    Language Models are Few-Shot Learners ↗

    Brown and colleagues. Describes the 175-billion-parameter model and learning from prompt examples with fixed model parameters.

  5. Primary source · First released

    GPT-4 research ↗

    OpenAI. Explains the multimodal model, initial availability and evaluation limitations.

  6. Primary source · First released

    Hello GPT-4o ↗

    OpenAI. Describes the model and the staged rollout of its capabilities.

  7. Primary source · First released

    Introducing GPT-5 ↗

    OpenAI. Describes the unified product system and labels its individual evaluations.

  8. Developer announcement · First released

    GPT-5.6 release ↗

    OpenAI. A dated account of the family, its Ultra orchestration and developer-reported evaluations. Pricing updates appear on the page.

  9. Living documentation · Living documentation

    Reasoning models ↗

    OpenAI. Explains reasoning effort and reasoning-token usage. Available settings depend on the model.

  10. Living documentation · Living documentation

    GPT-5.6 Sol model reference ↗

    OpenAI. Current endpoint modalities, supported features and separate input/output pricing. Reviewed on the date shown above.

  11. Living documentation · Living documentation

    How your data is used to improve model performance ↗

    OpenAI. Product-specific data controls, defaults, Temporary Chats and feedback exceptions.

← Back to my computing notes