Computing.
Computing
What is AI?
A photo app finds dogs
Artificial intelligence is the field of making computers perform tasks such as recognising objects or using language. Amina’s photo app uses an AI system to find pictures containing a dog.
One picture enters
Amina’s beach photograph enters the image-recognition system. The picture is stored as numbers describing tiny coloured squares, called pixels. Those numbers are the information the computer can work with.
Learning helps recognition
This particular system was trained using examples. Training shaped calculations that respond to patterns in pictures. The new beach photograph passes through those calculations, producing an estimate that a dog is present.
A mistaken dog appears
The app adds the beach photograph to Amina’s dog collection. It also includes a picture of a soft toy. The same recognition system has produced one useful match and one mistake.
Several methods share one field
The dog finder illustrates learning-based AI. Other AI systems use rules, search or combinations of methods. All these everyday systems depend on software running on physical computers, with abilities that vary by task.
Dog finder runs on → Computer hardware
AI brings recognition, language and other abilities into computer systems.
Ideas from John McCarthy, Stanford · What is artificial intelligence?
Source-backed explanation. The everyday teaching examples were created for Learn.
AI includes approaches based on rules, search and learning. A system can combine them.
Machine learning is one area within AI. Neural networks are one family of learning models. A chatbot is an application people talk to.
Ability on one task gives limited evidence about ability on other tasks.
What is machine learning?
An inbox learns patterns
Machine learning builds computer models using data. An email filter can learn patterns associated with unwanted messages, then use them to assess new mail arriving in Amina’s inbox.
Examples carry labels
The training collection contains messages marked wanted or unwanted. A delivery update is wanted; a deceptive prize message is unwanted. These recorded labels supply target answers for this learning method.
Delivery update wanted example → Learning filter
Mistakes change the model
The filter predicts labels for training messages. A training program compares those predictions with the recorded labels and adjusts the model. Repeated adjustments can improve its results across the collection.
Learning filter supplies prediction → Training adjustment
Prize message supplies target → Training adjustment
Training adjustment changes settings → Learning filter
Fresh mail tests learning
A new advertising message arrives with wording the filter has never seen. Learned patterns help it estimate that the message is unwanted. Separate test messages reveal how often that judgement succeeds.
The world can change
An important delivery message could resemble an advertisement and be mislabelled. Learning depends on examples and circumstances. Fresh evaluations matter because senders, writing styles and unwanted-mail tactics change over time.
Machine learning uses experience recorded as data to build useful models.
Ideas from Google · What is machine learning?
Source-backed explanation. The everyday teaching examples were created for Learn.
This example uses supervised learning: examples include target answers. Other methods discover patterns in unlabelled data or learn from consequences.
A system can improve its training results while learning patterns that work poorly on new examples. Separate testing helps reveal this.
Results depend on the training data, learning method and conditions of use.
What is an AI model?
A parcel gets an estimate
A model turns supplied information into an estimate or other result. A delivery model uses journey details to estimate when Leila’s parcel will arrive, and the delivery app displays its answer.
Past journeys provide evidence
Recorded journeys contain distances, traffic conditions and actual travel times. During training, these examples help establish relationships between the details of a journey and how long it tends to take.
Traffic changes the input
Leila’s parcel is five miles away. The morning roads are busy, so the app supplies distance and traffic information. The model combines these details using its stored relationships to estimate the journey.
Five miles supplies distance → Delivery model
Busy roads supplies traffic → Delivery model
Delivery model calculates estimate → Longer estimate
A quiet afternoon differs
For a similar journey on a quiet afternoon, the traffic input changes. The same stored model can produce a shorter estimate. A different result can come from different information supplied to it.
Five miles same distance → Same model
Quiet roads changed traffic → Same model
Same model revised estimate → Shorter estimate
A missing closure matters
An unexpected road closure delays the parcel. If that information never reaches the model, its estimate may be inaccurate. The app’s data collection and the model’s calculations both affect the final result.
A model stores relationships that software applies to new inputs.
Ideas from Google · Linear regression
Source-backed explanation. The everyday teaching examples were created for Learn.
A parameter is a stored setting within a model. Training a numerical model adjusts these settings to improve its results.
The wider application collects inputs and presents results. A single application can contain several models and other software.
Unexpected road closures can make an estimate inaccurate when the model lacks that information.
What does generative AI generate?
An invitation becomes possible
Generative AI creates content such as text, pictures or sound. Amir uses a writing system to turn a few birthday-party details into an invitation that friends can read.
Three details enter
Amir supplies Saturday, his garden and a two o’clock start. These details enter with a request for a friendly invitation. They form the information available for this particular writing task.
Saturday supplies day → Invitation request
The model creates wording
The model uses patterns learned during training to generate wording that fits the request. Its output develops into sentences describing the party, combining Amir’s supplied details with the model’s learned ways of writing invitations.
Cake appears in the draft
The draft mentions Saturday, the garden and two o’clock. It also promises cake, although Amir supplied no food details. A phrase can fit a birthday invitation while adding an unsupported claim.
Supplied details supported content → Invitation draft
Cake promised unsupported addition → Invitation draft
Creation and accuracy differ
Amir removes the cake promise before using the invitation. The system has helped create wording, while comparison with the original details catches an error. Generative systems can also create other kinds of content.
Amir corrects wording → Checked invitation
Generative AI creates content whose details still need assessment.
Ideas from Google · What is machine learning?
Source-backed explanation. The everyday teaching examples were created for Learn.
Generative AI describes what a system produces. The underlying methods differ between models and kinds of content.
Generated content can combine familiar patterns in new ways. It can also repeat material encountered during training.
A convincing result can contain invented details, errors or copied material.
What is a large language model?
A long message becomes shorter
A large language model is a large neural network trained to work with language. Amina’s writing app uses one to turn her long explanation of lateness into a shorter message.
The request includes a fact
Amina’s message says she will arrive at four because her train is delayed. The app supplies this text and the request to shorten it. Together they form the model’s current context.
Train delayed message content → Language model
Small pieces form words
Many chat language models generate text one token at a time. A token is a piece of text, sometimes part of a word. The model calculates possible next pieces from the available context.
The growing reply matters
The selected pieces become part of the sequence used for later calculations. The reply can grow into a short sentence about arriving at four. Each new piece is generated with the available earlier text.
Fluency can hide a change
The app might produce a smooth sentence saying five instead of four. That would change Amina’s meaning. Generating plausible language and preserving a supplied fact are related capabilities with different possible failures.
Original message original fact → Time mismatch
A language model uses learned relationships and current text to generate a response.
Ideas from Google · Introduction to large language models
Source-backed explanation. The everyday teaching examples were created for Learn.
Large refers to scale, including many learned settings. The term has no single universally agreed size boundary.
A language model can support many applications. A chatbot supplies a conversational interface around a model and other software.
Language fluency and factual accuracy are separate qualities that need separate assessment.
What is a prompt?
The request gives direction
A prompt is the input supplied to a generative AI system. Leila wants an invitation to a picnic. Her prompt tells the system what she wants written and which details it can use.
A brief request leaves gaps
Leila initially supplies only the words picnic invitation. The system knows the kind of text requested, while details such as the date, meeting place and intended audience remain unspecified.
Picnic invitation leaves open → Place unspecified
Concrete details narrow the task
The revised prompt includes Sunday at noon, the park gate and a friendly message for neighbours. These additions give the system information it can incorporate into the invitation it generates.
Sunday noon supplies time → Revised request
Park gate supplies meeting point → Revised request
Neighbours supplies audience → Revised request
The format shapes the reply
Leila’s prompt also asks for two sentences. The model now receives both the party facts and a requested length. Its draft can reflect these instructions while drawing wording from learned language patterns.
Specificity helps checking
The resulting invitation names Sunday, noon and the park gate. Those details can be compared with Leila’s prompt. A clear request supports a clear comparison, while the system can still misread or omit information.
Leila’s details intended facts → Details compared
A prompt supplies the task and information available for a generated response.
Ideas from Google Cloud · Introduction to prompting
Source-backed explanation. The everyday teaching examples were created for Learn.
A prompt can contain a task, relevant facts and a requested format. Each supplies a different part of the specification. The facts constrain the content, while the format constrains its presentation.
Explicit success criteria make the response easier to assess. An invitation can be checked against the supplied place, time, audience and length.
A clear prompt improves the specification of a task. It provides no guarantee of correctness.
What is RAG?
A handbook supplies the answer
Retrieval-augmented generation, called RAG, combines finding information with generating an answer. A workplace assistant can search a staff handbook before answering Amina’s question about how to report a broken laptop.
Amina asks about laptop → Work assistant
The search finds a section
The assistant’s search system looks for handbook passages about damaged equipment. It retrieves the section saying that staff report broken laptops through the help desk, with the device’s identifying number.
The passage joins the request
The retrieved section is supplied alongside Amina’s question. The language model now has specific workplace information available in its context, so its answer can use details from that passage.
Amina’s question question context → Language model
The reply names a route
The assistant generates a reply directing Amina to the help desk and mentioning the device number. A source link can let the reader inspect the handbook passage behind that answer.
An old handbook can mislead
If the retrieved handbook is outdated, the reply may repeat an old reporting route. Retrieval quality, source freshness and faithful wording all matter. Supplying a passage gives useful evidence whose relevance still needs assessment.
Current handbook reveals mismatch → Outdated route
Retrieval gives a model relevant source material for its current answer.
Ideas from Anthropic · Contextual retrieval
Source-backed explanation. The everyday teaching examples were created for Learn.
Put three imaginary notices on a desk: opening hours, parking and café prices. Choose which one helps with closing time. You have made the retrieval step visible.
Now change the hours on the notice. The source-backed answer should change too. This exercise helps separate finding information, reading it and checking the response.
Retrieval can miss the right document, find outdated information or be misread by the model. Source links still need checking.
What is an AI agent?
A goal needs several steps
An AI agent uses a model to choose steps and use available tools while pursuing a goal. Leila’s assistant searches a folder of receipts and calculates the total for a particular trip.
A search tool finds receipts
The model chooses to search for the trip’s dates. The application runs the permitted search tool and returns matching receipts. These results give the agent information for deciding what to do next.
Receipt assistant requests search → Folder search
Folder search returns matches → Matching receipts
Matching receipts supplies results → Receipt assistant
A calculator adds amounts
After reading the amounts, the model requests a calculation. The calculator adds the supplied numbers and returns a total. The tool performs the arithmetic while the agent coordinates the sequence of work.
Receipt assistant supplies amounts → Calculator tool
Calculator tool adds amounts → Calculated total
Calculated total returns result → Receipt assistant
One duplicate changes the result
Two files contain the same hotel receipt. If the assistant includes both, the calculator accurately adds an inflated list. Tool accuracy and the quality of the agent’s decisions both affect the final total.
Duplicate hotel bill counted twice → Receipt list
Receipt list supplies excess amount → Inflated total
Permissions set the boundary
The receipt assistant can search and calculate because those tools are available. Sending payments would require a different capability and permission. An agent’s practical reach depends on its surrounding application and tool access.
Search allowed available tool → Receipt assistant
Calculate allowed available tool → Receipt assistant
Payment permission controls payment access → Receipt assistant
An agent connects model decisions with tools and their returned results.
Ideas from Anthropic · Building effective agents
Source-backed explanation. The everyday teaching examples were created for Learn.
On paper, give the helper only a read-calendar card. Then add a book-appointment card. Discuss how the available actions and the need for permission change.
Agent definitions vary. Tool access expands what a system can do and what mistakes can affect, so boundaries and checks matter.
Why can AI sound certain and be wrong?
A confident answer can fail
An AI hallucination is generated content that is false or unsupported. Amir asks a writing assistant for a book about village history, and receives a confident-looking title with an author and publication date.
The details fit a pattern
The proposed title sounds like a history book. The author’s name and year also look ordinary. A language model can generate a convincing combination because it has learned patterns of how citations are written.
Plausible title fits citation form → Complete citation
A search finds no match
Amir checks the library catalogue for that exact title and author. The search produces no matching record. The generated statement has supplied more certainty than the available evidence supports.
The source claim stays unverified
An unsuccessful catalogue search alone cannot prove that every possible copy is absent. It does show that this check has failed to establish the source. Further evidence would be needed before relying on the citation.
No catalogue match limits confidence → Still unverified
Wording and evidence separate
The assistant’s fluent wording made the citation easy to believe. A verified catalogue entry or the book itself would provide different evidence. Generated confidence can be stronger than the support behind the claim.
Fluent wording presents confidently → Source claim
A plausible generated statement can still be unsupported or false.
Ideas from NIST · Generative AI Profile, confabulation
Source-backed explanation. The everyday teaching examples were created for Learn.
Split an answer into three claims. Beside each, record the source you actually checked. This converts a smooth paragraph into a small evidence exercise.
Ask what would change your decision if a detail were wrong. Spend the most checking effort on those details. A date on an invitation and a medical claim have very different consequences.
Errors have different causes. Checking sources reduces some risks; the reliability required depends on the task.
How do we know an AI system works?
A filter needs a test
Evaluation means checking how well a system performs its intended job. Amina’s team tests an email filter by comparing its suggested labels with known labels for messages kept outside its training collection.
Reserved messages fresh inputs → Email filter
Email filter predictions compared → Known labels
Different mistakes matter
The test collection includes unwanted advertisements and important delivery messages. Sending an advertisement to the inbox is one error; hiding a delivery message is another. Each mistake has a different practical consequence.
Advertisement unwanted example → Email filter
Predictions meet known answers
The team records the filter’s label beside each known answer. This reveals which messages were handled correctly and which kinds were missed. A single overall score can conceal an important pattern of failures.
Predicted labels compared with answers → Mistake list
One success has limited reach
The filter handles short advertisements well but hides several long delivery updates. A successful demonstration using one short message would have missed that weakness. The broader collection exposes a more useful picture of performance.
Short advertisements successful cases → Evaluation report
Fresh tests keep their value
After revisions, another reserved collection helps test whether the filter improved. Repeatedly tuning against the same examples can make that test overly familiar. Real-world changes can also reduce how well earlier results apply.
Fresh test set unseen test messages → Revised filter
Revised filter measured results → New evidence
Evaluation tests a defined capability against relevant evidence.
Ideas from Google · Training, validation and test sets
Source-backed explanation. The everyday teaching examples were created for Learn.
Try adding five examples unlike the first ten. Does the pattern of errors change? Your conclusion should state which cases you checked.
Test results describe the chosen test. New situations, changed data and repeated tuning against the same test can weaken what it tells you.
What is a computer?
A machine counts two bags
A computer is a machine that works with information by following instructions. A shopkeeper has two bags of apples. The computer can use the recorded counts to work out their total.
The apples become recorded counts
The shopkeeper enters 2 for the first bag and 3 for the second. The apples stay on the counter. The computer stores numbers representing how many apples each bag contains.
An instruction selects addition
A program tells the computer to add the two stored numbers. Its processor follows instructions implemented through physical electronic circuits. Those circuits change their states in ways that carry out the calculation.
Five appears on the screen
The calculation produces a stored result representing five. Display software turns that result into the visible symbol 5. The shopkeeper can interpret the symbol as the number of apples across both bags.
The same machine can compare
The shopkeeper can run different instructions to identify the larger bag. The hardware can now compare 2 and 3 and report the second bag. Programmability lets the same physical machine serve different information tasks.
A computer follows instructions to turn represented information into results.
Look through the word computer and ask what each part contributes. Something receives information. Something keeps information available. Something changes it according to instructions. Something makes results available. A keyboard, memory, calculating circuits and screen are familiar examples of these jobs. We will open each one gradually.
The computers in today’s phones and laptops belong to a wider history. People once earned a living doing calculations under the job title computer. Mechanical calculators used moving parts. Analogue computers represented quantities using changing physical amounts. Our main route follows programmable digital computers, which represent information with distinct states.
This opening describes the programmable digital computers in everyday devices; historical computers and other computing machines take several forms.
What does computing mean?
Counting is already computing
Computing means carrying out steps on information to reach a result. Asha has two apples in one bag and three in another. Finding how many she has altogether is a computation.
Two apples first quantity → Asha
Three apples second quantity → Asha
The first bag starts the count
Asha counts the first bag and records two. That number is her running total: the amount counted so far. Recording it lets her continue the task without starting from the beginning.
Second bag waiting still to add → Running total 2
The next bag adds three
She includes the second bag’s three apples. Following the addition rule changes the recorded total from two to five. The rule explains how the starting quantities determine the result.
Total starts at 2 starting amount → Total becomes 5
The result answers the task
Five is the result for the two bags together. Asha could obtain it by counting individual apples or by adding their recorded quantities. Both methods connect the given information with the same answer.
Two apples contributes two → Five altogether
A machine can follow the rule
The same addition can be carried out with fingers, a numbered wheel or electronic circuits. The physical method changes, while the represented quantities and the rule connecting them can remain the same.
Counting fingers counts quantity → Five altogether
Numbered wheel adds mechanically → Five altogether
Computer circuits adds electronically → Five altogether
Computation is rule-guided work that produces a result from given information.
Calculation usually refers to working with numbers, such as adding prices. Computation is broader: it can involve comparing letters, following a route through a map or changing a picture. These tasks become suitable for a machine when their operations can be expressed as steps the machine can carry out.
The starting information and the rule play different roles. Keep the rule “add the two counts” and replace two and three with four and one: the answer is still five. Keep two and three but change the rule to “choose the larger”: the result becomes three.
The examples use small, definite tasks; later lessons explain computations that involve uncertainty, ongoing interaction and practical limits.
When a computer was a person
Computer was a job title
People once worked as computers, carrying out calculations for organisations. In an imagined counting office, Asha receives records of apple deliveries and is paid to turn those figures into daily totals.
Delivery records calculation task → Asha the calculator
Asha the calculator written result → Daily total
Rows arrive on paper
The morning sheet lists deliveries of two, three and four crates. Asha reads each row and follows the office’s addition method. The sheet holds the information while she performs the calculation.
A colleague checks the sum
Asha records nine crates. Another worker calculates from the original sheet and compares the result with hers. Independent checking can expose a copied number or an arithmetic mistake in the office’s work.
Original sheet same source figures → Second calculation
Asha’s total 9 first result → Totals compared
Second calculation second result → Totals compared
A machine takes over addition
The office introduces a calculating machine. Workers enter the figures and read its result. The division of labour changes: the mechanism carries out arithmetic while people prepare inputs and assess the records.
Real workplaces used human computers
The apple office illustrates the job. NASA documents human computers at Langley working with aeronautical research figures before and alongside electronic computers. The history connects human calculation, checking, machinery and changing technical roles.
Computing was human work before the word became associated with electronic machines.
At Langley, women employed as computers analysed experimental measurements and prepared calculations and graphs. This work demanded training and judgement. NASA’s historical account also describes how some moved into programming as electronic machines arrived. Human expertise helped shape the transition to machine computing.
Several practical needs encouraged computing tools: reducing repetitive labour, handling more figures and checking results. Different communities pursued different solutions over centuries. The route to today’s computers includes people, workshops, scientific institutions, businesses and public funding, with failed projects and competing designs along the way.
The apple office is an invented illustration; the cited Langley account documents a particular historical workplace within a much longer history.
What is information?
A bag has several details
Information is the detail available for answering a question. An apple bag can have a quantity, an owner and a destination. Each detail says something different about the same physical bag.
Apple bag ownership described → Belongs to Asha
The label records a quantity
Asha writes 3 on the bag’s label. The label preserves the count after the bag is closed. Someone can read the recorded quantity without seeing each apple inside it.
A name adds another fact
Asha adds her name to the label. The number answers how many; the name identifies whose bag it is. Keeping those fields separate helps prevent a quantity from being confused with a person’s name.
Quantity 3 quantity field → Bag record
Owner Asha owner field → Bag record
A destination guides delivery
The record also names the north stall. A delivery worker uses that detail to choose where the bag goes. Information becomes useful through the relationship between a recorded detail and a task.
North stall names destination → Delivery worker
Delivery worker carries bag → Apple bag
Apple bag delivered to → North stall
The record leaves things out
The label says three apples, Asha and north stall. It says nothing about sweetness or bruising. A record preserves selected details, so its usefulness depends on what someone later needs to know.
Recorded details let people and computers answer questions about the world.
We use the word data for recorded values or symbols prepared for storage, communication or use. A computer can hold a name, a count and a photograph using agreed ways of representing them. The next lesson asks how marks and physical states can stand for those details.
Choosing what to record is already a decision. A shop may record counts and prices while leaving out the apples’ smell. The resulting record supports some questions and leaves other questions unanswered. This is why understanding the origin and meaning of a computer’s data matters.
This lesson uses the everyday meaning of information; information theory later gives a more specialised way to measure uncertainty.
How can a mark stand for something?
Circles can stand for apples
A representation is something that stands for something else. Asha draws one circle for each apple in a bag. Three circles on paper can preserve the quantity while the apples stay elsewhere.
Asha records quantity → Three drawn circles
Each mark has a partner
Asha matches the first apple with one circle, the second with another and the third with another. This one-to-one connection is what makes the drawing a record of the bag’s quantity.
Apple collection counted objects → One each
One each matching marks → Circle collection
A shorter symbol works too
She replaces the three circles with the symbol 3. People familiar with that numeral can recover the same quantity from a shorter mark. An agreed meaning connects the written symbol to the apples.
The record can travel
Asha sends the paper count to another stall while keeping the bag. The recipient can use the number to plan a purchase. The representation carries selected information without transporting the apples themselves.
Computers use agreed codes
The stall’s computer stores its own physical pattern for the number three. Software interprets that pattern as a count. Drawn circles, numerals and computer codes connect through the quantities they are chosen to represent.
A representation preserves an agreed feature of something else.
A machine needs representations its parts can reliably distinguish. A wheel can occupy numbered positions. A card can contain holes in chosen places. Electronic components can produce distinguishable electrical conditions. Designers connect those physical differences with values, then arrange mechanisms that transform them.
The same pattern can carry different meanings under different agreements. A number may represent a count, a letter or the brightness of a tiny part of an image. Keeping the agreement consistent lets physical operations on patterns produce useful results about the things those patterns represent.
A representation preserves the features chosen for a task; a count of apples leaves out features such as their size and flavour.
What is an instruction?
A task needs clear actions
An instruction specifies an action to carry out. Asha wants a computer to total two apple bags. A program breaks that job into actions the machine can perform in a defined order.
Asha supplies task → Counting program
The first count is read
The first instruction reads the count recorded for bag A. Its value is two. The computer keeps that value in a working location so a later instruction can use it.
The next count is added
The program reads three from bag B and adds it to the working value. The stored total changes from two to five. Each action uses information prepared by the preceding part of the sequence.
A display instruction follows
After the addition, another instruction asks the display system to show the total. Five appears on the screen. Displaying the value before addition would have shown an earlier state of the calculation.
Order connects the actions
Reading, adding and displaying form one small program. Their order connects inputs to a visible result. Larger programs organise many such actions, including choices about which instruction comes next and when repetition ends.
Instructions specify actions and their order for an available machine or person.
The level of detail depends on the worker. A person already knows how to read a handwritten number. A computing device needs that number supplied in a physical form it can receive. “Add” may itself require several smaller operations inside the machine. Opening those operations reveals another layer of instructions and mechanisms.
A method written as repeatable steps is called an algorithm. A program expresses instructions in a form intended for a computer. People build layers of language and translation so a useful instruction such as sorting names can eventually be carried out through small physical operations.
The bag-counting method assumes readable counts and a way to mark each bag once; those assumptions are part of the method’s setup.
How can physical parts calculate?
A wheel can hold a number
A mechanism is an arrangement of physical parts whose movements affect one another. A numbered wheel and pointer can represent a count: the digit under the pointer shows the wheel’s current position.
A click moves one place
The wheel starts at two. Its shaped teeth and holding mechanism allow one step at a time. Turning through one click brings the next digit, three, beneath the pointer.
One click turns one step → Wheel at 2
Wheel at 2 changes displayed position → Pointer reads 3
Three clicks produce five
Asha turns the wheel through three clicks from its starting position of two. It moves through three and four to five. The mechanical movement follows the same progression as counting three further objects.
The position keeps the result
When Asha releases the wheel, the holding mechanism keeps it in place. Five remains under the pointer. A physical position has both carried out a calculation and preserved its result for reading.
Holding mechanism prevents unwanted turning → Wheel held at 5
Wheel held at 5 stores readable result → Five remains visible
Reliable parts make useful rules
A dependable wheel needs accurate steps and controlled movement. A slipping tooth could produce a wrong result. Electronic computers also depend on reliable physical behaviour, with electrical states taking the role played here by wheel positions.
Mechanical positions implements counting → Represented calculation
A physical arrangement can make its changes correspond to a calculation.
This is the key bridge between an idea and a machine: choose physical states to represent values, then arrange causes that change those states in useful ways. The wheel positions are states. A tooth pushing another wheel is a cause. The arrangement makes the physical outcome follow the calculation’s rule.
Historical calculating engines used mechanical parts to represent numbers and carry out arithmetic. Electronic computers use electrical changes through circuits. The engineering changes, while the central question remains: how can physical behaviour reliably produce the transformations required by the task? Later lessons open circuits, switches and silicon.
The two-wheel example explains the principle; a dependable calculating machine also needs detailed arrangements for timing, carrying and preventing unwanted movement.
How can one machine do different jobs?
One machine can do two jobs
A programmable machine can follow different sets of instructions. Asha’s computer has the apple counts two and three. One program adds them; another identifies which count is larger using the same hardware.
Different programs selects work → Same computer
The first program adds
The addition program tells the processor to combine the two numbers. It uses the machine’s available arithmetic operations and returns five. The program selects which capability is used on the stored information.
The instructions are replaced
Asha switches to a comparison program. The counts remain two and three, but the new instructions ask the processor to compare them. The selected operation has changed while the physical computer remains available.
Same 2 and 3 same input → Comparison instructions
The second program reports larger
The comparison finds that three is greater than two. The screen reports bag B as larger. This answer comes from the new sequence of instructions acting on the same recorded counts.
Bag A 2 compared with B → Bag B larger
Hardware still sets limits
The computer can run each program because its parts support the required operations. Programs depend on available memory, processing and connections. Changing instructions gives flexibility within what the machine can physically carry out.
Selected program directs operations → Processor
Memory supplies stored values → Processor
Processor produces result → Displayed answer
Changing a program can change the job performed by the same hardware.
Programmability has taken several physical forms, including punched cards, switches and stored instructions. Babbage’s proposed Analytical Engine used cards to select operations. Later electronic computers could keep instructions in memory, the part of a computer that holds information for use. That made changing and organising programs much more convenient.
A general-purpose computer provides basic operations that can be combined into an enormous range of tasks. Practical success still depends on having enough time, storage and suitable inputs and outputs. The same underlying machine can support writing, games or calculations through different programs and representations.
Programmability comes in degrees; a device’s available actions and resources determine which jobs its programs can carry out.
Counting outside your head
A bowl remembers the count
Counting can be recorded outside someone’s head. At an imagined apple stall, Asha puts one pebble in a bowl for each bag sold. The bowl preserves how many sales have occurred.
Asha keeps record → Pebble bowl
The first sale adds one
One customer buys a bag. Asha places one pebble in the empty bowl. The bag leaves the stall, while a small physical object remains as a record of that sale.
One bag sold bag delivered → First customer
More sales grow the record
Two more customers buy one bag each. Asha adds two pebbles to the bowl. There are now three pebbles corresponding to three bags sold, even though the customers have carried the bags away.
The count can be checked
At closing time, Asha counts the pebbles and records three sales on paper. The bowl has allowed the quantity to survive across interruptions, provided each sale received exactly one pebble.
The matching rule matters
A missed pebble would undercount sales, and an extra pebble would overcount them. Every recording system depends on a reliable connection between events and records, including electronic systems recording purchases today.
Bag sold event occurs → One record each
One record each record created → Sales record
A recorded quantity lets a count survive beyond the moment of counting.
A representation works because people agree what its states mean. One pebble could mean one sheep, ten sheep or a completed delivery. The physical object stays the same while the interpretation changes. Computers also require rules connecting physical states with meaning.
Counting aids appeared in many cultures, including fingers, counting boards and abacuses. Surviving objects provide pieces of that history, with gaps and uncertain dates. This atlas starts with a teaching example of representation; it makes no claim about a single first inventor.
The pebble story is an invented teaching example, and the earliest human counting practices remain incompletely documented.
Place value and zero
Two hundred and three fits
Place value lets a digit represent different amounts in different positions. Asha’s warehouse records 203 apples. The two counts hundreds, the zero counts tens, and the three counts individual apples.
The apples form groups
Asha groups two hundred apples into two large crates, with three loose apples beside them. The written 2 records two groups of one hundred, while 3 records the loose apples.
Two hundred crates two hundreds → 203 apples
Zero keeps the tens place
There are no complete groups of ten between the hundreds and the loose apples in this representation. The zero holds that tens position, keeping the hundreds digit two places away from the ones digit.
Removing zero changes the amount
Writing 23 moves the two into the tens position. The new numeral describes two groups of ten and three ones: twenty-three apples. A change in position changes the amount represented by the same digit.
The pattern keeps extending
The warehouse can also record thousands by adding another position to the left. Each decimal place is ten times the place beside it. A small set of digits can therefore represent increasingly large quantities.
A digit’s position supplies part of its meaning, and zero preserves empty places.
Binary follows the same grouping idea using groups of two. Its columns are ones, twos, fours and eights. The pattern 101 therefore represents one four, zero twos and one one. Changing the base changes how quantities are written while preserving the quantities themselves.
Positional systems developed through several mathematical traditions. Babylonian place value used sixty as its base. The decimal system with zero developed in India and spread through Arabic scholarship and other exchanges. The historical story includes transmission, adaptation and independently developed ideas.
Zero as a placeholder and zero as a number have related histories with distinct stages.
An algorithm is a repeatable recipe
A method finds the largest
An algorithm is a precise method for carrying out a task. Asha has three apple prices, £2, £5 and £3, and wants the largest. A comparison method can find it reliably.
The first price becomes champion
The method starts by keeping £2 as the largest price seen so far. Only one price has been examined, so that value is the current answer while the remaining prices wait.
£5 and £3 still to compare → Largest so far £2
Five replaces two
The next price is £5. The rule compares it with the stored £2 and replaces the stored value because five is larger. The running answer now reflects both prices examined so far.
Next price £5 larger value kept → Stored £5
Three leaves five unchanged
The final price is £3. Comparing it with £5 leaves the stored answer unchanged. Every price has now been examined, so the method stops and reports five pounds as the largest.
End of list comparison complete → Largest remains £5
The same rule handles more
With a longer list, the method repeats the same comparison until the list ends. Clear starting conditions, a repeated rule and a stopping condition let a person or computer carry out the procedure.
An algorithm makes a task precise enough to carry out step by step.
An algorithm can be studied separately from its machinery. The same maximum-finding method can run on paper, in a mechanical device or inside a computer. Its input and rules determine the answer; the implementation determines practical details such as speed and energy use.
Written calculation procedures long predate electronic computers. Euclid described a systematic method for finding common divisors in ancient mathematics. Later machinery made specified procedures automatic. Historical priority depends on the procedure and definition being discussed, so the atlas avoids assigning one universal starting date.
A precise algorithm can still give an unsuitable result when the problem or input has been defined badly.
Gears that carry the answer
Two wheels display a count
A mechanical calculator uses moving parts to calculate. Two numbered wheels can show tens and ones. Asha’s example machine displays 09, meaning zero tens and nine ones before another apple is counted.
Tens wheel 0 tens digit → Display 09
One click crosses the boundary
Asha adds one by moving the ones wheel one step. After nine, that wheel returns to zero. A connection between the wheels must preserve the extra group of ten represented by that completed turn.
The carry moves another wheel
At the change from nine to zero, a carry mechanism advances the tens wheel one step. The ones wheel now shows zero and the tens wheel shows one, preserving the full count.
Ten appears in the window
The two wheel positions are read together as 10. Adding a single apple to nine has changed two displayed digits. The movement of linked parts has implemented the familiar rule of carrying.
Tens digit 1 one ten → Display 10
Further wheels extend the total
Another wheel could hold hundreds, with another carry connection from the tens wheel. Reliable timing prevents unwanted movement. Historical calculators used varied mechanisms, while this example isolates the physical idea behind decimal carrying.
Ones wheel carries each ten → Tens wheel
Tens wheel carries each hundred → Hundreds wheel
Connected digit wheels can carry a calculation across number positions.
Seventeenth-century calculators, including Pascal and Leibniz designs, explored ways to mechanise arithmetic. Babbage later designed machines for larger sequences of calculations. Their capabilities and construction histories differed. A functioning calculator, a proposed design and a programmable computer are separate historical achievements.
The important connection is physical embodiment. A rule can live in teeth, levers and motion. Electronic circuits later embodied arithmetic using electrical states. The need to preserve state, transform it and carry information between positions survives across these very different implementations.
The wheel example explains decimal carrying, while historical calculators used varied mechanisms with additional engineering complications.
Give one machine different jobs
Cards can direct a machine
Instructions can be stored outside a machine’s working mechanism. In an illustrative card-controlled calculator, a sequence of cards tells the mechanism which action to perform next on two apple counts.
Counts 2 and 3 supplies values → Calculating mechanism
Holes stand for a command
One card’s hole pattern is assigned the meaning add. A reader detects the pattern and activates the corresponding mechanism. The connection between a pattern and an action gives the card its instructional meaning.
The sequence supplies order
An input card loads two and three, the add card requests their sum, and an output card requests a display. Reading them in sequence organises several actions into a complete calculation.
A different card changes work
Replacing the add card with a compare card asks the machine to identify the larger count. The mechanism receives a different instruction sequence and reports three as larger, using its available comparison capability.
A historical design explored this
Babbage planned card-controlled operations for the Analytical Engine, and Ada Lovelace examined its wider possibilities. That historical work helps explain the distinction between a machine’s mechanisms and the instruction sequences that organise their use.
Planned card control planned instructions → Analytical Engine design
Lovelace’s analysis examined possibilities → Analytical Engine design
Separating instructions from machinery lets a machine follow different procedures.
The Analytical Engine was an ambitious nineteenth-century design with separate calculation and storage functions and control through punched cards. It remained unfinished during Babbage’s lifetime. Lovelace’s 1843 notes included a published procedure and explored how symbol manipulation could extend beyond numerical quantities.
Modern programmes preserve this separation at another level. Stored instruction patterns select operations already supported by hardware. Higher-level software combines those operations into increasingly elaborate behaviours. The toy card sequence is an illustration of programmability, with deliberately simplified instructions for easy tracing.
Programmability developed through several machines and designs, so the title of first programmer or first computer depends on historical criteria.
Turn conditions into yes or no
Two conditions control a lamp
Boolean logic combines values with two possibilities, such as true and false. In a demonstration cabinet, a lamp is allowed on only when the door is closed and an enable button is pressed.
Door sensor closed condition required → Cabinet lamp
The door is open
The enable button is pressed, but the door sensor reports open. Under the chosen AND rule, the lamp stays off because only one of the two required conditions is true.
Door open condition false → Lamp off
Both conditions become true
The door closes while the button remains pressed. Now both sensor statements are true. The AND rule gives a true output, so the demonstration circuit allows the lamp to turn on.
Door closed condition true → Lamp on
Releasing the button changes it
The door remains closed, but the button is released. Its statement becomes false and the AND output becomes false. The lamp turns off, showing that either missing condition changes the result.
Door closed condition true → Lamp off
The rule can be written down
All four input combinations can be listed before a circuit is built. Only closed-and-pressed produces on. Real installations also require suitable sensing and engineering, because a logical rule alone cannot guarantee reliable physical behaviour.
Closed and pressed required behaviour → Physical circuit
Boolean rules specify exactly how yes-or-no conditions combine.
Boole developed an algebraic treatment of logic in the nineteenth century. Shannon later connected Boolean algebra to electrical switching circuits in his 1937 master’s thesis. This connection mattered because designers could reason about switching behaviour mathematically before constructing the physical system.
A truth table describes a relationship between inputs and outputs. It leaves the physical implementation open: relays, transistors or a careful person can realise the same table. When you open the gates lesson, you will see how engineers implement this abstract rule using voltages.
A binary condition is useful only when its definition and measurement fit the real situation.
A machine for describing computation
A strip and a rulebook
A Turing machine is a mathematical model of computation. It has a strip of symbol positions, a reading head and rules. An illustrative machine will add one mark to a row of existing marks.
Rulebook specifies next action → Reading head
The head finds a mark
The strip begins with two marks followed by an empty square. The head is over the first mark. In its current state, the rule says to leave that mark and move one square right.
Mark means move move right → Head on first
The rule repeats once
The head reaches the second mark and applies the same rule. It leaves the mark unchanged and moves right again. The machine’s next action depends on the symbol beneath the head and its internal state.
An empty square ends it
The empty square selects another rule: write a mark and halt. The strip now contains three marks. A short rulebook has specified how to increase the represented quantity by one.
An ideal machine reveals limits
Turing introduced this kind of model in 1936 to study what mechanical calculation can accomplish. Its tape can expand as needed in the mathematical model, while physical computers always have finite resources.
Precisely stated rules defines computation → Expandable model tape
Expandable model tape abstract comparison → Finite physical memory
Turing’s abstract machine makes rule-following computation precise enough to study.
A universal Turing machine can read an encoded description of another machine and simulate its steps. That idea separates the general interpreter from the particular procedure being interpreted. It is a mathematical foundation for understanding why one programmable computer can perform many different jobs.
Turing also established limits on what such procedures can decide in general. A closely related modern statement is the halting problem: no algorithm correctly determines whether every possible programme will eventually halt on every input. These are logical limits, independent of processor speed.
The tape model idealises unlimited expandable storage, while every physical computer has finite resources.
One bit, two distinguishable states
Two states can carry a choice
A bit is a value with two possibilities, conventionally zero and one. Asha’s demonstration uses a small lamp: off represents zero and on represents one. The two states can record a simple choice.
Small lamp other possible state → On means 1
One lamp records one answer
Asha uses the lamp to record whether an apple order is ready. Off means waiting and on means ready. An agreed interpretation connects the lamp’s physical condition to that particular meaning.
A second lamp adds patterns
Two lamps provide four combinations: both off, first on, second on, or both on. The ordered pair can therefore distinguish four cases, such as four different stages of the same order.
First lamp first bit → Four ordered patterns
A code chooses the meaning
In Asha’s chosen code, first off and second on means packed. Another code could assign that same pattern a different meaning. Software needs the encoding to interpret the recorded state consistently.
First off first coded position → Order packed
Chips use smaller physical states
The order system could replace lamps with electrical states inside a chip. Other devices store bits through charge or magnetism. The physical method changes while the idea of two distinguishable values remains useful.
Lamp pattern 01 visible encoding → Order packed
Bits acquire meaning through their physical states, positions and agreed encoding.
Each added bit doubles the number of available patterns because every existing pattern can acquire either a zero or a one. Eight bits form a byte with 256 possible patterns. The counting grows by repeated multiplication by two, which the shorthand 2⁸ records.
Electrical signals vary continuously in the physical world. Digital circuits use allowed voltage ranges and tolerance margins to recover stable logical values despite small disturbances. This separation between a physical signal and its agreed interpretation is central to the reliability of digital systems.
A bit describes an abstract two-valued quantity; its physical storage can use voltages, charge, magnetism or other mechanisms.
What is the physical world made from?
A spoon contains tiny building blocks
Ordinary materials are made of atoms. A metal spoon contains vast numbers of them. Each atom has a small central nucleus and electrons occupying the surrounding region, giving us a starting point for understanding electricity.
The centre contains positive charge
Inside an atom, the nucleus contains positively charged protons and usually neutrons. Electrons carry negative charge. Their arrangement and interactions contribute to the behaviour of the material containing the atoms.
Many atoms share a material
In the spoon, atoms belong to a solid structure. Some electrons in the metal can move through the material. This mobility helps explain why a metal can carry electric current when conditions allow.
A field influences that movement
When a suitable electrical circuit creates an electric field in a metal wire, mobile electrons acquire an average drift. Their motion carries charge, connecting the small-scale description of matter with electrical current.
The drawing simplifies the atom
The atom picture explains useful parts of the story but greatly distorts sizes and distances. Electrons have quantum behaviour. Later semiconductor lessons use that behaviour to explain why carefully prepared silicon can support controllable circuits.
The behaviour of electrons within materials helps explain electrical devices.
A neutral atom has equal numbers of positively charged protons and negatively charged electrons. Here, positive and negative name two kinds of electric charge. Equal amounts balance. The next lesson explains how charge produces attractions and repulsions, giving us a physical mechanism to work with.
Our picture of atoms grew through experiments. Chemical measurements revealed repeating combinations; later experiments exposed electrons and a concentrated nucleus. Modern physics describes an electron in an atom using a spread of possible locations. The fuzzy cloud in a diagram represents that spread.
An atom drawing is a simplified model: its sizes and distances are usually exaggerated, and electrons have quantum behaviour.
What makes electricity possible?
Charged particles affect one another
Electric charge is a characteristic of particles such as electrons and protons. It comes in positive and negative signs. Two small charged objects can exert a push or pull on one another.
Opposite signs attract
In our simple drawing, one object has positive charge and another has negative charge. Each experiences an electrical force towards the other. That interaction can change their movement when they are free to move.
Matching signs repel
The second object is replaced by another positively charged object. Now the electrical forces point away from one another. The signs of the charges determine whether this simple pair attracts or repels.
A nearby electron responds
An electron brought near the positive object experiences attraction. An electric field describes the electrical influence at each location. It lets us describe the force that a charge would experience there.
Circuits organise these influences
A battery and connected conductors establish electrical conditions that influence mobile charges throughout a circuit. Understanding charge and force therefore helps explain how electrical devices move energy and represent signals.
Electric charge allows particles and objects to exert electrical forces.
An object can contain charged particles and still have zero total charge. Its positive and negative amounts balance. If electrons transfer between objects, one can gain an excess of negative charge while the other loses some. The total charge of the combined isolated system stays the same.
A field tells us what force a charge would experience at each location. You can picture arrows showing its direction. Engineers arrange materials and voltages to shape these fields. In a transistor, an electric field helps control how easily charge passes through a small region.
Positive and negative are names for opposite charge signs; ordinary objects contain vast numbers of both kinds of charged particle.
What are voltage and current?
A lamp needs electrical conditions
Voltage is an electrical energy difference per amount of charge between two points. Current is charge passing per second. A battery connected to a lamp gives us one concrete way to distinguish them.
Battery terminals voltage maintained → Connecting wires
Connecting wires possible conducting path → Small lamp
The open switch breaks the path
The battery has a voltage between its terminals while the switch is open. In this simple steady circuit, the gap prevents a sustained current through the lamp, so the lamp remains off.
Closing the path permits current
The switch closes and completes the conducting path. An electric field acts on mobile charges in the wire, producing an average drift. The lamp receives electrical energy and begins producing light and heat.
Current counts passing charge
Current measures how much charge passes a chosen point each second. A larger current means more charge crossing that point in the same time. It differs from the speed at which any individual electron drifts.
Charge passing crosses chosen point → Point on wire
One second measurement interval → Charge passing
The components set the result
The battery’s voltage and the connected components together determine the current. A different lamp can respond differently to the same battery. Voltage, current and circuit design must therefore be understood together.
Same battery voltage driving conditions → Resulting current
Voltage describes energy per charge; current describes charge passing per second.
For a tiny numerical picture, one volt means one unit of energy difference for each standard unit of charge. Physicists call those units the joule and coulomb. One ampere of current means one coulomb passes each second. You can understand the distinction without calculating either quantity.
Electrons already occupy the wire before you connect the battery. Their average drift is usually slow, while a change in the electrical conditions spreads much faster. Circuit arrows conventionally follow the direction positive charge would move; electrons in metal drift in the opposite direction.
The water picture helps separate driving conditions from flow; electric fields and charge obey their own physical laws.
How do we make electricity do something?
A circuit gives charge a route
A circuit is an arrangement of connected electrical components. A simple battery, switch and lamp circuit forms a complete path through wires and components, allowing the battery to supply energy to the lamp.
The switch starts open
The switch contacts are separated. That gap interrupts the external conducting path between the battery terminals. The simple circuit has no sustained lamp current, and the lamp is dark.
The contacts meet
Closing the switch joins its contacts. The wires, lamp and closed switch now connect the two terminals, completing the external path. The battery’s internal chemical processes maintain the conditions that drive the circuit.
The lamp changes energy form
While current flows, the lamp converts electrical energy into light and heat. Charge continues through the circuit’s conducting route. The battery’s stored chemical energy decreases as it supplies energy to the connected components.
A small change controls behaviour
Opening the switch again stops sustained current in this simple circuit. One physical change controls the lamp’s state. Computers build on controlled electrical changes, using tiny devices and connected circuits to represent and manipulate information.
Switch opens again introduces control principle → Switching circuits
A complete circuit connects an energy source and components through conducting paths.
The battery spends chemical energy as it powers the lamp. Charge continues through the circuit while energy is transferred into light and heat. In a simple steady loop, the same current passes each component. A battery's internal charge transport involves ions as well as electrons in the outer wires.
Volta's battery, introduced in 1800, gave experimenters a sustained source of current. Repeated experiments became easier than relying on occasional sparks. Over the following century, researchers developed ways to generate, measure and control electricity, creating tools that electronic computing would later use.
This example describes a simple battery circuit; rapidly changing circuits also involve temporary charge storage and electromagnetic effects.
Why choose silicon, copper or plastic?
A cable uses different materials
Electrical devices use materials with different jobs. A lamp cable has a metal centre that carries current and a plastic covering that restricts unwanted current paths. A computer chip adds carefully engineered semiconductor regions.
Metal cable centre conducting route → Connected lamp
Plastic covering insulates outside → Metal cable centre
Copper provides a conducting path
Inside the cable, copper contains mobile electrons that can respond to an electric field. This makes it useful for carrying current between the battery and lamp when the full circuit is connected.
Plastic separates neighbouring connections
The plastic covering strongly restricts charge passing through it under ordinary operating conditions. It helps keep the cable’s conductor electrically separated from neighbouring objects and other conductors that should remain distinct.
Plastic barrier insulates outer surface → First conductor
Silicon adds controllable regions
A semiconductor such as silicon can be prepared so its electrical behaviour changes usefully with composition and applied voltages. Engineers use that behaviour in devices that control current within computer chips.
Applied voltage changes electrical conditions → Prepared silicon
Prepared silicon controls conduction → Controlled current path
The materials work together
A useful circuit combines conducting paths, insulating separation and components that control electrical behaviour. Each material contributes something different. Temperature, composition and device design affect how those materials behave in the actual circuit.
Conducting connections carries current → Working circuit
Insulating separation separates paths → Working circuit
Controlling devices controls signals → Working circuit
Different electrical materials provide paths, separation and controllable behaviour.
Atoms in a solid influence one another's electrons. This produces allowed ranges of electron energy. In a semiconductor, an energy gap separates the usual occupied range from a range that supports conduction. Temperature, carefully added atoms and electric fields can change the available charge carriers.
Doping means introducing selected atoms into the semiconductor to change its electrical behaviour. Device structures then use voltages to control current. The value for computing is repeatable control: one small electrical change can alter another signal, allowing many such devices to be linked into useful circuits.
Conductivity depends on conditions such as temperature and composition, and silicon is one of several semiconductor materials.
Charge, voltage and current
A battery can light a lamp
Electricity involves electric charge and its behaviour. In a battery-powered torch, the battery provides electrical energy, wires provide a conducting route, and the lamp turns some of that energy into visible light.
The battery maintains a difference
Chemical reactions maintain a voltage between the battery’s two terminals. Voltage describes an electrical energy difference per amount of charge. This difference is part of the conditions that make the connected torch work.
Closing the torch completes a route
The torch switch closes a conducting path through the lamp and back to the battery. Mobile charges respond to the electric field in the circuit. Their average motion creates a current.
The lamp releases light and heat
Current passes through the lamp as electrical energy is transferred to it. The lamp produces light and heat. The circuit uses charge movement while the battery’s chemical energy supplies the energy being transformed.
Signals can control another circuit
The torch’s switch controls one electrical state. In computers, carefully designed circuits let electrical signals control other signals. That makes it possible for small physical changes to participate in larger calculations and stored records.
Electrical circuits use moving charge and energy differences to produce controlled changes.
A water-pressure analogy can help initially: a difference in pressure can drive flow through a pipe. For electricity, voltage and current are different quantities, just as pressure and flow rate differ. The analogy has limits because electric fields and circuit components follow their own physical laws.
Electrons drift through a wire much more slowly than changes in electrical signals propagate along a circuit. A computer therefore needs no single electron to rush through an entire programme. Coordinated changes in local fields and circuit states support the movement of information.
The battery-and-lamp example introduces circuit behaviour while leaving electromagnetic field calculations and semiconductor physics for deeper lessons.
What is silicon?
Silicon is a useful material
Silicon is a chemical element widely used to make computer chips. It belongs to the semiconductors, materials whose electrical behaviour allows engineers to create controllable devices. A chip begins with carefully prepared solid material.
Atoms form an ordered crystal
Silicon atoms bond into an ordered structure. The arrangement affects which electrical states electrons can occupy and how easily charge can move. A simple drawing of connected atoms introduces the structure without showing its full quantum behaviour.
Selected atoms change the carriers
Engineers introduce controlled amounts of other elements into selected regions. This process, called doping, changes the available mobile charge carriers. Some preparations supply extra mobile electrons; others create mobile electron vacancies called holes.
A field controls a small region
A transistor combines prepared semiconductor regions with contacts and insulation. A voltage on a nearby control terminal changes the electrical conditions in a small channel, affecting how easily current passes between other terminals.
Control terminal field through insulation → Silicon channel
Insulating layer separates from channel → Control terminal
Silicon channel connects conducting region → Two current terminals
Tiny devices become a chip
Many such devices are connected through conducting tracks on a chip. Silicon supplies useful semiconductor behaviour, while metal connections and insulating layers organise the devices into circuits that manipulate electrical signals.
Silicon devices controlled conduction → Complete chip
Metal connections connects devices → Complete chip
Insulating layers separates paths → Complete chip
Prepared silicon gives engineers a material whose conduction can be controlled.
Doping means deliberately adding selected impurity atoms. Donor atoms can supply extra electrons, creating n-type material. Acceptor atoms create conditions described through mobile holes, producing p-type material. A hole behaves as a positive charge carrier even though its motion results from electron rearrangement.
A band gap is an energy separation between allowed electronic states. Silicon’s band structure helps engineers control conduction using doping, temperature and electric fields. Its manufacturing advantages, including useful oxide formation, also helped make it the dominant material for many integrated circuits.
Silicon is one semiconductor material among several, and its detailed behaviour follows quantum mechanics beyond the seat analogy.
A signal controls a switch
A signal can control a path
A transistor is a device that lets one electrical signal control another conducting path. In a common chip transistor, a voltage on its gate changes how easily current can pass through a small semiconductor channel.
The gate has an insulating barrier
The gate sits beside a very thin insulating layer. That layer separates the gate from the channel beneath it. A gate voltage can influence the channel through an electric field across this insulation.
Gate field crosses insulation → Channel region
The channel becomes easier to cross
In this simplified transistor example, a suitable gate voltage attracts charge carriers into the channel region. A conducting path forms between the other terminals, allowing more current to flow under the circuit’s applied voltage.
Suitable gate voltage field changes carriers → Conducting channel
First current terminal channel connection → Conducting channel
Conducting channel channel connection → Second current terminal
The next circuit receives a change
The transistor is connected with other components so that changing its conduction changes an output voltage. That output can become the control signal for another transistor, linking one electrical decision to another.
Many controlled paths form logic
A chip combines many transistors into circuits for logic, memory and calculation. Digital diagrams call states on and off, while real devices also have transition times and small leakage currents that engineering must account for.
Individual transistors connected for storage → Memory circuit
A transistor lets an electrical signal control conduction in another part of a circuit.
Bell Labs demonstrated a transistor in 1947. Later field-effect devices and silicon manufacturing processes enabled enormous numbers of devices on integrated circuits. The transistor used in the doorway example is a simplified MOSFET, one influential device family with several modern structural variations.
Manufacturing repeatedly patterns, deposits, removes and modifies materials on a wafer. This builds transistors and the connections between them. Integrating devices reduces connection distances and supports scale. Heat, fabrication variability and power consumption still constrain how the resulting chip can operate.
Real transistors have gradual transitions, leakage and timing limits, even when digital diagrams depict ideal on and off states.
Build logic from switching circuits
A gate combines electrical inputs
A logic gate is a circuit whose output follows a defined rule. An AND gate in our lamp demonstration receives two input signals and produces a high output only when both inputs are high.
Door input first input wire → AND gate
Button input second input wire → AND gate
AND gate output wire → Lamp-control output
Voltage ranges represent bits
The door signal is low, representing zero; the button signal is high, representing one. Recognised voltage ranges stand for these bits. With only one high input, the AND gate’s output settles low.
Door low 0 AND input zero → Output low 0
Both inputs rise high
The door closes and its input becomes high. With both inputs representing one, the gate’s transistor arrangement makes the output settle into the high range. The circuit physically implements the specified AND relationship.
Door high 1 AND input one → Output high 1
The output feeds another input
The AND output can be wired to a later gate. That later circuit can combine it with another condition, such as a timer signal. Several small rules can therefore form a larger decision.
AND output high output-to-input wire → Later gate
Timer signal additional input → Later gate
Later gate larger decision → Combined output
Electrical timing still matters
The logical rule describes which output should follow each input combination. Physical outputs take time to settle after inputs change. Larger circuits must account for these delays while combining many gates into useful operations.
A logic gate physically implements a specified relationship between input and output bits.
The familiar gate symbols hide the transistor circuit beneath them. This is an abstraction: you can design a larger circuit using each gate’s input-output behaviour. When timing, power or reliability matters, you reopen the abstraction and inspect how the physical implementation actually behaves.
Certain gate types are sufficient for constructing every Boolean function when combined in suitable arrangements. NAND is one example. This gives a striking engineering result: a small repeated building block can support an enormous variety of digital behaviours, provided the circuit also has appropriate storage and control.
Gate outputs take time to settle, so circuit timing matters whenever connected stages change their inputs.
Make gates add numbers
One plus one needs carrying
An adder is a circuit that performs addition. In binary, each digit is zero or one. Adding one apple to one apple produces two, written as 10: one pair and zero extra ones.
One apple first one → One pair
Two input bits reach gates
The two apple counts enter an adder as bits, each representing one. Connected logic gates calculate two outputs: the bit for the current column and a carry for the next column.
Input A is 1 first input wire → Adder gates
The current column becomes zero
Together, the inputs make one complete pair. The current column therefore has zero single apples left over. The adder produces a sum bit of zero and a carry bit of one.
Adder gates pair carried onwards → Carry bit 1
The carry enters the next column
A wire carries the one into the next binary column, whose place represents pairs. Read together, the two output positions are 10. Their position values let that pattern represent the quantity two.
More columns add larger counts
Connecting column-sized adders allows larger binary numbers to be added. Each full adder can also accept a carry from the preceding column. Real processors use engineering refinements to manage the time these calculations take.
Adder circuits calculate a column’s sum and pass any carry to the next column.
A full adder has three one-bit inputs, so its total can range from zero to three. Two output bits represent those four possibilities. You can write its entire behaviour as a truth table, then construct a gate network that produces exactly that table.
Arithmetic units extend these ideas to subtraction and other operations. Engineers also improve speed by handling carries in more elaborate ways. The simple chain is useful for understanding dependency: a later column may need information generated by an earlier column before its result settles.
The simple carry chain teaches the principle, while high-performance processors use more sophisticated arithmetic circuits.
Keep a state after the input moves on
A running total needs somewhere
Memory preserves information for later use. Asha’s apple program has counted two apples but has another bag to include. It needs a place to keep the first total while the calculation continues.
Next bag waiting later addition pending → Working memory
A write sets a physical state
A write operation puts selected memory elements into states representing the number two. The physical method depends on the memory technology, such as stored charge or a circuit maintaining a state through feedback.
An address locates the record
The program associates the running total with a memory location. Its address identifies where the value can be read. This lets later instructions find the count among many other stored values.
The total is read and replaced
The processor reads two, adds the next bag’s three, and writes five to the total’s location. Memory now holds the updated count for the display instruction that will follow.
Persistence depends on the technology
Many working-memory technologies need power to preserve their contents. A saved file can use storage designed to retain information when power ends. The apple program’s temporary total and saved sales record can therefore have different lifetimes.
Memory keeps encoded information available between separate operations.
Registers sit close to a processor’s calculation units. Caches keep frequently needed data nearby. Main memory offers more space, while persistent storage preserves files across power cycles. This hierarchy balances access time, capacity and cost, and it strongly affects real programme performance.
A stored value and its address are different things. The address identifies where to look; the value is what you find there. Maintaining that distinction helps explain variables in software, instruction fetching and the movement of large model weights between memory and processors.
Memory technologies differ in their physical mechanisms, and many common working-memory types lose their contents without power.
The processor follows the instructions
The processor carries out instructions
The central processing unit, or CPU, executes a program’s instructions. Asha’s computer must add two apple counts. The CPU coordinates where the numbers come from, which operation happens and where its answer is stored.
Stored counts input values → Processor
The next instruction is fetched
The CPU retrieves the next encoded instruction from memory. In this example, it asks for addition. A stored instruction location helps the processor keep track of which part of the program comes next.
Control circuitry selects addition
The CPU interprets the instruction’s bit pattern. Its control circuitry directs the two values towards the arithmetic unit, the part that performs calculations. Other instructions would direct different supported actions.
Instruction decoded selects addition → Arithmetic unit
Five returns to storage
The arithmetic unit adds two and three, producing five. The processor stores that result in the designated location and updates its progress through the program. A later instruction can use the saved result.
Many small actions create behaviour
A display instruction can now use five to show the apple total. Reading instructions, directing operations and preserving results connect tiny electronic actions to visible program behaviour. Modern processors organise this work with far more overlap and complexity.
A processor executes instructions by coordinating movement, calculation and storage.
The familiar fetch-decode-execute sequence is a learning model. Modern processors overlap work, predict likely branches and execute some operations out of their original order while preserving required observable results. Opening that layer reveals performance engineering built around the same instruction-level contract.
A clock provides a timing reference for many digital circuits. One clock tick does not necessarily equal one completed instruction. Some instructions take several cycles, while several independent instructions may progress together. Clock frequency therefore gives only one part of the explanation for performance.
This single-register processor is an invented teaching machine with far fewer features than commercial processors.
Instructions become bit patterns
A processor recognises encoded commands
Machine code is instructions encoded as bit patterns that a processor design can execute. Our illustrative processor has an add command for combining the apple counts stored in two small working locations.
Apple counts available values → Illustrative processor
Parts of the pattern have roles
In this invented encoding, one field selects addition and other fields identify the two working locations. A field is a defined section of the instruction. The machine’s design determines how those sections are interpreted.
Operation field which operation → Complete instruction
First location field first value location → Complete instruction
Second location field second value location → Complete instruction
The stored instruction is fetched
Memory holds the complete instruction as physical states representing its bits. The processor fetches that pattern. Its decoding circuitry recognises the operation field and the working locations specified by the remaining fields.
Control signals route the numbers
The decoded instruction activates paths carrying two and three to an adding circuit. The result is written back as five. The instruction’s physical bit pattern has organised a particular change in the machine.
Working value 2 selected data path → Adding circuit
Working value 3 selected data path → Adding circuit
Adding circuit result write path → Stored result 5
Another design can use another code
A different processor may assign different meanings to the same bits. Executable code therefore belongs to an instruction-set design. Translation tools connect human-written programs to the encodings a chosen processor can run.
Machine code assigns executable meaning to particular instruction bit patterns.
Assembly language gives readable names to machine operations, often including names such as ADD or LOAD. An assembler translates those names and operands into encoded instructions. This creates a bridge between human-readable low-level text and the binary patterns the processor can execute.
An instruction set is a contract between software and hardware. Different processor designs can implement the same contract using different internal circuitry. Programmes compiled for another instruction set generally need translation, emulation or recompilation before they can execute on the new hardware.
The binary instruction shown here is invented for teaching and has no promised compatibility with any commercial processor.
Translate human instructions into machine work
Readable code needs a translation
A compiler translates a program into another form. Asha writes a small program that adds two apple counts and displays the total. A compiler can turn that readable description into instructions suitable for a target machine.
The program names its values
The source uses names such as firstBag, secondBag and total. These names help people express which quantities belong together. The compiler reads the program’s structure according to the programming language’s rules.
firstBag is 2 named input → total equals sum
The translation represents the work
The compiler builds an internal representation of the requested operations. It can check and reorganise that work while preserving the language’s required behaviour. For this program, the essential operation combines the two counts.
Compiler checks checks language rules → Represented addition
Target instructions carry it out
The compiler produces target instructions for obtaining the values, adding them and making the result available to display code. When the translated program runs, processor operations produce the total five.
Translation preserves mistakes too
If Asha’s source adds the first bag twice, a faithful translation can preserve that error. Successful compilation checks specific language requirements. The program’s behaviour still needs to match the task she intended to solve.
A compiler translates a program into another representation while preserving its intended language-defined behaviour.
Compilation can happen before a programme runs or while it is running. Interpreters execute a programme representation through another programme, and many language systems combine interpretation with compilation. The important question is which transformation or execution stage is happening at a particular moment.
The compiler is itself software. Early compilers could be written using lower-level tools, and later versions could compile their own language through a process called bootstrapping. This explains how increasingly convenient programming systems can grow from simpler instruction-writing capabilities over time.
A successful translation preserves specified behaviour, including language rules that can be subtler than the surface wording suggests.
Organise shared hardware
Music and a letter
Leila types a letter while music plays on her laptop. Both programmes need the same machine. Its operating system organises access to the processor, memory, files and devices so these activities can continue together.
Leila types words → Letter programme
Operating system provides computer resources → Letter programme
Operating system provides computer resources → Music programme
A turn to calculate
The letter programme needs calculations when Leila types. The music programme needs calculations to keep sound ready. The operating system arranges processor time for running work, switching between tasks or using available processor cores.
Letter programme requests work → Operating system
Music programme requests work → Operating system
Operating system schedules running work → Processor time
Room for each programme
The letter and music also occupy working memory. The operating system manages their memory and protection. Each programme can use its allocated space while controlled boundaries help prevent accidental interference with another programme’s data.
Memory management manages allocation → Working memory
Letter programme stores letter data → Working memory
Music programme stores sound data → Working memory
Saving the letter
Leila selects Save. Her writing programme asks the operating system to write the document into a file. The system coordinates storage access and reports the result, allowing the programme to show that saving has finished.
One shared foundation
Leila’s music continues while her letter is saved. Scheduling, memory management and file services support both activities. Other programmes can use those same services, which is why an operating system forms a shared foundation for applications.
Operating system schedules work → Processor time
Operating system manages space → Working memory
Operating system coordinates access → File storage
An operating system coordinates the computer resources that running programmes share.
A process is a running programme together with its execution state and managed resources. Virtual memory gives a process a useful view of addresses while hardware and operating-system mechanisms translate and protect actual memory access. This separation simplifies programme construction and resource sharing.
A system call is a controlled request from a programme to the operating system. Opening a file eventually crosses this boundary. Underneath, drivers and hardware protocols perform the detailed device work, creating another chain of abstractions between an application’s simple request and physical operations.
Resource management and protection mechanisms reduce interference, while bugs and configuration errors can still cause failures.
Send information in packets
A photograph travels
Amina sends a photograph from her laptop to a computer upstairs. A network connects the machines. Software packages the photograph’s data, and electrical, radio or light signals carry those pieces across the available connections.
Pieces with addresses
The sending software organises the photograph into manageable pieces. Network packets carry addressing information alongside data. This gives network equipment information it can use when forwarding each packet towards the destination computer.
Upstairs address identifies intended destination → Addressed packets
Signals cross a link
Amina’s laptop sends the first packets through its network connection. The data becomes changing physical signals. Equipment at the other end interprets those signals and recovers the packet information for the next part of the journey.
The photograph is rebuilt
The upstairs computer receives the arriving data. For a reliable file transfer, additional communication rules check delivery and request missing data when needed. The receiving application gathers the delivered bytes into the photograph file.
Many connected links
Amina can send the photograph much farther through a chain of networks. Routers forward packets from link to link. The message keeps its digital meaning while each connection carries it through its own kind of physical signal.
A computer network carries encoded messages through physical links using shared communication rules.
Internet Protocol supplies addressing and packet delivery across connected networks. Its basic service provides no guarantee that every packet arrives or that arrival order matches sending order. Protocols such as TCP add mechanisms for reliable ordered delivery of a byte stream when applications need that behaviour.
A layered design lets different jobs remain manageable: a physical link moves signals, network protocols route packets, and application protocols describe useful exchanges. Following a message through these layers reveals why a failure can occur in a cable, route, connection or application independently.
The envelope analogy omits congestion control, encryption, routing policies and many details of real communication protocols.
A network becomes a web of pages
Opening a recipe
Amir opens a recipe in his browser. The web is the system of linked pages and resources he is exploring. The internet supplies the connected networks that carry requests and replies between his device and other computers.
An address names it
The recipe has a web address. It identifies a website and the resource being requested. Amir’s browser uses the address, together with internet services that locate the server, to begin asking for the recipe.
The server sends content
The browser sends a request using the web’s communication rules. The server returns the recipe document and may supply images and other files. These replies travel through the internet as data carried across network connections.
Web browser requests recipe resource → Website server
Website server returns content → Recipe and pictures
Recipe and pictures arrives through internet → Web browser
The browser draws it
Amir’s browser reads the received page instructions and arranges the heading, ingredients and picture on screen. A link beside the recipe contains another address, leading to a separate page about preparing its sauce.
A web of pages
Amir opens the sauce link, starting another request and reply. Links connect resources into the web. The underlying internet also carries other services, including email and many online games, using their own communication arrangements.
Recipe page link identifies page → Sauce page
Internet connections carries web data → Recipe page
Internet connections carries web data → Sauce page
Internet connections carries email data → Email service
The internet connects networks; the web uses those connections to deliver linked pages and resources.
Internet development involved many researchers and institutions working on packet switching, interconnection and protocols. The web arrived later as a particular information-sharing system. Following the history requires separate timelines for network infrastructure and the applications that people built on top of it.
HTML describes document structure. HTTP governs a common form of request and response. URLs identify resources. Combined with browsers and servers, these agreements let one person publish something that another person’s independently implemented software can find and display across the network.
This basic request-and-response description leaves out browser caches, encrypted transport and other common implementation details.
Give stored patterns a meaning
A red apple picture
Leila photographs a red apple. Her phone records a grid of tiny coloured areas called pixels. These recorded values form image data, giving software a way to store, display and change a selected representation of the apple.
Each place has values
In a simple colour image format, each pixel has numbers describing amounts of red, green and blue. The order of the pixels and the meaning of those numbers let software reconstruct the arrangement of colours.
The numbers become bits
The image format specifies how numbers and other image details are encoded. The phone stores the resulting patterns of bits. Software needs those format rules to interpret the stored patterns as an image with the intended dimensions and colours.
A brighter apple
Leila increases the picture’s brightness. The editing software changes image values according to its calculation rules. The screen then uses the revised values to emit different amounts of coloured light, making the displayed apple look brighter.
The picture leaves things out
The picture preserves useful visual details of Leila’s apple. Its smell, weight and hidden underside require other observations. Digital representations always select what to record, so the information available depends on the measurements and format used.
Physical apple records visible light → Recorded picture
Physical apple requires separate measurement → Weight measurement
Physical apple requires different observation → Smell experience
Digital data represents selected information through agreed formats and patterns of bits.
Digitising a changing signal involves selecting measurements at particular times and storing values with finite precision. Those choices determine what details can be preserved. An audio recording is therefore shaped by its sampling and encoding choices, even before anyone trains a model on it.
Training data adds a further issue: which examples are selected, repeated, excluded or labelled? A system can learn regularities in its observations, including accidental shortcuts. Understanding a model’s behaviour therefore requires questions about both the representation and the process that produced the dataset.
Every practical representation selects and approximates aspects of the world, which can matter for later interpretation.
Do many similar calculations together
Brightening a photograph
Amina wants to brighten a photograph containing millions of pixels. Many pixels need similar calculations. A GPU is a processor designed to carry out large amounts of this kind of work through many calculations running together.
Divide the picture
The software assigns groups of pixels to groups of GPU workers. For this simple adjustment, one pixel’s new value can be calculated from its old value without waiting for every other pixel to finish.
Original pixels assigns pixel work → Pixel group A
Original pixels assigns pixel work → Pixel group B
Original pixels assigns pixel work → Pixel group C
Values must reach workers
The GPU needs pixel values from memory before it can calculate. Software arranges this movement so many workers can obtain useful data efficiently. Waiting for data can limit speed even when plenty of calculation capacity is available.
Gather the changed pixels
Each group calculates its assigned brighter pixels. The results are written into the output image in their correct positions. The display system can then present the completed photograph, preserving its arrangement while changing its colours.
Changed group A writes assigned positions → Output image
Changed group B writes assigned positions → Output image
Output image supplies display values → Screen
The same strength helps AI
Neural networks also contain large groups of numerical calculations. GPUs can perform many of them together, much like the photograph’s pixel work. The benefit depends on how calculations depend on each other and how efficiently their data moves.
Pixel calculations offers parallel work → Parallel GPU work
Network calculations offers parallel work → Parallel GPU work
Data movement supplies required values → Parallel GPU work
A GPU can carry out many similar calculations together when the work can be divided efficiently.
GPU performance depends on how much useful work can happen alongside other work and how quickly data arrives. Moving values between memory and arithmetic units can dominate the cost. Batching several examples sometimes helps reuse information and keep the hardware occupied with useful calculations.
CUDA, introduced by NVIDIA in 2006, is one influential programming platform for general computation on its GPUs. Many other hardware and software developments also contributed to modern AI. The historical connection is a combination of programmable hardware, algorithms, datasets and sustained engineering.
A GPU speed-up depends on the workload, data movement and software implementation; some tasks have limited exploitable parallelism.
Describe uncertainty with numbers
A counter from a bag
Amir’s bag holds three red counters and one blue counter. He mixes them and takes one without looking. Probability describes the chance of each possible colour before he sees which counter comes out.
Three red counters three possible counters → Mixed bag
One blue counter one possible counter → Mixed bag
Mixed bag one counter drawn → Unseen selection
Count the possibilities
Assume every counter has an equal chance of being selected. Three of the four possibilities are red. The chance of red is therefore three out of four, which can also be written as seventy-five per cent.
A likely outcome can lose
Amir draws the blue counter. That outcome had a twenty-five per cent chance under the assumptions. A higher chance for red allowed room for blue to appear on this particular draw, so the result fits the original probabilities.
The bag has changed
Amir leaves the blue counter outside the bag. Three red counters remain inside. On his next draw from this bag, every remaining possible counter is red, so the chance of red becomes one hundred per cent.
The assumptions matter
The calculation used known counter counts and equal selection chances. A hidden extra counter or a biased drawing method would change the situation. AI probabilities likewise depend on the information and calculation method used to produce them.
Known counter counts defines possible outcomes → Calculated probability
Equal selection chances sets selection assumptions → Calculated probability
Probability describes the chance of an outcome under stated assumptions and available information.
Conditional probability asks about a chance given particular information. The chance of a word after “cup of” differs from its chance after “city of”. A language model learns to estimate these context-dependent distributions over its possible next tokens, then a generation procedure selects from them.
A forecast can be tested for calibration. Across many comparable events assigned sixty per cent probability, roughly sixty per cent should occur if the forecasts are well calibrated. Small samples fluctuate, and changing conditions can undermine a previously useful probability model.
A model’s probability reflects its assumptions and learned estimates; an individual prediction carries no guarantee.
A rule connecting input to output
The price of gifts
Leila’s shop charges two pounds for each small gift plus one pound for delivery. A function describes the rule that turns the number of gifts into the total price for this particular offer.
Three gifts enter
A customer orders three gifts. Three becomes the input to the price rule. The shop multiplies that count by the two-pound price of each gift, giving six pounds for the gifts themselves.
Three gifts three groups counted → £6 for gifts
£2 each multiplies each group → £6 for gifts
Add the delivery charge
The rule then adds the fixed one-pound delivery charge. Six pounds for the gifts plus one pound for delivery gives seven pounds. Seven is the output produced by this rule for an input of three.
£6 for gifts adds gift cost → £7 total
£1 delivery adds delivery charge → £7 total
Four gifts enter
Another customer orders four gifts. The same rule produces eight pounds for the gifts and nine pounds including delivery. Changing the input changes the output while the calculation rule itself stays the same.
The formula stores the rule
Writing the rule as two times the gift count plus one makes it reusable. The shop can apply it to many orders. If prices or delivery arrangements change, the rule needs updating to match the offer.
Reusable price rule calculates one order → Three gifts: £7
Reusable price rule calculates another order → Four gifts: £9
New delivery charge requires updated rule → Reusable price rule
A function gives an output for each allowed input according to a defined rule.
A parameter is an adjustable number within a rule. In y = wx+b, w controls how strongly the input affects the result, and b provides an added offset. Training a simple model means finding parameter values that make its outputs useful for the chosen examples.
Functions can be chained. One rule converts an input into an intermediate value, and another uses that value to produce a final result. Neural networks exploit this composition repeatedly. Understanding a large network starts with tracing a small chain while keeping every intermediate value visible.
A simple formula captures selected relationships, and its usefulness depends on whether those relationships fit the task.
A list of numbers with an order
One basket, three counts
Amina’s basket contains two apples, one pear and three bananas. The counts can be recorded together as an ordered list: two, one, three. This is a simple way to picture a vector used in computing.
Amina’s basket apple count → Two apples
Amina’s basket pear count → One pear
Amina’s basket banana count → Three bananas
Positions carry meaning
The list uses a fixed order: apples first, pears second, bananas third. That order gives each number its meaning. A list of two, one, three now describes the basket without repeating every fruit name.
A second basket arrives
Amir brings one apple, two pears and no bananas. Using the same order gives the list one, two, zero. The two baskets can now be compared directly because matching positions describe matching kinds of fruit.
Apples, pears, bananas same position meanings → Amina: 2, 1, 3
Apples, pears, bananas same position meanings → Amir: 1, 2, 0
Combine matching positions
They combine their baskets. Apple counts add to three, pear counts add to three and banana counts stay at three. Adding the matching list positions produces the combined vector three, three, three.
Amina: 2, 1, 3 adds matching positions → Together: 3, 3, 3
Amir: 1, 2, 0 adds matching positions → Together: 3, 3, 3
Lists can describe more
Their fruit lists made several counts available for calculation at once. Computers also use vectors for measurements such as location or colour. Learned AI representations use numerical lists too, with meanings that can be spread across many positions.
Fruit count vector groups fruit counts → Ordered numerical lists
Location vector groups position values → Ordered numerical lists
Colour vector groups colour values → Ordered numerical lists
An ordered list can hold several measurements together while preserving what each position means.
A vector’s dimension is its number of entries. Two-dimensional vectors can be drawn on ordinary axes, while large language models use representations with many more entries. A two-dimensional picture can illustrate a relationship, with substantial compression of the actual mathematical space.
Some vectors use human-labelled measurements; learned embeddings develop through optimisation. Individual embedding positions usually have no simple dictionary definition. What matters is how the whole representation participates in useful computations. Treating every position as a named human concept would overstate our understanding.
An ordered list is the entry point here; the full mathematical definition of a vector includes additional structure and rules.
A table that transforms a list
Two kinds of fruit box
A shop sells small boxes with two apples and one pear, and large boxes with four apples and three pears. A table of numbers records these recipes so an order can be converted into the required fruit totals.
Small box records small recipe → Recipe table
Large box records large recipe → Recipe table
The order supplies counts
A customer orders three small boxes and two large boxes. These counts form the input list. The recipe table keeps its columns in the same order, so each count matches the correct kind of box.
The apple row calculates
The apple row contains two and four. Three small boxes need three times two apples, giving six. Two large boxes need two times four, giving eight. Adding these gives fourteen apples for the order.
Small boxes: 6 adds three times two → 14 apples total
Large boxes: 8 adds two times four → 14 apples total
The pear row follows
The pear row contains one and three. Three small boxes need three pears, and two large boxes need six. Adding these gives nine pears. Applying both rows has turned the order into fourteen apples and nine pears.
Small boxes: 3 adds three times one → 9 pears total
Large boxes: 6 adds two times three → 9 pears total
9 pears total supplies pear total → 14 apples, 9 pears
One table handles orders
The shop can reuse the recipe table for different orders. Each row multiplies matching entries and adds them, producing one output. Neural networks use matrix calculations of this form alongside other calculations to transform numerical representations.
A matrix can store several calculation rules in rows and apply them to an input list.
Changing a matrix entry changes how strongly one input contributes to one output. In many neural-network layers, training adjusts such entries. You can therefore picture a weight matrix as a bank of connected controls, where each control influences a specific route through the calculation.
Matrix multiplication combines these row-based calculations across many input columns. It appears repeatedly in neural networks because groups of examples and learned weights fit naturally into this structure. Hardware libraries exploit that regularity to organise efficient arithmetic and memory access on modern processors.
Matrix multiplication is one building block; useful neural networks also require other operations, including nonlinear transformations.
A network of adjustable calculations
Reading a handwritten seven
Amir writes the number seven on paper and photographs it. A neural network can help recognise the digit. Its connected layers of calculations turn the picture’s numerical values into scores for the possible answers.
The picture becomes numbers
The image is represented as a grid of pixels. Each pixel supplies a numerical value, such as its brightness. The network receives these values in a defined arrangement, preserving information about where the dark strokes appear.
Layers change the values
A layer combines incoming numbers using stored adjustable values, then applies further calculations. The next layer receives the resulting numbers. Repeating this produces increasingly transformed representations that help the network distinguish patterns in the photographed strokes.
Seven receives the highest score
The final layer produces a score for each possible digit. In this example, seven receives the highest score. The recognition software uses those results to display seven as its proposed reading of Amir’s writing.
Practice changes the network
During training, many labelled digit pictures guide changes to the network’s adjustable values. During ordinary use, the trained network calculates answers with those values. Unfamiliar handwriting or poor photographs can still produce mistaken readings.
Labelled digit pictures provides learning examples → Training calculations
Training calculations changes stored values → Adjusted network
New handwriting picture receives calculated reading → Adjusted network
A neural network transforms numerical inputs through connected layers of calculations to produce an output.
Weights control how strongly one calculated value affects another. Training changes weights and other settings across many examples.
The name draws on an early inspiration from brain cells. Artificial networks run mathematical operations on computer hardware.
Success on familiar handwriting can fall when writing styles or image conditions change.
Put a number on the error
A delivery prediction misses
Leila’s delivery model predicts twenty minutes, but the journey takes thirty. Training needs a numerical way to score that mistake. A loss function supplies the scoring rule, making some kinds of prediction error measurable.
20 minutes predicted supplies predicted time → Error scoring rule
30 minutes recorded supplies recorded time → Error scoring rule
Measure the difference
The prediction falls ten minutes short of the recorded journey time. One possible scoring rule squares this difference, multiplying ten by ten. That gives a loss of one hundred for this delivery example.
A closer prediction scores lower
After a training adjustment, the model predicts twenty-eight minutes for the same journey. The difference is now two minutes. Squaring that difference gives four, so this prediction receives a lower loss under the same scoring rule.
Several journeys contribute
Training usually combines scores from several examples. Leila’s model might improve on this journey while worsening on another. Combining their losses provides a broader signal for changing the model’s stored values during the next training step.
First journey loss contributes example score → Combined training loss
Second journey loss contributes example score → Combined training loss
Combined training loss guides training calculation → Model adjustment
The score selects a goal
Lower delivery-time error is useful for Leila’s service. Customers also care about damaged parcels, clear messages and safe driving. Those concerns require suitable measures and checks because the time-prediction loss scores a particular part of the service.
Time prediction loss separate service concern → Parcel condition
Time prediction loss separate service concern → Clear delivery messages
Time prediction loss separate service concern → Safe driving
A loss converts a chosen kind of prediction error into a number that training can try to reduce.
Squaring makes positive and negative errors contribute positively and penalises large deviations strongly. Other tasks need other loss functions. Language models commonly use a loss that penalises assigning low probability to the observed next token, which connects probability prediction to numerical optimisation.
Training loss measures performance on examples used for adjustment. Separate evaluation examples help estimate how well the model generalises. A falling training loss can accompany worsening performance elsewhere, especially if the system memorises peculiarities or encounters a different data distribution after deployment.
Reducing a chosen loss improves that numerical objective and can leave important real-world goals incompletely represented.
Which way reduces the error?
An adjustable journey rule
Amir’s simple model predicts journey time by multiplying distance by an adjustable minutes-per-kilometre value. A gradient helps training work out which small change to that value would reduce the model’s current error score.
Journey distance multiplied by rate → Predicted time
Adjustable travel rate sets minutes per kilometre → Predicted time
Predicted time compared with recorded time → Prediction error score
The prediction is too short
For a two-kilometre journey, the model currently uses two minutes per kilometre and predicts four minutes. The recorded journey took six minutes. Under a squared-error rule, the two-minute difference gives a loss of four.
2 minutes per kilometre multiplies two kilometres → 4 minutes predicted
4 minutes predicted supplies prediction → Loss: 4
6 minutes recorded supplies target → Loss: 4
Find the useful direction
Near this setting, a small increase in the travel rate raises the prediction towards six minutes and lowers the loss. The gradient describes that local sensitivity, giving the training method direction information for its adjustment.
Choose how far to move
The training method also needs a step size. A small step might raise the rate to 2.1, making the prediction 4.2 minutes. A very large step could overshoot the useful region and increase the error.
Calculate again at each step
After updating the rate, training calculates the loss and gradient again. The useful direction can change as values move. Larger models apply this idea to many adjustable values, using more examples than this single journey.
Current model values calculates predictions → Current loss
Current loss supports derivative calculation → Current sensitivities
Current sensitivities guides size and direction → Next adjustment
Next adjustment changes stored values → Current model values
A gradient describes how small changes to adjustable values affect the current loss.
The simplest update is new weight = old weight − learning rate × gradient. In the example, choose a learning rate of 0.1: new weight = 1 − 0.1×(-4) = 1.4. The new loss is (1.4−3)² = 2.56, lower than four.
A gradient is local information. A very large step can overshoot a useful region, and complicated loss surfaces can contain flat areas and many interacting directions. Modern optimisers modify the basic update using information such as past gradients, while still depending on carefully designed training procedures.
Gradient-based optimisation can improve an objective without guaranteeing the best possible solution or reliable real-world performance.
Trace sensitivity backwards
Trace an error backwards
Leila’s journey model first calculates travel time, then adds a stopping allowance. Its prediction depends on both adjustable values. Backpropagation traces the calculation backwards to determine how small changes to each value would affect the final error score.
Travel calculation supplies travel minutes → Predicted time
Stopping allowance adds stopping minutes → Predicted time
Predicted time compared with recorded time → Error score
First calculate forwards
The journey is two kilometres. At two minutes per kilometre, travel contributes four minutes. Adding a one-minute stopping allowance produces a five-minute prediction. The calculation keeps intermediate values available for the later backwards calculation.
2 kilometres multiplies rate of two → 4 travel minutes
4 travel minutes adds travel contribution → 5 minutes predicted
1 stopping minute adds stopping contribution → 5 minutes predicted
Start with the final score
The recorded journey took seven minutes. The prediction is two minutes short, giving a squared-error loss of four. The backwards calculation starts by finding how a small change in the prediction would change this loss.
5 minutes predicted supplies prediction → Loss: 4
7 minutes recorded supplies target → Loss: 4
Loss: 4 calculates local sensitivity → Prediction’s local effect
Follow each connection backwards
Adding a little stopping time adds the same amount to the prediction. Increasing minutes per kilometre has twice that effect on this two-kilometre journey. Backpropagation combines these local relationships with the prediction’s effect on loss.
Prediction’s effect on loss passes through addition → Stopping allowance effect
Prediction’s effect on loss passes through multiplication → Travel rate effect
Distance multiplier: 2 scales local effect → Travel rate effect
An optimiser makes the change
Backpropagation supplies a sensitivity for each adjustable value. A separate update rule uses these sensitivities to change the travel rate and stopping allowance. Repeating forward calculation, backwards calculation and updating is one way to train much larger networks.
Forward prediction provides saved calculations → Backwards sensitivities
Backwards sensitivities supplies parameter sensitivities → Update rule
Update rule changes adjustable values → Revised stored values
Revised stored values used on next example → Forward prediction
Backpropagation calculates how each adjustable value contributes to changes in the final loss.
The chain rule expresses this relationship in calculus. If one change is multiplied by two at one stage and by three at the next, its final influence is multiplied by six. Backpropagation applies that principle systematically across the complete computational graph, including branching and shared values.
Backpropagation calculates gradients; a separate update rule uses them to change parameters. Keeping these roles visible clarifies the training cycle: forward calculation, loss evaluation, backward gradient calculation and optimiser update. Ordinary generation usually needs the forward calculation while leaving stored weights fixed.
Backpropagation is a mathematical method for artificial models, and its direct equivalence to learning in biological brains remains unsettled.
Cut text into model-sized pieces
A sentence becomes pieces
Amina types “The cat sat.” A language model receives encoded pieces of this text called tokens. Depending on the tokenizer, a token can contain a word, part of a word, punctuation or other characters.
One possible split
For this illustration, the sentence becomes four pieces: “The”, “ cat”, “ sat” and “.”. Spaces are included with the following words. The exact split depends on the tokenizer’s vocabulary and encoding rules.
Identifiers point to entries
Each piece has an identifier in the tokenizer’s vocabulary. In an illustrative vocabulary, those identifiers might be ten, twenty, thirty and forty. They let the system look up the numerical representation associated with each text piece.
Order changes the sentence
The same word pieces can appear in a different order. A model receives a sequence, so their positions matter. “The cat sat.” supplies a different arrangement from a sequence with “sat” placed before “cat”.
Sequence positions records original order → The cat sat.
Sequence positions records changed order → The sat cat.
Pieces also become replies
When a language model generates a reply, it selects output tokens step by step. The tokenizer converts the resulting sequence back into readable text. The number of tokens can differ from the number of words Amina sees.
Tokens are encoded text pieces that give a language model a sequence it can calculate with.
Reusing smaller pieces helps a limited vocabulary represent unfamiliar words. Different tokenizers can split the same text differently.
The identifier acts as a lookup label. A later step supplies learned numerical representations used in the model's calculations.
Word counts and token counts can differ considerably across languages and kinds of text.
Give a token a useful numerical representation
A numerical description of cat
Amina’s sentence contains the word “cat”. The model needs numbers to calculate with. An embedding supplies a learned list of numbers for its text token, giving the token a starting representation inside the model.
The identifier finds a row
The tokenizer gives the cat token an identifier. That identifier selects a row in the model’s embedding table. Each entry in the row is a stored number, and the whole row forms the token’s initial vector.
Training shapes the numbers
During training, adjustments to these numbers can help the model predict language more successfully. Tokens used in related ways can develop useful numerical relationships. Those relationships give later calculations information beyond the token’s vocabulary identifier alone.
The sentence adds context
Amina’s sentence continues, “The cat sat on the mat.” Later layers combine information from its different pieces. The representation at “cat” can therefore include information supplied by the surrounding sentence as the model works through its layers.
Initial cat representation supplies starting values → Model layers
Surrounding text pieces contributes sentence information → Model layers
Model layers calculates updated representation → Contextual cat representation
A map simplifies many numbers
A diagram might place cat and dog close together to illustrate a learned relationship. Real embeddings usually contain many more than two numbers. A small word map shows selected relationships while leaving much of the full representation unseen.
Cat representation selected relationship displayed → Small illustrative map
Dog representation selected relationship displayed → Small illustrative map
Many numerical dimensions reduced for illustration → Small illustrative map
An embedding gives a token a learned numerical representation that later model calculations can use.
Earlier word-embedding research showed how training on language could produce useful relationships among vector representations. Neural language models and later Transformer systems developed these ideas in different ways. Input embeddings and the contextual representations inside later layers therefore deserve separate labels in a careful explanation.
Similarity depends on both the representation and the comparison rule. Two vectors close under one measure can still differ in ways important to a particular task. Embeddings can also reproduce biases and shortcuts in training data, so a useful geometric pattern needs interpretation and evaluation.
A two-dimensional word map simplifies high-dimensional learned representations and supplies no complete dictionary of a model’s internal concepts.
What is attention in AI?
Connecting it with context
Amina writes, “The dog chased the ball because it was playful.” Attention is a calculation that lets a text position use information from other positions. It can help a language model connect “it” with relevant surrounding words.
The dog possible contextual information → It
The ball possible contextual information → It
Was playful adds sentence context → It
The model compares number patterns
The model represents each piece of text with numbers. At “it”, a learned calculation compares these number patterns and gives available positions different scores. The model’s design controls which earlier or later positions are available.
It representation supplies current comparison → Comparison scores
Dog representation supplies other representation → Comparison scores
Ball representation supplies other representation → Comparison scores
Some words contribute more
The scores become shares of the information to combine. In this simplified example with only two inputs, dog contributes 70% and ball 30%. These invented numbers illustrate the calculation; a trained model computes its own shares.
The information is mixed
The model weights each input by its share and adds the results. That creates a new set of numbers for “it”, now carrying information from the surrounding text. Further calculations use this updated context.
Dog’s contribution adds weighted contribution → Updated “it”
Ball’s contribution adds weighted contribution → Updated “it”
Updated “it” supplies contextual values → Next calculation
Several calculations work together
A model can use several attention calculations and many layers. Different calculations can combine different relationships in Amina’s sentence. One attention picture reveals part of this work, while the complete answer also depends on other calculations and trained values.
One attention calculation contributes contextual values → Further layers
Other attention calculations contributes other values → Further layers
Further layers continues model calculation → Interpreting the sentence
Attention mixes information from relevant text positions, helping each piece gain context.
An attention result is a weighted mixture: some pieces contribute more than others. Training shapes how the comparisons are made.
Attention mechanisms were used in translation before the Transformer. Transformer models organise attention alongside other calculations.
Attention links illustrate one calculation. They provide only part of an explanation of the model's whole answer.
What is a transformer?
Translate a short message
Amir wants “Good morning” translated into French. A Transformer is a design for the neural network that can help with this task. It repeatedly combines contextual information and other calculations to produce useful representations of language.
Represent pieces and positions
The system divides the English message into tokens, turns them into numerical representations and includes information about their positions. This gives the network values it can calculate with while preserving the arrangement of the original message.
Attention shares useful information
Within a layer, attention lets the representation of one text piece use information from other permitted pieces. Further calculations transform the resulting values. Repeated layers continue these operations, building representations useful for the translation task.
Generate the French reply
In the original translation design, one part processes the source message and another generates the translation. The output side uses the source representation and its generated text so far to help produce the French response.
Processed English message supplies source information → Translation generator
French text so far supplies previous output → Translation generator
Translation generator generates translated text → Bonjour
The design has variations
The original 2017 Transformer described this two-part translation arrangement. Later designs use different combinations of its components for tasks such as text generation and classification. Amir’s translated greeting illustrates one application of the wider architectural idea.
Transformer components arranged for translation → Original translation design
Transformer components adapted for generation → Text generation designs
Transformer components adapted for classification → Text classification designs
A Transformer is a neural-network design that combines attention with repeated numerical transformations.
The original 2017 Transformer had an encoder to process input and a decoder to generate output.
Later designs use different arrangements. Decoder-style language models can generate text using the preceding context.
Different Transformer designs serve different tasks. Attention is one component of the complete design.
Learn from many prediction problems
Learning from existing text
A GPT-style model can begin learning from a large collection of text. This stage is called pretraining. It repeatedly predicts how text continues and changes its stored values using the continuations that actually appear in the examples.
Hide the next piece
One example reads, “She filled the kettle with water.” The model receives the earlier text and predicts the next token. For this illustration, the next piece is the word water, supplied by the existing example.
Kettle sentence supplies preceding text → Text before water
Kettle sentence supplies continuation target → Water token
Score its prediction
The model assigns probabilities to possible next tokens. Training checks how much probability it gave to the token appearing in the example. A low probability for that token produces a stronger error signal under the chosen training rule.
Earlier sentence text supplies input context → Current model
Current model calculates possible continuations → Next-token probabilities
Actual water token identifies scored continuation → Next-token probabilities
Adjust and repeat
The training software calculates changes to the model’s stored values, then continues through more examples. Repeated updates can build useful language patterns and factual associations. Individual training examples may also contain mistakes or misleading claims.
Training error signal guides adjustment → Update calculation
Update calculation changes stored values → Changed model values
Further text examples supplies next training input → Changed model values
Broad learning supports later tasks
The kettle sentence is one tiny contribution among many. The resulting model can then receive further training or be used to generate text. Its broad learned patterns support useful tasks, while testing remains necessary to assess factual accuracy and behaviour.
Many training examples shapes stored patterns → Pretrained model
Pretrained model provides starting model → Further training
Pretrained model supports language generation → Generating text
Pretraining adjusts a model across broad examples before it is adapted or used for particular tasks.
Many examples carry patterns of grammar, topics and relationships. Predicting them can build abilities useful across different tasks.
Next-token prediction is one pretraining method. Other models learn by recovering missing text or using other objectives.
Training data can pass on mistakes, biases and misleading patterns alongside useful information.
Learn the behaviour shown in examples
Shorter customer-service replies
Leila wants a language model to produce concise customer-service replies. Fine-tuning gives an existing model further training for this purpose. In supervised fine-tuning, selected examples pair customer messages with replies demonstrating the desired behaviour.
Existing language model supplies starting values → Further training
Message and reply examples demonstrates desired responses → Further training
Further training changes stored values → Adapted model
Show the desired response
One example pairs “My parcel is late” with a short reply acknowledging the delay and explaining the next step. The training collection contains many such pairs, showing both the content and style Leila wants the system to learn.
My parcel is late supplies example request → Training example collection
Concise helpful reply supplies desired response → Training example collection
Change the stored values
Training runs the existing model on the example requests and compares its predicted output with the supplied replies. Adjustments make those demonstrated responses more likely in suitable contexts. The resulting changes remain in the trained model’s stored values.
Example customer request model calculates response → Model output probabilities
Demonstrated reply provides training target → Model output probabilities
Model output probabilities error guides adjustment → Updated model values
Check new customer messages
Leila tests the adapted model on separate messages, including a damaged parcel and an unclear delivery address. These examples reveal whether the shorter style transfers to fresh requests while the replies still contain useful and accurate information.
Damaged parcel request supplies fresh request → Adapted model
Unclear address request supplies fresh request → Adapted model
Adapted model produces replies to assess → Response evaluation
Adaptation has trade-offs
The customer-service model may become more consistent at short replies while other behaviours change. Leila compares its results with the starting model. Additional facts supplied during a conversation affect that answer through context, a separate mechanism from these stored training changes.
Shared evaluation requests same evaluation inputs → Starting model
Shared evaluation requests same evaluation inputs → Adapted model
Starting model provides baseline replies → Behaviour comparison
Adapted model provides changed replies → Behaviour comparison
Fine-tuning further adjusts a trained model using examples or objectives chosen for a particular purpose.
Demonstrations show the behaviour being encouraged. Their quality and variety influence what the model learns.
Examples inside a single prompt guide that response through context. Fine-tuning changes model settings used across later requests.
Further training can improve some behaviours and weaken others, so results need evaluation.
Give outcomes a numerical score
Points for reaching a doorway
A simulated robot is learning to cross a room. Its training setup gives points when it reaches the doorway. This numerical reward provides a signal that a learning method can use to improve the robot’s future choices.
The rule chooses outcomes
The designer gives ten points for reaching the doorway and subtracts a point for each collision. These invented numbers express selected preferences for the task. Changing the scoring rule changes which outcomes the learning method is encouraged to pursue.
Doorway reached earns ten points → Chosen scoring rule
Collision occurs loses one point → Chosen scoring rule
An action produces feedback
The robot tries moving forward and hits a chair. The simulator records the collision and returns a negative reward. This experience gives the learning process information about that movement in the robot’s current situation.
Later rewards matter too
On another attempt, the robot first moves around the chair and then reaches the doorway. A learning method can use rewards across the sequence, helping earlier useful movements gain value even when the positive reward arrives later.
The score covers selected goals
Suppose the room contains a fragile vase that the score ignores. A route can earn doorway points while knocking it over. The scoring rule captures selected goals, so desirable behaviour also depends on what designers measure, constrain and evaluate.
High-scoring route earns arrival reward → Doorway reached
High-scoring route causes unscored damage → Fragile vase broken
Fragile vase broken reveals missing concern → Task design review
A reward is a numerical signal that helps a learning system judge the outcomes of actions.
Expected reward averages possible scores according to their chances. If a choice returns ten points with probability one half and zero otherwise, its expected reward is five. That expectation describes an average under the assumptions; a particular attempt can still receive ten or zero.
Human preferences can provide comparisons between behaviours when writing a direct scoring rule is difficult. A reward model learns to predict those comparisons and supplies scalable scores. Its training evidence, generalisation and vulnerabilities become additional parts of the system that must be examined.
A reward model is an estimate of selected feedback, with possible errors and gaps between its scores and the intended goal.
Learn from actions and consequences
Learning a route through experience
A simulated robot needs to reach a doorway across a room containing a chair. Reinforcement learning lets it improve its movement strategy through experience, using rewards for selected outcomes such as reaching the doorway.
Simulated robot must navigate around → Chair obstacle
Simulated robot seeks a route → Doorway target
Doorway target arrival earns points → Arrival reward
Its first move fails
The robot’s current strategy chooses a forward movement. It reaches the chair and receives a collision penalty. Training records the situation, chosen action, resulting situation and reward, creating one piece of experience for improving later choices.
Another route reaches the goal
On a later attempt, the robot tries moving to the side before advancing. It passes the chair and reaches the doorway. The successful sequence provides experience linking earlier movements with a reward received at the end.
Training changes future choices
The learning method combines these experiences to update the robot’s strategy or its estimates of useful actions. Successful routes can become more likely. Trying alternatives can still be valuable because an unexplored movement might lead to a better outcome.
Collision experience contributes observed outcome → Learning update
Doorway experience contributes observed outcome → Learning update
Learning update changes future decisions → Changed movement strategy
A moved chair changes things
The learned route works in the training room. If the chair moves, the robot may need different actions. Results depend on the reward rule, experiences and available observations, so success is evaluated across situations that resemble its intended use.
Chair in new position changes clear space → Different route needed
Learned strategy supplies learned behaviour → Evaluation in new layout
Different route needed reveals adaptation demands → Evaluation in new layout
Reinforcement learning uses actions, outcomes and rewards to improve a strategy through experience.
Trying unfamiliar actions can reveal better options. Reusing successful actions takes advantage of what has already been learned.
Rewards are chosen signals. A system can achieve a high score while missing something the designers actually care about.
Outcomes can be uncertain, and success in training conditions can fail to transfer elsewhere.
Turn human comparisons into training
People compare two explanations
Amir requests a simple explanation of rain. One answer is clear and accurate; another is confusing. Reinforcement learning from human feedback, or RLHF, can use people’s comparisons to help train a model towards preferred kinds of responses.
Explain rain simply model produces option → Clear explanation
Explain rain simply model produces option → Confusing explanation
Clear explanation compared for preference → Human reviewers
Confusing explanation compared for preference → Human reviewers
Comparisons become examples
Reviewers select the clearer rain explanation using the guidance provided for the task. Many comparisons form a training collection. Those choices reflect the reviewers, instructions and examples involved, which shapes the feedback available to the system.
Clear rain explanation preferred in this example → Reviewer’s preference
Confusing rain explanation compared alternative → Reviewer’s preference
Reviewer’s preference recorded as feedback → Comparison collection
A model learns to score
In the influential InstructGPT method, a separate reward model learns from these comparisons. It can then assign estimated preference scores to further replies, including rain explanations that the human reviewers have never directly compared.
Human comparison examples provides preference evidence → Reward-model training
Reward-model training adjusts scoring values → Learned reward model
Another rain explanation receives estimated score → Learned reward model
Scores guide further training
Reinforcement learning then adjusts the answering model towards replies receiving higher estimated scores. The method also limits departures from its earlier behaviour. In this example, that can encourage clearer rain explanations across further training requests.
Answering model generates candidate response → Generated rain explanation
Generated rain explanation receives estimated score → Reward model score
Reward model score guides training objective → Reinforcement-learning update
Reinforcement-learning update changes answering behaviour → Answering model
Preference needs accuracy checks
An appealing explanation of rain can still contain a factual mistake. Human comparisons and reward-model scores provide useful training signals with limitations. Evaluations therefore examine accuracy and other goals alongside the responses’ style and apparent helpfulness.
Appealing rain explanation assesses preferred presentation → Preference judgement
Appealing rain explanation checks factual claims → Scientific accuracy check
Preference judgement contributes one assessment → Broader evaluation
Scientific accuracy check contributes another assessment → Broader evaluation
RLHF uses human feedback to guide reinforcement learning, often through a model trained to estimate preferences.
The scoring model supplies estimated feedback during later training. Human reviewers provide the comparisons it learns from.
This describes a published historical approach. Feedback-based training can use other methods, with different stages and objectives.
Human preferences and predicted scores can favour convincing answers that contain factual mistakes.
Use the trained model to generate
Recognising a new photograph
Amina opens a new photograph of her dog in a photo app. Inference is the work of using a trained model to produce a result, such as the app’s suggested label for this particular picture.
Prepare the picture
The app prepares the photograph in the format the model expects. This can include resizing it and converting its pixel values into numerical inputs. These steps provide the image information used in the model’s calculations.
Run the learned calculations
Computer chips carry out the model’s calculations using the picture’s numbers and the stored values learned during training. Each layer contributes to the result. For ordinary inference, these learned values remain fixed throughout this use.
Picture input values supplies this picture → Processor calculations
Learned stored values supplies trained settings → Processor calculations
Processor calculations calculates output values → Category scores
Display the suggested label
The model’s output gives dog the highest score in this example. The app turns that output into a label Amina can read. The complete user experience includes the model calculation and the surrounding software that presents its result.
Using and training are separate
Amina’s photograph produced a label through inference. Changing the model’s learned values would involve a training update. Some systems perform both activities over time, while a single inference call applies the model’s current values to its supplied input.
Amina’s photograph input for inference → Current trained model
Current trained model output of inference → Calculated label
Separate training update changes learned values → Current trained model
Inference applies a trained model to an input to calculate an output.
For a text-generating model, inference can repeatedly select a token and use the growing sequence to calculate the next.
Stored settings ordinarily stay fixed during a request. Current context and intermediate calculations change as information is processed.
Continual-learning systems can also perform training updates. Updating is a separate operation from applying the current model.
The information available right now
A city with purple buses
Leila tells a language model that her fictional city has purple buses. The supplied sentence becomes part of the model’s context, the information available for its current calculation. It can use that detail when describing the city.
Include the relevant detail
The application prepares the conversation and instructions for the model. Leila’s sentence about purple buses is included alongside her request for a short city description. Both pieces are available during the next response calculation.
Buses are purple adds fictional fact → Assembled model context
Short city description adds requested task → Assembled model context
Calculate with the supplied sentence
The model’s layers calculate relationships among the supplied text pieces. The purple-bus detail influences the generated description. This use of information can happen with the model’s trained values staying fixed during the response.
Purple-bus context supplies current text → Response calculation
Fixed trained values supplies learned calculation → Response calculation
Response calculation generates contextual response → City description
A later request needs information
Leila later asks for a travel scene set in the same city. The model can use the bus detail if the application supplies it again. Systems manage a finite context capacity by selecting, summarising or retrieving information for later calls.
Earlier bus detail detail may be selected → Context preparation
Context preparation supplies retained information → Next call’s context
Later scene request adds current request → Next call’s context
Different routes preserve information
Keeping Leila’s city detail in the current context, storing it in an application’s memory and training a model on examples are separate mechanisms. Their effects depend on how the product retrieves information and whether model values are updated.
Purple-bus detail can be supplied now → Current call context
Purple-bus detail can be stored separately → Application’s saved information
Purple-bus detail could become training data → Model training process
Context is information supplied for the current model calculation that can shape the response.
The GPT-3 paper studied task examples supplied in context while keeping model parameters fixed during evaluation. This demonstrated a useful adaptation route: change the prompt and its examples, then observe how the model behaves. The underlying mechanism involves computation over those examples through already trained weights.
A context window has finite capacity, and having information somewhere in the window does not guarantee the model uses it accurately. Retrieval systems can fetch selected external material into the current input. Stored user memories in a product can also be supplied this way, depending on implementation.
Context use, stored product memory and later training are distinct mechanisms whose details depend on the system being used.
The layers meet in a conversation
A conversation with software
Amir asks ChatGPT to explain rainbows. ChatGPT is OpenAI’s conversational AI application. It accepts requests and presents generated responses, using trained models and supporting software running on physical computer chips behind the conversation.
The request reaches the system
Amir’s device sends his request through network connections. The application prepares the message and relevant conversation information. This supplied context gives the model the request it is answering and any accompanying instructions or details.
Chips carry out calculations
The model’s stored values and the prepared input are represented as numbers. Processors carry out calculations with those numbers. Those processors are physical electronic circuits, linking the conversation on Amir’s screen to the silicon-and-electricity foundations of computing.
Prepared input numbers supplies current input → Electronic processors
Trained model values supplies trained settings → Electronic processors
Electronic processors performs numerical calculations → Calculated output values
The explanation appears
The system returns generated text, and the application displays a rainbow explanation in the conversation. Amir can request a simpler version, supplying another message that shapes a later response through the conversation information provided to the model.
Many layers make one experience
Amir sees a conversation supported by networks, software, model calculations and electronic hardware. The original ChatGPT launched in November 2022; the product has evolved since. Useful generated explanations can still contain mistakes, so factual claims benefit from suitable evidence.
ChatGPT combines a conversational application, trained AI models, software and physical computing hardware.
ChatGPT launched on 30 November 2022. Its original GPT-3.5-family model used further training with conversation examples and human feedback.
The stack connects physical circuits, processors, software, trained models and a conversational interface. Each layer supports the next.
The 2022 account describes the original release. Current versions have evolved and some internal details remain undisclosed.
Ray Kurzweil
A page becomes speech
Leila uses a reading application to hear a printed page spoken aloud. This everyday extension of an ability helps introduce Ray Kurzweil, an inventor and futurist who argues that technology will increasingly expand what people can do.
Printed page text is captured → Reading application
Reading application produces spoken words → Leila hears speech
Leila hears speech illustrates expanded ability → Ray Kurzweil’s argument
A useful ability crosses boundaries
The application connects visual information with spoken language. In this invented example, Leila can listen while her hands are occupied. Kurzweil’s broader outlook treats such extensions of human abilities as part of a long technological development.
Better tools support later tools
Kurzweil studies improvements in computing and argues that earlier technologies help produce more capable successors. Applied to the reading example, better recognition, language processing and hardware could support future tools with broader abilities and lower costs.
Text recognition can improve captured text → More capable reading tools
Language processing can improve language handling → More capable reading tools
Computing hardware can support more computation → More capable reading tools
His forecast extends much further
Kurzweil’s forecast extends beyond reading applications to close integration between people and advanced AI. He predicts human-level AI by 2029 and a much wider transformation around 2045. These dates belong to his proposed account of the future.
Today’s reading example introduces broader argument → Kurzweil’s future account
Kurzweil’s future account proposes dated milestone → 2029 AI forecast
Kurzweil’s future account proposes wider transformation → 2045 transformation forecast
An example and a prediction
Leila’s reading tool illustrates technology extending an ability. Kurzweil’s larger forecast adds assumptions about future capability, cost and human adoption. Its timing and consequences remain uncertain, so present achievements and proposed future milestones receive separate kinds of assessment.
Existing reading example can be directly assessed → Observed capability
Future integration forecast requires forecast evaluation → Assumptions and uncertainty
Ray Kurzweil argues that increasingly capable technologies will expand human abilities and eventually integrate closely with people.
The Singularity Is Near appeared in 2005; The Singularity Is Nearer followed in 2024.
His outlook is an influential futurist argument. The timing, feasibility and consequences of advanced AI remain uncertain.
The law of accelerating returns
Tools help build better tools
A fictional chip-design team uses computers to test new chip layouts. Ray Kurzweil’s accelerating-returns argument proposes that better tools can help create the next generation of tools faster, allowing improvements to compound across successive generations.
One improvement feeds the next
The team’s improved computer can run more design calculations in a given time. If those calculations help produce another improvement, the new tool contributes to its own successors. This feedback is central to Kurzweil’s proposed mechanism.
Current computing tool supports design work → More design calculations
More design calculations can enable improvement → Next improved tool
Next improved tool becomes next starting tool → Current computing tool
Repeated doubling changes scale
Suppose an invented performance measure doubles each generation. It goes from one unit to two, then four, then eight. Equal multiplications produce increasingly large additions, which explains why a sustained exponential pattern can change scale rapidly.
- Generation one: 1
- Generation two: 2
- Generation three: 4
- Generation four: 8
The measure needs a meaning
For the chip team, performance might mean calculations completed per pound spent. That differs from power consumption, manufacturing yield or success on a real design task. A rising curve has meaning only alongside its chosen measure and conditions.
Calculations per pound different assessment dimension → Energy consumed
Calculations per pound different assessment dimension → Usable manufactured chips
Calculations per pound different assessment dimension → Design task results
Future growth depends on conditions
The team’s next chip also depends on materials, energy, manufacturing and successful ideas. Kurzweil argues that new approaches can sustain broader progress as older approaches slow. Extending the curve therefore requires assumptions about how future constraints and innovations will interact.
Materials and manufacturing constrains feasible production → Proposed future growth
Energy availability supports physical operation → Proposed future growth
Successful new designs may enable improvement → Proposed future growth
Kurzweil’s accelerating-returns argument proposes that improving tools can help generate further improvements faster.
Kurzweil proposes that new technological approaches can continue a wider trend as earlier approaches reach their limits.
This is Kurzweil’s proposed pattern of technological change. Extending a historical curve requires assumptions about future progress.
The technological singularity
A future assistant redesigns itself
In a hypothetical future, Amina works with an AI assistant that helps design better AI systems. A technological singularity describes a proposed transformation in which advancing machine intelligence changes society so profoundly that its later direction becomes difficult to predict.
Each successor helps research
Suppose the assistant’s improved successor becomes better at the research needed for another improvement. Amina’s imagined laboratory now has a feedback loop: increasing ability contributes to the work that may increase ability again. Whether this continues depends on many assumptions.
Current advanced assistant supports further research → AI research work
AI research work may produce improvement → More capable successor
More capable successor becomes next research tool → Current advanced assistant
Kurzweil includes people too
Ray Kurzweil’s version includes increasingly close connections between people and advanced computing. In that proposed future, Amina’s relationship with technology could change far beyond using a screen. These claims describe his vision of human and machine integration.
Amina in imagined future human abilities involved → Proposed close integration
Advanced computing proposed computational connection → Proposed close integration
A date belongs to a forecast
Kurzweil places this wider transformation around 2045. The imagined laboratory gives one way to picture his argument, while the actual route, pace and consequences depend on scientific, engineering and social developments whose outcomes remain uncertain.
Scientific advances affects possible pathway → Kurzweil’s 2045 forecast
Engineering capability affects achievable systems → Kurzweil’s 2045 forecast
Social decisions affects adoption and consequences → Kurzweil’s 2045 forecast
The scenario keeps open questions
Amina’s imagined assistant makes the proposed feedback loop concrete. Evaluating such a future involves definitions of intelligence, evidence about improvement, practical limits and choices about use. Different answers can produce very different timelines and social outcomes.
Meaning of intelligence changes milestone meaning → Imagined assistant scenario
Evidence of improvement supports or weakens mechanism → Imagined assistant scenario
Limits and human choices shapes possible outcomes → Imagined assistant scenario
A technological singularity is a hypothetical transformation in which rapidly advancing intelligence makes the future unusually difficult to predict.
Kurzweil uses singularity as a metaphor for an immense transformation that is difficult to imagine with today’s abilities.
A singularity remains hypothetical. Timing, definitions of intelligence and the path from research to widespread use affect any forecast.