CPU Microarchitecture Interview Questions: Pipelines, Hazards, and Branch Prediction

Someone hands you six lines of assembly in a loop and asks, "How many cycles does one iteration take?" The honest answer starts with a question back: which pipeline, which forwarding paths, and how is the branch handled?

Stating assumptions is most of what a microarchitecture interview rewards. This guide draws on public lecture notes from UMBC (CMSC 411, Lecture 17), Carnegie Mellon (18-344, Lecture 6), UC San Diego (CSE 240A, "The Reorder Buffer"), and the University of San Francisco (CS 631, lecture 12). The practice prompts and the worked example are ours, not questions reported from any employer.

How do you answer a pipeline hazard question?

A classic five stage pipeline (fetch, decode, execute, memory, write back) approaches one instruction per cycle once full. The USF notes name three things that break that: structural, data, and control hazards. Start any answer by naming which one you are looking at.

A structural hazard means two instructions need the same hardware in the same cycle, such as one memory port serving both fetch and data access. A data hazard means an instruction needs a value an earlier instruction has not produced yet.

The UMBC notes show how forwarding handles most data hazards: a forwarding unit compares destination registers held in later pipeline registers with the source registers of the instruction in execute, and routes the result straight to the ALU input.

Loads are the exception. The data is not available until after the memory stage, so an instruction that uses a loaded value right away must wait. The notes describe a hazard detection unit that spots this case and inserts a bubble, delaying the pipeline by one cycle.

Control hazards and the cost of guessing wrong

While a branch resolves, the pipeline keeps fetching. UMBC explains that in the simplest design the branch resolves in the memory stage, and a wrong guess means flushing everything fetched behind it. Moving the decision into decode cuts the penalty to one cycle but creates new data hazards when the branch depends on a register that was just computed or loaded. Deeper pipelines pay more per flush.

Prediction fills that gap. The UMBC notes describe a one bit predictor, which mispredicts a loop branch at the end of the loop and again on the first iteration the next time the loop runs. The CMU notes go on to the two bit saturating counter, a branch history table indexed by a hash of the branch address, and its limits: different branches colliding in the table, and a single branch's history missing correlation with other branches. They then add a branch target buffer, a tagged table that remembers where a branch goes.

Why do out-of-order cores retire in order?

When interviewers move past the five stage pipeline, the question is usually "why retire in order?" The UCSD slides answer it directly. Instructions can execute out of order, but results sit in a reorder buffer, whose entries hold an instruction type, a destination, and a value, until they commit in order. Because speculative results are not yet architectural state, a mispredicted branch means flushing the buffer, and the same mechanism gives precise exceptions.

Pair that with renaming, which removes false dependencies between instructions that reuse a register name, and reservation stations, which hold instructions until their operands arrive. Out-of-order is not free: USF's lecture describes typical gains as modest against a well tuned in-order core, so be ready to discuss cost as well as benefit.

How do IPC and the performance equation measure a CPU?

Time per program is instruction count times cycles per instruction times clock period, and IPC is the reciprocal of CPI. The USF lecture makes the point that clock rate alone misleads: its example is a 3 GHz core at IPC 4 retiring 12 billion instructions per second against a 4 GHz core at IPC 2 retiring 8 billion. It also says branches show up roughly every 5 to 6 instructions, so misprediction rates show up in IPC.

Topic	What the source says	Practice prompt
Forwarding	A forwarding unit routes results from later pipeline registers to the ALU input (UMBC)	Which hazards does forwarding remove, and which can it never remove?
Load-use	A load followed by a dependent instruction costs one bubble (UMBC, USF)	Show where the bubble appears on a pipeline diagram
Branch penalty	Resolving in decode instead of memory shortens the flush (UMBC)	What new hazard does early resolution create?
Predictors	Two bit counters, history tables, and a target buffer (CMU)	Why does a one bit predictor fail twice per loop?
Reorder buffer	Execute out of order, commit in order, flush on mispredict (UCSD)	How does the buffer give precise exceptions?
IPC	Time is instructions times CPI times clock period (USF)	When is a slower clock the faster machine?

Illustrative scenario: one loop, two problems

These numbers are ours, under stated assumptions: a five stage in-order pipeline with full forwarding, separate instruction and data memory ports, and a branch that resolves in execute, so a mispredict wastes two cycles. Consider:

loop: lw   t0, 0(a0)
      add  t1, t0, t2
      sw   t1, 0(a1)
      addi a0, a0, 4
      addi a1, a1, 4
      bne  a0, a2, loop

1. The add uses t0 right after the load. Forwarding cannot help in time, so every iteration takes one bubble: 6 instructions in 7 cycles, a CPI of about 1.17 and an IPC of about 0.86.
2. A compiler or scheduler can move addi a0, a0, 4 between the load and the add. The bubble disappears and the loop runs at 6 cycles per iteration.
3. The loop branch is taken until the last iteration. A predictor that learned "taken" is wrong exactly once, at the exit, costing two cycles.
4. Over 100 iterations, ignoring pipeline fill: the original loop takes about 702 cycles for 600 instructions (IPC about 0.85), the reordered loop about 602 (IPC about 0.99).
5. An out-of-order core could issue the independent addi instructions while the add waits, and the reorder buffer would still retire everything in program order, flushing the wrong path at the loop exit.

Say your assumptions out loud, then give the number.

Where this fits in your preparation

Microarchitecture shows up in CPU design, performance modeling, and some verification roles. See Arm's CPU architecture and design interviews, RTL designer versus microarchitect, and CPU versus GPU design interviews. Pipelining also overlaps with digital design questions on pipelining and timing and static timing analysis, and the memory side continues in cache coherence and memory system questions.

Draw the pipeline diagram on paper, and practice saying the assumptions first. You can book a mock interview on MockVise to work through this loop with someone who will push on the arithmetic.

Sources: UMBC CMSC 411 Lecture 17, CMU 18-344 Lecture 6, UCSD CSE 240A, The Reorder Buffer, USF CS 631 Lecture 12. This article is general preparation guidance based on public course notes, not an official interview process or reported company questions, and the worked example uses simplified assumptions that real processors do not share. Confirm the interview format with your recruiter. MockVise is not affiliated with or endorsed by UMBC, Carnegie Mellon University, UC San Diego, or the University of San Francisco.

FAQ

Q: What is the first thing to do when asked how many cycles a loop iteration takes? Ask back which pipeline, which forwarding paths, and how the branch is handled. Stating assumptions is most of what a microarchitecture interview rewards.

Q: Why does a load followed by a dependent instruction cost a bubble? The loaded data is not available until after the memory stage, so forwarding cannot deliver it in time. A hazard detection unit spots this case and inserts a bubble, delaying the pipeline by one cycle.

Q: Why does a one bit branch predictor fail twice per loop? It mispredicts a loop branch at the end of the loop, and again on the first iteration the next time the loop runs. The two bit saturating counter addresses this.

Q: Why is clock rate alone a misleading measure of performance? Time per program is instruction count times CPI times clock period. The USF example compares a 3 GHz core at IPC 4 retiring 12 billion instructions per second against a 4 GHz core at IPC 2 retiring 8 billion.