Assembly 2: Calling convention and control flow

Overview

We discuss calling conventions and control flow in machine code.

Full lecture notes on assembly — Textbook readings

Calling convention

• Some aspects of machine code are fixed by the processor manufacturer
  • Intel decided 0xc3 is the representation of ret
• Some aspects of machine code are set by agreement among compiler and operating system developers
  • Intel did not decide which register holds return values
• We call this agreement the calling convention since it governs function calls
  • Think Geneva Convention, not Comic Convention
  • Different conventions can exist for the same processor (e.g., Unix vs. Windows)
  • Only codes with the same conventions can safely interact

Elements of a calling convention

• Function arguments
• Function return values
• Local variable storage
• Stack alignment
• Memory and processor state when the program begins

Let’s explore: cc01.cc–cc03.cc

Arguments

• Argument registers are %rdi, %rsi, %rdx, %rcx, %r8, %r9, in that order
• Large objects are passed in up to 2 registers if they fit, stack otherwise

Return values

• Return register is %rax
• Large objects are returned in %rax+%rdx if they fit, otherwise first argument points to space for return value

Sidebar: Type-safe linkage and mangled names

• The name of a C++ function encodes the types of its arguments
• This makes C++ compilations safer and supports overloading (functions with different behavior based on argument types)
• Example: f(int) ⟶ _Z1fi
  • _Z: This is a mangled name
  • 1: Function name is 1 character long
  • f: Actual function name
  • i: First argument is int
• To demangle, try c++filt MANGLEDNAME

cc04.cc

Function entry and exit sequence

• Call sequence: Steps required before calling a function
  • Set up arguments
• Entry sequence: Steps required to enter a function
  • Prepare environment for callee function
• Exit sequence: Steps required to exit a function
  • Set up return value
  • Restore environment for caller function
• Return sequence: Steps required after function returns
  • Clean up any argument space

Action of call FUNCTION instruction

subq $8, %rsp
movq %NEXT_rip, (%rsp)
movq FUNCTION, %rip

Action of ret instruction

addq $8, %rsp
movq -8(%rsp), %rip

cc05.cc

Local variables

• Local variables are stored in the stack
• The function entry sequence reserves enough space for locals
• Small local variables may be stored in the red zone
  • 128 bytes below %rsp
  • Does not need explicit reservation
  • Generally seen in leaf functions (functions that do not call other functions)