You would keep a frame pointer, which would tell you where to find the caller's stack frame. Most systems already do this, to aid debugging and tracing tools, to support variable-length stack allocations, or for any number of other reasons.