vm86(2) / vm86old(2)
| Field | Value |
|---|---|
| Numbers (x64 / i386 / arm64-generic / arm32) | vm86 – / 166 / – / –; vm86old – / 113 / – / – |
| Since (kernel) | vm86old (originally vm86) 1.0 (0.97p2 per the man page HISTORY note); renamed/added 2.0.28 per the syscalls(2) note – “Was ‘vm86’; renamed in …” |
| Man page | man2/vm86.2 (man-pages 6.19 HTML set, fetched 2026-10-17) covers both |
| glibc wrapper | none on x86_64 (i386-only calls; no vm86 symbol in glibc 2.41, nm -D) |
| Aliases covered | vm86old (SC-vm86old, grouped); vm86 is the primary (SC-vm86) |
| Ledger ID | SC-vm86 |
Purpose
vm86() makes the calling process enter virtual-8086 mode – Intel’s emulation of real mode inside a protected-mode task – the mechanism DOSEMU used to run DOS binaries against real hardware semantics (vm86.2, DESCRIPTION). The naming history is in the man page: vm86() appeared in Linux 0.97p2, was renamed vm86old() in 2.1.15/2.0.28 when a new vm86() with the vm86plus structure arrived, and the underlying struct changed in 1.1.8/1.1.9 (vm86.2, DESCRIPTION).
Synopsis
#include <sys/vm86.h>
int vm86old(struct vm86_struct *info);
int vm86(unsigned long fn, struct vm86plus_struct *v86);
Arguments and behavior
info/v86 carry the VM86 state: the 8086 register set, screen memory and BIOS-mapping layout, and control flags; the modern vm86() takes a function selector fn (VM86_ENTER, VM86_ENTER_NO_BYPASS_STRICT, or the VM86_FREE_RETURN family) (vm86.2, DESCRIPTION/SYNOPSIS). The kernel returns to the caller when VM86 mode exits for a reason userspace must handle (interrupts that are not reflected, GPFs on privileged instructions), which is what the return path encodes. On x86_64 kernels the calls do not exist at all – long mode dropped VM86 (v86 mode via emulation only).
Return value and errors
0 on success; -1 with errno (vm86.2, RETURN VALUE). Errors: EFAULT (i386-specific: problem fetching user-space data), ENOSYS (not implemented on this architecture), EPERM (“Saved kernel stack exists” – a kernel sanity check, since the saved stack should only exist inside vm86 mode) (vm86.2, ERRORS).
Flags and constants
VM86_ENTER, VM86_ENTER_NO_BYPASS_STRICT and the return-code family (<sys/vm86.h>).
Security and notes
VM86 mode hands userspace a fake ring-0-ish world with the kernel intercepting privileged operations – historically both a compatibility triumph and a bottomless kernel-attack surface (dozens of CVEs in vm86 handling). The modern replacements: v86 emulation in userspace (QEMU) and, for the one legit kernel-mode use, the vDSO/vsyscall page. Interview angle: “why is there no vm86 on x86_64?” – long mode removed the mode entirely, and Linux never re-implemented it. Cross-reference: modify_ldt, ioperm, iopl.
References
- https://man7.org/linux/man-pages/man2/vm86.2.html
- man-pages syscalls(2) (vm86old entry: “Was ‘vm86’; renamed in 2.0.28/2.1.15”)