When I try to research about return values of system calls of the kernel, I find tables that describe them and what do I need to put in the different registers to let them work. However, I don't find any documentation where it states what is that return value I get from the system call. I'm just finding in different places that what I receive will be in the EAX register.
The result is usually returned in the EAX register.
- Look at sys_read's return value in EAX
- Copy sys_read return value for safe keeping
C is the language of Unix systems programming, so all the documentation is in terms of C. And then there's documentation for the minor differences between the C interface and the asm on any given platform, usually in the Notes section of man pages.
sys_read is a way to talk about the raw system call (as opposed to the libc wrapper function), because
syscall.h defines constants like
SYS_read with the actual system call number. (The value you put in EAX before an
int 0x80 or
read can also refer to the raw system call, or to the libc wrapper function. In this context, for the rest of this answer, I'm talking about the raw system call.
Linux system call return values (in
EAX on x86) are either a positive value for success, or a negative error code. e.g.
-EFAULT if you pass an invalid pointer.
This behaviour is documented in the
syscalls(2) man page.
To find the actual numeric values of constants for a specific platform, you need to find the C header file where they're
#defined. See my answer on a question about that for details.
The meanings of return values for each sys call are documented in the section 2 man pages, like
sys_read is the raw system call that the glibc
read() function is a very thin wrapper for.) Most man pages have a whole section for the return value. e.g.
On success, the number of bytes read is returned (zero indicates end of file), and the file position is advanced by this number. It is not an error if this number is smaller than the number of bytes requested; this may happen for example because fewer bytes are
actually available right now (maybe because we were close to end-of-
file, or because we are reading from a pipe, or from a terminal), or
because read() was interrupted by a signal. See also NOTES.
On error, -1 is returned, and errno is set appropriately. In this case, it is left unspecified whether the file position (if any)
Note that the last paragraph describes how the glibc wrapper decodes the value and sets errno to
-EAX if the raw system call's return value is negative, so
errno=EFAULT and return
-1 if the raw system call returned
And there's a whole section listing all the possible error codes that
read() is allowed to return, and what they mean specifically for
read(). (POSIX standardizes most of this behaviour.)
I'm not sure exactly how the return value is decoded for a system call like
mmap(2), where the return value is not a signed type. (In this case a
void*). So it's not obvious how to distinguish an error return from a valid pointer that just happens to have the high bits set. Presumably there's some kind of way for the libc wrapper function to decode the value, and glibc knows the method.