This chapter describes how processes manage and use memory in a system that uses the GNU C library.
The GNU C Library has several functions for dynamically allocating virtual memory in various ways. They vary in generality and in efficiency. The library also provides functions for controlling paging and allocation of real memory.
Memory mapped I/O is not discussed in this chapter. See section Memory-mapped I/O.
One of the most basic resources a process has available to it is memory. There are a lot of different ways systems organize memory, but in a typical one, each process has one linear virtual address space, with addresses running from zero to some huge maximum. It need not be contiguous; i.e. not all of these addresses actually can be used to store data.
The virtual memory is divided into pages (4 kilobytes is typical). Backing each page of virtual memory is a page of real memory (called a frame) or some secondary storage, usually disk space. The disk space might be swap space or just some ordinary disk file. Actually, a page of all zeroes sometimes has nothing at all backing it -- there's just a flag saying it is all zeroes.
The same frame of real memory or backing store can back multiple virtual
pages belonging to multiple processes. This is normally the case, for
example, with virtual memory occupied by GNU C library code. The same
real memory frame containing the printf function backs a virtual
memory page in each of the existing processes that has a printf
call in its program.
In order for a program to access any part of a virtual page, the page must at that moment be backed by ("connected to") a real frame. But because there is usually a lot more virtual memory than real memory, the pages must move back and forth between real memory and backing store regularly, coming into real memory when a process needs to access them and then retreating to backing store when not needed anymore. This movement is called paging.
When a program attempts to access a page which is not at that moment backed by real memory, this is known as a page fault. When a page fault occurs, the kernel suspends the process, places the page into a real page frame (this is called "paging in" or "faulting in"), then resumes the process so that from the process' point of view, the page was in real memory all along. In fact, to the process, all pages always seem to be in real memory. Except for one thing: the elapsed execution time of an instruction that would normally be a few nanoseconds is suddenly much, much, longer (because the kernel normally has to do I/O to complete the page-in). For programs sensitive to that, the functions described in section Locking Pages can control it.
Within each virtual address space, a process has to keep track of what is at which addresses, and that process is called memory allocation. Allocation usually brings to mind meting out scarce resources, but in the case of virtual memory, that's not a major goal, because there is generally much more of it than anyone needs. Memory allocation within a process is mainly just a matter of making sure that the same byte of memory isn't used to store two different things.
Processes allocate memory in two major ways: by exec and programmatically. Actually, forking is a third way, but it's not very interesting. See section Creating a Process.
Exec is the operation of creating a virtual address space for a process,
loading its basic program into it, and executing the program. It is
done by the "exec" family of functions (e.g. execl). The
operation takes a program file (an executable), it allocates space to
load all the data in the executable, loads it, and transfers control to
it. That data is most notably the instructions of the program (the
text), but also literals and constants in the program and even
some variables: C variables with the static storage class (see section Memory Allocation in C Programs).
Once that program begins to execute, it uses programmatic allocation to gain additional memory. In a C program with the GNU C library, there are two kinds of programmatic allocation: automatic and dynamic. See section Memory Allocation in C Programs.
Memory-mapped I/O is another form of dynamic virtual memory allocation. Mapping memory to a file means declaring that the contents of certain range of a process' addresses shall be identical to the contents of a specified regular file. The system makes the virtual memory initially contain the contents of the file, and if you modify the memory, the system writes the same modification to the file. Note that due to the magic of virtual memory and page faults, there is no reason for the system to do I/O to read the file, or allocate real memory for its contents, until the program accesses the virtual memory. See section Memory-mapped I/O.
Just as it programmatically allocates memory, the program can programmatically deallocate (free) it. You can't free the memory that was allocated by exec. When the program exits or execs, you might say that all its memory gets freed, but since in both cases the address space ceases to exist, the point is really moot. See section Program Termination.
A process' virtual address space is divided into segments. A segment is a contiguous range of virtual addresses. Three important segments are:
This section covers how ordinary programs manage storage for their data,
including the famous malloc function and some fancier facilities
special the GNU C library and GNU Compiler.
The C language supports two kinds of memory allocation through the variables in C programs:
A third important kind of memory allocation, dynamic allocation, is not supported by C variables but is available via GNU C library functions.
Dynamic memory allocation is a technique in which programs determine as they are running where to store some information. You need dynamic allocation when the amount of memory you need, or how long you continue to need it, depends on factors that are not known before the program runs.
For example, you may need a block to store a line read from an input file; since there is no limit to how long a line can be, you must allocate the memory dynamically and make it dynamically larger as you read more of the line.
Or, you may need a block for each record or each definition in the input data; since you can't know in advance how many there will be, you must allocate a new block for each record or definition as you read it.
When you use dynamic allocation, the allocation of a block of memory is an action that the program requests explicitly. You call a function or macro when you want to allocate space, and specify the size with an argument. If you want to free the space, you do so by calling another function or macro. You can do these things whenever you want, as often as you want.
Dynamic allocation is not supported by C variables; there is no storage class "dynamic", and there can never be a C variable whose value is stored in dynamically allocated space. The only way to get dynamically allocated memory is via a system call (which is generally via a GNU C library function call), and the only way to refer to dynamically allocated space is through a pointer. Because it is less convenient, and because the actual process of dynamic allocation requires more computation time, programmers generally use dynamic allocation only when neither static nor automatic allocation will serve.
For example, if you want to allocate dynamically some space to hold a
struct foobar, you cannot declare a variable of type struct
foobar whose contents are the dynamically allocated space. But you can
declare a variable of pointer type struct foobar * and assign it the
address of the space. Then you can use the operators `*' and
`->' on this pointer variable to refer to the contents of the space:
{
struct foobar *ptr
= (struct foobar *) malloc (sizeof (struct foobar));
ptr->name = x;
ptr->next = current_foobar;
current_foobar = ptr;
}
The most general dynamic allocation facility is malloc. It
allows you to allocate blocks of memory of any size at any time, make
them bigger or smaller at any time, and free the blocks individually at
any time (or never).
To allocate a block of memory, call malloc. The prototype for
this function is in `stdlib.h'.
The contents of the block are undefined; you must initialize it yourself
(or use calloc instead; see section Allocating Cleared Space).
Normally you would cast the value as a pointer to the kind of object
that you want to store in the block. Here we show an example of doing
so, and of initializing the space with zeros using the library function
memset (see section Copying and Concatenation):
struct foo *ptr; ... ptr = (struct foo *) malloc (sizeof (struct foo)); if (ptr == 0) abort (); memset (ptr, 0, sizeof (struct foo));
You can store the result of malloc into any pointer variable
without a cast, because ISO C automatically converts the type
void * to another type of pointer when necessary. But the cast
is necessary in contexts other than assignment operators or if you might
want your code to run in traditional C.
Remember that when allocating space for a string, the argument to
malloc must be one plus the length of the string. This is
because a string is terminated with a null character that doesn't count
in the "length" of the string but does need space. For example:
char *ptr; ... ptr = (char *) malloc (length + 1);
See section Representation of Strings, for more information about this.
malloc
If no more space is available, malloc returns a null pointer.
You should check the value of every call to malloc. It is
useful to write a subroutine that calls malloc and reports an
error if the value is a null pointer, returning only if the value is
nonzero. This function is conventionally called xmalloc. Here
it is:
void *
xmalloc (size_t size)
{
register void *value = malloc (size);
if (value == 0)
fatal ("virtual memory exhausted");
return value;
}
Here is a real example of using malloc (by way of xmalloc).
The function savestring will copy a sequence of characters into
a newly allocated null-terminated string:
char *
savestring (const char *ptr, size_t len)
{
register char *value = (char *) xmalloc (len + 1);
value[len] = '\0';
return (char *) memcpy (value, ptr, len);
}
The block that malloc gives you is guaranteed to be aligned so
that it can hold any type of data. In the GNU system, the address is
always a multiple of eight on most systems, and a multiple of 16 on
64-bit systems. Only rarely is any higher boundary (such as a page
boundary) necessary; for those cases, use memalign,
posix_memalign or valloc (see section Allocating Aligned Memory Blocks).
Note that the memory located after the end of the block is likely to be
in use for something else; perhaps a block already allocated by another
call to malloc. If you attempt to treat the block as longer than
you asked for it to be, you are liable to destroy the data that
malloc uses to keep track of its blocks, or you may destroy the
contents of another block. If you have already allocated a block and
discover you want it to be bigger, use realloc (see section Changing the Size of a Block).
malloc
When you no longer need a block that you got with malloc, use the
function free to make the block available to be allocated again.
The prototype for this function is in `stdlib.h'.
free function deallocates the block of memory pointed at
by ptr.
free. It's provided for
backward compatibility with SunOS; you should use free instead.
Freeing a block alters the contents of the block. Do not expect to find any data (such as a pointer to the next block in a chain of blocks) in the block after freeing it. Copy whatever you need out of the block before freeing it! Here is an example of the proper way to free all the blocks in a chain, and the strings that they point to:
struct chain
{
struct chain *next;
char *name;
}
void
free_chain (struct chain *chain)
{
while (chain != 0)
{
struct chain *next = chain->next;
free (chain->name);
free (chain);
chain = next;
}
}
Occasionally, free can actually return memory to the operating
system and make the process smaller. Usually, all it can do is allow a
later call to malloc to reuse the space. In the meantime, the
space remains in your program as part of a free-list used internally by
malloc.
There is no point in freeing blocks at the end of a program, because all of the program's space is given back to the system when the process terminates.
Often you do not know for certain how big a block you will ultimately need at the time you must begin to use the block. For example, the block might be a buffer that you use to hold a line being read from a file; no matter how long you make the buffer initially, you may encounter a line that is longer.
You can make the block longer by calling realloc. This function
is declared in `stdlib.h'.
realloc function changes the size of the block whose address is
ptr to be newsize.
Since the space after the end of the block may be in use, realloc
may find it necessary to copy the block to a new address where more free
space is available. The value of realloc is the new address of the
block. If the block needs to be moved, realloc copies the old
contents.
If you pass a null pointer for ptr, realloc behaves just
like `malloc (newsize)'. This can be convenient, but beware
that older implementations (before ISO C) may not support this
behavior, and will probably crash when realloc is passed a null
pointer.
Like malloc, realloc may return a null pointer if no
memory space is available to make the block bigger. When this happens,
the original block is untouched; it has not been modified or relocated.
In most cases it makes no difference what happens to the original block
when realloc fails, because the application program cannot continue
when it is out of memory, and the only thing to do is to give a fatal error
message. Often it is convenient to write and use a subroutine,
conventionally called xrealloc, that takes care of the error message
as xmalloc does for malloc:
void *
xrealloc (void *ptr, size_t size)
{
register void *value = realloc (ptr, size);
if (value == 0)
fatal ("Virtual memory exhausted");
return value;
}
You can also use realloc to make a block smaller. The reason you
would do this is to avoid tying up a lot of memory space when only a little
is needed.
In several allocation implementations, making a block smaller sometimes
necessitates copying it, so it can fail if no other space is available.
If the new size you specify is the same as the old size, realloc
is guaranteed to change nothing and return the same address that you gave.
The function calloc allocates memory and clears it to zero. It
is declared in `stdlib.h'.
calloc returns.
You could define calloc as follows:
void *
calloc (size_t count, size_t eltsize)
{
size_t size = count * eltsize;
void *value = malloc (size);
if (value != 0)
memset (value, 0, size);
return value;
}
But in general, it is not guaranteed that calloc calls
malloc internally. Therefore, if an application provides its own
malloc/realloc/free outside the C library, it
should always define calloc, too.
malloc
As opposed to other versions, the malloc in the GNU C Library
does not round up block sizes to powers of two, neither for large nor
for small sizes. Neighboring chunks can be coalesced on a free
no matter what their size is. This makes the implementation suitable
for all kinds of allocation patterns without generally incurring high
memory waste through fragmentation.
Very large blocks (much larger than a page) are allocated with
mmap (anonymous or via /dev/zero) by this implementation.
This has the great advantage that these chunks are returned to the
system immediately when they are freed. Therefore, it cannot happen
that a large chunk becomes "locked" in between smaller ones and even
after calling free wastes memory. The size threshold for
mmap to be used can be adjusted with mallopt. The use of
mmap can also be disabled completely.
The address of a block returned by malloc or realloc in
the GNU system is always a multiple of eight (or sixteen on 64-bit
systems). If you need a block whose address is a multiple of a higher
power of two than that, use memalign, posix_memalign, or
valloc. These functions are declared in `stdlib.h'.
With the GNU library, you can use free to free the blocks that
memalign, posix_memalign, and valloc return. That
does not work in BSD, however--BSD does not provide any way to free
such blocks.
memalign function allocates a block of size bytes whose
address is a multiple of boundary. The boundary must be a
power of two! The function memalign works by allocating a
somewhat larger block, and then returning an address within the block
that is on the specified boundary.
posix_memalign function is similar to the memalign
function in that it returns a buffer of size bytes aligned to a
multiple of alignment. But it adds one requirement to the
parameter alignment: the value must be a power of two multiple of
sizeof (void *).
If the function succeeds in allocation memory a pointer to the allocated
memory is returned in *memptr and the return value is zero.
Otherwise the function returns an error value indicating the problem.
This function was introduced in POSIX 1003.1d.
valloc is like using memalign and passing the page size
as the value of the second argument. It is implemented like this:
void *
valloc (size_t size)
{
return memalign (getpagesize (), size);
}
section How to get information about the memory subsystem? for more information about the memory subsystem.
You can adjust some parameters for dynamic memory allocation with the
mallopt function. This function is the general SVID/XPG
interface, defined in `malloc.h'.
mallopt, the param argument specifies the
parameter to be set, and value the new value to be set. Possible
choices for param, as defined in `malloc.h', are:
M_TRIM_THRESHOLD
sbrk to be called with a negative argument in
order to return memory to the system.
M_TOP_PAD
sbrk is required. It also specifies the
number of bytes to retain when shrinking the heap by calling sbrk
with a negative argument. This provides the necessary hysteresis in
heap size such that excessive amounts of system calls can be avoided.
M_MMAP_THRESHOLD
mmap system call. This way it is guaranteed
that the memory for these chunks can be returned to the system on
free.
M_MMAP_MAX
mmap. Setting this
to zero disables all use of mmap.
You can ask malloc to check the consistency of dynamic memory by
using the mcheck function. This function is a GNU extension,
declared in `mcheck.h'.
mcheck tells malloc to perform occasional
consistency checks. These will catch things such as writing
past the end of a block that was allocated with malloc.
The abortfn argument is the function to call when an inconsistency
is found. If you supply a null pointer, then mcheck uses a
default function which prints a message and calls abort
(see section Aborting a Program). The function you supply is called with
one argument, which says what sort of inconsistency was detected; its
type is described below.
It is too late to begin allocation checking once you have allocated
anything with malloc. So mcheck does nothing in that
case. The function returns -1 if you call it too late, and
0 otherwise (when it is successful).
The easiest way to arrange to call mcheck early enough is to use
the option `-lmcheck' when you link your program; then you don't
need to modify your program source at all. Alternatively you might use
a debugger to insert a call to mcheck whenever the program is
started, for example these gdb commands will automatically call mcheck
whenever the program starts:
(gdb) break main Breakpoint 1, main (argc=2, argv=0xbffff964) at whatever.c:10 (gdb) command 1 Type commands for when breakpoint 1 is hit, one per line. End with a line saying just "end". >call mcheck(0) >continue >end (gdb) ...
This will however only work if no initialization function of any object
involved calls any of the malloc functions since mcheck
must be called before the first such function.
mprobe function lets you explicitly check for inconsistencies
in a particular allocated block. You must have already called
mcheck at the beginning of the program, to do its occasional
checks; calling mprobe requests an additional consistency check
to be done at the time of the call.
The argument pointer must be a pointer returned by malloc
or realloc. mprobe returns a value that says what
inconsistency, if any, was found. The values are described below.
MCHECK_DISABLED
mcheck was not called before the first allocation.
No consistency checking can be done.
MCHECK_OK
MCHECK_HEAD
MCHECK_TAIL
MCHECK_FREE
Another possibility to check for and guard against bugs in the use of
malloc, realloc and free is to set the environment
variable MALLOC_CHECK_. When MALLOC_CHECK_ is set, a
special (less efficient) implementation is used which is designed to be
tolerant against simple errors, such as double calls of free with
the same argument, or overruns of a single byte (off-by-one bugs). Not
all such errors can be protected against, however, and memory leaks can
result. If MALLOC_CHECK_ is set to 0, any detected heap
corruption is silently ignored; if set to 1, a diagnostic is
printed on stderr; if set to 2, abort is called
immediately. This can be useful because otherwise a crash may happen
much later, and the true cause for the problem is then very hard to
track down.
There is one problem with MALLOC_CHECK_: in SUID or SGID binaries
it could possibly be exploited since diverging from the normal programs
behaviour it now writes something to the standard error desriptor.
Therefore the use of MALLOC_CHECK_ is disabled by default for
SUID and SGID binaries. It can be enabled again by the system
administrator by adding a file `/etc/suid-debug' (the content is
not important it could be empty).
So, what's the difference between using MALLOC_CHECK_ and linking
with `-lmcheck'? MALLOC_CHECK_ is orthogonal with respect to
`-lmcheck'. `-lmcheck' has been added for backward
compatibility. Both MALLOC_CHECK_ and `-lmcheck' should
uncover the same bugs - but using MALLOC_CHECK_ you don't need to
recompile your application.
The GNU C library lets you modify the behavior of malloc,
realloc, and free by specifying appropriate hook
functions. You can use these hooks to help you debug programs that use
dynamic memory allocation, for example.
The hook variables are declared in `malloc.h'.
malloc uses whenever it is called. You should define this
function to look like malloc; that is, like:
void *function (size_t size, const void *caller)
The value of caller is the return address found on the stack when
the malloc function was called. This value allows you to trace
the memory consumption of the program.
realloc
uses whenever it is called. You should define this function to look
like realloc; that is, like:
void *function (void *ptr, size_t size, const void *caller)
The value of caller is the return address found on the stack when
the realloc function was called. This value allows you to trace the
memory consumption of the program.
free
uses whenever it is called. You should define this function to look
like free; that is, like:
void function (void *ptr, const void *caller)
The value of caller is the return address found on the stack when
the free function was called. This value allows you to trace the
memory consumption of the program.
memalign
uses whenever it is called. You should define this function to look
like memalign; that is, like:
void *function (size_t size, size_t alignment, const void *caller)
The value of caller is the return address found on the stack when
the memalign function was called. This value allows you to trace the
memory consumption of the program.
You must make sure that the function you install as a hook for one of these functions does not call that function recursively without restoring the old value of the hook first! Otherwise, your program will get stuck in an infinite recursion. Before calling the function recursively, one should make sure to restore all the hooks to their previous value. When coming back from the recursive call, all the hooks should be resaved since a hook might modify itself.
void (*__malloc_initialize_hook) (void) = my_init_hook;
An issue to look out for is the time at which the malloc hook functions
can be safely installed. If the hook functions call the malloc-related
functions recursively, it is necessary that malloc has already properly
initialized itself at the time when __malloc_hook etc. is
assigned to. On the other hand, if the hook functions provide a
complete malloc implementation of their own, it is vital that the hooks
are assigned to before the very first malloc call has
completed, because otherwise a chunk obtained from the ordinary,
un-hooked malloc may later be handed to __free_hook, for example.
In both cases, the problem can be solved by setting up the hooks from
within a user-defined function pointed to by
__malloc_initialize_hook---then the hooks will be set up safely
at the right time.
Here is an example showing how to use __malloc_hook and
__free_hook properly. It installs a function that prints out
information every time malloc or free is called. We just
assume here that realloc and memalign are not used in our
program.
/* Prototypes for __malloc_hook, __free_hook */
#include <malloc.h>
/* Prototypes for our hooks. */
static void *my_init_hook (void);
static void *my_malloc_hook (size_t, const void *);
static void my_free_hook (void*, const void *);
/* Override initializing hook from the C library. */
void (*__malloc_initialize_hook) (void) = my_init_hook;
static void
my_init_hook (void)
{
old_malloc_hook = __malloc_hook;
old_free_hook = __free_hook;
__malloc_hook = my_malloc_hook;
__free_hook = my_free_hook;
}
static void *
my_malloc_hook (size_t size, const void *caller)
{
void *result;
/* Restore all old hooks */
__malloc_hook = old_malloc_hook;
__free_hook = old_free_hook;
/* Call recursively */
result = malloc (size);
/* Save underlaying hooks */
old_malloc_hook = __malloc_hook;
old_free_hook = __free_hook;
/* printf might call malloc, so protect it too. */
printf ("malloc (%u) returns %p\n", (unsigned int) size, result);
/* Restore our own hooks */
__malloc_hook = my_malloc_hook;
__free_hook = my_free_hook;
return result;
}
static void *
my_free_hook (void *ptr, const void *caller)
{
/* Restore all old hooks */
__malloc_hook = old_malloc_hook;
__free_hook = old_free_hook;
/* Call recursively */
free (ptr);
/* Save underlaying hooks */
old_malloc_hook = __malloc_hook;
old_free_hook = __free_hook;
/* printf might call free, so protect it too. */
printf ("freed pointer %p\n", ptr);
/* Restore our own hooks */
__malloc_hook = my_malloc_hook;
__free_hook = my_free_hook;
}
main ()
{
...
}
The mcheck function (see section Heap Consistency Checking) works by
installing such hooks.
malloc
You can get information about dynamic memory allocation by calling the
mallinfo function. This function and its associated data type
are declared in `malloc.h'; they are an extension of the standard
SVID/XPG version.
int arena
sbrk by
malloc, in bytes.
int ordblks
malloc requests; see
section Efficiency Considerations for malloc.)
int smblks
int hblks
mmap.
int hblkhd
mmap, in bytes.
int usmblks
int fsmblks
int uordblks
malloc.
int fordblks
int keepcost
struct mallinfo.
malloc-Related Functions
Here is a summary of the functions that work with malloc:
void *malloc (size_t size)
void free (void *addr)
malloc. See section Freeing Memory Allocated with malloc.
void *realloc (void *addr, size_t size)
malloc larger or smaller,
possibly by copying it to a new location. See section Changing the Size of a Block.
void *calloc (size_t count, size_t eltsize)
malloc, and set its contents to zero. See section Allocating Cleared Space.
void *valloc (size_t size)
void *memalign (size_t size, size_t boundary)
int mallopt (int param, int value)
int mcheck (void (*abortfn) (void))
malloc to perform occasional consistency checks on
dynamically allocated memory, and to call abortfn when an
inconsistency is found. See section Heap Consistency Checking.
void *(*__malloc_hook) (size_t size, const void *caller)
malloc uses whenever it is called.
void *(*__realloc_hook) (void *ptr, size_t size, const void *caller)
realloc uses whenever it is called.
void (*__free_hook) (void *ptr, const void *caller)
free uses whenever it is called.
void (*__memalign_hook) (size_t size, size_t alignment, const void *caller)
memalign uses whenever it is called.
struct mallinfo mallinfo (void)
malloc.
A complicated task when programming with languages which do not use garbage collected dynamic memory allocation is to find memory leaks. Long running programs must assure that dynamically allocated objects are freed at the end of their lifetime. If this does not happen the system runs out of memory, sooner or later.
The malloc implementation in the GNU C library provides some
simple means to detect such leaks and obtain some information to find
the location. To do this the application must be started in a special
mode which is enabled by an environment variable. There are no speed
penalties for the program if the debugging mode is not enabled.
mtrace function is called it looks for an environment
variable named MALLOC_TRACE. This variable is supposed to
contain a valid file name. The user must have write access. If the
file already exists it is truncated. If the environment variable is not
set or it does not name a valid file which can be opened for writing
nothing is done. The behaviour of malloc etc. is not changed.
For obvious reasons this also happens if the application is installed
with the SUID or SGID bit set.
If the named file is successfully opened, mtrace installs special
handlers for the functions malloc, realloc, and
free (see section Memory Allocation Hooks). From then on, all uses of these
functions are traced and protocolled into the file. There is now of
course a speed penalty for all calls to the traced functions so tracing
should not be enabled during normal use.
This function is a GNU extension and generally not available on other systems. The prototype can be found in `mcheck.h'.
muntrace function can be called after mtrace was used
to enable tracing the malloc calls. If no (succesful) call of
mtrace was made muntrace does nothing.
Otherwise it deinstalls the handlers for malloc, realloc,
and free and then closes the protocol file. No calls are
protocolled anymore and the program runs again at full speed.
This function is a GNU extension and generally not available on other systems. The prototype can be found in `mcheck.h'.
Even though the tracing functionality does not influence the runtime
behaviour of the program it is not a good idea to call mtrace in
all programs. Just imagine that you debug a program using mtrace
and all other programs used in the debugging session also trace their
malloc calls. The output file would be the same for all programs
and thus is unusable. Therefore one should call mtrace only if
compiled for debugging. A program could therefore start like this:
#include <mcheck.h>
int
main (int argc, char *argv[])
{
#ifdef DEBUGGING
mtrace ();
#endif
...
}
This is all what is needed if you want to trace the calls during the
whole runtime of the program. Alternatively you can stop the tracing at
any time with a call to muntrace. It is even possible to restart
the tracing again with a new call to mtrace. But this can cause
unreliable results since there may be calls of the functions which are
not called. Please note that not only the application uses the traced
functions, also libraries (including the C library itself) use these
functions.
This last point is also why it is no good idea to call muntrace
before the program terminated. The libraries are informed about the
termination of the program only after the program returns from
main or calls exit and so cannot free the memory they use
before this time.
So the best thing one can do is to call mtrace as the very first
function in the program and never call muntrace. So the program
traces almost all uses of the malloc functions (except those
calls which are executed by constructors of the program or used
libraries).
You know the situation. The program is prepared for debugging and in all debugging sessions it runs well. But once it is started without debugging the error shows up. A typical example is a memory leak that becomes visible only when we turn off the debugging. If you foresee such situations you can still win. Simply use something equivalent to the following little program:
#include <mcheck.h>
#include <signal.h>
static void
enable (int sig)
{
mtrace ();
signal (SIGUSR1, enable);
}
static void
disable (int sig)
{
muntrace ();
signal (SIGUSR2, disable);
}
int
main (int argc, char *argv[])
{
...
signal (SIGUSR1, enable);
signal (SIGUSR2, disable);
...
}
I.e., the user can start the memory debugger any time s/he wants if the
program was started with MALLOC_TRACE set in the environment.
The output will of course not show the allocations which happened before
the first signal but if there is a memory leak this will show up
nevertheless.
If you take a look at the output it will look similar to this:
= Start [0x8048209] - 0x8064cc8 [0x8048209] - 0x8064ce0 [0x8048209] - 0x8064cf8 [0x80481eb] + 0x8064c48 0x14 [0x80481eb] + 0x8064c60 0x14 [0x80481eb] + 0x8064c78 0x14 [0x80481eb] + 0x8064c90 0x14 = End
What this all means is not really important since the trace file is not
meant to be read by a human. Therefore no attention is given to
readability. Instead there is a program which comes with the GNU C
library which interprets the traces and outputs a summary in an
user-friendly way. The program is called mtrace (it is in fact a
Perl script) and it takes one or two arguments. In any case the name of
the file with the trace output must be specified. If an optional
argument precedes the name of the trace file this must be the name of
the program which generated the trace.
drepper$ mtrace tst-mtrace log No memory leaks.
In this case the program tst-mtrace was run and it produced a
trace file `log'. The message printed by mtrace shows there
are no problems with the code, all allocated memory was freed
afterwards.
If we call mtrace on the example trace given above we would get a
different outout:
drepper$ mtrace errlog - 0x08064cc8 Free 2 was never alloc'd 0x8048209 - 0x08064ce0 Free 3 was never alloc'd 0x8048209 - 0x08064cf8 Free 4 was never alloc'd 0x8048209 Memory not freed: ----------------- Address Size Caller 0x08064c48 0x14 at 0x80481eb 0x08064c60 0x14 at 0x80481eb 0x08064c78 0x14 at 0x80481eb 0x08064c90 0x14 at 0x80481eb
We have called mtrace with only one argument and so the script
has no chance to find out what is meant with the addresses given in the
trace. We can do better:
drepper$ mtrace tst errlog - 0x08064cc8 Free 2 was never alloc'd /home/drepper/tst.c:39 - 0x08064ce0 Free 3 was never alloc'd /home/drepper/tst.c:39 - 0x08064cf8 Free 4 was never alloc'd /home/drepper/tst.c:39 Memory not freed: ----------------- Address Size Caller 0x08064c48 0x14 at /home/drepper/tst.c:33 0x08064c60 0x14 at /home/drepper/tst.c:33 0x08064c78 0x14 at /home/drepper/tst.c:33 0x08064c90 0x14 at /home/drepper/tst.c:33
Suddenly the output makes much more sense and the user can see immediately where the function calls causing the trouble can be found.
Interpreting this output is not complicated. There are at most two
different situations being detected. First, free was called for
pointers which were never returned by one of the allocation functions.
This is usually a very bad problem and what this looks like is shown in
the first three lines of the output. Situations like this are quite
rare and if they appear they show up very drastically: the program
normally crashes.
The other situation which is much harder to detect are memory leaks. As
you can see in the output the mtrace function collects all this
information and so can say that the program calls an allocation function
from line 33 in the source file `/home/drepper/tst-mtrace.c' four
times without freeing this memory before the program terminates.
Whether this is a real problem remains to be investigated.
An obstack is a pool of memory containing a stack of objects. You can create any number of separate obstacks, and then allocate objects in specified obstacks. Within each obstack, the last object allocated must always be the first one freed, but distinct obstacks are independent of each other.
Aside from this one constraint of order of freeing, obstacks are totally general: an obstack can contain any number of objects of any size. They are implemented with macros, so allocation is usually very fast as long as the objects are usually small. And the only space overhead per object is the padding needed to start each object on a suitable boundary.
The utilities for manipulating obstacks are declared in the header file `obstack.h'.
struct
obstack. This structure has a small fixed size; it records the status
of the obstack and how to find the space in which objects are allocated.
It does not contain any of the objects themselves. You should not try
to access the contents of the structure directly; use only the functions
described in this chapter.
You can declare variables of type struct obstack and use them as
obstacks, or you can allocate obstacks dynamically like any other kind
of object. Dynamic allocation of obstacks allows your program to have a
variable number of different stacks. (You can even allocate an
obstack structure in another obstack, but this is rarely useful.)
All the functions that work with obstacks require you to specify which
obstack to use. You do this with a pointer of type struct obstack
*. In the following, we often say "an obstack" when strictly
speaking the object at hand is such a pointer.
The objects in the obstack are packed into large blocks called
chunks. The struct obstack structure points to a chain of
the chunks currently in use.
The obstack library obtains a new chunk whenever you allocate an object
that won't fit in the previous chunk. Since the obstack library manages
chunks automatically, you don't need to pay much attention to them, but
you do need to supply a function which the obstack library should use to
get a chunk. Usually you supply a function which uses malloc
directly or indirectly. You must also supply a function to free a chunk.
These matters are described in the following section.
Each source file in which you plan to use the obstack functions must include the header file `obstack.h', like this:
#include <obstack.h>
Also, if the source file uses the macro obstack_init, it must
declare or define two functions or macros that will be called by the
obstack library. One, obstack_chunk_alloc, is used to allocate
the chunks of memory into which objects are packed. The other,
obstack_chunk_free, is used to return chunks when the objects in
them are freed. These macros should appear before any use of obstacks
in the source file.
Usually these are defined to use malloc via the intermediary
xmalloc (see section Unconstrained Allocation). This is done with
the following pair of macro definitions:
#define obstack_chunk_alloc xmalloc #define obstack_chunk_free free
Though the memory you get using obstacks really comes from malloc,
using obstacks is faster because malloc is called less often, for
larger blocks of memory. See section Obstack Chunks, for full details.
At run time, before the program can use a struct obstack object
as an obstack, it must initialize the obstack by calling
obstack_init.
obstack_chunk_alloc function. If
allocation of memory fails, the function pointed to by
obstack_alloc_failed_handler is called. The obstack_init
function always returns 1 (Compatibility notice: Former versions of
obstack returned 0 if allocation failed).
Here are two examples of how to allocate the space for an obstack and initialize it. First, an obstack that is a static variable:
static struct obstack myobstack; ... obstack_init (&myobstack);
Second, an obstack that is itself dynamically allocated:
struct obstack *myobstack_ptr = (struct obstack *) xmalloc (sizeof (struct obstack)); obstack_init (myobstack_ptr);
obstack uses when obstack_chunk_alloc fails to allocate
memory. The default action is to print a message and abort.
You should supply a function that either calls exit
(see section Program Termination) or longjmp (see section Non-Local Exits) and doesn't return.
void my_obstack_alloc_failed (void) ... obstack_alloc_failed_handler = &my_obstack_alloc_failed;
The most direct way to allocate an object in an obstack is with
obstack_alloc, which is invoked almost like malloc.
struct obstack
object which represents the obstack. Each obstack function or macro
requires you to specify an obstack-ptr as the first argument.
This function calls the obstack's obstack_chunk_alloc function if
it needs to allocate a new chunk of memory; it calls
obstack_alloc_failed_handler if allocation of memory by
obstack_chunk_alloc failed.
For example, here is a function that allocates a copy of a string str
in a specific obstack, which is in the variable string_obstack:
struct obstack string_obstack;
char *
copystring (char *string)
{
size_t len = strlen (string) + 1;
char *s = (char *) obstack_alloc (&string_obstack, len);
memcpy (s, string, len);
return s;
}
To allocate a block with specified contents, use the function
obstack_copy, declared like this:
obstack_alloc_failed_handler if allocation of memory by
obstack_chunk_alloc failed.
obstack_copy, but appends an extra byte containing a null
character. This extra byte is not counted in the argument size.
The obstack_copy0 function is convenient for copying a sequence
of characters into an obstack as a null-terminated string. Here is an
example of its use:
char *
obstack_savestring (char *addr, int size)
{
return obstack_copy0 (&myobstack, addr, size);
}
Contrast this with the previous example of savestring using
malloc (see section Basic Memory Allocation).
To free an object allocated in an obstack, use the function
obstack_free. Since the obstack is a stack of objects, freeing
one object automatically frees all other objects allocated more recently
in the same obstack.
Note that if object is a null pointer, the result is an
uninitialized obstack. To free all memory in an obstack but leave it
valid for further allocation, call obstack_free with the address
of the first object allocated on the obstack:
obstack_free (obstack_ptr, first_object_allocated_ptr);
Recall that the objects in an obstack are grouped into chunks. When all the objects in a chunk become free, the obstack library automatically frees the chunk (see section Preparing for Using Obstacks). Then other obstacks, or non-obstack allocation, can reuse the space of the chunk.
The interfaces for using obstacks may be defined either as functions or as macros, depending on the compiler. The obstack facility works with all C compilers, including both ISO C and traditional C, but there are precautions you must take if you plan to use compilers other than GNU C.
If you are using an old-fashioned non-ISO C compiler, all the obstack "functions" are actually defined only as macros. You can call these macros like functions, but you cannot use them in any other way (for example, you cannot take their address).
Calling the macros requires a special precaution: namely, the first operand (the obstack pointer) may not contain any side effects, because it may be computed more than once. For example, if you write this:
obstack_alloc (get_obstack (), 4);
you will find that get_obstack may be called several times.
If you use *obstack_list_ptr++ as the obstack pointer argument,
you will get very strange results since the incrementation may occur
several times.
In ISO C, each function has both a macro definition and a function definition. The function definition is used if you take the address of the function without calling it. An ordinary call uses the macro definition by default, but you can request the function definition instead by writing the function name in parentheses, as shown here:
char *x; void *(*funcp) (); /* Use the macro. */ x = (char *) obstack_alloc (obptr, size); /* Call the function. */ x = (char *) (obstack_alloc) (obptr, size); /* Take the address of the function. */ funcp = obstack_alloc;
This is the same situation that exists in ISO C for the standard library functions. See section Macro Definitions of Functions.
Warning: When you do use the macros, you must observe the precaution of avoiding side effects in the first operand, even in ISO C.
If you use the GNU C compiler, this precaution is not necessary, because various language extensions in GNU C permit defining the macros so as to compute each argument only once.
Because memory in obstack chunks is used sequentially, it is possible to build up an object step by step, adding one or more bytes at a time to the end of the object. With this technique, you do not need to know how much data you will put in the object until you come to the end of it. We call this the technique of growing objects. The special functions for adding data to the growing object are described in this section.
You don't need to do anything special when you start to grow an object.
Using one of the functions to add data to the object automatically
starts it. However, it is necessary to say explicitly when the object is
finished. This is done with the function obstack_finish.
The actual address of the object thus built up is not known until the object is finished. Until then, it always remains possible that you will add so much data that the object must be copied into a new chunk.
While the obstack is in use for a growing object, you cannot use it for ordinary allocation of another object. If you try to do so, the space already added to the growing object will become part of the other object.
obstack_blank, which adds space without initializing it.
obstack_grow, which is
the growing-object analogue of obstack_copy. It adds size
bytes of data to the growing object, copying the contents from
data.
obstack_copy0. It adds
size bytes copied from data, followed by an additional null
character.
obstack_1grow.
It adds a single byte containing c to the growing object.
obstack_ptr_grow. It adds sizeof (void *) bytes
containing the value of data.
int can be added by using the
obstack_int_grow function. It adds sizeof (int) bytes to
the growing object and initializes them with the value of data.
obstack_finish to close it off and return its final address.
Once you have finished the object, the obstack is available for ordinary allocation or for growing another object.
This function can return a null pointer under the same conditions as
obstack_alloc (see section Allocation in an Obstack).
When you build an object by growing it, you will probably need to know
afterward how long it became. You need not keep track of this as you grow
the object, because you can find out the length from the obstack just
before finishing the object with the function obstack_object_size,
declared as follows:
obstack_object_size will return zero.
If you have started growing an object and wish to cancel it, you should finish it and then free it, like this:
obstack_free (obstack_ptr, obstack_finish (obstack_ptr));
This has no effect if no object was growing.
You can use obstack_blank with a negative size argument to make
the current object smaller. Just don't try to shrink it beyond zero
length--there's no telling what will happen if you do that.
The usual functions for growing objects incur overhead for checking whether there is room for the new growth in the current chunk. If you are frequently constructing objects in small steps of growth, this overhead can be significant.
You can reduce the overhead by using special "fast growth" functions that grow the object without checking. In order to have a robust program, you must do the checking yourself. If you do this checking in the simplest way each time you are about to add data to the object, you have not saved anything, because that is what the ordinary growth functions do. But if you can arrange to check less often, or check more efficiently, then you make the program faster.
The function obstack_room returns the amount of room available
in the current chunk. It is declared as follows:
While you know there is room, you can use these fast growth functions for adding data to a growing object:
obstack_1grow_fast adds one byte containing the
character c to the growing object in obstack obstack-ptr.
obstack_ptr_grow_fast adds sizeof (void *)
bytes containing the value of data to the growing object in
obstack obstack-ptr.
obstack_int_grow_fast adds sizeof (int) bytes
containing the value of data to the growing object in obstack
obstack-ptr.
obstack_blank_fast adds size bytes to the
growing object in obstack obstack-ptr without initializing them.
When you check for space using obstack_room and there is not
enough room for what you want to add, the fast growth functions
are not safe. In this case, simply use the corresponding ordinary
growth function instead. Very soon this will copy the object to a
new chunk; then there will be lots of room available again.
So, each time you use an ordinary growth function, check afterward for
sufficient space using obstack_room. Once the object is copied
to a new chunk, there will be plenty of space again, so the program will
start using the fast growth functions again.
Here is an example:
void
add_string (struct obstack *obstack, const char *ptr, int len)
{
while (len > 0)
{
int room = obstack_room (obstack);
if (room == 0)
{
/* Not enough room. Add one character slowly,
which may copy to a new chunk and make room. */
obstack_1grow (obstack, *ptr++);
len--;
}
else
{
if (room > len)
room = len;
/* Add fast as much as we have room for. */
len -= room;
while (room-- > 0)
obstack_1grow_fast (obstack, *ptr++);
}
}
}
Here are functions that provide information on the current status of allocation in an obstack. You can use them to learn about an object while still growing it.
If no object is growing, this value says where the next object you allocate will start (once again assuming it fits in the current chunk).
obstack_next_free
returns the same value as obstack_base.
obstack_next_free (obstack-ptr) - obstack_base (obstack-ptr)
Each obstack has an alignment boundary; each object allocated in the obstack automatically starts on an address that is a multiple of the specified boundary. By default, this boundary is 4 bytes.
To access an obstack's alignment boundary, use the macro
obstack_alignment_mask, whose function prototype looks like
this:
The expansion of the macro obstack_alignment_mask is an lvalue,
so you can alter the mask by assignment. For example, this statement:
obstack_alignment_mask (obstack_ptr) = 0;
has the effect of turning off alignment processing in the specified obstack.
Note that a change in alignment mask does not take effect until
after the next time an object is allocated or finished in the
obstack. If you are not growing an object, you can make the new
alignment mask take effect immediately by calling obstack_finish.
This will finish a zero-length object and then do proper alignment for
the next object.
Obstacks work by allocating space for themselves in large chunks, and then parceling out space in the chunks to satisfy your requests. Chunks are normally 4096 bytes long unless you specify a different chunk size. The chunk size includes 8 bytes of overhead that are not actually used for storing objects. Regardless of the specified size, longer chunks will be allocated when necessary for long objects.
The obstack library allocates chunks by calling the function
obstack_chunk_alloc, which you must define. When a chunk is no
longer needed because you have freed all the objects in it, the obstack
library frees the chunk by calling obstack_chunk_free, which you
must also define.
These two must be defined (as macros) or declared (as functions) in each
source file that uses obstack_init (see section Creating Obstacks).
Most often they are defined as macros like this:
#define obstack_chunk_alloc malloc #define obstack_chunk_free free
Note that these are simple macros (no arguments). Macro definitions with
arguments will not work! It is necessary that obstack_chunk_alloc
or obstack_chunk_free, alone, expand into a function name if it is
not itself a function name.
If you allocate chunks with malloc, the chunk size should be a
power of 2. The default chunk size, 4096, was chosen because it is long
enough to satisfy many typical requests on the obstack yet short enough
not to waste too much memory in the portion of the last chunk not yet used.
Since this macro expands to an lvalue, you can specify a new chunk size by assigning it a new value. Doing so does not affect the chunks already allocated, but will change the size of chunks allocated for that particular obstack in the future. It is unlikely to be useful to make the chunk size smaller, but making it larger might improve efficiency if you are allocating many objects whose size is comparable to the chunk size. Here is how to do so cleanly:
if (obstack_chunk_size (obstack_ptr) < new-chunk-size) obstack_chunk_size (obstack_ptr) = new-chunk-size;
Here is a summary of all the functions associated with obstacks. Each
takes the address of an obstack (struct obstack *) as its first
argument.
void obstack_init (struct obstack *obstack-ptr)
void *obstack_alloc (struct obstack *obstack-ptr, int size)
void *obstack_copy (struct obstack *obstack-ptr, void *address, int size)
void *obstack_copy0 (struct obstack *obstack-ptr, void *address, int size)
void obstack_free (struct obstack *obstack-ptr, void *object)
void obstack_blank (struct obstack *obstack-ptr, int size)
void obstack_grow (struct obstack *obstack-ptr, void *address, int size)
void obstack_grow0 (struct obstack *obstack-ptr, void *address, int size)
void obstack_1grow (struct obstack *obstack-ptr, char data-char)
void *obstack_finish (struct obstack *obstack-ptr)
int obstack_object_size (struct obstack *obstack-ptr)
void obstack_blank_fast (struct obstack *obstack-ptr, int size)
void obstack_1grow_fast (struct obstack *obstack-ptr, char data-char)
int obstack_room (struct obstack *obstack-ptr)
int obstack_alignment_mask (struct obstack *obstack-ptr)
int obstack_chunk_size (struct obstack *obstack-ptr)
void *obstack_base (struct obstack *obstack-ptr)
void *obstack_next_free (struct obstack *obstack-ptr)
The function alloca supports a kind of half-dynamic allocation in
which blocks are allocated dynamically but freed automatically.
Allocating a block with alloca is an explicit action; you can
allocate as many blocks as you wish, and compute the size at run time. But
all the blocks are freed when you exit the function that alloca was
called from, just as if they were automatic variables declared in that
function. There is no way to free the space explicitly.
The prototype for alloca is in `stdlib.h'. This function is
a BSD extension.
alloca is the address of a block of size
bytes of memory, allocated in the stack frame of the calling function.
Do not use alloca inside the arguments of a function call--you
will get unpredictable results, because the stack space for the
alloca would appear on the stack in the middle of the space for
the function arguments. An example of what to avoid is foo (x,
alloca (4), y).
alloca Example
As an example of the use of alloca, here is a function that opens
a file name made from concatenating two argument strings, and returns a
file descriptor or minus one signifying failure:
int
open2 (char *str1, char *str2, int flags, int mode)
{
char *name = (char *) alloca (strlen (str1) + strlen (str2) + 1);
stpcpy (stpcpy (name, str1), str2);
return open (name, flags, mode);
}
Here is how you would get the same results with malloc and
free:
int
open2 (char *str1, char *str2, int flags, int mode)
{
char *name = (char *) malloc (strlen (str1) + strlen (str2) + 1);
int desc;
if (name == 0)
fatal ("virtual memory exceeded");
stpcpy (stpcpy (name, str1), str2);
desc = open (name, flags, mode);
free (name);
return desc;
}
As you can see, it is simpler with alloca. But alloca has
other, more important advantages, and some disadvantages.
alloca
Here are the reasons why alloca may be preferable to malloc:
alloca wastes very little space and is very fast. (It is
open-coded by the GNU C compiler.)
alloca does not have separate pools for different sizes of
block, space used for any size block can be reused for any other size.
alloca does not cause memory fragmentation.
longjmp (see section Non-Local Exits)
automatically free the space allocated with alloca when they exit
through the function that called alloca. This is the most
important reason to use alloca.
To illustrate this, suppose you have a function
open_or_report_error which returns a descriptor, like
open, if it succeeds, but does not return to its caller if it
fails. If the file cannot be opened, it prints an error message and
jumps out to the command level of your program using longjmp.
Let's change open2 (see section alloca Example) to use this
subroutine:
int
open2 (char *str1, char *str2, int flags, int mode)
{
char *name = (char *) alloca (strlen (str1) + strlen (str2) + 1);
stpcpy (stpcpy (name, str1), str2);
return open_or_report_error (name, flags, mode);
}
Because of the way alloca works, the memory it allocates is
freed even when an error occurs, with no special effort required.
By contrast, the previous definition of open2 (which uses
malloc and free) would develop a memory leak if it were
changed in this way. Even if you are willing to make more changes to
fix it, there is no easy way to do so.
alloca
These are the disadvantages of alloca in comparison with
malloc:
alloca, so it is less
portable. However, a slower emulation of alloca written in C
is available for use on systems with this deficiency.
In GNU C, you can replace most uses of alloca with an array of
variable size. Here is how open2 would look then:
int open2 (char *str1, char *str2, int flags, int mode)
{
char name[strlen (str1) + strlen (str2) + 1];
stpcpy (stpcpy (name, str1), str2);
return open (name, flags, mode);
}
But alloca is not always equivalent to a variable-sized array, for
several reasons:
alloca
remains until the end of the function.
alloca within a loop, allocating an
additional block on each iteration. This is impossible with
variable-sized arrays.
Note: If you mix use of alloca and variable-sized arrays
within one function, exiting a scope in which a variable-sized array was
declared frees all blocks allocated with alloca during the
execution of that scope.
The symbols in this section are declared in `unistd.h'.
You will not normally use the functions in this section, because the functions described in section Allocating Storage For Program Data are easier to use. Those are interfaces to a GNU C Library memory allocator that uses the functions below itself. The functions below are simple interfaces to system calls.
brk sets the high end of the calling process' data segment to
addr.
The address of the end of a segment is defined to be the address of the last byte in the segment plus 1.
The function has no effect if addr is lower than the low end of the data segment. (This is considered success, by the way).
The function fails if it would cause the data segment to overlap another segment or exceed the process' data storage limit (see section Limiting Resource Usage).
The function is named for a common historical case where data storage and the stack are in the same segment. Data storage allocation grows upward from the bottom of the segment while the stack grows downward toward it from the top of the segment and the curtain between them is called the break.
The return value is zero on success. On failure, the return value is
-1 and errno is set accordingly. The following errno
values are specific to this function:
ENOMEM
brk except that you specify the new
end of the data segment as an offset delta from the current end
and on success the return value is the address of the resulting end of
the data segment instead of zero.
This means you can use `sbrk(0)' to find out what the current end of the data segment is.
You can tell the system to associate a particular virtual memory page with a real page frame and keep it that way -- i.e. cause the page to be paged in if it isn't already and mark it so it will never be paged out and consequently will never cause a page fault. This is called locking a page.
The functions in this chapter lock and unlock the calling process' pages.
Because page faults cause paged out pages to be paged in transparently, a process rarely needs to be concerned about locking pages. However, there are two reasons people sometimes are:
Be aware that when you lock a page, that's one fewer page frame that can be used to back other virtual memory (by the same or other processes), which can mean more page faults, which means the system runs more slowly. In fact, if you lock enough memory, some programs may not be able to run at all for lack of real memory.
A memory lock is associated with a virtual page, not a real frame. The paging rule is: If a frame backs at least one locked page, don't page it out.
Memory locks do not stack. I.e. you can't lock a particular page twice so that it has to be unlocked twice before it is truly unlocked. It is either locked or it isn't.
A memory lock persists until the process that owns the memory explicitly unlocks it. (But process termination and exec cause the virtual memory to cease to exist, which you might say means it isn't locked any more).
Memory locks are not inherited by child processes. (But note that on a modern Unix system, immediately after a fork, the parent's and the child's virtual address space are backed by the same real page frames, so the child enjoys the parent's locks). See section Creating a Process.
Because of its ability to impact other processes, only the superuser can lock a page. Any process can unlock its own page.
The system sets limits on the amount of memory a process can have locked and the amount of real memory it can have dedicated to it. See section Limiting Resource Usage.
In Linux, locked pages aren't as locked as you might think. Two virtual pages that are not shared memory can nonetheless be backed by the same real frame. The kernel does this in the name of efficiency when it knows both virtual pages contain identical data, and does it even if one or both of the virtual pages are locked.
But when a process modifies one of those pages, the kernel must get it a separate frame and fill it with the page's data. This is known as a copy-on-write page fault. It takes a small amount of time and in a pathological case, getting that frame may require I/O.
To make sure this doesn't happen to your program, don't just lock the pages. Write to them as well, unless you know you won't write to them ever. And to make sure you have pre-allocated frames for your stack, enter a scope that declares a C automatic variable larger than the maximum stack size you will need, set it to something, then return from its scope.
The symbols in this section are declared in `sys/mman.h'. These functions are defined by POSIX.1b, but their availability depends on your kernel. If your kernel doesn't allow these functions, they exist but always fail. They are available with a Linux kernel.
Portability Note: POSIX.1b requires that when the mlock
and munlock functions are available, the file `unistd.h'
define the macro _POSIX_MEMLOCK_RANGE and the file
limits.h define the macro PAGESIZE to be the size of a
memory page in bytes. It requires that when the mlockall and
munlockall functions are available, the `unistd.h' file
define the macro _POSIX_MEMLOCK. The GNU C library conforms to
this requirement.
mlock locks a range of the calling process' virtual pages.
The range of memory starts at address addr and is len bytes long. Actually, since you must lock whole pages, it is the range of pages that include any part of the specified range.
When the function returns successfully, each of those pages is backed by (connected to) a real frame (is resident) and is marked to stay that way. This means the function may cause page-ins and have to wait for them.
When the function fails, it does not affect the lock status of any pages.
The return value is zero if the function succeeds. Otherwise, it is
-1 and errno is set accordingly. errno values
specific to this function are:
ENOMEM
EPERM
EINVAL
ENOSYS
mlock capability.
You can lock all a process' memory with mlockall. You
unlock memory with munlock or munlockall.
To avoid all page faults in a C program, you have to use
mlockall, because some of the memory a program uses is hidden
from the C code, e.g. the stack and automatic variables, and you
wouldn't know what address to tell mlock.
mlock unlocks a range of the calling process' virtual pages.
munlock is the inverse of mlock and functions completely
analogously to mlock, except that there is no EPERM
failure.
mlockall locks all the pages in a process' virtual memory address
space, and/or any that are added to it in the future. This includes the
pages of the code, data and stack segment, as well as shared libraries,
user space kernel data, shared memory, and memory mapped files.
flags is a string of single bit flags represented by the following
macros. They tell mlockall which of its functions you want. All
other bits must be zero.
MCL_CURRENT
MCL_FUTURE
MCL_FUTURE.
See section Executing a File.
When the function returns successfully, and you specified
MCL_CURRENT, all of the process' pages are backed by (connected
to) real frames (they are resident) and are marked to stay that way.
This means the function may cause page-ins and have to wait for them.
When the process is in MCL_FUTURE mode because it successfully
executed this function and specified MCL_CURRENT, any system call
by the process that requires space be added to its virtual address space
fails with errno = ENOMEM if locking the additional space
would cause the process to exceed its locked page limit. In the case
that the address space addition that can't be accomodated is stack
expansion, the stack expansion fails and the kernel sends a
SIGSEGV signal to the process.
When the function fails, it does not affect the lock status of any pages or the future locking mode.
The return value is zero if the function succeeds. Otherwise, it is
-1 and errno is set accordingly. errno values
specific to this function are:
ENOMEM
EPERM
EINVAL
ENOSYS
mlockall capability.
You can lock just specific pages with mlock. You unlock pages
with munlockall and munlock.
munlockall unlocks every page in the calling process' virtual
address space and turn off MCL_FUTURE future locking mode.
The return value is zero if the function succeeds. Otherwise, it is
-1 and errno is set accordingly. The only way this
function can fail is for generic reasons that all functions and system
calls can fail, so there are no specific errno values.
Go to the first, previous, next, last section, table of contents.