If your code just uses other libraries etc., then it's usually very simple to just mock those parts, especially if you're using a dynamic language. I won't waste your time by talking about that. Instead, I'll talk about when it gets interesting - when you're writing system-level tools in a static language.
Since the tool support (and by "tool support" I mean Valgrind) is best on Linux, I first try to make the code build as a self-contained executable in Linux, no matter what the target OS is. For small embedded OSes, you can probably just reimplement the OS functions and you're set. For Windows, suit your self. That leaves POSIX-like OSes, in which case the code should be fairly easily buildable in Linux. Here's where the fun starts.
So you have some code that calls open(2) on devices that don't exist on your workstation, connects sockets using address families unknown to civilization, or does all kinds of strange things that requires root privileges and can't be easily chrooted. How can you possibly write a harness for that? LD_PRELOAD, that's how.
LD_PRELOAD is a little-used feature of the GNU dynamic linker (and others, e.g. the one in Mac OS X) that lets you specify a dynamic library that is injected into an executable before other libraries are loaded. That, in combination with the rule that whoever first defines a symbol wins, means that you can reimplement any function you like in a library that is part of your harness and have those functions be used instead of the versions defined in, say, libc. If the component you're testing opens /dev/thingamajig, then just add a function that looks like open(2), but instead of actually opening the device node just tells your test scripts about it. One useful way of doing that is to have the test running as a separate process and have a Unix Domain Socket (or pair of pipes if you prefer) where you can send messages about what the component tried to do and receive instructions about what to do with the call.
Since reimplementing everything can be both tedious and slow, you may want to forward uninteresting calls to the normal versions of the functions you've overridden. This can be done using dlsym(RTLD_NEXT, "some_function"). That will make the dynamic linker look up the next library that has a symbol called "some_function" and give you a pointer to that. Assigning that to a function pointer variable gives you a way to call, say, the plain old libc open(2) from your magic open(2) for any file that the test scripts deem uninteresting. Something like this:
static int (*real_open)(const char *path, int oflag, ... );
__attribute__((constructor))
void init_hackery(void)
{
real_open = dlsym(RTLD_NEXT, "open");
}
int open(const char *path, int oflag, ... )
{
int mode = 0;
va_list va;
va_start(va, oflag);
if(oflag & O_CREAT)
{
mode = va_arg(va, int);
}
va_end(va);
inform_scripts_about_open(path, oflag, mode);
switch(ask_scripts_what_to_do())
{
case HandleInScripts:
{
return hanlde_open_in_scripts(path, oflag, mode);
} break;
case PassOnToLibc:
{
if(oflag & O_CREAT) return real_open(path, oflag, mode);
else return real_open(path, oflag);
}
}
}
read, write, ioctl and close in the same manner (mapping the file descriptor to some mock object in your scripts), and the component can get its devices and whatnot and you get your tests. Have fun!
