Clarify test/Makefile and snippet testing documentation

Hopefully this makes it easier for future development work to augment
the snippet-based testing.
This commit is contained in:
Daniel Kahn Gillmor
2021-03-01 21:40:57 -05:00
parent 0bfb72b627
commit 986e6e1cdc
2 changed files with 69 additions and 30 deletions

View File

@@ -31,16 +31,18 @@ functest:
randomtest: repeat_random
./randomtest.sh
## test variables
# run snippet tests
snippets: test_variable_data test_library_constructors
## test snippet behavior across env var setting over time:
test_variable_data: test_variable_data.sh $(foreach f,${EXPECTATIONS},run_${f})
./test_variable_data.sh ${EXPECTATIONS}
run_%: _run_test.c snippets/%.c
sed s/SNIPPET_NAME/$*/g < _run_test.c | ${CC} -o $@ ${CFLAGS} -x c -
## testing when interception points get called in library constructors:
## test snippets in other library constructors:
test_library_constructors: $(foreach f,${TEST_SNIPPETS},test_lib_${f})
test_lib_%: test_constructors.sh use_lib_% lib%.so
./test_constructors.sh $*
@@ -60,4 +62,4 @@ clean:
distclean: clean
@echo
.PHONY: all test clean distclean randomtest
.PHONY: all test clean distclean randomtest snippets test_variable_data test_library_constructors

View File

@@ -1,35 +1,72 @@
Bulk testing of function interception
=====================================
Testing Interception with Snippets
==================================
Faketime intercepts some C library functions. We want to make it
easier to apply certain generic tests across many different functions.
Faketime intercepts some C library functions and system calls. We
want to make it easier to apply certain generic tests across many
different functions. We do that with snippets of C, which each can be
applied in multiple testing frameworks.
Including a new function
------------------------
Most snippets are just a minimalist invocation of a single function or
syscall that is likely to be intercepted by libfaketime, though it's
possible to test more complex snippets too.
To test a function FOO, supply a C snippet in this directory named
<FOO.c>. This should be a small bit of code that exercises the
function FOO. It should be self-contained, and it should produce some
kind of description of what happened to stdout. Take a look at
getpid.c for a simple example.
Including a New Snippet
-----------------------
The data sent to stdout should include the const char* "where" value
(which is an indication of which test framework is using the snippet).
To cover a new bit of intercepted functionality, supply a C snippet in
this directory named `FOO.c` (the name FOO should conform to C
function names -- letters, numbers, and underscores; if you're testing
interception of a single function, the simplest thing is to name
snippet file after the intercepted function name).
If the output of the snippet is expected to be stable when the
associated variable (e.g. FAKETIME, FAKETIME_FAKEPID, or
FAKERANDOM_SEED) is set, and is likely to vary otherwise, especially
across a delay of a second or more, add a single-line file
FOO.variable that contains the name of the variable separated by a
space and then the value of the variable.
This file should contain a small bit of code that exercises the
functionality in question. It should be self-contained, and it should
produce some kind of description of what happened to stdout. The data
sent to stdout should include the const char* "where" value (which is
an indication of which test framework is using the snippet). Take a
look at getpid.c for a simple example.
If the snippet needs to additional #include headers, please add them
in include_headers.h. These #includes will be used by every snippet,
If the snippet needs additional #include headers, please add them
in `include_headers.h`. These #includes will be used by every snippet,
so try to keep it minimal.
Testing across functions
------------------------
Snippet Testing Frameworks
--------------------------
If you want to test something systemically, autogenerate code that
uses each snippet. See test_variable_data or
test_library_constructors in ../Makefile for an example.
We have the following frameworks that use the snippets:
### Variable Data
Most functionality intercepted by libfaketime will normally produce
variant output when invoked multiple times across the span of a few
seconds by different processes. But if `FAKETIME` (or an analogous
variable like `FAKERANDOM_SEED`) is set, the output data should remain
constant.
If this describes the functionality in a new snippet `FOO.c`, please
also drop a single-line file `FOO.variable` in this directory, where
the first word of the line is the variable that should hold the output
constant, and the rest of the line is the value of that variable.
See the `test_variable_data` target in `test/Makefile` for how this is
implemented.
### Library Constructors
Library constructor routines are run by ld.so before the main process
is launched. If the LD_PRELOADed libfaketime is masking a symbol from
libc, and another library has a constructor routine that invokes that
symbol, it might get called before libfaketime has had a chance to
initialize its followup pointers to the actual libc functionality.
This framework is applied automatically to all snippets.
See the `test_library_constructors` target in `test/Makefile` for how
this is implemented.
Adding a New Framework
----------------------
If you want to add a new framework that tests across all snippets,
implement it in a few lines of `test/Makefile` and document it in the
section above.