This is a unit test library and configurable tester for the 42 project ft_printf.
It is designed to be a regression testing utility--that is, it is meant to be used while you are developing. It lets you select specific tests and test groups to run, enable and disable entire blocks of tests at once, or even select tests based on previous run/pass/fail history. It's quick and easy to add your own tests, which I must recommend for the principle of it--but its original name (printftester2000) reflected that there are over 2000 tests available already. Regardless, it's built to be flexible and highly configurable, so you can use it how you wish.
You have to have a Makefile in your project directory that will compile libftprintf.a as the default make option, and your libftprintf.a has to have ft_printf inside.
Other than this, it should be completely general to all ft_printf projects.
Requires PHP. All 42 lab computers should have PHP installed. See also Compatibility with Other Systems.
In the root of your repo, run this command:
git clone https://github.com/gavinfielder/pft.git pft && echo "pft/" >> .gitignore
For most users, that is all that is required.
If you include all required .o files (including your libft) in libftprintf.a, this is not necessary. If you do NOT, and require your libft separate, you must set USE_SEPARATE_LIBFT=1
in options-config.ini, and make sure the path is correct. See options-config.ini, and it should be self-explanatory.
For non-standard installation options, see Installation.
Change INCLUDE_LIBPTHREAD=0
in options-config.ini
to INCLUDE_LIBPTHREAD=1
.
This could apply to other systems--if you get a make error with undefined refrences to pthread_
functions, make this change and it will work.
Every test has a name and a number. You select tests or groups of tests to run based on either a name search or a number range. Name searches will look for tests that start with the given string. Groups of tests are organized with their names so that they can be selected easily in this way.
./test s
runs all the tests that start with 's'. As you might guess, the%s
tests start with 's'. Tests for the other specifiers (%d
,%p
, etc.) can be selected the same way../test 42
runs test #42 only./test 42 84
runs all the enabled tests from #42 to #84./test
runs all the enabled tests./test help
shows examples and other help information.
You can also run specific types of tests by using Wildcard Search, for example ./test "d*prec"
will run all the %d
tests that have prec
in their name, which selects %d
tests that use precision (.
). See Test Naming Conventions for a reference on selecting tests in more detail.
When you fail a test, the file results.txt
will show the results of the test including the first line of code for the test (most of them are one line anyway), the return values, what printf printed, and what ft_printf printed.
I have provided scripts that make it easy to enable and disable tests. "Disabled" tests mean they will never run unless you force them to with -a
. These enable-test and disable-test scripts accept the same queries as the ./test
executable, that is, querying on the name or using a numeric range.
./disable-test s
Disables all tests that start with 's'./enable-test nocrash
Enables all tests that start with 'nocrash'./disable-test 42 84
Disables all tests from #42 to #84./disable-test && ./enable-test s
Disables all tests except tests that start with 's'.
You can call ./enable-test
(with no arguments) to enable all tests, but since you are probably not crazy enough to actually implement everything that the test library can possibly test, that is generally a bad idea.
If you want to reset to the default state of enabled tests, ./reset-to-default-enabled-tests
./enable-test bonus
will enable all tests that test bonuses./enable-test bonus && ./disable-test bonus_notrequired
will enable bonus tests, but not tests that are not necessary to pass for moulinette to validate the bonus./enable-test "bonus*_f_"
will enable%f
tests. (Similarly for%g
and%e
)./enable-test bonus_length
will enable tests forl
ll
h
hh
./enable-test bonus_af
will enable tests for#
(af
is short for alternate form)./enable-test bonus_sp
will enable tests forsp
is short for space padding)./enable-test bonus_as
will enable tests for+
(as
is short for always sign)./enable-test nocrash
will enable tests that test yourft_printf
's ability to handle bad input (will pass as long as it doesn't crash) While they are disabled by default and may be irrelevant to moulinette, I suggest that handling bad input is in fact a mandatory requirement of the subject which evaluators may opt to grade you on, so I personally recommend that everyone enable these.
There are many tests not covered by the above examples.
./show-disabled-tests
will print a list of all tests currently disabled. This may help to know what non-mandatory tests are available, which you can use the enable- and disable-test scripts to select by name or wildcard search as in the above examples. See also Test Naming Conventions.
Debugger compatibility mode ( -d
) is automatically turned on for single tests:
lldb ./test 42
The 2020 update added *
tests to the required features. Tests were added from https://github.com/cclaude42/PFT_2019, but these tests don't include enough combinations with other flags--looking for current 42 students to add such tests.
Fork mode (-x
) used in conjunction with IGNORE_RETURN_VALUE=0
is currently not properly reporting expected return value in results.txt for many tests (issue #11). This bug does not affect the pass/fail result of a test. Running in non-fork mode (-X
) will show the correct return values. The default configuration has been set to IGNORE_RETURN_VALUE=1
. If this issue might affect you, a warning will be printed in results.txt.
The Makefile creates two versions of each unit test function, one that uses ft_printf, and one that uses printf. For each test, it redirects stdout to a file, calls the function. Once each version returns, it opens both files and reads each one byte by byte until both reach EOF. If any single byte differs, the test fails. If IGNORE_RETURN_VALUE
is set to 0
, the test will also fail if the return values between printf and ft_printf differ.
Tests starting with nocrash
are handled differently: these tests will pass as long as ft_printf does not crash while executing them.
Feel free to contribute tests for these:
- Size modifiers
t
,z
,j
, andq
. %n
'
(thousands separator flag)$
for dynamic precision. ($
for argument selection is covered underargnum_
)
*
(There are some tests, but not many combinations with other flags)%a
(some tests exist in themoul
block, but they are not rigorously tested with flag combinations like the other tests)
There are differences between printf(3) between systems. One common difference is the printing of null pointers--there can be others.
ft_printf is now being developed on other systems due to 42 working remotely as it deals with the Covid-19 pandemic. Moulinette tests your ft_printf against its own system's printf. That makes the project system-dependent and since PFT uses the same testing method as moulinette, it will not be able to tell you if your ft_printf will fail because you neglected to test on lab computers.
You must test on your lab's computers prior to submission.
The full documentation is available in the Wiki. Here are some quick links:
- Installation
- Usage
- What's Not Covered
- Workflow with PFT
- Options and Configuration
- Additional Features
- Troubleshooting
- Compatibility With Other Systems
- Contributing to this repo (and possible future features)
I occasionally get suggestions for the unit test library. I keep this list to keep track of what I want to do the next time I modify unit_tests.c
Todo List for the Unit Test Library
By default, PFT collects some usage statistics on every make
. The full list of data collected is:
- A SHA-1 hash of your
whoami
username. - Whether you're on the Fremont, Paris, or Moscow campus, or your
hostname
if none of those. - Whether it is a first install or a re-make.
- The name of your configuration (
CONFIG_NAME
in options-config.ini)
You can disable this behavior in options-config.ini, and/or see exactly what the script does in src/usage_statistics.php. My reason in doing this is to have real usage data to show recruiters and hiring managers. If you're a 42 student and want to know more or see the collected data, feel free to message me on slack.
The test method itself was adapted from outdated moulinette test files a buddy gave me, from which the author was [email protected]. The vast majority of code was written by me. The tests prefixed moul_ were adapted from the moulinette test files, the tests with _ftfc_ were adapted from 42FileChecker. Many tests were written by me; tests were contributed by phtruong, akharrou, robbie, and mgiraldo. Big thanks to cclaude for doing the first update for the new curriculum, and bigger to apuchill for reorganizing the test library and adding tests for the new curriculum.
Also thanks to: