Include Checker 1.0.0

Shahzad Muzaffar (Shahzad.Muzaffar@cern.ch)

Northeastern University, Boston, USA

What is Include Checker:

Include Checker is a perl script which can parse any c/c++ files (source/

header) and checks if any include statement is needed or not. It can tell you

- Which files you do not need to include (because they are not need or might

be included by some other files).

- Which files you need to include

- Numbers of lines in each file

- Number of comment lines

- Number of empty lines

- Number of actual code lines (If a line has code and comments then that

will be counted once for code and once for comments).

- Number of included files

Your code should not have any compilation errors and each file should be

self parsed.

This script assumes that a .cc file always needs its own .h. e.g. A.cc will

always need A.h.

There is also another script available to generate some statistics of the code

e.g. number of code/empty/comments lines for project upto the file level. The

script is codestats.pl and to run it you have to first run includechecker.pl and

save its output in a log file then run codestats.pl of that log file. e.g.

> includechecker.pl ... > mylog.txt

> codestats.pl --log mylog.txt

How does it work:

Include Checker parses the c/c++ (source/header) file and collects the list

of files included in it. Then it compiles the original file and collects the

warnings generated by the compiler (if there are any). After this, it comments

the 1st include line, compiles and checks the compiler warnings/errors with the

original results. If there is mismatch then this tool assumes that this include

line is needed otherwise it keeps that include line commented (this is all done

on a copy of original file). It does this for all the includes and at the end

it copies the modified file in a temp area. Compilation is done in a way that

for each file the already modified files are used. e.g. if file A/file1.h

includes A/file2.h then first A/file1.h will be checked and copied to

/tmp/A/file1.h. Now when A/file2.h is tested then -I/tmp will be added to the

start of compilation command for A/file2.h so that it get the modified

A/file.h.

Some times developers do not include a header file because it is indirectly

included by some other header they have included. e.g. A/file2.h assumes

that A/file1.h has already included X/file3.h, so just including A/file1.h

in A/file2.h would be enough. Now what will happen if include checker decided

to remove X/file3.h from A/file1.h? Will the compilation of A/file2.h failed.

The answer is no, before compiling A/file2.h, include checker will add all the

removed include statement from A/file1.h (or all other dependent includes) into

A/file2.h. If X/file3.h is really needed by A/file2.h the include checker will

suggest to add it otherwise it silently remove it from A/file2.h.

If for some reasons the compilation of original file failed (most of time it

happens when a file is not parsed by itself) then include checker will stop its

processing. Its better to use "--keep" command line option which will save you

some time. e.g. if you have 300 files to check and at file number 250 the

include checker decided to stop then all the time checking 250 files will be

wasted if --keep was not ON. Please see --keep command line argument detail.

How to run:

Here are the command line options you can use

--config <file>

Please see "How to create a config file" section

--recursive

If this flag is used then all the included files will be check first (only

if they exist under any BASE_DIR, matched by the INCLUDE_FILTER and not

explicitly skipped via SKIP_FILES key).

--keep <num>

Do not delete the tmp files.

num < 0: Internal cache will be saved only once at the time of exit

num > 0: Internal cache will be saved after checking num files

num = 0: Same as if keep was not passed to command line argument i.e. do

not keep the tmp area.

By default the tmp directory used by the include checker is deleted at the

end. If you want to keep it then use this option. So if you think that include

checker has done a good job, you can copy updated files from this tmp area to

your original area.

Also --keep is useful if you are going to check many files and in middle

include checker stops due to errors in your original files. In this case

you can fix the error and start include checker again with --tmpdir <prev_dir>

This will save you a lot of time.

Include checker writes its internal cache in this tmp area, so next time

when you start include checker with --tmpdir <dir> then it reads that

previously saved cache and start the processing where it left.

--tmpdir <dir>

One can provide its own tmp directory. This option will also enable the

--keep option. If your <dir> already has a config_cache file then include

checker will ignore your config file passed via --config command line option.

At the end of include checker processing you can find an updated version of

your file in <dir>/includechecker/src. If you are happy with include checker

results then you can directly copy these file from this tmp area to your

original area.

--detail

It will prints more detail messages of include checker. e.g. why include

checker thinks a included file is needed etc.

--help

To see the include checker help message.

How to create a config file:

One needs to create a config file which then can be passed to include checker

via command line argument. Here is what one needs to provide in this config

file. Format of this file is KEY=VALUE. Some keys are optional. Here are the

list of keys

- BASE_DIR=<space separated list of base directories>

Files only exist under these directories will be checked. One can use this

key multiple times to provide more BASE_DIR. e.g.

BASE_DIR=/my/dev/area

BASE_DIR=/project/release/area

OR

BASE_DIR=/my/dev/area /project/release/area

Files will be search in these BASE_DIRs according to the order of these

BASE_DIR. So always provide your dev area first and then the release area.

- COMPILER=<command to compile the c/c++ files>

It is an optional key. Default value is "c++".

- COMPILER_FLAGS=flags

It is an optional key. Default value is "".

You can provide all the include path and defines to pass to compiler to

compile your file here. e.g. if your files needs qt, coin3d then you can

have something like (in one line)

COMPILER_FLAGS=-I/my/qt/installation/include

-I/my/coin3d/installation/include

-DMYDEFINE -DMYDEFINE2

NOTE: Define this key before any FILES key (see FILES key)

- HEADER_EXT=<regexp>

It is an optional key. Default is \.h

A Perl regular expression to to check if a file is c/c++ header or not.

- SOURCE_EXT=<regexp>

It is an optional key. Default is \.(cc|CC|cpp|C|c|CPP|cxx|CXX)

A Perl regular expression to to check if a file is c/c++ source or not.

- INCLUDE_FILTER=<regexp>

It is an optional key. Default is .+

This filter will be used to limit the recursive checking. e.g. if your project

consists of many packages (e.g. A/A1, A/A2, B/B1...) and you are only

interested to check files which exists in you package A/A1 then you can have

INCLUDE_FILTER=A/A1/.+

Now if A/A1/file1.cc depends on A/A1/file1.h, A/A2/file3.h, B/B1/file4.h then

only A/A1/file1.cc and A/A1/file1.h will be checked.

- FILES=<space separated file names (relative path within any BASE_DIR)>

You at least need one key like this in your config file. e.g.

FILES=A/A1/file1.cc A/A1/file2.cc

OR

FILES=A/A/file1.cc

FILEs/A/A1/file2.cc

A file will only be checked if it exists under any of the BASE_DIR.

Please remember that if you want to use any COMPILER_FLAGS to compile your

files then those must be defined before the FILES=<file> key. e.g. if you

have something like this in your config file

COMPILER_FLAGS=-DMYDEFINE1

FILES=A/A1/file1.cc A/A1/file2.cc

COMPILER_FLAGS=-I/my/qt/installation/include -DMYDEFINE1

FILES=B/B1/file4.cc

then files A/A1/file1.cc and A/A1/file2.cc will be compiled with "-DMYDEFINE1"

compiler flags and file B/B1/file4.cc will be compiled with

"-I/my/qt/installation/include -DMYDEFINE1" compiler flags.

- SKIP_FILES=<space separated regexp (relative path within any BASE_DIR)>

It is an optional key. If for some reason you want to skip a file to be

checked by the include checker then add that file in here. e.g. sometime

developers create header files which only export things for others to use

(like common headers used by every one etc).

- SKIP_INCLUDES=<space separated regexp (relative path within any BASE_DIR)>

It is an optional key. This key will skip the checking of a include directive

for a give file. e.g. if a file B/b.h includes files A/a1.h, A/a2.h and if you

think that file A/a1.h is really needed by B/b.h then you can add the following

line in the config file

SKIP_INCLUDES=B/b.h:A/a1.h

Now include checker will only test A/a2.h while checking for B/b.h.

Config File Generation for SCRAM-Based projects:

For SCRAM-based projects this config file can be generated by the

createconfig.pl script. e.g. If you are only interested to check the source

available in /PROJECT/src/SubSystem/Package then just run

./createconfig.pl /PROJECT/src/SubSystem/Package > myconfig.txt

If you are interested in whole project then do

./createconfig.pl /PROJECT/src > myconfig.txt

After generating this config file you can edit it according to your needs e.g.

you can add INCLUDE_FILTER, SKIP_FILES, SKIP_INCLUDES etc. in it.

Modified by: Shahzad Muzaffar

Dated: 20 July, 2005