Goals

Our goal is to create a portable library for face recognition algorithms, that abstracts the actual algorithms from the application. Once a few algorithms are implemented as plugins for the library and sufficiently tuned, this could attract enough interest to get a lot of experimenting and innovation in this sector going. This library will provide application developers with an easy way to integrate face reognition services into their applications and may serve as a testbed for new algorithms and algorithm tuning for researchers and hobbyists.

How does face recognition usually work?

• Find the face
• Crop the image to face
• Find eyes and mouth
• Position eyes and mouth on predefined spots, thereby transforming the picture slightly
• Apply a mask
• Normalize color/brightness

The recognition algorithms can be very different. Some need to train faces they have to recognize, others just compute different characteristic features of faces from a sample set and yet others don't require training whatsoever. When it comes to recognizing, some of the algorithms produce a template or trait that can be compared with other traits to get a match probability.

For the impatient - CLI (command line interface) tools walkthrough

For now, our command line tool for setting up recognition parameters only supports eigenfaces, so this tool will get a more complex interface later, since it has to be algorithm agnostic.

• Setup:

  To setup the system, you have to use a trainingset of at least 3 images, preferrably more. You have to call faceinit -o <parameter file> <image list>. This will initiate the training phase of the eigenfaces algorithm and generate a parameter file which can be used to compare faces.

  If you want to actually see the eigenfaces that were generated, you can add the -v switch at the beginning of the command. This will cause the programm to generate several .png pictures in the current directory. The first of these pictures is the average picture of your input set, all following pictures are visualization of the eigenfaces generated. Please note that these pictures are just a visualization of the eigenvectors, which are vectors of float values.

• Comparing faces:

  After creating your parameter file, you are ready to compare images. This is done via the facecmp program. You have to supply a parameter file and the two pictures to compare like this: facecmp -p <parameter file> <image1> <image2>

  The program will then output the distance between the two pictures and the match probability. Note that the probability will always be zero, since we have not set a correct threshold for the distance.

Our library, nomenclature, limitations

Interoperability, Compatibility, ...

Currently the library is implemented in what I believe to be ANSI C because we feel this ensures maximum portability and flexibility. Bindings to other languages will surely emerge, if these languages do not provide C interoperability out of the box. As of now, the library should compile on any machine, but files written on one architecutre are not yet compatible with files written by other architectures.

Using the library

Currently the library needs OpenIL and OpenCV to be present on the system. You have to include IL.h and ILU.h in every file that includes fr.h. Before calling any library functions, you have to initialize IL and then libface by calling ilInit, iluInit and frInit.

Before you start throwing images at the library, you have to setup a FRrecognitionParameter structure. It describes what algorithms (and eventually which preprocessing steps) a picture should go through. Nearly every function requires a pointer to this structure. After you set up this structure, you can compute traits from images. These traits can be compared (only in the context of the FRrecognitionParameters struct used for creating them) to get a match probability.

There are several groups of functions:

• frImg*: used to manage images
• frRecoParams*: used to set up a FRrecoParams structure and to save/load this structure
• frTrait*: used to compoute and serialize traits
• frCmp compares two traits and returns a match probability

Almost all functions return an error code from enum FRerror, those few that don't are guranteed to succeed.

Current Limitations / TODOS:

This is by no means a complete list, if you feel that i've omitted something particular important feel free to contact me..

• dynamically load available plugins (currently eigenfaces is the only available algo and is hardcoded in the list of algorithms)
• preproccessing framework
• support for algorithms, that don't compute templates which are compared (maybe needed to integrate neural net aproaches?)
• serialization helpers to get endianness and machine word size agnostic serialized data
• user supplied logging stream / function / whatever
• more thorough error checking
• support for different image formats (currently float brightness is used)

How to contribute

If you are still not scared and wish to contribute, I'd be glad to answer all your questions and help you out with any problems you might encounter (as far as I _can_ help anyway ;). If you want to implement an algorithm, the best place to start might be recoalgo.h. You will have to provide functions for this structure. In eigenfaces.h/.c you'll see how i did it. Especially the serialization functions might be interesting to you, so you see how they could be written (not that I am that good a coder).

Preprocessing freaks will have to wait a little while, before they can provide their algorithms as plugins, since we have not come up with a reasonable design here. Any ideas appreciated.