Table of Contents

The original idea

In order to engage users for longer term relationships it's important to be able to tell if you've met the user before, and recall past engagements.

Also provide a test competency to try out reusable standard stuff for lirec (yarp, cross platform development, etc)

Initial version 0.1 [27th Feb 2009]

Calibration Mode

Look for all faces in the image, compare with stored faces and create a new identity if the difference is over a given threshold. Do this each frame until switched into detect mode.

Detection Mode

Look for all faces in the image, compare with stored faces and print the id number of the most similar stored face, if it's over a given threshold.

Demo Movie

Thinking about usage [19th March 2009]

* Adding user inteface for recording up to 10 people by pressing the number keys * Better output - rendering the recorded faces, and draws lines between them and the detected faces.

YARP interface [20th March 2009]

Added a simple message to tell other programs which user is in view, with a confidence metric.

/faceident user-id confidence-value

The program still needs to be driven by user input, calibrating it by recording people at startup.

Scene state [32rd March 2009]

Rather than simply outputting the user id's every frame, it's more useful if the program tracks the state of the scene internally, and outputs changes. I've added the following yarp messages:

Output port Bottle contents Meaning
/faceident “user appeared” ID confidence A user has entered the view of the camera
/faceident “user disappeared” ID The user has left the camera view

More YARP, and persistence for INESC-ID [15 April 2009]

New messages

Input port Bottle contents Meaning
/faceident-ctrl “train” ID Train for this user
/faceident-ctrl “detect” Switch out of training mode
/faceident-ctrl “save” sessionname Save the detected faces
/faceident-ctrl “load” sessionname Load previously detected faces
/faceident-ctrl “clear” Clears all faces

Loading and saving is done quickly, for the review - and just consists of saving images as png files and a quickly hacked together ascii format for mapping the images to the id numbers.

Persistence across time via load and save to disk brings up problems - the computer vision will break when lighting changes too much, the saved faces will be too different from the visible ones. One possible workaround would be to lower the error threshold and sneakily recalibrate for the most likely user at the start of a new session.

Idle mode [15 April 2009]

Due to a request from INESC-ID, which helps thinking about this more in terms of the wider architecture, it's important that competencies can be tured on or off easily, particularly if they use up cpu, as computer vision competencies will. To this end I've added an idle command:

Input port Bottle contents Meaning
/faceident-ctrl “idle” Switch to idle mode, mostly frees up cpu

XML and experimental multi image mode [23 April 2009]

Using TinyXML as a source library for making the persistence much more scalable and user editable.

I've also added another yarp message:

Input port Bottle contents Meaning
/faceident-ctrl “multiimages” 1/0 Turn on or off multiple images per face, experimental

This is the first step to dealing with lighting changes, just allowing images of people in multiple lighting situations to be used at once. The system looks through each image for each person in order to match the closest.

In calibration mode, the system will add new images to the user if the detected ones are above the error threshold. This is slightly dangerous, as you would be able to get images for different people in the same id… Not sure if there is a way to cope with that automatically.