Monday, April 12, 2010

[OpenCV] Face Detection (人臉偵測)


何謂OpenCV?
Open代表Open Source,CV為Computer Vision,Computer Vision被廣泛應用在計算機科學及醫學等領域,OpenCV是由Intel公司參與及開發出來的,感謝Intel的Open Source :)。OpenCV支援Windows跟Linux兩種平台,並具有強大的圖形跟矩陣的運算。然而,近幾年OpenCV為人垢病的是,它並不再更新,因此無法提供一些新的演算法。But,這個問題似乎得到了回應,最近OpenCV已經開始在改版了,加入了一些新的函數,像是近年來在電腦視覺領域上,極常被使用及提及的Scale-invariant feature transform(SIFT)

下載及安裝OpenCV
最新版的OpenCV可至以下網址下載:
http://sourceforge.net/projects/opencvlibrary/

Face Detection
There are numerous researches making efforts toward how to correctly detect faces in digital images. 而OpenCV所使用的方法是由Viola & Jones」所發表的AdaBoost Learning with Haar-Like Features來實現人臉偵測。下面就簡單介紹一下這個方法:
  1. 首先定義一些Haar-Like Features
  2. 再來,我們必須給定一些sample例如: 假如我們要偵測的是人臉,那麼我們就要輸入一些人臉的sample有了這些sample我們將會利用AdaBoost learning algorithm來挑出某幾個代表性的Haar-Like Features,以這個例子來講,我們可以想成這些被挑選出來的feature就是代表人臉的feature
  3. 每個被挑選出來的feature都代表一種classifier,許多種被挑選出來的feature因此構成了一連串的classifier,我們稱為strong classifier。而每個classifier皆用來判斷所輸入的圖片是否為人臉,並且回傳,最後,通過所有classifier的圖片將被判定是一張人臉
以上就是Viola與Jones所提出的Object Detection方法,除了人臉當然它還可以拿來用在其它的物件偵測,只要我們改變一開始輸入訓練用的sample型態即可。

「Source Code
OpenCV 有提供Face Detection的source code,不過它註解很少,並且有點難懂,所以我試著修改它,並加上一些註解

/*  
 * Face Detection in Digital Photographs  
 *  
 * We leverage the AdaBoost learning-based face detection method with Haar-like features to detect faces in photos.  
 * However, OpenCV supplies Haar Cascade classifier for object (face) detection.
 * 
 * (C) Matt Swanson & TrekLee 
 * http://seeyababy.blogspot.com/  
 *  
*/ 

// Path setting for OpenCV Library
#pragma comment(lib,"../OpenCV/lib/cv.lib")
#pragma comment(lib,"../OpenCV/lib/cxcore.lib")
#pragma comment(lib,"../OpenCV/lib/highgui.lib")

// Path setting for OpenCV Header files
#include "../OpenCV/include/cv.h"
#include "../OpenCV/include/cxcore.h"
#include "../OpenCV/include/highgui.h"
  
#include <stdio.h>
#include <iostream>

using namespace std;

// the minimum object size
int min_face_height    = 20;
int min_face_width    = 10;

//Face Detection
int runFaceDetection(IplImage* image_detect);

void main()
{    
    IplImage* pImg = cvLoadImage( "image.jpg", 1);    
    runFaceDetection(pImg);
    cvReleaseImage( &pImg ); 
}

int runFaceDetection(IplImage* image_detect)
{ 
    // Load the pre-trained Haar classifier data.
    CvHaarClassifierCascade* classifier = (CvHaarClassifierCascade*)cvLoad("haarcascade_frontalface_alt.xml", 0, 0, 0);
 
    // Quit the application if the input source or the classifier data failed to load properly.
    if( !classifier)
    {
        cerr << "ERROR: Could not load classifier cascade." << endl;
        return -1;
    }

    // Create a CvMemStorage object for use by the face detection function.
    CvMemStorage* facesMemStorage = cvCreateMemStorage(0);

    IplImage* tempFrame = cvCreateImage(cvSize(image_detect->width, 
        image_detect->height), IPL_DEPTH_8U, image_detect->nChannels);

    // Copy the current frame into the temporary image.  Also, make 
    // sure the images have the same orientation.
    if(image_detect->origin == IPL_ORIGIN_TL)
    {
        cvCopy(image_detect, tempFrame, 0);
    }
    else
    {
        cvFlip(image_detect, tempFrame, 0);
    }

    /* Perform face detection on the temporary image, adding a rectangle around the detected face. */

    // "Resets" the memory but does not deallocate it.
    cvClearMemStorage(facesMemStorage);
 
    // Run the main object recognition function.  The arguments are: 
    // 1. the image to use
    // 2. the pre-trained Haar classifier cascade data
    // 3. memory storage for rectangles around recognized objects
    // 4. a scale factor "by which the search window is scaled between the 
    //    subsequent scans, for example, 1.1 means increasing window by 10%"
    // 5. the "minimum number (minus 1) of neighbor rectangles that makes up 
    //    an object. All the groups of a smaller number of rectangles than 
    //    min_neighbors-1 are rejected. If min_neighbors is 0, the function 
    //    does not any grouping at all and returns all the detected candidate 
    //    rectangles, which may be useful if the user wants to apply a 
    //    customized grouping procedure."
    // 6. flags which determine the mode of operation
    // 7. the minimum object size (if possible, increasing this will 
    //    really speed up the process)
    CvSeq* faces = cvHaarDetectObjects(tempFrame, classifier, facesMemStorage, 1.1, 
        2, CV_HAAR_DO_CANNY_PRUNING, cvSize(min_face_width, min_face_height));

    // If any faces were detected, draw rectangles around them.
    if (faces)
    {    
        for(int i = 0; i < faces->total; ++i)
        {
            // Setup two points that define the extremes of the rectangle, 
            // then draw it to the image..
            CvPoint point1, point2;
            CvRect* rectangle = (CvRect*)cvGetSeqElem(faces, i);
            
            point1.x = rectangle->x;
            point2.x = rectangle->x + rectangle->width;
            point1.y = rectangle->y;
            point2.y = rectangle->y + rectangle->height;

            cvRectangle(tempFrame, point1, point2, CV_RGB(255,0,0), 3, 8, 0);
        }
    }
    
    // Show the result in the window.
    cvNamedWindow("Face Detection Result", 1);
    cvShowImage("Face Detection Result", tempFrame);
    cvWaitKey(0);
    cvDestroyWindow("Face Detection Result");

    // Clean up allocated OpenCV objects.
    cvReleaseMemStorage(&facesMemStorage);
    cvReleaseImage(&tempFrame);
    cvReleaseHaarClassifierCascade( &classifier );

    return 0;
}


「Experiment
接下來我挑張照片來做個簡單的實驗,下圖是做完人臉偵測,並框出系統判斷為人臉的結果



















我們可以發現接近右下角的地方有個誤判的情況發生,但不得不說的是,仔細看它真的長的很像一張邪惡的人臉...囧|||...,回歸正題,我們該如何解決這個誤判? 除了加入膚色判斷,或是讓face detector更準確之外,其實有個立即就可以針對這個問題來解決的方法,當我們觀察以下這個函式

 CvSeq* faces = cvHaarDetectObjects(tempFrame, classifier, facesMemStorage, 1.1, 
        2, CV_HAAR_DO_CANNY_PRUNING, cvSize(min_face_width, min_face_height));

其中cvSize(min_face_width, min_face_height)設定被偵測人臉的最小size,上面被誤判為人臉的圖片明顯小於其它正常的人臉,所以我們可以考慮將size放大,看看是否可以解決這個問題,因此我將min_face_heightmin_face_width由原本的size各加上10個pixel,重新偵測所得到的結果如下


















我們發現誤判果然被解決了



「Conclusion
本篇文章首先介紹了Viola與Jones所提出的利用AdaBoost Learning with Haar-Like Features來做人臉偵測的方法,並且提供了source code,這份程式改進了OpenCV所提供的source code其難讀(for me)、註解少的缺失,最後,做了一個簡單的實驗及討論


Coming soon...
接下來,我希望有機會及時間可以介紹人臉辨識(Face Recognition)的部份,也就是判斷某張人臉是哪個人,如此,加上Face Detection,便可以讓我們很方便、很快的從數以千或萬計的數位照片當中挑出某個人出現過的照片了