Real-time face tracking and recognition by sparse eigentracker with associative mapping to 3D shape

Research output: Contribution to journalArticle

8 Citations (Scopus)

Abstract

This paper proposes a novel framework of real-time face tracking and recognition by combining two eigen-based methods. The first method is a novel extension of eigenface called augmented eigenface and the second method is a sparse 3D eigentemplate tracker controlled by a particle filter. The augmented eigenface is an eigenface augmented by an associative mapping to 3D shape that is specified by a set of volumetric face models. This paper discusses how to make up the augmented eigenface and how it can be used for inference of 3D shape from partial images. The associative mapping is also generalized to subspace-to-one mappings to cover photometric image changes for a fixed shape. A novel technique, called photometric adjustment, is introduced for simple implementation of associative mapping when an image subspace should be combined to a shape. The sparse 3D eigentemplate tracker is an extension of the 3D template tracker proposed by Oka et al. In combination with the augmented eigenface, the sparse 3D eigentemplate tracker facilitates real-time 3D tracking and recognition when a monocular image sequence is provided. In the tracking, sparse 3D eigentemplate is updated by the augmented eigenface while face pose is estimated by the sparse eigentracker. Since the augmented eigenface is constructed on the conventional eigenfaces, face identification and expression recognition are also accomplished efficiently during the tracking. In the experiment, an augmented eigenface was constructed from 25 faces where 24 images were taken in different lighting conditions for each face. Experimental results show that the augmented eigenface works with the 3D eigentemplate tracker for real-time tracking and recognition.

Original languageEnglish
Pages (from-to)147-158
Number of pages12
JournalImage and Vision Computing
Volume30
Issue number3
DOIs
Publication statusPublished - Mar 2012

Fingerprint

Lighting
Experiments

Keywords

  • 3D eigentemplate
  • Augmented eigenface
  • Face tracking and recognition
  • Photometric adjustment
  • Shape inference from image

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Cite this

Real-time face tracking and recognition by sparse eigentracker with associative mapping to 3D shape. / Oka, Yuki; Shakunaga, Takeshi.

In: Image and Vision Computing, Vol. 30, No. 3, 03.2012, p. 147-158.

Research output: Contribution to journalArticle

@article{cf703d2d8ae640baad1851ba43523ccc,
title = "Real-time face tracking and recognition by sparse eigentracker with associative mapping to 3D shape",
abstract = "This paper proposes a novel framework of real-time face tracking and recognition by combining two eigen-based methods. The first method is a novel extension of eigenface called augmented eigenface and the second method is a sparse 3D eigentemplate tracker controlled by a particle filter. The augmented eigenface is an eigenface augmented by an associative mapping to 3D shape that is specified by a set of volumetric face models. This paper discusses how to make up the augmented eigenface and how it can be used for inference of 3D shape from partial images. The associative mapping is also generalized to subspace-to-one mappings to cover photometric image changes for a fixed shape. A novel technique, called photometric adjustment, is introduced for simple implementation of associative mapping when an image subspace should be combined to a shape. The sparse 3D eigentemplate tracker is an extension of the 3D template tracker proposed by Oka et al. In combination with the augmented eigenface, the sparse 3D eigentemplate tracker facilitates real-time 3D tracking and recognition when a monocular image sequence is provided. In the tracking, sparse 3D eigentemplate is updated by the augmented eigenface while face pose is estimated by the sparse eigentracker. Since the augmented eigenface is constructed on the conventional eigenfaces, face identification and expression recognition are also accomplished efficiently during the tracking. In the experiment, an augmented eigenface was constructed from 25 faces where 24 images were taken in different lighting conditions for each face. Experimental results show that the augmented eigenface works with the 3D eigentemplate tracker for real-time tracking and recognition.",
keywords = "3D eigentemplate, Augmented eigenface, Face tracking and recognition, Photometric adjustment, Shape inference from image",
author = "Yuki Oka and Takeshi Shakunaga",
year = "2012",
month = "3",
doi = "10.1016/j.imavis.2011.12.010",
language = "English",
volume = "30",
pages = "147--158",
journal = "Image and Vision Computing",
issn = "0262-8856",
publisher = "Elsevier Limited",
number = "3",

}

TY - JOUR

T1 - Real-time face tracking and recognition by sparse eigentracker with associative mapping to 3D shape

AU - Oka, Yuki

AU - Shakunaga, Takeshi

PY - 2012/3

Y1 - 2012/3

N2 - This paper proposes a novel framework of real-time face tracking and recognition by combining two eigen-based methods. The first method is a novel extension of eigenface called augmented eigenface and the second method is a sparse 3D eigentemplate tracker controlled by a particle filter. The augmented eigenface is an eigenface augmented by an associative mapping to 3D shape that is specified by a set of volumetric face models. This paper discusses how to make up the augmented eigenface and how it can be used for inference of 3D shape from partial images. The associative mapping is also generalized to subspace-to-one mappings to cover photometric image changes for a fixed shape. A novel technique, called photometric adjustment, is introduced for simple implementation of associative mapping when an image subspace should be combined to a shape. The sparse 3D eigentemplate tracker is an extension of the 3D template tracker proposed by Oka et al. In combination with the augmented eigenface, the sparse 3D eigentemplate tracker facilitates real-time 3D tracking and recognition when a monocular image sequence is provided. In the tracking, sparse 3D eigentemplate is updated by the augmented eigenface while face pose is estimated by the sparse eigentracker. Since the augmented eigenface is constructed on the conventional eigenfaces, face identification and expression recognition are also accomplished efficiently during the tracking. In the experiment, an augmented eigenface was constructed from 25 faces where 24 images were taken in different lighting conditions for each face. Experimental results show that the augmented eigenface works with the 3D eigentemplate tracker for real-time tracking and recognition.

AB - This paper proposes a novel framework of real-time face tracking and recognition by combining two eigen-based methods. The first method is a novel extension of eigenface called augmented eigenface and the second method is a sparse 3D eigentemplate tracker controlled by a particle filter. The augmented eigenface is an eigenface augmented by an associative mapping to 3D shape that is specified by a set of volumetric face models. This paper discusses how to make up the augmented eigenface and how it can be used for inference of 3D shape from partial images. The associative mapping is also generalized to subspace-to-one mappings to cover photometric image changes for a fixed shape. A novel technique, called photometric adjustment, is introduced for simple implementation of associative mapping when an image subspace should be combined to a shape. The sparse 3D eigentemplate tracker is an extension of the 3D template tracker proposed by Oka et al. In combination with the augmented eigenface, the sparse 3D eigentemplate tracker facilitates real-time 3D tracking and recognition when a monocular image sequence is provided. In the tracking, sparse 3D eigentemplate is updated by the augmented eigenface while face pose is estimated by the sparse eigentracker. Since the augmented eigenface is constructed on the conventional eigenfaces, face identification and expression recognition are also accomplished efficiently during the tracking. In the experiment, an augmented eigenface was constructed from 25 faces where 24 images were taken in different lighting conditions for each face. Experimental results show that the augmented eigenface works with the 3D eigentemplate tracker for real-time tracking and recognition.

KW - 3D eigentemplate

KW - Augmented eigenface

KW - Face tracking and recognition

KW - Photometric adjustment

KW - Shape inference from image

UR - http://www.scopus.com/inward/record.url?scp=84859841982&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84859841982&partnerID=8YFLogxK

U2 - 10.1016/j.imavis.2011.12.010

DO - 10.1016/j.imavis.2011.12.010

M3 - Article

AN - SCOPUS:84859841982

VL - 30

SP - 147

EP - 158

JO - Image and Vision Computing

JF - Image and Vision Computing

SN - 0262-8856

IS - 3

ER -