Advances in video face recognition

Contents
Preface
1 Video face recognition 1
1.1 Why study video face recognition 1
1.2 Factors affecting video face recognition 5
1.2.1 Image acquisition and imaging conditions 5
1.2.2 Video face gathering and acquisition 9
1.2.3 Some difficulties of video face recognition 10
1.3 Applications of video face recognition 11
1.4 Approaches to video face recognition 13
1.4.1 Approaches using holistic features 13
1.4.2 Approaches using local features 16
1.4.3 3D-based approaches 17
1.4.4 Video-based approaches 19
1.5 Future research directions 22
2 Local face alignment 25
2.1 Local face alignment for video faces 25
2.2 Classification of local face alignment methods 27
2.2.1 Image alignment 27
2.2.2 Groupwise alignment 31
2.2.3 Face alignment considering continuity among images 35
2.3 Development of video face alignment 40
2.3.1 Development of image alignment 41
2.3.2 Development of groupwise alignment 43
2.3.3 Development of face alignment considering continuity 44
2.4 Future research directions about face alignment in a video 46
2.4.1 Application fields 48
2.4.2 Problem description 49
2.4.3 Representation of model and fitting 49
2.4.4 Further study 50
3 Joint face alignment 51
3.1 Significance of joint face alignment 51
3.2 Rigid and non-rigid joint alignment 54
3.2.1 Rigid joint face alignment 55
3.2.2 Non-rigid joint face alignment 57
3.3 Variants 59
3.4 Future research directions of joint alignment 61
4 Binocular face alignment 65
4.1 Importance of binocular face alignment 65
4.2 Approaches for binocular face alignment 67
4.2.1 Approaches using epipolar geometry 67
4.2.2 Approaches of using 3D face model constraint 71
4.3 Factors influencing binocular face alignment 74
4.4 Development of binocular face alignment 76
4.4.1 Using joint alignment to align binocular faces 76
4.4.2 Using epipolar geometry to align binocular faces 77
4.4.3 Using 3D face model to align binocular faces 82
4.5 Future research directions 83
5 Local texture feature descriptors 85
5.1 Classification of local texture feature descriptors 85
5.2 SIFT feature descriptor 86
5.2.1 Original SIFT 87
5.2.2 SIFT variants 87
5.3 LBP feature descriptor 89
5.3.1 Original LBP 89
5.3.2 LBP variants 90
5.4 Gabor wavelets feature descriptor 93
5.4.1 Original gabor 93
5.4.2 Gabor variants 94
5.5 Performance analysis of three texture feature descriptors 96
5.5.1 Performance of regressor using single feature 97
5.5.2 Performance of regressor using fusion features 99
5.5.3 Comparisons of face recognition using Gabor wavelets 100
5.6 Future research directions 102
6 Considering facial symmetry 104
6.1 Why use facial symmetry 104
6.2 Significance of symmetry 106
6.3 Factors influencing symmetry 107
6.3.1 Effect of rotating reference point variation 108
6.3.2 Effect of eye center shifting 109
6.3.3 Effect of face classifiers 110
6.3.4 Effect of eyebrows 111
6.4 How to solve symmetry violation problems 111
6.5 Eye center location method 112
6.5.1 Removing detection window overlay and location offset 113
6.5.2 Obtaining unknown eye center 115
6.5.3 Optimizing pupil center 118
6.6 Experiments and performance analysis 118
6.6.1 Process and measurement 119
6.6.2 Test results for BioID database 119
6.6.3 Test results for open head pose database 121
6.6.4 Test results for unconstrained face database 121
6.7 Future research directions 122
7 Illumination problem 124
7.1 Classification of approaches for the illumination problem 124
7.2 Influence of illumination variation 125
7.3 Lambertian reflectance model 126
7.3.1 Model description 127
7.3.2 Simplified form 128
7.4 Illumination pre-processing or normalization 129
7.4.1 Image processing methods 129
7.4.2 Retinex and its variants 130
7.4.3 Transform domain methods 136
7.5 Illumination insensitive features 136
7.6 Illumination generation (re-lighting) 137
7.6.1 Linear subspace 138
7.6.2 Illumination cone 139
7.6.3 Spherical harmonic function 140
7.6.4 3D methods 141
7.6.5 Generating face images 143
7.7 Future research directions 143
8 Score level fusion 145
8.1 Necessity of score level fusion 145
8.2 Score level fusion using Quasiconvex optimization 146
8.2.1 Definitions and preliminaries 146
8.2.2 Quasiconvex optimization-based score fusion 151
8.3 Experiments 156
8.3.1 Score fusion in face recognition 156
8.3.2 Score fusion in multimodal biometrics 163
9 Facial analysis based on subspace 168
9.1 Subspace construction and analysis 168
9.2 AAM 171
9.2.1 AAM modeling 171
9.2.2 Model fitting 172
9.3 Face analysis by using submanifold 173
9.3.1 Submanifold decomposition 174
9.3.2 Tests and analysis for submanifold decomposition method 177
9.4 Facial analysis by using Bayesian infe