Multimodal biometrics Fusion based on TER and Hybrid Intelligent Multiple Hidden Layer Probabilistic Extreme Learning Machine

Di Wu; Qin Wan

doi:10.2991/ijcis.11.1.71

<Previous Article In Issue

Download article (PDF)

Next Article In Issue>

Volume 11, Issue 1, 2018, Pages 936 - 950

Multimodal biometrics Fusion based on TER and Hybrid Intelligent Multiple Hidden Layer Probabilistic Extreme Learning Machine

Authors

Di Wu¹^,wudi6152007@163.com, Qin Wan²^,15798021@qq.com

¹College of Electrical and Information, Hunan Institute of Engineering, Xiang Tan, 411104, China

²College of Electrical and Information, Hunan Institute of Engineering, Xiang Tan, 411104, China

Received 7 November 2017, Accepted 17 April 2018, Available Online 2 May 2018.

DOI: 10.2991/ijcis.11.1.71 How to use a DOI?
Keywords: MultiBiometrics; Total Error Rate(TER); Extreme Learning Machine(ELM); Differential Evolution(DE); Particle Swarm Optimization(PSO)
Abstract: In this paper, a novel fusion method based on Total Error Rate (TER) and multiple hidden layer probabilistic extreme learning machine is proposed. At first, the study transfers the matching scores into TER based on corresponding False Reject Rates (FRR) and False Accept Rates (FAR) aims at avoiding to calculating the posterior probability. At the second, a new fusion strategy based on multiple hidden layer probabilistic extreme learning machine is introduced, which optimizes the architecture of hidden nodes by weighted calculation of different output matrices and then transforms the numeric output of ELM to the probabilistic outputs and unifies the outputs in a fixed range, the matrices weights and the output weights are optimized using a hybrid intelligent algorithm based on differential evolution and particle swarm optimization. Experiment result shown that the proposed method renders very good performance as it is quite computationally and outperforms the traditional score level fusion schemes, the experimental result also confirms the effectiveness of the proposed method to improve the performance of multibiometric system.
Copyright: © 2018, the Authors. Published by Atlantis Press.
Open Access: This is an open access article under the CC BY-NC license (http://creativecommons.org/licences/by-nc/4.0/).

1. Introduction

Face, fingerprint and iris have been explored to recognize humans in some extent [1]. However, the recognition performance of these unimodal biometric recognition system are not ideal in some case because they are plagued by some drawbacks. For example, noise in the input samples, nonuniversality, usceptibility of the result to the quality of the sample, its orientation/rotation and distortion, risk of spoofing and others [2]. confronted by the limitation of the unimodal recognition systems, multimodal biometric recognition systems try to alleviate these disadvantages by consolidating the evidence presented by multiple biometric sources in order to in order to improve the recognition performance. Compared to the unimodal recognition system, the multimodal biometric system have several merits:1) it is address the problem of nonuniversality which encountered by unimodal system; 2)they limit the ability of an impostor to spoof multiple biometric features of a legitimately enrolled sample;3) it is can solve the problem of noisy input data effectively;4) it can viewed as fault tolerant recognition systems due to the reason of they can operate continuously even when some biometric resources become unbelievable and unreliable.

According to the level of information fused, we can subdivide them in five categories [3]: sensor level fusion, feature level fusion, matching level fusion, rank level fusion and decision level fusion. Among them, score level fusion is the most frequently utilized because of easy availability of the scores and contains ample information to discriminate between genuine and impostor scores. Already existing score level fusion approaches can be categorized into three classes: transformation-based, density-based and classifier-based.

In conclusion, the transformation-based fusion approach, with no training process and few consideration of distribution of matching scores and it is easy to implement [7]. On the contrary, density-based fusion method, which requires accurate estimation of density and huge number of training samples, is hard to carry out for the following reasons: firstly, positive samples, namely genuine matching scores are limited in today’s multi-biometric systems, secondly, it is difficult to estimate the density of matching scores in that they may not obey a certain distribution model. Classifier-based fusion has its advantage of freeing from the restriction of different distribution of matching scores, but it needs sufficient training samples and new training for a different system.

In our previous works, we had explored the potential of False Reject Rates (FRR) and False Accept Rates(FAR) using in the score level fusion. In [8], we proposed a fusion method based on FAR and FRR using triangular norm avoiding to calculate the posterior probability of a certain source. In [9], a novel fusion strategy based on total error rate (TER) and feature correlation adaptive fusion is given. From the results in [8] and [9] we find that the improvement of the recognition performance are not obviously, and we are not good use of the feature correlation information.

Based on the above analysis of the advantages and drawbacks of the existing methods and our previous works, in this paper, a novel hybrid fusion method is proposed which based on TER and Multiple Hidden Layer Probabilistic Extreme Learning Machine (MHP-ELM). At first, the study transfers the matching scores into TER based on corresponding False Reject Rates (FRR) and False Accept Rates(FAR) aims at avoiding to calculating the posterior probability. At the second, a new fusion strategy based on hybrid intelligent multiple hidden layer probabilistic extreme learning machine(HMP-ELM) is introduced, which optimizes the architecture of hidden nodes by weighted calculation of different output matrices and then transforms the numeric output of ELM to the probabilistic outputs and unifies the outputs in a fixed range, in this strategy, the matrices weights μ of the Multiple Hidden Layer Extreme Learning Machine(M-ELM) and the output weights β belongs to the ELM are optimized using a hybrid intelligent algorithm based on Differential Evolution and Particle Swarm Optimization(DEPSO).

The main contributions of this paper can be summarized as follows: 1) Based on our previous works, this paper still using the total error rate (TER) instead of matching scores as the matching character in fusion process. 2) A novel fusion scheme based on Hybrid Intelligent Multiple Hidden Layer Probabilistic Extreme Learning Machine (HMP-ELM) is given in this paper. To our knowledge, this is the first time to utilize the ELM for information fusion. 3) In order to explore a optimal parameter belong to the Extreme Learning Machine, a new optimization strategy to the two weighted parameters based on Differential Evolution and Particle Swarm Optimization(DEPSO) is proposed.

The remainder of this paper is organized as follows: Section II reviews some related works. Section III provides the background knowledge required for proposing the new fusion method. In Section IV, the detail of the Hybrid Intelligent Multiple Hidden Layer Probabilistic Extreme Learning Machine (HMP-ELM) is briefly discussed. The proposed new fusion method is given in Section V. Hence, the experiments result is presented in Section VI. Section VII conclude our work and outlines our future work to generalize the method to multibiometric recognition system.

2. Related Works

In the previous researcher’s works^[10], most multibiometric fusion system always use fixed fusion rules with some parameters, these fusion method tuned to reach the desired recognition accuracy for a fixed security level. However, we had found most multimodal biometric recognition system select the the best fusion methods or the fusion parameters automatically in order to enhance the recognition performance.

In [11], Tronci et al. Proposed a dynamic score selection scheme for multiple biometric matchers fusion, the best matching score is selected based on the likelihood of the users when is being a genuine or an impostor, but the fusion performance of this method is poor. In [12], Veeramachaneni et al. Introduced a novel fusion method called “ Adaptive Multiple Biometric Fusion Algorithm(AMBF)” which based on the combination of Bayesian decision and Particle Swarm Optimization(PSO), in this fusion method, the author have considered that the multibiometric recognition system is composed of N classifiers, so there are 2^{2^N} possible fusion rules will be used but only a part of these fusion rules can be used to improve the fusion performance under a Bayesian framework to fuse decisions returned from the classifiers.

In [13], Srinivas et al also presented a new fusion scheme in order to enhancing the recognition performance of correlated biometric classifiers, in this fusion technique, the fusion weight is associated with each classifier and then computed by PSO, in the final, the weight is adapted based on the correlation between different classifiers. The author have claimed that the fusion technique proposed in this paper is outperforms the traditional weighted sum fusion rules.

In [14], Kumar et al proposed a fusion approach based on a hybrid PSO modal which used to determine the optimal fusion parameters, in this method, it can ensure an optimal fusion performance in a dynamic architecture which include multiple fusion rules such as sum, product, exponential sum and tan-sum. The author final claimed that the method proposed in this paper can achieve a better recognition performance than the decision level adaptive fusion method.

In [15], Anzar et al proposed a PSO scheme in the weighted sum fusion rule. It uses d-prime statistic to enhance the recognition performance under various noise conditions which the weights are computed as the ratio between the genuine and the impostor score distribution while the best weights are founded by PSO.

In [16], Kien et al proposed a score level fusion technique based on D-S theory and uncertain factors. The research seeks to develop a unified framework for multimodal biometric fusion to take advantages of the uncertainty concept of the D-S theory. In the fusion process, the quality measures and classifiers performance are combined.

In [17], Lamia et al proposed a score level fusion algorithm using Differential Evolution and Proportional Conflict Redistribution Rule. DE technique is used to find the best confidence factors of the belief assignments of the difference modalities and Proportional Conflict Redistribution Rule is used to compute the weight.

In our previous work, a novel approach is proposed for the fusion at score level fusion based on False Reject Rate(FRR) and False Accept Rate(FRR) using triangular norms(t-norms). This study aims at tapping the potential of t-norms for information fusion at first, at the second, it transfers scores into Transfer function based on corresponding FRRs and FARs, thus avoiding calculating posterior probability of a certain score. In the [9], A novel approach is proposed based on Total Error Rate (TER) and feature correlation adaptive fusion method. The new adaptive fusion method adjusting the fusion weights adaptive between weight sum fusion scheme and multiplicative fusion scheme using the characteristic of correlation coefficient.

3. Preliminary

In this part, we will give the notation of Extreme Learning Machine(ELM) - Multiple Hidden Extreme Learning Machine(M-ELM) and Probabilistic Extreme Learning Machine(P-ELM).

3.1. Extreme Learning Machine (ELM)

Extreme Learning Machine^[18] is a high efficient learning algorithm that proposed on the singer-hidden layer neural network, unlike other different traditional neural network, all the parameters in the Extreme Learning Machine are generate randomly and the complicated iteration process is avoid. Suppose the training set {xi,ti}i=1N is compose of N training samples, the input is x_i which the dimension is d, t_i is the label of the output, then the output of the ELM is^[19]:

(1)fL(x)=∑j=1LβjG(aj,bj,xi)=h(xi)β

In equation (1), the parameter a_j is the input weight of the jth hidden node, b_j is the deviation of the jth hidden node, and β_j is the weight of the jth hidden node to the output node of the ELM. G(a_j, b_j, x_i) is the output function of the jth hidden node, from the equation (1), we will obtain that h(x_i) = [G(a₁, b₁, x_i),...,G(a₁, b₁, x_i)] is the output of hidden layer regard to training sample x_i. In order to enhance the ability of generalization of the ELM, we describe the ELM to the optimization problem when we using for classification problem in the form of equation (2):

(2)Min:LELM=12‖β‖2+C12∑i=1N‖ξi‖2s.t.h(xi)=tiT−ξiT, i=1,…,N}

In order to eliminate the excessive fitting phenomenon, we introduce a training error ξ_i, and the corresponding punish factor is C, so this optimization problem can transform into the dual problem:

(3)LELM=12‖β‖2+C12∑i=1N‖ξi‖2−∑i=1N∑j=1Lαi,j(h(xi)βj−ti,j+ξi,j)

(4)∂LELM∂βj=0→βj=∑i=1Nαi,jhT(xi)→β=HTα

(5)∂LELM∂ξi=0→αi=Cξi

(6)∂LELM∂αi=0→h(xi)β−tiT+ξiT=0

So from the equation (4) to equation (6), we can obtain:

(7)(1C+HHT)α=T

(8)β=HT(1C+HHT)−1T

We can compute the output weight β Using equation (8) directly, then the output equation of the ELM can describe as:

(9)f(x)=h(x)β=h(x)HT(1C+HHT)−1T

(10)H=[G(a1,b1,x1)…G(aL,bL,x1)⋮⋮G(a1,b1,xN)…G(aL,bL,xN)]N*L =[h(x1)⋮h(xN)]

In the training process of the ELM, there is only one parameter to be determined, i.e. the punish coefficient C.

3.2. Multiple Hidden Extreme Learning Machine (M-ELM)

In traditional Extreme Learning Machine method, there is only single form about its hidden output matric and the parameter will be computed randomly. But in the Multiple Hidden Extreme Learning Machine, if there are C different basic matric, then we will generate a new hidden output matric through a weighting strategy as follows^[25]:

(11)H=∑k=1CμkHk(xi)

In this notation, the coefficients μ_k is called weighting coefficients which using to weighting different basic matric. In the traditional Extreme Learning Machine method, the core of the optimization problem is to solve the output weight β, but in this M-ELM method, we also need to compute the coefficients μ_k. Then, a new optimization problem will be tackling as follows:

(12)minLM−ELM=∑i=1N‖∑k=1CμkHk(xi)β−ti‖2+β2

so the output of the M-ELM method given as follows:

(13)f(x)=∑k=1Cμkhk(x)β

3.3. Probabilistic Extreme Learning Machine (P-ELM)

The traditional Extreme Learning Machine method is always using for classification problem, in this part, we introducing a Probabilistic Extreme Learning Machine (P-ELM).

In the P-ELM, we modify the project process at first, so the output of the ELM given in the equation (9) will be transformed as^[26]:

(14)P(fi(x))=11+exp(−fi(x))

Through this operation, we transfer the output of the traditional ELM into the probabilistic form, further more, and normalize the probabilistic output P(f_i(x)) so we will obtain the probability of the input sample x belongs to the label i in the numerical sense and we mark it as p_i(x) :

(15)pi(x)=P(fi(x))∑i=1mP(fi(x))

Using the equation (16), we can transfer the output of the PELM into the stiff discriminate way:

(16)label(x)=argmaxpi(x)

4. The proposed Hybrid Intelligent Multiple Hidden Layer Probabilistic Extreme Learning Machine (HMP-ELM)

In this part, the proposed Hybrid Intelligent Multiple Hidden Layer Probabilistic Extreme Learning Machine (HMP-ELM) will be given. At first, we are using the Multiple Hidden Layer Extreme Learning Machine instead of traditional Extreme Learning Machine. For the kth classifier in the recognition system, we will obtain its output f_ik(x), while f_ik(x) stands for the output belongs to the label i in the numerical sense and kth classifier. Then, the outputs of different classifiers that come from the M-ELM are transfer into probabilistic output forms using equation (14). After this process, the numerical output will change into p_ik(x). In the next fusion process, the p_ik(x) will be using for fusion.

In our proposed new Hybrid Intelligent Multiple Hidden Layer Probabilistic Extreme Learning Machine (HMP-ELM), the key of the method is the computation of the matrices weights μ of the Multiple Hidden Layer Extreme Learning Machine (M-ELM) and the output weights β belongs to the ELM. In this part, a improved hybrid intelligent optimized strategy called Differential Evolution Particle Swarm Optimization (DEPSO) which inspired from the idea of Frog Leaping algorithm (FLA) is proposed while based on differential evolution (DE) and particle swarm optimization (PSO).

The iteration process of the traditional Frog Leaping algorithm (FLA) is consisting of three parts: partial meme group, evolve in different meme group isolated and hybrid meme group, the iterative evolution process has been running until the termination condition is met. The new proposed DEPSO optimization algorithm using the DE and PSO as evolution method and has the ability of meme evolution while derive from Frog Leaping algorithm (FLA) in order to improve the performance when taking the advantage of the two algorithms.

In this paper, the detailed steps of the Differential Evolution Particle Swarm Optimization which using for the computation of the matrices weights μ of the Multiple Hidden Layer Extreme Learning Machine (M-ELM) and the output weights β belongs to the ELM is stated as follows:

Step 1: Generate N_P solutions as the initial population P={Xj}j=1NP in the dimension space randomly, the total number of iterations is I_iter_max, the number of iterations belong each subpopulation is I_iter.
Step 2: Divide the population P into N_k subpopulations averagely.
Step 3: Choose k subpopulations randomly while 1 < k < N_k, using the DE algorithm to compute the I_iter generation in the iteration process respectively. Regarding to the rest N_k ‒ k subpopulations, using the PSO algorithm to compute the I_iter generation in the iteration process respectively. In the entire iteration process, recording all the change of the optimal value to the all population.
Step 4: Mix N_k subpopulations in order to obtain the new population P, judge the number of the iteration of local search reaches the designated number of iterations.
Step 5: The algorithm is termination.

The flowchart of the proposed Hybrid Intelligent Multiple Hidden Layer Probabilistic Extreme Learning Machine (HMP-ELM) is given as follows.

5. The Proposed Novel Fusion Strategy Based on HMP-ELM

In traditional Extreme Learning Machine strategy, it is always using for classification problems, in this paper, we using it for information fusion problem. To our knowledge, this is the first time to utilize the Extreme Learning Machine for information fusion problem.

The base of using the Extreme Learning Machine for information fusion problem is the introduction of Probabilistic Extreme Learning Machine, aimed to the multi classes classification problem, when we utilize M different classifier, we can compute the probabilistic output matric:

(17)P(x)=[p11(x)…p1M(x)⋮⋱⋮pm1(x)…pmM(x)]m*M

In this matric, each column stands for a probabilistic output of a classifier, the element p_ij(x) in the matric represent that the input sample x while discriminate it belong to the ith label using the jth classifier. While the computing of the probabilistic output matric equal to a normalization process to the traditional ELM. At the end, computing the fusion result.

Suppose there are M different classifier in the fusion recognition system, according to the equation (31), the probabilistic output of the the jth classifier to random test samples is the utmost value of all the output belong to this classifier, that is:

(18)Pj=maxi=1,2,…,m[pij(x)]

Then the fusion weight is computing using the adaptive computing strategy as:

(19)αj=Pj∑j=1MPj

The fusion output of the fusion system will stated as follows:

(20)label(x)=arg maxi=1,2,…,m[∑j=1Mαjpij(x)]=arg maxi=1,2,…,m[∑j=1Mαj11+exp(−fij(x))]=arg maxi=1,2,…,m[∑j=1Mαj11+exp(−∑k=1Cμkhkβ)]

Specially, if there are two classifier in the fusion recognition system, the fusion strategy proposed by the equation (20) can be simplified as :

(21)label(x)=arg maxi=1,2,…,m[α1pi1+α2pi2]=arg maxi=1,2,…,m[α111+exp(−fi1(x))+α211+exp(−fi2(x))]=arg maxi=1,2,…,m[α111+exp(−∑k=1Cμkhkβ)+α211+exp(−∑k=1Cμkhkβ)]

The pseudecode of this proposed fusion approach is stated as follows:

Pseudecode 1: The proposed fusion method using HMP-ELM.

Input: the matching score from M classifiers

1.
Transfer the matching scores into Total Error Rate.
2.
Compute the output of the M-ELM for different classifier using (28). Then, we will obtain f_ij(x).
3.
Using P-ELM, transfer the output into probabilistic form using (29).
4.
Fusing different classifier output using (33)∼(35).

Output: fusion recognition result

The flowchart of this fusion method is illustrated in Figure 2.

6. Result and Discussion

6.1. Experimental settings

In this section, we will provide a wide range of different experimental results in different quarters to access the effectiveness of the proposed new fusion strategy.

Because of the main level of the interest in this paper is multi-biometric fusion recognition, so we put the multi-biometric as the main target in the experiments. In order to access the effectiveness of the proposed new method, we using the XM2VTS-Benchmark multi-biometric database in our experiments.

The XM2VTS-Benchmark multi-biometric database contains 295 people face image data and voice data, while all the face data and voice data are collected four times synchronically, each time be arranged at intervals of one months. In the database, there are five face recognition classifiers and three-speaker recognition classifiers. in our experiment, we set five face recognition classifiers as XF₁, XF₂, XF₃, XF₄, XF₅; and the three-speaker classifier as XS₁, XS₂, XS₃.

In these experiments, we choose several popular fusion methods for comparison, to evaluate the performance of the proposed novel fusion methods, such as LR method, Min-Max method, Tanh method, ELM method and the fusion method proposed in Reference [9] which it our former working. In the comparison experiments, we put the fusion method proposed in this paper as T method.

The experiments are include four parts:

(1)
To evaluate the performance and robustness of the HMP-ELM. At first, we test the performance of the DEPSO, which proposed in this paper while using for normalize the matrices weights μ of the Multiple Hidden Layer Extreme Learning Machine (M-ELM) and the output weights β belongs to the ELM. Then, we test the performance of the HYP-ELM.
(2)
We compare the recognition performance between different classify criteria. In this experiment, we conduct the raw matching scores and TER as the classify criteria respectively, and then to test the fusing recognition performance using different fusion methods in order to certificate that recognition performance which using TER is better than using raw matching scores.
(3)
We test the recognition performance based on the fusion method proposed in this paper. In this experiment, we using the TER as the classify criteria and compare the recognition performance between the proposed fusion method and the other five fusion methods mentioned before.
(4)
We compare the recognition accuracy and recognition time between different fusion methods. In this experiment, we using the TER as the classify criteria and compare the recognition accuracy and recognition time between the proposed fusion method and the other five fusion methods mentioned before.

6.2. evaluate the performance and robustness of the HYP-ELM

In this section, we will evaluate the performance and robustness of the Hybrid Intelligent Multiple Hidden Layer Probabilistic Extreme Learning Machine (HMP-ELM) that proposed in this paper. Firstly, we test and verify the optimization ability of the Differential Evolution Particle Swarm Optimization (DEPSO) algorithm that given in this paper which using for optimize the matrices weights μ of the Multiple Hidden Layer Extreme Learning Machine (M-ELM) and the output weights β belongs to the ELM. Secondly, we check the performance of the HMP-ELM algorithm proposed in this paper, in this experiment, we test the regression ability of the HMP-ELM and compare it to other ELM methods.

At first, we present the optimization ability of the Differential Evolution Particle Swarm Optimization (DEPSO) algorithm. In this experiment, we are using 10 typical functions, which stated in Table 1 in order to check the optimization ability. The dimension of the solution in every typical function D is setting as 30, the range of the solution of the typical function F_n₆ is setting as [–100, 100], F_n₉ is setting as [–500, 500], the remains is setting as [–30, 30].

Function Name	Equations
Sphere	Fn1=∑i=1Dxi2
Quadric	Fn2=(∑i=1Dxi2)2
Griewank	Fn3=∑i=1Dxi24000−∏i=1Dcos(xii)+1
Ackley	Fn4=−20exp(−0.21D∑i=1Dxi2)− exp(1D∑i=1Dcos(2πxi)+20+e
Rosenbrock	Fn5=∑i=1D−1(100(xi2−xi+1)2+(xi−1)2)
Rastrigin	Fn6=∑i=1D(xi2−10cos(2πxi)+10)
Schwefel’s 1.2	Fn7=(∑i=1D(∑j=1ixj)2)(1+0.4\|N(0,1)\|)
Griewank with noise	Fn8=∑i=1D(xi−N(0,1))24000
Schwefel	Fn9=418.9828D−∑i=1Dxisin(\|xi\|1/2)
Schwefel’s 2.22	Fn10=∑i=1D\|xi\|+∏i=1D\|xi\|

Table 1.

The typical optimization functions

In order to compare the performance the DEPSO algorithm, we compare it with three basic optimization algorithms: DE, PSO and FLA. The parameters in each method are showed in Table 2.

Method	Parameter
DE	F = 0.5 C_R = 0.3
PSO	w = 0.729 c₁ = c₂ = 1.495
FLA	C = 1.2
DEPSO	N_k = 4 I_iter = 10 k = 4

Table 2.

The parameters in each method

In the numerical experiment, the scales of the population in four algorithms are all the same that means N_p = 40, the number of the iteration in each method is 2000. For each typical function, the times of the optimization which using four optimization algorithm is 50 and the average optimization value will be utilized as the final optimization result. Table 3 outlined the optimization result using DE, PSO, FLA and DEPSO methods while make use of 10 typical functions.

Function	Performance	DE	PSO	FLA	DEPSO
F_n₁	Mean	1.7667e+002	1.2494e+003	1.1951e+002	1.2008e−009
F_n₁	Variance	7.3438e+001	2.5787e+002	4.1270e+001	7.4548e−010

F_n₂	Mean	0.0000e+000	1.2494e+000	0.0000e+000	4.3528e−007
F_n₂	Variance	0.0000e+000	1.9059e−002	0.0000e+000	8.2034e−007

F_n₃	Mean	9.8590e−001	1.2928e+000	1.0011e+000	8.4011e−004
F_n₃	Variance	9.2600e−002	6.0500e−002	3.5900e−002	1.5000e−003

F_n₄	Mean	9.1589e+000	1.6711e+001	7.8190e−000	8.6480e−004
F_n₄	Variance	1.4730e+000	7.4780e−001	1.1279e−000	7.0593e−006

F_n₅	Mean	2.9764e+005	1.3255e+007	1.2866e+005	2.6678e+001
F_n₅	Variance	1.9895e+00	6.2356e+006	7.4297e+004	9.3772e+000

F_n₆	Mean	3.6770e+002	1.6137e+003	8.1555e+002	1.4644e+002
F_n₆	Variance	7.3999e+001	3.3505e+00	1.2285e+001	1.2235e+001

F_n₇	Mean	2.3981e+004	3.7054e+005	7.7842e+002	2.3319e+002
F_n₇	Variance	9.3795e+003	9.5826e+004	3.7931e+002	1.4052e+001

F_n₈	Mean	3.8400e−002	3.1190e−001	1.4609e−003	8.9489e−004
F_n₈	Variance	1.3400e−002	6.6700e−002	5.2516e−002	1.3616e−004

F_n₉	Mean	6.8456e+003	7.4513e+003	7.5097e+003	4.4974e+003
F_n₉	Variance	1.4465e+003	6.2601e+002	6.1316e+002	3.6771e+002

F_n₁₀	Mean	5.5864e+001	2.0366e+002	8.8238e+002	1.0276e−004
F_n₁₀	Variance	1.2322e+001	2.3497e+001	4.0068e+001	3.7921e−005

Table 3.

The optimization result using different methods

From the optimization results which shown in Table .3, we can found that when we taking optimization experiment to the typical function F_n₂, the searching performance obtained using four optimization algorithms all can reach a ideal result. While the optimization result taking another nine typical functions can be summary as follows: (1). For the precision of searching results, the proposed DEPSO optimization method is better than the DE, PSO and the FLA methods obviously when it taking to the other nine typical functions, and its can obtain more precise solution. (2). For the ability of out of local minimum, the PSO algorithm falls into the minimum value point quickly and the length which play the major role in the time domain in the optimization periods is very short; for the proposed DEPSO strategy, its can out of the local minimum continuously in the iteration process in order to search the optimal solution and its have a better searching ability. In conclusion, the proposed DEPSO optimization method have a better improvement in the searching optimization ability and its have a good balance to the searching optimization precision and convergence speed.

After testing the performance of DEPSO optimization, we evaluate the regression computing ability of the proposed HMP-ELM. In this experiment, six classical real databases are being used: Servo database, Cancer database, Autompg database, CPU database, Price database and Housing database and all the database are download from http://www.ics.uci.edu . In the experiment, all the database are divided into two categories: training database and testing database which the specification are given in Table.4:

Database	Training Data	Testing Data
Servo	80	87
Cancer	100	94
Autompg	220	178
CPU	100	109
Price	80	79
Housing	250	256

Table 4.

The classical real database

In the regression computing ability testing experiment of the proposed HMP-ELM method, the number of hidden point is setting as 20, and I_iter = 10, I_iter_max = 1000. The average optimal value and mean variance of the proposed HMP-ELM which taking 50 times regression computing simulation under the six classical real database are stated in Table.5. In the experiment, we compared the proposed HMP-ELM method to the Differential Equation Extreme Learning method (DE-ELM) and Self-Adaptive 8Evolutionary Extreme Learning Machine method. (SAE-ELM).

Database	Criteria	DE-ELM	SAE-ELM	HMP-ELM
Servo	Mean	0.1051	0.0084	0.0735
Servo	Variance	0.0025	0.0161	0.0109

Cancer	Mean	0.2663	0.2561	0.2254
Cancer	Variance	0.0208	0.0217	0.0116

Autompg	Mean	0.0691	0.0631	0.0598
Autompg	Variance	0.0129	0.0100	0.0044

CPU	Mean	0.0528	0.0396	0.0035
CPU	Variance	0.0238	0.0083	0.0004

Price	Mean	0.0494	0.0437	0.0432
Price	Variance	0.0094	0.0094	0.0068

Housing	Mean	0.0965	0.0904	0.0767
Housing	Variance	0.0137	0.0132	0.0087

Table 5.

The regression computing results under six classical real database

From the experiments shown in the Table.5, we can found that the computation results which obtained using proposed HMP-ELM are more precise than the DE-ELM and SAE-ELM methods, that means the HMP-ELM method proposed in this paper have a good regression computing ability.

6.3. compare the recognition performance between different classify criteria

In this section, we will evaluate the performance of Total Error Rate as the fusion classify criteria, In this experiment, we conduct the raw matching scores and TER as the classify criteria respectively, and then to test the fusing recognition performance using different fusion methods in order to certificate that recognition performance which using TER is better than using raw matching scores. And in the comparing experiment, the purpose is to evaluate the fusion classify criteria, so we taking five basic fusion strategy: LR method, Min-Max method, Tanh method, ELM method and the fusion method proposed in Reference [9] which it our former working. Table 6 to Table 8 are the fusion recognition results based on different biometric system which using the raw matching score as the classify criteria, Table 9 to Table 11 are the fusion recognition results based on different biometric system which using the TER as the classify criteria. In the recognition results shown in Table 7 and Table 10, the symbol XF1-XS1 represents the fusion of modal XF1 and modal XS1, and others are and so on and so forth. {XS} represents the fusion of all the speech modals, that means the fusion of XS1, XS2 and XS3, in the same way, the symbol {XF} represents the fusion of all the face modals, that the fusion of XF1, XF2, XF3, XF4 and XF5. The symbol {XS}-XF1 means the fusion of muti-modals: all the speech modals and face modal XF1. In the experiments, we are using the Equal Error Rate (EER) as the evaluation criteria in the fusion recognition system.

	XF1	XF2	XF3	XF4	XF5	XS1	XS2	XS3
EER (%)	2.615	5.038	5.988	5.052	7.663	2.586	8.308	6.675

Table. 6.

EER of each unimodal biometric recognition system based on matching score

EER(%)	XF1-XS1	XF1-XS2	XF1-XS3	XF2-XS1	XF2-XS2	XF2-XS3	XF3-XS1	XF3-XS2	XF3-XS3	XF4-XS1	XF4-XS2	XF4-XS3	XF5-XS1	XF5-XS2	XF5-XS3
Reference [9]	0.332	0.961	0.952	0.952	2.225	1.454	0.635	1.611	1.205	0.919	1.270	0.994	0.668	3.817	2.223
ELM	0.935	1.969	1.405	1.318	1.970	2.271	1.266	1.531	1.367	1.172	1.259	1.126	1.446	3.374	2.027
LR	0.823	1.489	1.276	1.341	2.157	1.915	1.069	1.546	1.368	0.889	1.266	1.508	1.173	3.323	2.376
Min-Max	1.379	1.783	1.601	0.883	2.247	1.761	0.830	1.783	1.171	1.696	1.822	2.083	3.421	4.522	4.840
Tanh	1.188	2.072	1.172	0.873	2.374	1.772	0.875	2.071	1.479	1.173	1.776	1.772	1.649	3.871	3.425

Table. 7.

The EER comparison between different fusion methods on double modals recognition system based on matching score

EER(%)	{XS}	{XF}	{XS} XF1	{XS} XF2	{XS} XF3	{XS} XF4	{XS} XF5	{XF} XS1	{XF} XS2	{XF} XS3
Reference [9]	3.254	1.988	1.231	1.520	1.344	1.347	1.629	1.341	1.476	1.433
ELM	2.996	4.203	1.535	2.182	2.562	1.645	2.922	3.091	2.073	2.524
LR	1.922	2.434	0.692	1.017	1.068	0.825	1.017	1.341	1.664	1.645
Min-Max	2.756	2.312	1.124	1.042	1.016	1.055	1.743	1.956	1.988	1.988
Tanh	3.181	2.152	1.017	1.344	1.278	1.037	1.807	1.664	1.635	1.654

Table. 8.

The EER comparison between different fusion methods on each muti-modals recognition system based on matching score

	XF1	XF2	XF3	XF4	XF5	XS1	XS2	XS3
EER (%)	1.814	4.115	1.767	3.500	6.500	1.109	6.500	4.500

Table. 9.

EER of each biometric recognition system based on TER

EER(%)	XF1-XS1	XF1-XS2	XF1-XS3	XF2-XS1	XF2-XS2	XF2-XS3	XF3-XS1	XF3-XS2	XF3-XS3	XF4-XS1	XF4-XS2	XF4-XS3	XF5-XS1	XF5-XS2	XF5-XS3
Reference [9]	0.260	0.756	0.750	0.750	1.750	1.146	0.500	1.269	0.949	0.718	1.000	0.783	0.526	3.000	1.750
ELM	0.466	1.328	0.858	0.785	1.328	1.579	0.742	0.962	0.826	0.664	0.736	0.625	0.892	2.499	1.376
LR	0.373	0.928	0.750	0.804	1.484	1.283	0.578	0.975	0.827	0.427	0.742	0.944	0.664	2.456	1.666
Min-Max	0.914	1.250	1.098	0.500	1.637	1.232	0.456	1.250	0.740	1.178	1.283	1.500	2.615	3.532	3.798
Tanh	0.764	1.500	0.750	0.500	1.750	1.250	0.500	1.500	1.000	0.750	1.250	1.250	1.148	3.000	2.628

Table. 10.

The EER comparison between different fusion method on each double modals recognition system based on TER

EER(%)	{XS}	{XF}	{XS} XF1	{XS} XF2	{XS} XF3	{XS} XF4	{XS} XF5	{XF} XS1	{XF} XS2	{XF} XS3
Reference [9]	2.227	1.250	0.666	0.889	0.753	0.750	0.973	0.750	0.855	0.830
ELM	2.028	2.960	0.900	1.400	1.693	0.985	1.971	2.038	1.316	1.664
LR	1.199	1.594	0.250	0.500	0.500	0.352	0.500	0.750	1.000	0.985
Min-Max	1.843	1.500	0.583	0.520	0.500	0.526	1.061	1.225	1.250	1.250
Tanh	2.170	1.377	0.500	0.722	0.702	0.516	1.110	1.000	1.000	1.000

Table.11.

The EER comparison between different fusion methods on each muiti-modals recognition system based on TER

From the experiment shown in Table.6 and Table.9, we can find that the value of EER in the unimodal recognition system are very high due to the reason of the unimodal recognition system often influenced by the unpredicted noise and other factors, in the biometric recognition system, the more bigger value of EER is, the more worse is.

When we using the TER as the classify criteria, from the experiment results shown in Table.9, in the unimodal recognition system, for the face recognition system, the system XF3 has the best recognition performance, the value of EER only 1.767%, for the speech recognition system, the XS1 has the best recognition performance that the value of EER only 1.109%.

From the results shown in Table.7, Table.8, Table.10, Table.11, in the bio-modals fusion recognition system and multi-modals fusion recognition system, no matter which fusion method we choose, the recognition performance is better than the unimodal recognition system and the value of EER is more smaller than its in the unimodal system. For the bio-modal fusion recognition system, from the results shown in Table.7, when we using the raw matching scores as the classify criteria, the average value of EER of the five fusion methods is 1.7005%, from the results shown in Table.10, when we using the TER as the classify criteria, the average value of EER of the five fusion methods is 1.1866%. For the multi-modal fusion recognition system, from the results shown in Table.8, when we using the raw matching scores as the classify criteria, the average value of EER of the five fusion methods is 1.7934%, from the results shown in Table.10, when we using the TER as the classify criteria, the average value of EER of the five fusion methods is 1.0977%.

According to the average value of EER shown in the Tables above, when we using TER as the classify criteria compared to the traditional matching scores, the evaluation criteria EER has dropped in some extent. For example, in the bio-modals fusion recognition system, the average EER has dropped 0.5139% when using TER as the classify criteria compared to the traditional matching scores, in the multi-modals fusion recognition system, he average EER has dropped 0.6957% when using TER as the classify criteria compared to the traditional matching scores. So the experiment results demonstrate that using the TER as the classify criteria can improve the fusion recognition performance compared to the traditional matching scores.

6.4. test the recognition performance based on the HMP-ELM and TER

In this section, we test the recognition performance based on the fusion method proposed in this paper. In this experiment, we using the TER as the classify criteria and compare the recognition performance between the proposed fusion method and the other five fusion methods mentioned before. The experiment results shown in Table.12 and Table.13. In this experiment, we put the fusion method that based on HMP-ELM proposed in this paper and TER as T method.

EER(%)	XF1-XS1	XF1-XS2	XF1-XS3	XF2-XS1	XF2-XS2	XF2-XS3	XF3-XS1	XF3-XS2	XF3-XS3	XF4-XS1	XF4-XS2	XF4-XS3	XF5-XS1	XF5-XS2	XF5-XS3
T	0.232	0.665	0.847	0.435	1.258	0.997	0.398	0.882	0.856	0.557	0.985	0.703	0.483	2.247	1.473
Reference [9]	0.260	0.756	0.750	0.750	1.750	1.146	0.500	1.269	0.949	0.718	1.000	0.783	0.526	3.000	1.750
ELM	0.466	1.328	0.858	0.785	1.328	1.579	0.742	0.962	0.826	0.664	0.736	0.625	0.892	2.499	1.376
LR	0.373	0.928	0.750	0.804	1.484	1.283	0.578	0.975	0.827	0.427	0.742	0.944	0.664	2.456	1.666
Min-Max	0.914	1.250	1.098	0.500	1.637	1.232	0.456	1.250	0.740	1.178	1.283	1.500	2.615	3.532	3.798
Tanh	0.764	1.500	0.750	0.500	1.750	1.250	0.500	1.500	1.000	0.750	1.250	1.250	1.148	3.000	2.628

Table.12.

The EER comparison between different fusion methods on each double modals recognition system

EER(%)	{XS}	{XF}	{XS} XF1	{XS} XF2	{XS} XF3	{XS} XF4	{XS} XF5	{XF} XS1	{XF} XS2	{XF} XS3
T	1.053	1.242	0.457	0.428	0.393	0.418	0.527	0.647	0.793	0.716
Reference [9]	2.227	1.250	0.666	0.889	0.753	0.750	0.973	0.750	0.855	0.830
ELM	2.028	2.960	0.900	1.400	1.693	0.985	1.971	2.038	1.316	1.664
LR	1.199	1.594	0.250	0.500	0.500	0.352	0.500	0.750	1.000	0.985
Min-Max	1.843	1.500	0.583	0.520	0.500	0.526	1.061	1.225	1.250	1.250
Tanh	2.170	1.377	0.500	0.722	0.702	0.516	1.110	1.000	1.000	1.000

Table.13.

The EER comparison between different fusion methods on each multi-modals recognition system

The experiment results shown in the Table.12 is the EER derived from bio-modals fusion recognition systems based on the classify criteria TER which using different fusion methods and the fusion method proposed in this paper, from the results, we can find that the performance based on the proposed T method is the best and the reference [9] method take the second place. In the bio-modals fusion recognition systems which based on proposed T fusion method, there are 11 systems have the lowest EER that means have the best recognition performance, there are 3 systems have the second rank lowest EER. On the whole, the fusion recognition performance is better than the LR fusion method and reference [9] that we proposed before. In the experiment, the XF1-XS1 bio-modals fusion recognition system has the best recognition performance based on proposed T method while the value of EER only 0.232%, the XF5-XS2 bio-modals fusion recognition system has the worst recognition performance based on proposed T method while the value of EER is 2.247%, but still lower than the unimodal recognition system XF5 and XS2, and the value of EER of the bio-modals recognition system are lower than the uni-modal recognition system 4.253% and 4.253% respectively.

The experiment results shown in the Table.12 is the EER derived from multi-modals fusion recognition systems based on the classify criteria TER which using different fusion methods and the fusion method proposed in this paper, from the results, we can find that the performance based on the proposed T method still best. From the results, we can still find that there 6 times the value of EER are lowest, that means its have the best recognition performance, and 2 times the value of EER are second rank lowest that means the performance take the second place. On the whole, in the multi-modals fusion recognition systems, the performance based on proposed T method is better than the reference [9] method and LR fusion method. In this experiment, the {XS}-XF3 multi-modals fusion recognition system has the best recognition performance based on proposed T method while the value of EER only 0.393%, the {XF}-XS2 multi-modals fusion recognition system has the worst recognition performance based on proposed T method while the value of EER is 20.793%, but still lower than the unimodal recognition system.

We compare the results from the Table.12 and Table.13 comprehensively; we can still find that the fusion recognition performance based on proposed T method achieve the best result. Because the proposed fusion method not only utilize the criteria TER to depict the distribution of the matching scores of the recognition system, but also utilize the correlation of different features to fusing the criteria TER derived from different uni-modal recognition systems. The LR fusion method is based on density estimation that estimate the joint density distribution of the samples based on GMM modal, the performance of the LR method can get a ideal result when the scale of the training samples is very high, otherwise, the performance is very poor. The Tanh fusion method and Min-Max fusion method cannot depict the distribution of the matching scores properly, but they can reflect some laws of the matching scores in some respect, the best advantages of these two fusion methods is easy to realize and its can be make use for the foundation for other complex fusion methods.

6.5. The verification of recognition time and accuracy

We verify the recognition time and recognition accuracy of different fusion methods on the XM2VTS-Benchmark database, the experiment result are shown in the Table.14.

methods	Time(S)		Recognition Rate
methods	Training Time	Recognition Time	Recognition Rate
T	55.609	18.689	95.61%
Reference [9]	38.625	16.235	93.14%
ELM	49.252	17.227	91.78%
LR	32.844	12.172	89.44%
Min-Max	29.672	4.898	87.33%
Tanh	22.032	8.278	80.67%

Tab 14.

The Recognition Time and Recognition Rate comparison different fusion method on each double modals recognition system

From the experiment results shown in Table.14, we can found that the average training time and average recognition time are 55.609S and 18.689S respectively, it’s the highest value in all the fusion methods, but its have the highest recognition accuracy that is 95.61%. compared to the reference [9] fusion method, the proposed fusion method is higher than 3.47%; compared to the Tanh fusion method, the proposed fusion method is higher than 14.94%. That means the recognition performance are improved but at the sacrifice of the training time and recognition time.

7. Conclusion

In this paper, a new fusion method based on based on TER and Multiple Hidden Layer Probabilistic Extreme Learning Machine (MHP-ELM) is proposed. A new fusion strategy based on hybrid intelligent multiple hidden layer probabilistic extreme learning machine(HMP-ELM) is introduced, which optimizes the architecture of hidden nodes by weighted calculation of different output matrices and then transforms the numeric output of ELM to the probabilistic outputs and unifies the outputs in a fixed range, in this strategy, the matrices weights μ of the Multiple Hidden Layer Extreme Learning Machine(M-ELM) and the output weights β belongs to the ELM are optimized using a hybrid intelligent algorithm based on Differential Evolution and Particle Swarm Optimization(DEPSO).

The main contributions of this paper can be summarized as follows: 1) Based on our previous works, this paper still using the total error rate (TER) instead of matching scores as the matching character in fusion process. 2) A novel fusion scheme based on Hybrid Intelligent Multiple Hidden Layer Probabilistic Extreme Learning Machine (HMP-ELM) is given in this paper. To our knowledge, this is the first time to utilize the ELM for information fusion. 3) In order to explore a optimal parameter belong to the Extreme Learning Machine, a new optimization strategy to the two weighted parameters based on Differential Evolution and Particle Swarm Optimization(DEPSO) is proposed.

Acknowledgements

The work was supported by the National Natural Science Foundation of China under Grant 61263031 and 61563030, by the by the National Natural Science Foundation of Hunan Province of China under Grant 14JJ2041and 16JJ6025, by the Key Project of Education Department of Hunan Province of China under Grant 15A044, 17A048.

References

[1]AK Jain, A Ross, and S Pankanti, “Biometrics: A tool for information security”, IEEE Trans. Inf. Forensics Security, Vol. 1, No. 2, 2006, pp. 125-143.

[2]Bahador Khaleghi, Alaa Khamis, O Fakhreddine, and Saiedeh N Razavi, Multisensor data fusion: A review of the state-of-the-art, Information Fusion, Vol. 14, 2013, pp. 28-44.

[3]B Khaleghi, A Khamis, and F Karray, Random finite set theoretic based soft/hard data fusion with application for target tracking, in Proc. of the IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (New York, 2010), pp. 50-55.

[4]MP Down and RJ Sands, Biometrics: An overview of the technology, challenges and control considerations, Inf. Syst. Control J, Vol. 4, 2004, pp. 53-56.

[5]A Ross and R Govindarajan, Feature level fusion using hand and face biometrics, in Proc. SPIE 2nd Conf. Biometric Technol. Human Identification (Orlando, 2005), pp. 196-204.

[6]Romain Giot and Christophe Rosenberger, Genetic programming for multibiometrics, Expert Systems with Applications, Vol. 39, 2012, pp. 1837-1847.

[7]Wu Di, Multimodel biometrics Fusion based on FAR and FRR using Triangular Norm[J], International Journal of Computational Intelligence Systems, Vol. 8, No. 4, 2015, pp. 779-786.

[8]Wu Di and Tang Yongqi, Multibiometric recognition based on total error rate and feature correlation adaptive fusion, Journal of Optoelectronocs. Laster, Vol. 27, No. 1, 2016, pp. 67-76.

[9]A Ross, K Nandakumar, and AK Jain, Handbook of Multibiometrics, Springer, New York, 2006.

[10]R Tronci, G Giacinto, and F Roli, Dynamic score selection for fusion of multiple biometric matchers, in Proc. 14th IEEE Int. Conf. Image Anal. Process (Modena, 2007), pp. 15-20.

[11]K Veeramachaneni, LA Osadciw, and PK Varshney, An adaptive multimodal biometric management algorithm, IEEE Trans. Sys. Man, Cybern. C, Appl. Rev, Vol. 35, No. 3, 2005, pp. 344-356.

[12]N Srinivas, K Veeramachaneni, and LA Osadciw, Fusing correlated data from multiple classifiers for improved biometric verification, in Proc. 12th Int. Conf. Inf. Fusion (New York, 2009), pp. 1504-1511.

[13]A Kumar, V Kanhangad, and D Zhang, A new framework for adaptive multimodal biometrics management, IEEE Trans. Inf. Forensics Security, Vol. 5, No. 1, 2010, pp. 92-102.

[14]M Anzar and PS Sathidevi, An efficient PSO optimized integration weight estimation using d-prime statistics for a multibiometric system, Int. J. Bioinformat. Biosci, Vol. 2, No. 3, 2012, pp. 31-33.

[15]Kien Nguyen and Simon Denman, Score-Level Multibiometric Fusion Based on Dempster-Shafer Theory Incorporating Uncertainty Factors, IEEE Transcations On Human Machines Systems, Vol. 45, No. 1, 2015, pp. 132-140.

[16]Lamia Mezai, Score Level Fusion Algorithm Using Differential Evolution and Proportional Conflict Redistribution Rule, in 2016 IEEE International Conference on S • SMC 2016 (Budapest, 2016), pp. 121-123.

[17]G Huang, G-B Huang, S Song, and K You, Trends in extreme learning machines: A review, Neural Network, Vol. 61, No. 1, 2015, pp. 32-48.

[18]G-B Huang, H Zhou, X Ding, and R Zhang, Extreme learning machine for regression and multiclass classification, IEEE Trans. Syst., Man, Cybern. B, Cybern, Vol. 42, No. 2, 2012, pp. 513-529.

[19]R Storn and K Price, Differential evolution a simple and efficient heuristic for global optimization over continuous spaces, Journal of Global Optimization, Vol. 11, No. 4, 1997, pp. 341-359.

[20]AK Qin, AL Huang, and PN Suganthan, Differential Evolution Algorithm with Strategy Adaptation For Global Numerical Optimization, IEEE Trans on Evolutionary Computation, Vol. 13, No. 2, 2009, pp. 398-417.

<Previous Article In Issue

Download article (PDF)

Next Article In Issue>

Journal: International Journal of Computational Intelligence Systems
Volume-Issue: 11 - 1
Pages: 936 - 950
Publication Date: 2018/05/02
ISSN (Online): 1875-6883
ISSN (Print): 1875-6891
DOI: 10.2991/ijcis.11.1.71 How to use a DOI?
Open Access: This is an open access article under the CC BY-NC license (http://creativecommons.org/licences/by-nc/4.0/).

Cite this article

ris enw bib

TY  - JOUR
AU  - Di Wu
AU  - Qin Wan
PY  - 2018
DA  - 2018/05/02
TI  - Multimodal biometrics Fusion based on TER and Hybrid Intelligent Multiple Hidden Layer Probabilistic Extreme Learning Machine
JO  - International Journal of Computational Intelligence Systems
SP  - 936
EP  - 950
VL  - 11
IS  - 1
SN  - 1875-6883
UR  - https://doi.org/10.2991/ijcis.11.1.71
DO  - 10.2991/ijcis.11.1.71
ID  - Wu2018
ER  -

download .riscopy to clipboard