CPP-ELM: Cryptographically Privacy-Preserving Extreme Learning Machine for Cloud Systems

Ferhat Özgür Çatak; Ahmet Fatih Mustacoglu

doi:10.2991/ijcis.11.1.3

<Previous Article In Issue

Download article (PDF)

Next Article In Issue>

Volume 11, Issue 1, 2018, Pages 33 - 44

CPP-ELM: Cryptographically Privacy-Preserving Extreme Learning Machine for Cloud Systems

Authors

Ferhat Özgür Çatak¹^{, *}^,ozgur.catak@tubitak.gov.tr, Ahmet Fatih Mustacoglu¹^,afatih.mustacoglu@tubitak.gov.tr

¹TÜBİTAK - BİLGEM, Baris Mh., Kocaeli, Turkey

^*

TÜBİTAK - BİLGEM Cyber Security Institute, Kocaeli, Turkey

Received 9 June 2017, Accepted 18 September 2017, Available Online 1 January 2018.

DOI: 10.2991/ijcis.11.1.3 How to use a DOI?
Keywords: Extreme learning machine; Privacy-preserving machine learning; Homomorphic encryption
Abstract: The training techniques of the distributed machine learning approach replace the traditional methods with a cloud computing infrastructure and provide flexible computing services to clients. Moreover, machine learning-based classification methods are used in many diverse applications such as medical predictions, speech/face recognition, and financial applications. Most of the application areas require security and confidentiality for both the data and the classifier model. In order to prevent the risk of confidential data disclosure while outsourcing the data analysis, we propose a privacy-preserving protocol approach for the extreme learning machine algorithm and give private classification protocols. The proposed protocols compute the hidden layer output matrix H in an encrypted form by using a distributed multi-party computation (or cloud computing model) approach. This paper shows how to build a privacy-preserving classification model from encrypted data.
Copyright: © 2018, the Authors. Published by Atlantis Press.
Open Access: This is an open access article under the CC BY-NC license (http://creativecommons.org/licences/by-nc/4.0/).

1. Introduction

Classification algorithms are used for predicting the label of new instances by a classifier model that is built using correctly identified input observations. In order to achieve high performance results for accurate classification, the classification algorithm needs a large number of valid input instances.

Today, many organizations such as financial and medical institutions produce large-scale data sets. Machine learning-based data analysis tools are used by these organizations. The main problems with these data sets are privacy and security concerns. All of these tools want full authority to access private and confidential data to build a supervised or unsupervised model.

Even though various anonymization techniques are applied to confidential data sets to protect privacy, the confidential data itself can be still retrieved by an adversary via various method ¹⁵. Imagine a scenario in which two or more institutions want to build a classification model ¹⁹ by combining each other’s confidential dataset. In such a case, it is crucial to find a new privacy-based training method for classification algorithms that can run together on multiple confidential data sets without retrieving the private data.

In machine learning, a classification algorithm basically consists of two sequential steps: model building and labeling of new instances. The model building phase of a classification algorithm creates a classifier hypothesis, h, from the correctly labeled input data set 𝒟. In the phase of labeling new instances, the classifier hypothesis h predicts the label of unseen input instances 𝒳. The traditional classifier algorithms require direct access to data, which poses security and privacy risks as mentioned above.

Extreme learning machines (ELMs) were recently introduced as a probabilistic neural network learning algorithm, proposed by Huang et al. ³ based on generalized single-hidden layer feed-forward networks (SLFNs). The major properties of the ELM method are the high generalization property of multi-class labeling of unseen input instances, small model building time in comparison to the traditional gradient-based learning methods, and being parameter-free with randomly generated hidden layer nodes.

In this research, we propose a privacy-preserving ELM classification model building method based on a Paillier cryptosystem²² that constructs the global neural network classification model from encrypted and partitioned data sets from different sites. The labeled input data set is vertically partitioned and distributed among the different sites, and then the intermediate computation results of each party are aggregated by an independent trusted party (or at the client-side of the computation) to privately build a classification model that is used for predicting the correct label.

The contributions of our paper are as follows:

•
The Paillier cryptosystem encryption-based ELM training model is proposed for learning and thus private classification model training is achieved.
•
The computation of the hidden layer output matrix of the probabilistic neural network model is distributed to independent sites, thus minimizing the data communication.

Our paper is organized as follows: in Section 2, we briefly introduce some of the related works for privacy-preserving training. In Section 3, we describe the ELM, homomorphic encryption, the Paillier cryptosystem, and the arbitrary data partitioning technique. In Section 4, we describe the proposed secure multi-party training and model building method. Section 5 evaluates our proposed training and classification model. Section 6 concludes the paper.

2. Related Works

In this section, we review the existing works that have been developed for different security and privacy-preserving machine learning methods. We highlight detailed differences between our proposed learning model and the existing works. Encryption methods can be employed to address the security and privacy concerns in machine learning ^16,17.

Recently, numerous significant privacy-preserving machine learning models have been proposed. Secretans et al. ²⁶ proposed a new probabilistic neural network (PNN) training model. In their proposed method, the PNN is built with an approximation to the optimal classifier. The theoretically optimal classifier is known as the Bayesian optimal classifier. Their proposed method needs at least three sites to be involved in the computation of the secure matrix summation. The secure matrix is used for adding the partial class conditional probability vectors.

Aggarwal et al. ¹ proposed a new condensation-based training method. In their paper, it was shown that the anonymized data closely matched the characteristics of the original data. Their proposed approach produces disclosable data that satisfy privacy concerns by providing utility for data analysis. Each instance in the data is randomized while a few statistical properties of these instances are kept.

Graepel et al. ⁸ implemented two binary classification algorithms for homomorphically encrypted data: linear means and Fisher’s linear discriminant. They make scaling adjustments that preserve the results, but leave the fundamental methodology unchanged.

Samet et al. ²⁵ proposed new privacy-preserving algorithms for both back-propagation training and ELM classification between several sites. Their proposed algorithms are applied to the perceptron learning algorithm and presented for only single-layer models.

Bost et al. ⁴ developed a two-party computation framework and used a mix of different partially and fully homomorphic encryption schemes, which allowed them to use machine learning techniques based on hyperplane decisions, naive Bayes, and binary decision trees. The fundamental methodologies are still unchanged, but substantial communication between two “honest, but curious” parties is required.

Oliveria et al. ²⁰ developed techniques for distorting private numerical attributes to protect confidentiality for clustering analysis methods. Their approach transforms data by using several different types of geometric transformations such as translation, rotation, and scaling while keeping the Euclidean distance.

Guang et al. ⁹ developed a new training method for a privacy-preserving back-propagation algorithm using horizontally partitioned data sets for a multisite learning case. Their approach uses homomorphic encryption-based secure summation in its protocols.

Yu et al. ³³ developed a new privacy-preserving training method for a support vector machine (SVM) classification algorithm. Their protocols build the global SVM classifier hypothesis from the distributed data at multiple sites, without retrieving or pooling the data of each site with others. Homomorphic encryption-based secure multi-party integer sums are applied in their protocols to compute the Gram matrix of the SVM algorithm formulation in a distributed manner from the encrypted partitioned data held by several different sites.

Yang et al. ³¹ proposed a privacy-preserving schema using homomorphic encryption in image processing. A secret key homomorphic encryption was constructed for encrypting image. Then the encrypted image can be processed homomorphically.

Tso et al. ²⁹ proposed a new practical approach to prevent data disclosure from inside attack. Their approach is based on FairplayMP framework which enables programmers who are not experts in the theory of secure computation to implement such protocols. The method supports privacy-preserving machine learning for medical data stored in parties.

3. Preliminaries

In this section, we briefly introduce preliminary information about the ELM, homomorphic encryption, the Paillier cryptosystem, secure multi-party computation, and arbitrarily partitioned data.

3.1. Extreme Learning Machine

As mentioned before, a probabilistic neural network-based ELM classification algorithm was proposed for single-layer feed-forward neural networks ^3,12,14,6. Later, the ELM algorithm was expanded to a new model of generalized single-layer feed-forward networks. In this model, the hidden layer of the ELM may not be a neuron like in previous papers ^10,11. The first advantage of the ELM classification algorithm is its training time performance. The training phase of the ELM algorithm can be a hundred times faster than the standard SVM algorithm or the classical neural network algorithm. The reason for this rapid model building is that its hidden node biases vectors and input weight vectors are arbitrarily generated, and then the output layer weights of the model can be calculated by using a least-squares method ^28,13. As a result, the most appreciable property of the ELM training phase is based on the creation of probabilistic hidden layer parameters for performance issues.

Let us give a set of input instances 𝒟 = {(x_i,y_i) | i = 1, ⋯, n}, x_i ∈ ℝ^p, y_i {1,2, ⋯ , K}} from some unknown distribution 𝒳; they are sampled independently and identically distributed (i.i.d.). The main objective of a neural network model is to find a hypothesis h : 𝒳 → 𝒴 where 𝒳 is the input instances and 𝒴 is the set of all possible outcomes. The outcome of a single-layer feed-forward neural network with N hidden nodes in an n-dimensional vector space can be shown as:

(1)fN(x)=∑i=1NβiG(ai,bi,x),x∈ℝn,ai∈ℝn

where a_i and b_i are the input weights and bias values of hidden nodes, respectively, and β_i is the weight that connects the ith hidden node to the output node.

The output function of the ELM algorithm for generalized SLFNs can be identified by:

(2)fN(x)=∑i=1NβiG(ai,bi,x)=β×h(x)

For binary classification problems, the decision function of the ELM model becomes:

(3)fN(x)=sign(∑i=1NβiG(ai,bi,x))=sign(β×h(x))

We can rewrite Eq. 2 in another form, as shown in Eq. 4:

(4)Hβ=T

where H and T are the hidden layer matrix and the output matrix, respectively.

(5)β=H†T

where H^† is the Moore–Penrose generalized inverse matrix of the hidden layer matrix H. The hidden layer matrix can be described as:

(6)H(a˜,b˜,x˜)=[ G(a1,b1,x1)⋯G(aL,bL,x1)⋮⋱⋮G(a1,b1,xN)⋯G(aL,bL,xN) ]N×L

where ã = a₁, …,a_L, b˜=b1,…,bL, x˜=x1,…,xN. Then the output matrix of the ELM algorithm can be shown as:

(7)T=[ t1…tN ]T

The hidden nodes of the ELM can be arbitrarily created and they can be independent from the input data set.

3.2. Homomorphic Encryption

Homomorphic encryption schemes enable operations on plaintexts to be performed on their respective ciphertexts without revealing the plaintexts when data are divided between two or more parties as it facilitates computations with ciphertexts. One can say that a public-key encryption scheme is additively homomorphic if, given two encrypted messages such as ⟦a⟧ and ⟦b⟧ there exists a public-key summation operation such that ⟦a⟧ ⊕ ⟦b⟧ is an encryption of the plaintext of a + b. The formal definition is that an encryption scheme is additively homomorphic if for any private key, public key (sk, pk), the plaintext space 𝒫 = ℤ_N for a,b ℝ_N.

(8)Encpk(a+b mod N)=Encpk(x)×Encpk(y)Encpk(x⋅y mod N)=Encpk(x)y

3.2.1. Paillier’s Encryption Scheme

The Paillier cryptosystem²² is a probabilistic and asymmetric encryption algorithm. The Paillier cryptosystem is based on the problem of deciding whether a given number is an nth residue modulo n² where n is the product of two large primes²¹. The problem of computing the nth residue classes is believed to be computationally hard.

Let us give a set of possible plaintext messages M and a set of secret and public key pairs K = pk × sk, where pk is the public key and sk is the secret key of the cryptosytem. Then the Paillier homomorphic encryption cryptosystem satisfies the following property of any two plaintext messages m₁ and m₂ and a constant value a:

(9)Decsk(Encpk(m1)×Encpk(m1))=m1+m2

(10)Decsk(Encpk(m1)a)=a×m1

One of the main properties of the Paillier cryptosystem is that it is based on a probabilistic encryption algorithm. Large-scale data sets are sparse matrices in which most of the elements are zero. In order to prevent the guessing of elements of the input data set, the Paillier cryptosystem has probabilistic encryption that does not encrypt two equal plaintexts with the same encryption key into the same cipher-text.

3.3. Arbitrarily Partitioned Data

In this work, arbitrarily partitioned data exist between multplei sites (K), where K > 2 is considered. In the arbitrarily partitioned data scheme, there is not any specific order of how the data set is split between the multiple sites. More specifically, let us give a data set X = {x₁, ⋯ x_n}, consisting of n instances, and each instance in X contains m numeric features xi={x i1⋯x im}. X ij is the subset of the data set owned by party P_j, and then we have X i1∪X i2⋯∪X iK=Xi and X i1∩X i2⋯∩X ik=∅. Then, in each instance (X_i), site P_k has a number of features t ik, where ∑p=1Kt ik=m and each site’s feature size does not have to be equal. If a site has the same features in each instance, then the arbitrary partition becomes a vertical partition.

4. Cryptographically Privacy-Preserving ELM (CPP-ELM)

In this section, we briefly introduce our assumptions in the security model, our notations, and the sequence diagram of the overall method. In Section 4.6, we explain our proposed privacy-preserving training of the ELM algorithm in detail.

4.1. Security Model

In this paper, the aim is to enable multiple parties (or servers) to cooperatively build the ELM classification model without retrieving their own confidential input data set. Our main assumption is that the input data set is divided between two or more parties that are willing to train an ELM classifier if nothing beyond the expected end results is revealed ². Our other assumption is that all parties follow the protocol; this is called a semi-honest security model ^18,32.

4.2. Notations

The input instance x of data set 𝒳 is shown in the form of x = {x₁, ⋯·x_n}. In this work, two or more parties that hold vertically partitioned data are considered.

•
Boldface lowercase letters denote the vectors (e.g., x).
•
⟦m⟧ denotes the ciphertext of a message m.
•
sign(m) denotes the sign of the plaintext number m.
•
⊕ denotes encrypted addition and ⊗ denotes encrypted multiplication with the public-key operation.

4.3. Floating Point Numbers

The Paillier encryption scheme operates only on integer numbers. Thus, the proposed protocols manipulate only integers. However, the ELM classification algorithm is typically applied to continuous data. Nonetheless, in the case of an input data set with real numbers in the protocol, we need to map floating point input data vectors into the discrete domain with a conversion function (i.e. scaling).

Let ConvertToInteger : ℝ^m → ℤ^m be the corresponding function that multiplies its floating point number argument by an exponent (K : 2^K) and then rounds them to the nearest integer value and thus supports finite precision. Eq. 11 shows the conversion function:

(11)x^←ConvertInteger (x) where x∈ℝm,x^∈ℤm

4.4. Vertically Partitioned Data

The input data set that is used for finding an ELM-based classifier consists of m rows in n-dimensional vector space shown with 𝒟 ∈ ℝ^m×n. Input matrix 𝒟 is arbitrarily and vertically partitioned into k different sites of P₀, P₁, ⋯ , P_k and each attribute of rows is owned by a private site as depicted in Fig. 1. As denoted in Eq. 6, in the second phase of the ELM training, hidden layer output matrix H is computed using randomly generated hidden node parameters w, b, and β. The ELM training phase then computes the output weight vector, β, by multiplying H and T. Each element of matrix H is computed with activation function g. The global G matrix is such that G(w_i,x_i,b_i) = g(x_i · w_i + b_i) for sigmoid or G(w_i,x_i,b_i) = g(b_i − ||x_i − w_i||) for radial-based functions. The (i, j)^th element of H is:

(12)G(wi,xi,bi)=sign(xi⋅wi+bi)

where x_i is the i^th row of the input data set 𝒟, w_i is the hidden node input weight vector of the i^th row, and x_i, w_i ℝⁿ.

Let x i1,⋯x ik be vertically partitioned vectors of an input instance x_i, let w j1,⋯w jk be vertically partitioned vectors of the j^th hidden node input weight w_j, and let b j0,⋯,b jk be the j^th node input bias over k different parties. Then the output of the j^th input node with the i^th instance of an input data set using k different parties is:

(13)sign(xi⋅wj+bj)=sign((x i0⋅w j0+b j0)+⋯+(x jk⋅w jk+b jk))

From Eq. 13, the computation of the hidden layer output matrix H can be distributed into k different parties by using the secure sum of matrices, such that:

(14)H=sign(T1+⋯+Tk)

where:

(15)Ti=[ (x 1i⋅w 1i+b 1i)⋯(x 1i⋅w Li+b Li)⋮⋱⋮(x Ni⋅w 1i+b 1i)⋯(x Li⋅w Ni+b Ni) ]N×L

4.5. Sequence Diagram of Overall Method

In this section, we present the conceptual level of the CPP-ELM. We define the converting of the flow of messages into the flow of methods. In the sequence diagram, we use the following notation for the actions executed by an actor when receiving a signal:

•
M: Method invocation.
•
:User, :ClientApp, :CloudApp: Actors

The CPP-ELM is composed of four steps, as shown in Fig. 2 - 3:

•
Client initialization
•
Information sharing
•
Hidden layer output matrix computation
•
Classifier model building

4.6. Privacy-Preserving ELM Algorithm

In this section, a cloud-based privacy-preserving multi-party CPP-ELM learning algorithm over arbitrarily partitioned data is presented.

4.6.1. Client Initialization

In the client initialization phase of the proposed method, first the client normalizes the input data set 𝒟 = {(x,y)|x ℝⁿ,y ℝ}. Let x_min and x_max be an input data set of minimum and maximum values of each corresponding feature. Hence, the normalized data set can be written as:

(16)𝒟=𝒟−xminxmax−xmin

Next, the client splits the whole data set 𝒟 based on the server size of the cloud systems and then it encrypts each element of the input data set 𝒟 individually by using its public-key through the Paillier cryptosystem as shown in Eq. 17:

(17)⟦𝒟i⟧=Epk(𝒟i)∀i∈k

Algorithm 1 shows the overall process in the initialization phase.

4.6.2. Information Sharing

In the second phase of the CPP-ELM, first the client sends the encrypted sub-data set ⟦𝒟_i⟧ to each party Pⁱ.

Second, the client generates public and secret keys, as described in Algorithm 1: the client generates and assigns random weights w ℝⁿ and b ℝⁿ, and then sends the weight vectors’ corresponding items to each P_k. Algorithm 2 shows the overall process.

4.6.3. Hidden Layer Output Matrix Computation

The encrypted sub-data sets ⟦ 𝒟 ⟧ of each server and weight vectors w,b are used for computing the encrypted hidden layer output matrix for each data chunk ⟦ H_i ⟧, ∀i ∈ k. The computed ⟦H_i⟧ is sent to the client. Considering a case where a party k = 1, we have:

(18)⟦H1⟧=[ (⟦𝒟1,s⟧⋅w)+bs ], where 𝒟 ∈ℝm×n,s=1,⋯,k

The party (server) knows the plaintext weight vectors w,b. It can perform the required multiplication and additions with its own encrypted sub-data set ⟦𝒟_i⟧ by performing dot multiplication of each row with w and addition of b.

Algorithm 3 shows the calculation of the hidden layer output matrix for each server in the encrypted domain.

4.6.4. Classifier Model Building

The client can now compute the last step given in Fig. 3, i.e. compute the value of the ELM-based classifier function in the private domain. First, the client decrypts each encrypted hidden layer output matrix ⟦H_i⟧ received from each party by using its own private key and then computes the global hidden layer output matrix H. The smallest norm least-squares solution of the linear system⁵ is:

(19)β^=H+T

where H⁺ is the Moore–Penrose generalized inverse of matrix H ²⁴.

Algorithm 4 shows the classifier model building in the decrypted domain.

5. Experiments

5.1. Experimental Setup

We have implemented our proposed protocols and the classifier training phase in Python by using the scikit-learn library for machine learning and the PyPhe library for the partially homomorphic encryption implementation.

To show the training phase time performance of the proposed protocols, we tested different public data sets with parties and encryption/decryption keys that have different lengths. For the experiments, we consider four different data sets from the UCI Machine Learning Repository called Australian, Breast Cancer, Ionosphere, and Sonar. Table 5.1 shows the details of the data sets.

Data Set	Sample	Class	Attributes
Ionosphere	351	2	34
Sonar	208	2	60
Breast Cancer	683	2	10
Australian	690	2	14

Table 1.

Details of Data Sets Used in Experiments.

Each data set is arbitrarily vertically partitioned among each party (K ∈ {2,3,5,7,10}), and then the results in the encrypted-domain are compared with the results of the plain-domain. In plain-domain training, the conventional ELM training phase is performed for all data sets in a single pass. All experiments are repeated 5 times and the results are averaged. All data sets are publicly available in SVM-light format on the LIBSVM website.

The Ionosphere data set consists of a phased array of 16 high-frequency antennas with a total transmitted power on the order of 6.4 kilowatts. “Good” radar returns are those showing evidence of some type of structure in the ionosphere. “Bad” returns are those that do not; their signals pass through the ionosphere ²⁷. The data set contains 351 training instances.

The Sonar data set is used for the study of the classification of sonar signals using a neural network. The task is to train a network to discriminate between sonar signals bounced off a metal cylinder and those bounced off a roughly cylindrical rock ⁷. The data set contains 208 training instances.

The Breast Cancer data set contains 699 instances, with 458 benign (65.5%) and 241 (34.5%) malignant cases. Each instance is described by 9 attributes with an integer value ³⁰.

The Australian data set concerns credit card applications. The data set contains 690 instances and each instance is described by 14 attributes ²³.

5.2. Experimental Results

Table 2 shows the best performance of the conventional ELM method of each experimental data set.

Data Set	Accuracy
Ionosphere	0.8613
Sonar	0.7293
Breast Cancer	0.9592
Australian	0.8438

Table 2.

Accuracy Results in the Plain-Domain for Each Data Set.

The proposed CPP-ELM training method has been implemented in Python 2.7 using the PyPhe library, version 1.0. Both the server and the clients are modeled as different process with the Python multiprocessing library. Each process sends the variables to each other via file exchange. The developed software is tested on a computer with 1.6 GHz Intel(R) is (R) processor and 4 GB of RAM running on a Windows 64-bit operating system. The system has been tested while varying the party size of each data set from 3 to 10, and with key lengths of 512 and 1024 bits. Performance results are shown in Table 3.

Data	Key	Number of Parties

		2	3	5	7	10
Ion.	512	7477,43	8280,77	9743,51	11481,17	12542,17
Ion.	1024	39550,93	40020,45	45148,68	56751,82	82240,62
Sonar	512	8345,99	7824,33	8621,37	10192,59	11047,74
Sonar	1024	33775,32	30149,92	35418,41	38780,89	44357,66
Breast	512	7681,71	9245,26	10630,49	13672,9	18651,20
Breast	1024	44035,76	48736,21	58554,61	72524,98	99986,21
Aust.	512	8709,58	10463,64	11629,93	15406,08	17817,07
Aust.	1024	39188,52	45621,76	52264,14	71074,53	79875,20

Table 3.

Performance Results (in Seconds) for Each Data Set.

It is noted that the average training and model building time depends on the total number of instances used for classification. Fig. 4 shows the encryption time for each data set, which is affected by the number input instances and feature size. This process is computed at the client-side and can be pre-computed offline.

In the proposed method, CPP-ELM, there are three different factors (party size, complexity of the input data set, and key bit length) that affect the computation time according to the experimental results.

As show in Fig. 5, when the number of parties varies from 3 to 10, then the overall hidden layer output matrix H computation stays relatively stable in the proposed model, at about 5 minutes to 8.5 minutes for 512-bit key length encryption and 20 minutes to 43 minutes for 1024-bit key length encryption.

In cryptographic systems, all computations work with integers. Thus, all real numbers used in our algorithms are mapped into finite fields by using a scaling procedure. Before starting the learning phase, the client converts the input data set to integer representation by multiplying and rounding the results. In the last stage, in Algorithm 4, the client scales down the hidden layer output matrix H. This scaling operation causes accuracy loss in the final classifier model. Table 4 shows the best performance of the CPP-ELM method for each experimental data set.

Data Set	512 Bits	1024 Bits
Ionosphere	0.7311	0.6598
Sonar	0.5976	0.5856
Breast Cancer	0.9433	0.9204
Australian	0.6877	0.6444

Table 4.

Accuracy Results in the Encrypted-Domain on Each Data Set.

6. Conclusion

In this work, we proposed a privacy-preserving and practical multi-party ELM learning scheme over arbitrarily vertically partitioned data between two or more parties. We have also provided cryptographically secure protocols for computing the hidden layer output matrix and have shown how to aggregate encrypted intermediate matrices securely. In our proposed approach, the client encrypts its input data, creates arbitrarily and vertically partitioned data, and then uploads the encrypted messages to a cloud system. The cloud system can execute the most time-consuming operation of ELM training without knowing any confidential information. One interesting future work is to extend the privacy-preserving training to other existing classification algorithms.

We suggest that the method proposed here is appropriate for areas such as healthcare where data privacy is highly sensitive. Data privacy is protected by transferring the identification information to the remote cloud service provider in an encrypted manner. Using the high computational power provided by the cloud service provider, the classification model is built on the encrypted data set. In this way, the cloud service provider can create a classification model without reaching plaintext data.

References

1.Charu C Aggarwal and Philip S Yu, A condensation approach to privacy preserving data mining, Elisa Bertino, Stavros Christodoulakis, Dimitris Plexousakis, Vassilis Christophides, Manolis Koubarakis, Klemens Bhm, and Elena Ferrari (editors), Advances in Database Technology - EDBT 2004 volume 2992 of Lecture Notes in Computer Science, Springer, Berlin Heidelberg, 2004, pp. 183-199.

2.Ankur Bansal, Tingting Chen, and Sheng Zhong, Privacy preserving back-propagation neural network learning over arbitrarily partitioned data, Neural Computing and Applications, Vol. 20, No. 1, 2011, pp. 143-150.

3.Guang bin Huang, Qin yu Zhu, and Chee kheong Siew, Extreme learning machine: A new learning scheme of feedforward neural networks, in IN PROC. INT. JOINT CONF. NEURAL NETW (2006), pp. 985-990.

4.Raphael Bost, Raluca Ada Popa, Stephen Tu, and Shafi Goldwasser, Machine learning classification over encrypted data, IACR Cryptology ePrint Archive, Vol. 2014, 2014, pp. 331.

5.A Castaño, F Fernández-Navarro, and C Hervás-Martínez, Pca-elm: A robust and pruned extreme learning machine approach based on principal component analysis, Neural Processing Letters, Vol. 37, No. 3, 2012, pp. 377-392.

6.Yao-Wei Fu, Hui-Ling Chen, Su-Jie Chen, Li-Juan Li, Shan-Shan Huang, and Zhen-Nao Cai, Advances in Swarm Intelligence: 5th International Conference, ICSI 2014, Hefei, China, October 17–20, 2014, Proceedings, Part I, chapter A Hybrid Extreme Learning Machine Approach for Early Diagnosis of Parkinson’s Disease, Springer International Publishing, Cham, 2014, pp. 342-349.

7.R Paul Gorman and Terrence J Sejnowski, Analysis of hidden units in a layered network trained to classify sonar targets, Neural networks, Vol. 1, No. 1, 1988, pp. 75-89.

8.Thore Graepel, Kristin Lauter, and Michael Naehrig, Ml confidential: Machine learning on encrypted data, Information Security and Cryptology–ICISC 2012, Springer, 2012, pp. 1-21.

9.Li Guang, Wang Ya-Dong, and Su Xiao-Hong, A privacy preserving neural network learning algorithm for horizontally partitioned databases, Inform. Technol. J, Vol. 9, 2009, pp. 1-10.

10.Guang-Bin Huang and Lei Chen, Convex incremental extreme learning machine, Neurocomputing, Vol. 70, No. 1618, 2007, pp. 3056-3062. Neural Network Applications in Electrical Engineering Selected papers from the 3rd International Work-Conference on Artificial Neural Networks (IWANN 2005) 3rd International Work-Conference on Artificial Neural Networks (IWANN 2005).

11.Guang-Bin Huang and Lei Chen, Enhanced random search based incremental extreme learning machine, Neurocomputing, Vol. 71, No. 1618, 2008, pp. 3460-3468. Advances in Neural Information Processing (ICONIP 2006) / Brazilian Symposium on Neural Networks (SBRN 2006).

12.Guang-Bin Huang, Lei Chen, and Chee-Kheong Siew, Universal approximation using incremental constructive feedforward networks with random hidden nodes, Neural Networks, IEEE Transactions on, Vol. 17, No. 4, July 2006, pp. 879-892.

13.Guang-Bin Huang, Ming-Bin Li, Lei Chen, and Chee-Kheong Siew, Incremental extreme learning machine with fully complex hidden nodes, Neurocomputing, Vol. 71, No. 46, 2008, pp. 576-583. Neural Networks: Algorithms and Applications 4th International Symposium on Neural Networks 50 Years of Artificial Intelligence: a Neuronal Approach Campus Multidisciplinary in Perception and Intelligence.

14.Guang-Bin Huang, Qin-Yu Zhu, and Chee-Kheong Siew, Extreme learning machine: Theory and applications, Neurocomputing, Vol. 70, No. 13, 2006, pp. 489-501. Neural Networks Selected Papers from the 7th Brazilian Symposium on Neural Networks (SBRN ’04) 7th Brazilian Symposium on Neural Networks.

15.Zhanglong Ji, Zachary Lipton, and Charles Elkan, Differential privacy and machine learning: a survey and review, 2014. arXiv preprint arXiv:1412.7584,

16.Hiroaki Kikuchi, Kei Nagai, Wakaha Ogata, and Masakatsu Nishigaki, Privacy-preserving similarity evaluation and application to remote biometrics authentication, Soft Computing, Vol. 14, No. 5, 2009, pp. 529-536.

17.Xuan Li, Jin Li, and Faliang Huang, A secure cloud storage system supporting privacy-preserving fuzzy deduplication, Soft Computing, Vol. 20, No. 4, 2015, pp. 1437-1448.

18.Yehuda Lindell and Benny Pinkas, Privacy preserving data mining, Advances in CryptologyCRYPTO 2000, Springer, 2000, pp. 36-54.

19.Yehuda Lindell and Benny Pinkas, Secure multiparty computation for privacy-preserving data mining, Journal of Privacy and Confidentiality, Vol. 1, No. 1, 2009, pp. 5.

20.Stanley RM Oliveira and Osmar R Zaiane, Privacy preserving clustering by data transformation, Journal of Information and Data Management, Vol. 1, No. 1, 2010, pp. 37.

21.Claudio Orlandi, Alessandro Piva, and Mauro Barni, Oblivious neural network computing via homomorphic encryption, EURASIP Journal on Information Security, Vol. 2007, 2007, pp. 18.

22.Pascal Paillier, Public-key cryptosystems based on composite degree residuosity classes, Advances in cryptologyEUROCRYPT99, Springer, 1999, pp. 223-238.

23.J Ross Quinlan, Simplifying decision trees, International journal of man-machine studies, Vol. 27, No. 3, 1987, pp. 221-234.

24.Calyampudi Radhakrishna Rao and Sujit Kumar Mitra, Generalized inverse of matrices and its applications, Wiley, New York, Vol. 7, 1971.

25.Saeed Samet and Ali Miri, Privacy-preserving back-propagation and extreme learning machine algorithms, Data Knowl. Eng, Vol. 79–80, September 2012, pp. 40-61.

26.Jimmy Secretan, Michael Georgiopoulos, and Jose Castro, A privacy preserving probabilistic neural network for horizontally partitioned databases, Neural Networks, 2007. IJCNN 2007, Aug 2007, pp. 1554-1559. http://dx.doi.org/10.1109/ijcnn.2007.4371189

27.Vincent G Sigillito, Simon P Wing, Larrie V Hutton, and Kile B Baker, Classification of radar returns from the ionosphere using neural networks, Johns Hopkins APL Technical Digest, Vol. 10, No. 3, 1989, pp. 262-266.

28.Jiexiong Tang, Chenwei Deng, Guang-Bin Huang, and Baojun Zhao, Compressed-domain ship detection on spaceborne optical image using deep neural network and extreme learning machine, Geoscience and Remote Sensing, IEEE Transactions on, Vol. 53, No. 3, March 2015, pp. 1174-1185.

29.Raylin Tso, Abdulhameed Alelaiwi, Sk Md Mizanur Rahman, Mu-En Wu, and M Shamim Hossain, Privacy-preserving data communication through secure multi-party computation in healthcare sensor cloud, Journal of Signal Processing Systems, Vol. 89, No. 1, Oct 2017, pp. 51-59.

30.William H Wolberg and Olvi L Mangasarian, Multisurface method of pattern separation for medical diagnosis applied to breast cytology, Proceedings of the national academy of sciences, Vol. 87, No. 23, 1990, pp. 9193-9196.

31.Pan Yang, Xiaolin Gui, Jian An, and Feng Tian, An efficient secret key homomorphic encryption used in image processing service, Security and Communication Networks, 2017.

32.Zhiqiang Yang, Sheng Zhong, and Rebecca N Wright, Privacy-preserving classification of customer data without loss of accuracy, SDM, SIAM, 2005, pp. 92-102.

33.Hwanjo Yu, Xiaoqian Jiang, and Jaideep Vaidya, Privacy-preserving svm using nonlinear kernels on horizontally partitioned data, ACM, in Proceedings of the 2006 ACM Symposium on Applied Computing (New York, NY, USA, 2006), pp. 603-610. SAC ’06,

<Previous Article In Issue

Download article (PDF)

Next Article In Issue>

Journal: International Journal of Computational Intelligence Systems
Volume-Issue: 11 - 1
Pages: 33 - 44
Publication Date: 2018/01/01
ISSN (Online): 1875-6883
ISSN (Print): 1875-6891
DOI: 10.2991/ijcis.11.1.3 How to use a DOI?
Open Access: This is an open access article under the CC BY-NC license (http://creativecommons.org/licences/by-nc/4.0/).

Cite this article

ris enw bib

TY  - JOUR
AU  - Ferhat Özgür Çatak
AU  - Ahmet Fatih Mustacoglu
PY  - 2018
DA  - 2018/01/01
TI  - CPP-ELM: Cryptographically Privacy-Preserving Extreme Learning Machine for Cloud Systems
JO  - International Journal of Computational Intelligence Systems
SP  - 33
EP  - 44
VL  - 11
IS  - 1
SN  - 1875-6883
UR  - https://doi.org/10.2991/ijcis.11.1.3
DO  - 10.2991/ijcis.11.1.3
ID  - Çatak2018
ER  -

download .riscopy to clipboard