International Journal of Computational Intelligence Systems

Volume 10, Issue 1, 2017, Pages 266 - 276

Design and Implementation of a Speller based on EMG Signal

Authors
P M Shyam Prasad1, shyam.prasad@students.vnit.ac.in, Radhika Swarnkar2, radhika.swarnkar@students.vnit.ac.in, Mohammad Farukh Hashmi3, farooq786engg@gmail.com, Avinash G. Keskar4, agkeskar@ece.vnit.ac.in
Received 19 April 2016, Accepted 3 September 2016, Available Online 1 January 2017.
DOI
10.2991/ijcis.2017.10.1.18How to use a DOI?
Keywords
EMG Signal; Fast Walsh Hadamard Transform; Naïve Bayes Classifier; Emotiv Epoc headset
Abstract

A speller is a communication device designed for those suffering from neuromuscular disorders having difficulty to speak. An EMG based design is proposed which uses eye blinks for character selection that offers high accuracy and more comfort to the user. The eye blink signals are feature extracted using Fast Walsh Hadamard Transform (FWHT) and classified using Naive Bayes Classifier. The proposed design has achieved an accuracy of 100% for all users. The average values achieved for spelling rate was 12.12 characters/minute and ITR was 71.39 bits/minute.

Copyright
© 2017, the Authors. Published by Atlantis Press.
Open Access
This is an open access article under the CC BY-NC license (http://creativecommons.org/licences/by-nc/4.0/).

1. Introduction

Providing communication aid for the differently abled people suffering from neuromuscular disorders has been an important area of research. BCI allow users to communicate directly with their surroundings making use of their brain signals with the aid of systems like BCI Speller [1], [2]. Researchers have been using physiological data such as Electroencephalography (EEG), Electrooculography (EOG), Electromyography (EMG), Eelectrocorticography (ECoG), Electrodermal Activity (EDA), Blood Oxygen Saturation, Respiration, Skin Temperature etc. of users for various real time applications [3]–[6]. The data is captured using sensors attached to the body parts.

Biosignals such as EEG and EMG have extensively been made use of in the design of various aiding systems for the disabled. EEG signals such as P300 and SSVEP have been widely used in the design of speller [7], [8]. In this paper we have used EMG signals generated through eye blink for the design of a speller.

Most of the existing spellers make use of EEG signals such as P300 and SSVEP. These are highly subject dependent and very difficult to get evoked. The amplitude of P300 signals might be very low in some people making its detection cumbersome. Also in order to evoke P300 and SSVEP signals the visual stimuli is flashed at a high frequency in front of the eyes of the user. This procedure can be very annoying for the user’s eyes and also have the potential to induce seizure in some cases. Eye blinks are caused by voluntary muscle movements associated with the eyes. This involves the generation of EMG signals which can be effectively made use of in BCI applications. The visual display presented to the user here doesn’t involve constant flickering which causes irritation to eyes. This makes EMG signal one of the best candidate for developing speller. Studies reveal that eye muscles are directly connected to the brain and not through the spinal cord. So paralysis in most cases doesn’t affect eye movement. This proves that this speller can be used by the same group of patients on which existing spellers (like P300 and SSVEP based spellers) have been proved to work. To design a speller which is more accurate at the same time less painful to the eyes was the motivation behind this paper.

The already existing designs have compromised some aspect in order to enhance another. For example higher spelling rates have come at the cost of accuracy and vice-versa. So we have tried to come up with a design where optimum results in terms of all aspects like accuracy, spelling rate, comfort etc. are achieved.

The proposed design makes use of the eye blinks. In this method the focus shifts from one character to another in a predefined fashion. The user can select the target character by blinking twice whenever the focus is on that character. The proposed methods offer much better overall performance as compared to most of the existing techniques. Features like back space button to erase the wrongly spelled characters, exit button to stop the speller and provision to go back to the previous screen have been provided. In addition to these features, use of adaptive speed technique by which the speed of the speller adjusts itself according to the user using it, makes the proposed system very user friendly.

The rest of the paper is organized as follows: Section I is literature survey. Section II explains methods and materials. Section III discusses proposed methodology. Section IV deals with results and discussions. Section V has conclusion and future scope.

2. Literature Survey

Many methods have been used so far for the design of spellers. Kawai and Tomohiro came up with an asynchronous P300 speller. In that the user had the control to switch the speller ON/OFF all by himself [9]. Vasiljevas and Turcinas designed an EMG speller. This had adaptive stimulus rate and also dictionary support [10]. Postelnicu and Talaba designed a hybrid speller making use of EEG and ECoG signals. The design used half checkboard paradigm (HCBP) [11]. Zhengui Gu and Zhuliang Yu proposed an online semi- supervised speller. The algorithm used was based on LS-SVM [12]. A dynamically optimized SSVEP speller was developed by Erwei and Zongtan. They used real time biofeedback mechanism to improve accuracy [13]. Hubert Cecotti came up with a self-paced and calibration less speller. This was based on SSVEP signals [14]. Yalda and Abbas designed a P300 based speller. They used subspace based filtering approach for improved performance [15]. Pires and Nunes suggested a novel P300 based speller called lateral single character. Here a comparison was made with the standard row-column speller [16]. Laura and Benjamin developed a gaze independent speller. This was based on rapid serial visual presentation [17]. An SSVEP based speller called DTU speller was introduced by Adnan and Troels. The speller included dictionary support as well [18]. Jaeyoung and Kee-Eung proposed a new approach for the design of a P300 speller. They used partially observable Markov decision process (POMDP) approach in this design for optimization [19]. Erwei and Zongtan designed a speedy hybrid BCI speller. They used a combination of P300 and SSVEP signals [20].

3. Methods and Materials

3.1. Hardware

The hardware consists of an Emotiv Epoc headset (Fig.1) which was used for signal acquisition. It consists of 14 electrodes and has a sampling rate of 128 Hz [21].Of these we have used only four electrodes namely AF3, F7, F8 and AF4. The headset is also provided with a Bluetooth unit through which data can be transferred to any system for processing. An LCD monitor of size 23” with a refresh rate of 60 Hz was used for displaying the visual interface. This was run on a pc with 32GB RAM powered by Intel i7 processor. The EMG signal from the headset, acquired via Bluetooth, was processed using a laptop having 8GB RAM and Intel i5 processor. The two systems (PC and Laptop) were connected with each other using a LAN (Local Area Network) cable and they communicated with each other using UDP (User Datagram Protocol) for time synchronization. The hardware setup is shown in Fig.2.

Fig. 1.

Emotiv Epoc Headset with 14 electrodes.

Fig. 2.

Hardware Setup of the Experiment.

3.2. Visual Interface

The visual interface was created using Cogent Graphics tool box in MATLAB. The visual interface consists three main screens namely Screen 1, Screen 2 and Screen 3. Screen 1 consists of 64 characters arranged in groups of four quadrants and each quadrant consists of 16 characters as shown in Fig.3. On the selection of any quadrant, the letters belonging to that group again split to form four new groups of four characters each which will be displayed in screen 2. The group selected in Screen 2 will further divide in the same manner as in Screens 1 and 2 and will be displayed in Screen 3. Three such layers (three screens) have to be selected to spell a particular character. English alphabets from a to z, numbers from 0 to 9 and some of the most commonly used English articles (such as is, was, and, the etc.) have been included in the 64 characters displayed in Screen 1. The interface is easily modifiable and can be provided with words depending on the user using the speller. For a paralyzed person words like ‘eat’, ‘drink’ etc. would be of more use for which the interface can be customized easily.

Fig. 3.

Visual Interface of the Proposed Speller.

3.3. Fast Walsh Hadamard Transform (FWHT)

The Walsh Hadamard Transform is an orthogonal transformation which decomposes a signal into a set of basis functions [22]. These basis functions take up the values of 1 and -1 and are known as Walsh functions. The Hadamard unitary matrix of order n is generated as follows:

Hn=H1Hn1

Where, Hn is a 2n × 2n matrix,

H1=12[1111]
and ⊗ denotes the Kronecker product of two matrices calculated as follows:
PQ=[P(1,1)QP(1,2)QP(1,N)QP(N,1)QP(N,2)QP(N,N)Q]

Where P is a matrix of size N × N , Q of size M × M and PQ of size MN × MN.

For an input signal vector U of N samples where N= 2n , the Hadamard transform is given as follows:

V=HnU

Discrete Hadamard transform of a real signal in one dimension is given as [23]:

V(j)=1Ni=0N1(U(i).H(j,i)
  • U(i) is the ith discrete sample of the signal

  • V(j) is the jth Hadamard coefficient.

  • H(j,i) is the jth Hadamard function.

FWHT is an efficient algorithm for computing Hadamard transform which involves just NlogN additions or subtractions as compared with the native method which has computations of the order of N2.

FWHT has the advantage that it is faster than other sinusoidal transforms and it requires less storage space. This makes it ideal for real time applications. This was the motivation behind choosing FWHT for processing the input signal. FWHT was applied on the input EMG signal for feature extraction after which it was classified using Bayesian classifier.

3.4. Naive Bayesian Classifier (NBC)

In a Bayesian Classifier, the learning agent builds a probabilistic model of the features and uses that model to predict the classification of a new example [24], [25]. Naïve Bayesian Classifier is the simplest case which assumes that the input features are independent of each other for a given condition of classification features.

This can be mathematically modelled as follows:

P(X1,,Xn/Y)=i=1nP(Xi/Y)

Where X1,…,Xn are the features and Y is the given class label.

In our experiment we have two classes of data namely blink data (b) and non-blink (nb) data. Blink data consists of those signals in which the user has purposefully blinked his eyes to spell a target character. The rest of the time when the user doesn’t blink is considered as the non-blink data. In order to classify a new data X as blink or non-blink data the following steps have to be performed.

  • Step 1:

    Prior probability of class b:

    P(Y=b)=Count(Y=b)Totaldata

  • Step 2:

    Conditional probability of class data X given class b:

    P(Xi=at|Y=b)|=Count(Xi=uY=b)Count(Y=b)

  • Step 3:

    Similarly perform the above two steps for Y= nb.

  • Step 4:

    After this the new data X is classified as follows:

    YNaive Bayes=argmaxY[b,nb]P(Y)P(X|Y)
    NBC is much faster as compared with most of the other classification algorithms. It consumes limited resources in terms of CPU and memory. These qualities of NBC have urged us to use it as the classifier in our real time application.

4. Proposed Methodology

The speller consists of two main parts. The visual interface which is the front end and the signal processing unit which forms the back end. The synchronized working of the front end and the back has to be ensured for the speller to provide desired outputs. These two parts were connected using a LAN cable and their synchronized working was ensured using a time synchronization program developed using UDP protocol in MATLAB. The detailed working of the different units has been explained as follows:

4.1. Design and Working of the Visual Interface

The visual interface consists of three layers namely Screen 1, Screen 2 and Screen 3. The work flow of the speller is as shown in the Fig.5. The target character is spelled in a three step process as follows:

  • Step 1:

    The user is presented initially with Screen 1 in which 64 characters are displayed which are divided into four groups. Each group consists of 16 characters each arranged in four quadrants as shown in the Fig.4. Each of the four groups is highlighted; one after the other in a fixed pattern, for a fixed duration of t seconds each. Every time when a group gets highlighted, recording of the EMG signal starts simultaneously for the same duration of ‘t’ seconds during which the group remains highlighted. The user is required to blink two times voluntarily whenever the group containing the target letter is highlighted. The recorded signal corresponding to each highlighted block is processed every time by the signal processing unit to detect the presence of voluntary eye blinks. The detection of eye blinks in a particular recorded data suggests the presence of the target character in that corresponding group. And hence that particular group is selected for next step.

  • Step 2:

    In this step the characters present in the selected group in Screen 1 are further divided into four groups consisting of four characters each arranged in four quadrants in Screen 2 as shown in Fig.6. Then the same process as mentioned in Step 1 is repeated and the group which contains the target is selected.

    Displayed equations should be numbered consecutively in each section, with the number set flush right and enclosed in parentheses.

  • Step 3:

    This is the final step which will isolate out the target character. The characters present in the selected group in Screen 2 are further divided into four groups consisting of only a single character this time and each arranged in four quadrants in Screen 3 as shown in Fig.7. Then the same process as mentioned in Step 1 is repeated and the target character is selected. The selected character in Step 3 is displayed in the middle of the screen in all subsequent steps.

    In our method two blinks are used to overcome the effect of the natural blinks that might occur in between the process.

    Back space and exit options are also provided for the user to take full control of the speller. If the user blinks on a non-target group and goes to the next screen which does not include his target character, a provision for going back to the previous screen has been provided.

    This can be done by simply not blinking for two complete rounds in the screen which does not contain the target character. The inactivity of the user for two rounds is interpreted as the choice of a wrong screen and automatically the previous screen is displayed so that the user can start afresh. This facility saves much time since the user need not continue up to Screen 3 in case of a wrong selection in Screen 1 or Screen 2 and then apply back space in the subsequent step. Instead he can go to the immediate previous screen and reselect the target group. Hence the use of backspace is reduced to a large extent thus decreasing the spelling time.

    Some of the most frequently used words are also provided in the interface which greatly increases the spelling speed.

Fig. 4.

Screen 1containing 64 characters arranged in 4 groups.

Fig. 5.

Algorithm Flow of the proposed methodology

Fig. 6.

Screen 2 containing 16 characters arranged in 4 groups.

Fig. 7.

Screen 3 containing 4 characters arranged in 4 groups.

4.2. Signal Analysis

The signal processing step involves the analysis of the EMG signal obtained from the Emotive Epoc headset and classifying it as a blink or a non-blink signal. MATLAB plots of blink and non-blink signals are shown in Fig.8 and Fig.9 respectively.

Fig. 8.

Non-blink Signal captured using epoc headset.

Fig. 9.

Blink Signal generated by two consecutive blinks.

The signal processing algorithms involved need to be very fast since this is a real time application. So we have chosen FWHT which is relatively very fast as it involves only additions and subtractions and no multiplications. The Bayes classifier was also chosen with the same intend i.e. to reduce the computing time. Since the accuracy obtained with Bayes was sufficiently high we chose to make use of the same in the design.

The flow diagram of the signal processing step is as shown in Fig.10:

Fig. 10.

Signal Processing Steps involved in the method.

The acquired signal from the headset has to be processed to detect the presence of voluntary eye blinks for target detection. Each input signal has a sample size of t*128.

The input signal is transformed using FWHT for feature extraction. The transformed signal itself is used as feature for classification using Naive Bayes classifier. The classifier then identifies it as a blink or non-blink data which is used for target detection.

The Emotive Epoc headset has an inbuilt notch filter which cancels the 50 Hz noise. Also the normal preprocessing algorithms are time consuming. Due to these reasons no preprocessing step has been done in the proposed method. The results obtained shows that the design works well in spite of any preprocessing step involved.

4.3. Time synchronization

The recording of the data in the laptop should begin exactly at the same time as the onset of display of stimuli in the pc.

To synchronize the timing of the aforesaid operations, we have made use of UDP protocol. A flag value is sent by the visual interface to the signal processing unit just before the starting of the display of stimuli in the LCD screen. The laptop starts recording the data only after receiving this flag value. Processing of the acquired data immediately follows signal acquisition. A flag value is sent by the signal processing unit to the visual interface whenever a blink signal is detected so as to immediately display the next screen. Hence time synchronization is achieved.

4.4. Experimental setup

The recording of the data in the laptop should begin exactly at the same time as the onset of display of stimuli in the pc.

To synchronize the timing of the aforesaid operations, we have made use of UDP protocol. A flag value is sent by the visual interface to the signal processing unit just before the starting of the display of stimuli in the LCD screen. The laptop starts recording the data only after receiving this flag value. Processing of the acquired data immediately follows signal acquisition. A flag value is sent by the signal processing unit to the visual interface whenever a blink signal is detected so as to immediately display the next screen. Hence time synchronization is achieved.

Fig. 11.

Experimental Setup with user wearing epoc headset.

The headset is provided with a Bluetooth module through which the acquired data is sent to a laptop nearby for processing. The pc and the laptop are connected via LAN cable and they communicate with each other using UDP protocol for time synchronization. Each of the users was given a sentence to be spelled. The sentence consisted of 110 characters including spaces. Spelling a sentence was considered a trial. A total of five trials were conducted for each subject and the average and the peak values of the different performance parameters were calculated.

5. Results and Discussion

One of the most important parameter of a speller is its accuracy. The designed speller has succeeded in achieving a very high accuracy as compared with the existing real time spellers at a comparable speed. The results obtained are based on the experiment conducted on a total of 6 subjects.

5.1. Performance Parameters

The performance of the proposed design has been analyzed based on the three parameters namely accuracy, information transfer rate (bits per minute) and spelling rate (characters per minute).

3.1.1. Accuracy in Signal Detection

For a single word to be spelled correctly three correct detections (one in each Screen) of blink in the signal are necessary. The user going back to the previous window or the use of backspace indicates an incorrectly detected signal. A count of the total number of true and false detections are made and the accuracy is computed. The average value of accuracy in signal detection for all subjects obtained is 99.12%.

3.1.2. Spelling Rate

The time taken by each subject to correctly spell out each of the given sentences was noted. Based on the total number of characters in each sentence and the total time taken, spelling rate was computed. Spelling rate is expressed as the total number of correctly spelt characters in a minute. The average value of spelling rate for all subjects obtained is 12.12 characters/minute.

3.1.3. Spelling Rate

ITR was calculated based on the following formula [1]:

ITR=(log2M+PlogP+(1P)log2[1PM1])*(60T)

Where

  • M is the number of target characters. Here it is 64.

  • P is the accuracy of classification.

  • T is the average time for spelling a character in minutes.

The average value of ITR for all subjects obtained is 71.53%.

5.2. Variation of Performance Parameters with Time‘t’

The speed of the speller can be increased by reducing the time duration‘t’ for which each group is highlighted. This reduces the overall time in which any letter is spelt.

But ‘t’ can’t be reduced beyond a limit as it keeps decreasing the interval in which user can respond by blinking. Hence the probability that user blinks before or after the target group is more with reduced value of‘t’. A very small value of‘t’ might induce more strain to the user. The time duration‘t’ for which a particular group remains highlighted was varied and the number of correct detections were noted. This procedure was done with all the six subjects and the average values were taken down. Based on the detections, the performance parameters were calculated as shown in Table.1.

Parameters Time duration ‘t’ in seconds
t=0.5 t=1 t=1.5 t=2.5
Accuracy in Signal Detection (%) 88.47 98.86 99.02 100.00
ITR (bits/minute) 69.74 70.51 47.868 29.64
Spelling Rate (characters/minute) 14.4 12.06 8.16 4.94
Table 1.

Variation of parameters with time ‘t’.

The accuracy in the signal detection reduces as ‘t’ is decreased which can be observed from Table.1.

Below t=1 sec accuracy has decreased drastically. At the same time no much variation in the accuracy has been observed as ‘t’ changes from t=2.5 sec to t=1 sec. Also ITR and Spelling rates are also very high at smaller values of ‘t’. Taking into account all these factors an optimum range for t from t=1sec to t=2sec was chosen which gives the best overall performance with high values of ITR and Spelling rate with a high degree of accuracy. Hence in the adaptive speed mode of the speller, ‘t’ varies within this range automatically according to the user input (number of mistakes made). So the speed of the speller is adjusted so as to suit the user making it more comfortable to the user using it.

5.3. Variation of Performance Parameters with Subjects

Adaptability and the ability to use a new method varies with individual. Therefore the performance also varies with a large extend. By analyzing the performance of the proposed design over a sample of individuals the credibility of the design can be claimed. Performance of different subjects w.r.t different parameters is shown in this section. An optimum range of ‘t’ (1sec=< t =<2sec) was chosen with regard to all the performance parameters to attain maximum performance (from section A) and was used for calculating the performance of different subjects.

Then the average of three trials was taken as the observation for various performance parameters which is shown in Table.2.

Subjects Accuracy in Signal Detection (%) ITR (bits/minute) Spelling Rate (characters/minute)
Subject 1 100 77.88 12.98
Subject 2 99.14 73.33 12.47
Subject 3 98.70 67.83 11.64
Subject 4 98.36 69.69 12.04
Subject 5 100 76.80 12.80
Subject 6 98.52 62.83 10.82
Average 99.12 71.39 12.12
Table 2.

Variation of parameters with subjects.

Table.2 shows that Subject 1 and Subject 5 has attained 100% accuracy. Maximum ITR observed is 77.88% and maximum Spelling rate 12.98%. Lowest values observed are accuracy 98.36%, ITR 62.83 bits/minute and Spelling Rate 10.82 characters/minute.

5.4. Comparison of Proposed Speller with the Existing Spellers

The proposed design has also been compared with the existing designs of spellers. This comparison has been shown in Table.3.

Methods Accuracy ITR Spelling Rate
Self-Paced Speller [6] 92.25 37.62 5.51
Semi-supervised Speller 95.7 24.33 5.07
HCBP Speller 90.63 27.98 5.23
EMG speller With dictionary 89.16 58.69 11.35
Proposed Method 99.12 71.39 12.12
% Improvement from the best method 9.96 21.63 6.78
Table 3.

Comparison with existing Spellers.

The evaluation of the proposed design has been summarized in Fig. 12 to Fig. 14.

Fig. 12.

Comparison of information transfer rate of various methods.

Fig. 13

Comparison of spelling rate of various methods.

Fig. 14

Comparison of accuracy of various methods.

Fig.12 shows the comparison of ITR of the proposed speller with some of the existing spellers. The graph shows that the proposed design has achieved the best ITR as compared with other designs.

A comparison of the spelling rate of the proposed design has been done with that of the existing methods in Fig.13. Each plot in the graph shows the average value of many subjects. It can be inferred from the graph that spelling rate is highest in the proposed design.

The accuracy of various existing designs has been compared with that of the proposed design in Fig.14. The graph shows the clear dominance of the proposed design over the existing techniques in terms of accuracy.

The percentage increase of the performance parameters of the proposed design has been shown w.r.t the most efficient existing method. Accuracy has been improved by 9.96%, ITR by 21.63% and Spelling rate by 6.78%. The results show that significant increase has been attained with respect to accuracy and ITR. The spelling rate has improved as well. This states that the proposed design has achieved a superior performance with respect to most of the existing designs.

6. Conclusions and Future Work

An EMG speller was designed which makes use of eye blinks of the user to spell out the intended characters. The device was made using Emotive Epoc headset in which only four channels were used for data collection.

The study shows that one of the main problems associated with the existing designs is that they have not been able to obtain optimum values for all performance parameters. These methods have compromised on some parameters so as to achieve good results. For example [10] has achieved high spelling rate (11.35 characters/minute) while the signal detection accuracy obtained is comparatively low (89.16%). Whereas the proposed method has achieved good results with respect to all performance parameters without compromising on any factor. The suggested method has obtained a higher value of spelling rate (12.12 characters/minute) at a very high value of signal detection accuracy (99.12%).

One of the greatest advantages of this speller is that it causes very less irritation to the eyes as compared with other EEG spellers. The reason behind this is no flickering of the letters in the screen is involved in the proposed design unlike in EEG spellers which are highly irritating to the eyes. Also for the same reason this design is more suitable for prolonged use with more comfort as compared with its EEG counterparts. Its interface is simple to understand making it easier for the users especially the old aged and the differently abled people to use it with ease.

The design proposed here does not need any training so that the user can directly start using the speller. Also we plan to use our design on paralyzed people to know the credibility of the design and further improvements.

Acknowledgment

This study is a part of M.Tech project done by authors Shyam Prasad PM and Radhika Swarnkar who have contributed equally in this work. Authors would also like to express their sincere gratitude to Dr Md Farukh Hashmi and Prof A.G. Keskar for their valuable suggestions and guidance in this work.

References

25.Kevin P Murphy, Naive bayes classifiers, University of British Columbia, 2006.
Journal
International Journal of Computational Intelligence Systems
Volume-Issue
10 - 1
Pages
266 - 276
Publication Date
2017/01/01
ISSN (Online)
1875-6883
ISSN (Print)
1875-6891
DOI
10.2991/ijcis.2017.10.1.18How to use a DOI?
Copyright
© 2017, the Authors. Published by Atlantis Press.
Open Access
This is an open access article under the CC BY-NC license (http://creativecommons.org/licences/by-nc/4.0/).

Cite this article

TY  - JOUR
AU  - P M Shyam Prasad
AU  - Radhika Swarnkar
AU  - Mohammad Farukh Hashmi
AU  - Avinash G. Keskar
PY  - 2017
DA  - 2017/01/01
TI  - Design and Implementation of a Speller based on EMG Signal
JO  - International Journal of Computational Intelligence Systems
SP  - 266
EP  - 276
VL  - 10
IS  - 1
SN  - 1875-6883
UR  - https://doi.org/10.2991/ijcis.2017.10.1.18
DO  - 10.2991/ijcis.2017.10.1.18
ID  - ShyamPrasad2017
ER  -