An Optimization Method Using Clustering Technique for the Human Emotions Detection Artificial Neuro-Fuzzy Logic System

This paper utilizes clustering tool in MATLAB to find an optimal set of input parameters for the detection of human emotions using a neuro-fuzzy logic system. Previous studies have relied on a total of 14 physiological factors to detect one or more of 22 different human emotions. In this paper, we use clustering techniques to rank the factors in terms of their significance and impact on the system, and thus find a smaller subset of the factors for the detection of emotions. The clustering method shows that the stroke volume factor (SV) has the lowest impact in the model and as such can be eliminated from the set of factors. The electroencephalography (EEG), heart rate (HR), systolic blood pressure (SBP) and diastolic blood pressure (DBP) are shown to have the highest impact on the model, and must be include in the input set of the model. We compare the clustering method with exhaustive methods for finding the optimal set of factors.


I Introduction
The human emotional status is rather intangible "intangible things can't be manufactured in factories but they can be simulated" (Owaied H & Abu-Arr'a) [14]. However, these emotions can be correlated to external and/or internal factors, which are rather tangible things, and hence they can be measured and analyzed. The internal factors come from different parts of the body in several forms such as electroencephalography (EEG), heart rate (HR), heart rate variability(HRV), preejection period (PEP), stroke volume (SV), systolic blood pressure (SBP), diastolic blood pressure (DBP), skin conductance response (SCR), tidal volume (Vt), oscillatory resistance (Ros), respiration rate (RR), nonspecific skin conductance response rate (nSRR), skin conductance level (SCL), finger temperature (FT), and others (Kreibig) [9].
The factors' measurements are provided in wide ranges and often their impacts vary from a person to another and for different postures for the same person. For example, a given measurement of some factors may relate to a person being "Happy", while the same measurements may reveal a rather "Sad" status for another person. This kind of behavior lends itself naturally to fuzzy sets and fuzzy logic (Zadah) [ [18]. In (Malkawi & Murad) [10], the authors used fourteen factors in a neural/fuzzy logic model to identify and detect one of 22 different emotions (divided into 11 positive and 11 negative emotions) . In (Malkawi et al, 2015) [2], a reduction model was introduced to minimize the number of used factors in the neuro-fuzzy system for emotion detection.
In (Malkawi & Murad) [10], the authors used fuzzy operations to represent the knowledge about each factor. This enabled the detection of the emotion of a person using fuzzy inputs of the various factors. Although fuzzy sets and operations are useful for representing the knowledge base, they fail to model the individual behavior of each and every person. Obviously, a model that is able to adapt to various categories of human responses would be preferred. Consequently, an adaptive learning mechanism was used to adjust the model in order to cater for the differences in emotions between various humans. Consequently, artificial neural networks (ANN) (Abraham) [1] was used to accommodate this need. In order to utilize the benefits of both fuzzy logic and artificial neural networks, a combination of fuzzy logic and artificial neural networks in a single model was used. [12] In this paper, we describe a new method for clustering the various factors in order to reduce the total number of factors required to detect a set of emotions. The purpose of this approach is to provide a cost effective method for human emotion detection. Note that using 14 different factors means that we have to use a relatively large number of sensors and/or measuring devices to feed the emotion detection system. Furthermore, the measurement devices for some factors are very expensive, such as the EEG measurement device [19] Our approach relies on the clustering tools provided in Matlab. An exhaustive search method was used by Albashiti and Malkawi [2] to determine optimal sets of input factors. In this paper, we will show that the clustering method can provide an effective optimization method at a much lower computational cost than the exhaustive method used in [2].
The clustering method used in this work relies on clustering the data before building the theoretical artificial neuro-fuzzy inference system (ANFIS) model; this will help finding the most proper factors needed for the detection of a given set of emotions; the set of emotions will be chosen such that these emotions collectively identify one area of interest, such as security, social networks, gaming, health or other.
A pre-processing of the factors' data will be performed before building the ANFIS theoretical model. The pre-processing of the data is carried out using data clustering technique, a well-known technique in data mining. In our model, data clustering proceeds as follows : First, data will be classified into a group of clusters, each cluster will hold similar data. The clusters are formed using the initial data distribution. Second, every time an input item is determined to belong to a cluster based on a certain criteria, a cluster hit counter will be incremented and its weight will be increased. The cluster hit counter is initialized to zero. In our experiment, input data will be entered as samples of a group of elements while each element presents an input factor of the model. In our system, there will be 14 elements, since we have 14 factors representing 14 vectors. During clustering process, each data input in the factor vector will be classified in order to be entered in the proper cluster. At the end of the processing the cluster, each input data will join at least one cluster.
The factors whose data joined more clusters, will be classified as prime input factors in the theoretical model because they have relatively large number of hits according to their input values. On the other hand, factors whose input values joined less number of clusters will become candidates to be eliminated from the input factors of the neuro-fuzzy model; these factors will be considered as prime factors. The smaller the number of prime factors, the lower the cost and effort needed to build the physical model.
The rest of this paper is organized as follows: Related work is presented in section II. Section III provides an overview of the experimental results and discussions. Section IV will compare our results with the exhaustive method provided in [2]. Conclusions are presented in section V.

Related work
Human emotion detection and analysis has been an important field of study for scientists for a long time. Some researches concentrated on the external factors to detect human emotions. Authors in (Burkhardt F, et al 2009) [4], provided analysis of human emotions based on spoken dialog contexts. While in (Bakhtiyari K, Husain H 2013) [3], authors discussed a fuzzy model for multi-level human emotions recognition by computer systems through keyboard keystrokes, mouse and touch-screen interactions. In (Nagpal R, Nagpal P and Kaur S 2010) [13], authors built an automatic system for the recognition of facial expressions; they analyzed human emotions based on a representation of the expression, using a training set of pre-selected features.
In the medical field, (Silberstein) in [15] the authors focused on the ability to track the patients' emotions and responses. (Timmons and Scanlon ) in [16] introduced a new medical instrument, which allows doctors to monitor their patients remotely. They used a wireless sensor network organized as a star network. Sensors included insulin sensors among others. (Timmons and Scanlon ) [16].
The authors in (Malkawi & Murad) [10], used various physiological factors to detect human emotions; they used the factors identified by Kreibig [9], where he collected information from many research papers and summarized them in a table, which shows the impact of every factor on every some negative and positive human emotions.
(Mohammad Malkawi, Mohammad Khasawneh, and Abeer Bashiti) in [2], developed an optimization method to identify an optimal subset of factors to detect the same human emotions described in [10]. The authors used exhaustive search to get a subset of the 14 factors.

III Experimental Results and Discussions
In [10], authors evaluated all factors and their correlation to human emotions. While using 14 factors can lead to an accurate identification of emotions, it can be complicated, costly and often unnecessary to utilize on measurements obtained from all factors (e.g., 14 factors) to identify the current emotion. However, the developed ANFIS model can be used in different applications with different requirements for emotion detection. For example, applications related to social networks, health systems, security systems and gaming, industry have different sets of correlated emotions. As a result, emotion detection models can be distinguished from each other based on the set of emotions required for each model or application. For example, the emotions required for the security applications could be anger, fear and anxiety, while the emotions required for use in the gaming industry can be happiness, sadness, anxiety and depression. Our approach will attempt to identify the factors, which are suitable to a particular application and may not be necessary for other applications. For example, we can find the factors related to anger if the target application is criminal behavior for security purpose.
In this paper, we utilize use data clustering technique, a well known powerful tool in data mining, machine learning, pattern recognition, social network and many others [2], [5], [10]. We will cluster the input data sets before building the theoretical models using nctool from MATLAB [5][6] [7]. Nctool is an excellent application which allows the grouping of data by similarity. This tool uses SOM (self-organizing maps) which is one of the well-known unsupervised learning algorithms [Sarazin, T., Azzag, H.] [15], which is useful for clustering large amounts of data.
SOM is a powerful tool of visualization, that allows projection of large amount of data into small spaces, which are generally organized in two dimensional arrays. The basic model was presented by Kohonen that present a set C of cells/grids called maps. A priori size of the grid C should be determined and usually called k. Distributing the data into the clustering is done by neighborhood functions as described in [18][[8] [11]. Nctool allows a user to enter the data sets as a matrix which holds the elements and their samples. For example, in our model the elements are the physiological factors used in the neuro-fuzzy system while the samples are the data values for each factor. After entering the data sets (composed of the factors and their relevant values), the nctool will cluster the data according to the similarity of their values and generate output in the form of figures representing the data clusters. One of the figure types is a self organizing map of weight planes. The map, shows a figure for each element and its correlation with the clusters. Using these figures, thereafter we can decide which element is the most useful one and which is not. Moreover, the map shows the relation between the factors, and how they are correlated to each other.
Four training arrays are used to build the clusters. The data set contains 15 columns, each column represents one factor, except the last one which is used as a weight for the current input values. Each row represents a set of values for each of the factors. The first array of data sets is the NTA (Noisy Training Array); it has (814×15) records and holds a set of data values, which do not necessarily match the model rules; hence it is called the noisy data. The second data set is the correct values training array (CTA), it also has (814×15) records. The data included in CTA more accurately match the ANFIS rules used in the model. The third set of data is the small training array (STA) has (465×15) records. The last set of data is very small training array (VTA) which has (156×15). records; the four arrays (NTA, CTA, STA, and VTA) were used in the original model [10] [Murad and Malkwai] Using Nctool terminology in MATLAB, the 14 factors will be equivalent to the iris flower attributes which will act as inputs to the self-organizing map Neural Networks (SOM). SOM will map the factors into 2-dimensional layer of 100 neurons arranged in a 10×10 hexagonal grid, then it will be trained for 100 epochs.
The self organizing map has 4 different ways of plotting the clusters such as SOM Neighbor Distances, SOM Weight Planes, SOM Sample Hits and SOM Weight Positions . SOM weight planes which shows a weight plane for each of the 14 input factors. These planes visualize the weights that connect each factor and the 100 neurons in the 10×10 hexagonal grid.
Another way of plotting the clusters is SOM plot sample hits as shown in Figure 1, which shows the number of hits of each cluster; the number of hits is increased whenever the cluster is used irrespective of the factor making the hit. These numbers reflect the distribution of the data set on the map; the figure shows every neuron location in the topology, which indicates the number of training data associated with each cluster, for example the number 109 in Figure 1 means there is 109 input vector in that cluster.
This relation can be further illustrated using SOM weight planes with different colors, which shows the weights of usage for each factor over the clusters using different colors ( Figure 2). Each factor is represented in Figure 2 by a color map. The colors have the following interpretation: black color means that the factor was never used to increase the number of hits for the specific cluster. On the other hand darker (non-black) colors represent large weights. The correlation between the input factors can be observed by the similarity of their weight planes. For example, in Figure 2(the CTA data set), the first factor EEG (shown in the first color map) was heavily used, whereas the fifth factor SV shown in black color was never used in most of the clusters.   For further analysis of the output presented in Figures 2-5, we zoom into factors weight representation. Figures 6-9 show the weight map for the factors stroke volume (SV) for each of the data set arrays.
By inspecting figures [6][7][8][9], we can conclude that the weight plane for the fifth factor which relates to stroke volume (SV) has mostly black color in Figures 6-9; this implies SV factor has a low weight in all the array data sets. Low weight means SV has a relatively low connection with most 100 neurons in the 10×10 hexagonal grid. Consequently, SV can be eliminated from the optimal model. However, examining the factors 1, 2, 3, 6 and 7 electroencephalography (EEG), heart rate (HR), heart rate variability(HRV), systolic blood pressure (SBP), diastolic blood pressure (DBP) respectively, we notice that they have the largest correlation value between the factors. See Figures [10][11][12][13], show the results for the factors, systolic blood pressure (SBP) and diastolic blood pressure (DBP) factors respectively for each of the data sets (CTA, NTA, STA, VTA). We first observe, that the weight plane for each of the factors (SBP and DBP) is not dominated with black color, and thus are considered significant factors. Moreover, the shapes and colors for both factors for all data sets are rather similar, and thus it can be concluded that there is a strong correlation between these two factors. Similar observations are made for the EEG and HRV factors.
Finally, we can categorize the factors according to their effects on the clusters within three categories low, medium and high. The low category will hold the factors which have least effects on the clusters hits, on the other hand high category will hold the factors with the highest effect and finally the last category will hold the factors whose effects are in the medium range. The type of the category will be defined according to the number of the black clusters in each factor's weight; whenever the factor has a large number of black clusters it will be a candidate of the low category. Table 1 presents the categories to which each of the factors belong; SV, Vt, HRV and PEP belong to the low category, while EEG, HR, RR and nSRR belong to the high category and the rest were placed in the medium category.

IV. Comparison with Exhaustive Method
In [2], authors used the exhaustive search to find all the possible candidate subsets of the fourteen factors presented and used to detect human emotions [10].  [6,9].
For the case of using only 4 factors in the ANFIS model (k = 4) total 623 combinations were examined, Analyzing all possible combinations, the result showed that the most frequent factor in all combination was EEG ; this is consistent with our findings using the clustering technique. The factor appeared in 180 combination (28.89% as a total number of permutations). Similarly, HR and HRV achieved high rate of occurrence (21.99% and 17.97% respectively). On the other hand, Ros appears only in 2 combinations with 0.32 % as a percentage of appearance, which also line with the in results obtained using clustering method.
Using the exhaustive method, with 4 required factors, EEG, HR, HRV and PEP achieved the highest percentage among all factors. (EEG), (HR), (SBP) and (DBP) are the most frequent and the most correlated factors. The factors SBP and DBP showed up in the list of highest rate factors using the exhaustive method when the optimization method used the 14 choose 6 factors. In other words, the factors obtained using the clustering method is a good approximation of the exhaustive method, which considers all possible permutations.
MoreoverHowever, the main advantage of the clustering method is its ability to find a set of the most relevant factors in a relatively short time compared to the exhaustive method, which requires quite a long time to process all possible combinations of k factors, where k = 2, 3, 4, … 13.

V. Conclusions
In this paper, a pre-processing of an Artificial Neuro-Fuzzy Logic System for detecting human emotions is introduced which helps deciding which input factor can be eliminated and which are not; also it can give a view of how the factors are correlated to each others. Four distinct data sets were used by MATLAB nctool to build 10×10 hexagonal clusters and trained for 100 epochs.