Dynamic Analysis of Emotions through Artificial Intelligence
Análisis dinámico de las emociones mediante inteligencia artificial
Análise dinâmica das emoções através da inteligência artificial
Susana Mejía M., Olga Lucía Quintero M., Jaime Castro M.
Dynamic Analysis of Emotions through Artificial Intelligence
Avances en Psicología Latinoamericana, vol. 34, no. 2, 2016
Universidad del Rosario
Susana Mejía M.
Universidad EAFIT, Colombia
Olga Lucía Quintero M.
Universidad EAFIT, Colombia
Received: January 14, 2014
Accepted: August 05, 2015
Abstract: Emotions have been demonstrated to be an important aspect of human intelligence and to play a significant role in human decision-making processes. Emotions are not only feelings but also processes of establishing, maintaining or disrupting the relation between the organism and the environment. In the present paper, several features of social and developmental Psychology are introduced, especially concepts that are related to Theories of Emotions and the Mathematical Tools applied in psychology (i.e., Dynamic Systems and Fuzzy Logic). Later, five models that infer emotions from a single event, in AV-Space, are presented and discussed along with the finding that fuzzy logic can measure human emotional states.
Keywords theory of emotion, emotions, dynamic systems, fuzzy logic, Analysis and applications [Slides], Class, New York.
Resumen: Se ha comprobado que las emociones son un aspecto importante en la inteligencia humana y que desempeñan un rol significativo en el proceso humano de toma de decisiones. Las emociones no son solo sentimientos, sino también procesos de establecimiento, mantenimiento o interrupción de la relación existente entre el organismo y el ambiente. En el presente trabajo se describen algunas características de la psicología social y del desarrollo, especialmente los conceptos relacionados con las emociones y las teorías de la emoción, así como las herramientas matemáticas aplicadas en la psicología (i. e., sistemas dinámicos y lógica difusa). Luego se presentan y se discuten cinco modelos que infieren la emoción a partir de un evento, en el espacio Arousal-Valence (A-V), para encontrar que es posible usar la lógica difusa para medir los estados emocionales humanos.
Palabras clave: teoría de la emoción, emociones, sistemas dinámicos, lógica difusa.
Resumo: Se tem comprovado que as emoções são um aspeto importante na inteligência humana e que desempenha um papel significativo no processo de tomada de decisões humano. As emoções não são só sentimentos, mas também processos de estabelecimento, manutenção ou interrupção da relação existente entre o organismo e o ambiente. No presente trabalho descrevem-se algumas características da psicologia social e do desenvolvimento, especialmente os conceitos relacionados com emoções e as teorias da Emoção e as ferramentas matemáticas aplicadas na Psicologia (i.e., Sistemas dinámicos y Lógica difusa). Após, se apresentam e se discutem cinco modelos que inferem a emoção a partir de um evento, no espaço Arousal-Valence (A-V), encontrando que a lógica difusa pode usar-se para medir os estados emocionais humanos.
Palavras-chave: teoria da emoção, emoções, sistemas dinâmicos, lógica difusa.
Studying emotions is important for a number of reasons: first, emotions are the most unpredictable aspect of a person but also a common aspect; second, an emotion shows the way that a person perceives the world; and third, because there are several sciences that are interested in understanding how the human brain works and emotions have been demonstrated to be an important aspect of human intelligence that plays a significant role in human decision-making processes.
Experience has demonstrated that the emergence of new points of view in science is preceded by a period of deep professional insecurity and that if a crisis appears, the reason is that it is time to redesign the tools (Redorta, Obiols, & Bisquerra, 2006). In the study of emotions, the large body of theories, the broad field of study, the current boom and the difficulties of parameterizing the emotions suggest that it is the moment for obtaining new answers from using new tools such as systems theory, Ethology, Organizational approaches and neuroscience (Campos, Campos, & Caplovitz, 1989), which will allow the proper development of Psychology.
This research attempts to show the advantage of studying Psychology with tools from the ‘hard’ sciences and that some tools can be improved or redesigned using Psychology theories because of the information about human thinking that these theories have developed. The question to address is the following: Can two completely different branches of knowledge, such as mathematics and psychology, be studied and applied together for the benefit and growth of both?
The present paper commits to identifying the structure of emotions by reviewing the concept of emotion and the main theories, to propose a model to identify the emotion that governs a person according to an event. This paper investigates mathematical tools for measuring human emotional states in social and developmental psychology. In particular, some fuzzy clustering techniques and the concept of Russel’s AV-Space are applied to estimate the emotional state of a person with respect to a single event. In the experimental evaluation, the results show that this approach is effective for measuring human emotional states by using mathematical tools, clustering techniques, and the concept of Russel’s AV-Space.
In the section entitled “Theoretical Framework”, the theoretical framework of Dynamical Systems, Social Psychology, Theory of emotions, and Fuzzy Logic appears. The section entitled “Previous Studies” aims to show a brief review of the state-ofthe- art in Dynamic Systems and Artificial Intelligence, as applied to emotions. The “Methodology” section describes the methodology that is followed to reach the aim of this work. In the section entitled “Results”, the results are presented and the models are proposed, and finally, in the “Discussion” and “Conclusions” section, there is a discussion of the results and the relevant aspects of this research.
Dynamical Systems and Social Psychology
The theoretical basis of dynamical systems comes from the ‘hard’ sciences, such as mathematics and physics, among others; however, the systems approach has shown to have a high number of interdisciplinary aspects, which enables a large number of applications in other fields of knowledge. A dynamic systems approach has been proven in many applications, especially in economics, biology and sociology, where this approach has led to new and useful ideas.
A conceptual transposition between different areas of knowledge can be explained by the scientific rationality principle, which declares that the same laws and principles operate in the universe, and it is the basis of systems theory, which aims to unify science’s rules (García, Talavera & González, 1997).
Following this line of thought and assuming that systems theory reached its objective of unifying science’s rules, one could argue that because psychology is a science, then psychology postulates can be studied with a systemic approach to provide new and good answers to old questions (Puche, 2009); nonetheless, before making this statement, we must account for the peculiar features that distinguish psychology from the other sciences.
Luce (1997) and Van Geert (1994, cited by Puche, 2009) described Psychology’s features, which hinder any mathematical approach to psychology: First, the study subjects have individual differences, and then, it is impossible to reproduce change because there is no way to erase what has been learned from an experience, and finally, some psychological aspects cannot been generalized with statistics because they fail Quetelet’s assumptions (especially the first of Quetelet’s premises: The causes are proportional to the effects) (Van Geert, 1994). Similarly, studies were based on the cause- effect direct relation, but reality is not consistent with this premise because individuals present different behaviors in response to the same cause; indeed, the plot thickens to the point that each individual can present different behaviors in response to the same cause under different circumstances. Certainly, similar to Mathematics in the nineteenth century with Poincare’s studies, Psychology is passing from linear to non-linear thinking. For this reason, Dynamical Systems applied to Psychology could present a new vision that is more in line with reality (Castro, Sierra, & Flórez, 2012).
“The word ‘emotion’ is used to designate at least three or four different kind of things” (Ryle, 1949, p. 83). There is little information about what emotions are; the boundaries to the domain of what experts have called emotion are so blurry that it sometimes appears that everything is an emotion. They do not agree on what is and what is not (Russell & Feldman-Barret, 1999).
A few years ago, studying emotions appeared to be irrelevant because rationalism eliminated all of those aspects that were considered to be irrational or those that generated difficulties in parameterizing them, and clearly, emotions fit into both categories (Redorta et al., 2006; Castro, 2011). Additionally, experts did not think about a subjective feeling state as a component of emotion; however, with respect to the emotion itself, this idea was consistent with Layman’s implicit theory of emotions, common linguistic allusions to feelings and classical theories of emotion such as James’ or Cannon’s theories. Therefore, a hypothesis of measurability and a structuring of the emotions appeared to be untestable with this approach (Campos et al., 1989).
Recently, the relevance of studying emotions was discovered, when it was proven that emotions are an important aspect of human intelligence and play a significant role in the human decision-making process (El-Nasr, Yen, & Loerger, 2000). It can be concluded, then, that emotions are not only feelings but also the process of establishing, maintaining or disrupting the relation between the organism and the environment (Campos et al., 1989), i.e., emotions are responses to an internal or external occurrence that has a function (Redorta et al., 2006).
This new approach of how emotions are perceived renders the study of emotions incomplete (Campos et al., 1989). For this reason, it is important to study the structure of emotions (Russell & Feldman-Barret, 1999), work on emotional alphabetization and regulation (Redorta et al., 2006) and reach a consensus between different theories (Vallacher & Nowak, 1993), both for psychology’s development and for personal evolution, brain understanding and artificial intelligence machine building.
The concept of emotion
It has already been mentioned that emotions appear to be an answer to an internal or external event. Basically, emotions have four fundamental aspects: subjective, biological, functional, and social. Responses to an event are going to be related to at least one of these aspects.
Figure 1 shows the concept of emotion. There are four elements that allow conceptualizing an emotion: (1) Event, which could be internal, such as a previous emotional state or a change in reality’s perception or an external event that is an abnormal occurrence perceived by the senses; (2) Appraisement, when the subject measures the relevance and utility of the event, as affecting the individual, and the resources to face it (3) The answer, which has three components: the physiological component, which are involuntary biological answers that are impossible to control; the behavioral component, which is associated with the memory of previous events and the human-learning process, i.e., according to the event perceived, there is going to appear a characteristic behavior to those similar events and the cognitive component, which is the feeling itself, and (4) Action
From figures 1 and 2, it can be deduced that there is an important relation among judgment, emotion and action: a thought generates an emotion, which predisposes to action, which changes the thought. It is important to understand that a human’s rational aspect is weaker than the emotion aspect and that emotions have intensity degrees, and thus, when an emotional answer is generated, it is almost impossible to change the emotional state until the intensity is low, and to make this process lower the intensity faster, an emotion might be generated that is opposite to the first.
Classification of emotions
It should be evident that waiting for a consensus on how experts classify emotions should be quite unrealistic currently because the study of emotions is still incomplete. However, there is some acceptance of classifying emotions according to the emotion function, i.e., depending on the goal that each emotion has, as an adaptive, a motivational, an informative or a social emotion. According to the evaluation of the event with regard to their own welfare (Redorta et al., 2006) as negatives or positives, it must be considered that an emotion is not necessarily good or bad and that there also exists “ambiguous” or “grey” emotions, such as surprise, hope or compassion. Finally, emotions could be classified according to the “universality” as basic or secondary. However, the definition of which emotion is basic or not generates considerable controversy in the literature because each author proposes his own prototypical emotions list that is comprised of 2 to 18 basic emotions. There is still substantial agreement on Anger, Displeasure, Fear, Happiness, Sadness and Surprise as basic emotions (Peter & Herbon, 2006).
Despite the differences, some authors prefer to say “rather than an emotion, as a static chamber, as in a family of emotions, as communicating vessels, assuming that somehow there is a continuum between emotions and states” (Redorta et al., 2006, p. 36). In Appendix A, there is a chart that shows the prototypical emotions and their families, some features and the function of each emotion.
Theory of emotions
Current theories of emotions can be grouped into theories that focus on how emotions arise and how they are perceived and theories that focus on the emotion’s structure and how emotions could be categorized (Peter & Herbon, 2006). For both approaches, there is a number of possibilities, and again, there is no agreement among authors; however, some authors consider that these possibilities are actually complementary instead of opposite and explain different aspects of the same phenomenon (Russell & Feldman-Barret, 1999).
The theories that focus on how emotions arise and are perceived appear on Table 1. There is no agreement between the biological and cognitive components, the number of emotions and the relation between motivation and emotion.
The theories that focus on the structure of an emotion have two main approaches: the discrete approach, which is based on the existence of universal basic emotions (a review of the seven main suggestions of the discrete approach is presented in Appendix B), and the dimensional approach, which is based on the idea that emotions are not independent but instead are correlated in some specific way and, in the proposal, that emotions could be defined by the “Valence/Evaluation/Pleasure-Displeasure/Positivity and the Arousal-Sleepiness/Tension-Relaxation axes” (Peter & Herbon, 2006). There are two main suggestions of the dimensional approach: The circumplex model and the hierarchy model (Russell & Feldman-Barret, 1999).
Arousal – Valence Space
The main problem for structuring emotions is language because there are some emotion words that have different meanings in different countries; nevertheless, structuring emotions means implicitly the emotion words, which is why the categories are different in different cultures. Affect means “the emotion represented in language as a word”, and the dimensional approach means that the circumplex or the hierarchy model attempts to find the affect space.
Many studies have found evidence for monopolar factors; however, those studies have been proven to be biased because of the response format, the acquiescence, the inadequate sampling, and other factors (Russell, 1979). For this reason, some authors think that rather than discrete structures, emotions are not independent and are correlated in a systematic way (Russell, 1980).
Even when there are spaces that are proposed in the literature that have three or more dimensions (Russell, 1979), there is an agreement in Valence and Arousal (Russell, Weiss, & Mendelsohn, 1989). Valence is the evaluation of the emotions: Positive- negative or pleasure-displeasure, and Arousal is the degree of activation: arousal-sleepiness or tension-relaxation. Every two-dimensional model has these basic bipolar dimensions even when the structure varies, for example, Lang (1995) proposed a hierarchy model of Arousal/Valence using percentages. On the other hand, Russell (1980) based his model on Schlosberg’s proposal (1952) of organizing emotions around a circle and in Layman’s representation of the emotions because the boundaries of the emotions are fuzzy. Figure 3 shows Russell’s Circumplex Model of Affect.
Nevertheless, this approach is not very far from the discrete approach. In 1999, Russell and Feldman Barret focus on the study of two types of emotions, to blend both approaches: the prototypical emotions (Discrete approach) and the core affect (Dimensional approach). They clarify that emotions cannot be divided into core affect and prototypical emotional episodes, but some emotion could refer more to one than the other. Figure 4 shows a model that blends the prototypical emotions and the core affect: the inner circle shows a schematic map of the core affect, and the outer circle shows where the main prototypical emotions fall.
Concepts. Fuzzy logic provides a solution for problems that are characterized by handling multi-valued values that are typical of human thought; it is a method for specifying real problems in probabilistic terms without resorting to a mathematical model and with a higher level of abstraction.
Fuzzy logic is characterized by naturally handling imprecision and conceptual simplicity, besides the use of linguistic expressions that are associated with numerical data. It has been mathematically proven that it works with no linear processes and that there are many applications in control designing, prediction of time series, data mining, and other subjects.
Every fuzzy model has four key elements: First is the Universe of discourse, which is the domain of the variable. Second is the variable that is under study, and third is the membership functions μf (x) → [0,1], which allows, through the membership degree, to determine the associated linguistic term, which is the last element.
Membership functions are fuzzy sets and are defined by the expert using knowledge or experimentation with a variety of algorithms that allow constructing the membership functions, such as clustering algorithms. The literature describes many shapes for the membership functions, with triangular and trapezoidal shapes being the most common. The assignment of the form of the function is subjective in nature but is not arbitrary.
Finally, the membership functions are defined in a data base that is the knowledge base of the fuzzy system. To build the inference, fuzzy logic uses the operators of bi-valued logic, specifically, AND, OR and NOT, but the definition of those operators is through a function’s family that is fulfilled with a norm. According to the operator, there are two norms defined: the s-norms and the t-norms. Figure 5 provides an example that shows the four elements of a fuzzy system.
There are several ways to employ fuzzy logic in the modeling of systems (Babuska, 1998); one method is using rule-based fuzzy systems, in which relationships between variables are represented by rules of the form
In rule-based fuzzy systems, the membership functions are defined with both a rule base and a data base. To make an inference in a rule-based fuzzy system, fuzzy logic uses a modification of the inference rules that use bi-valued logic: The Modus Ponens rule, the Modus Tollens rule and the Hypothetical Syllogism. In addition, in rule-based systems, the membership functions are defined with both a rule base and a data base, and there are components that transform the inputs and the outputs. The inputs are transformed to fuzzy values in the fuzzifier, to be processed in the inference engine, and the outputs are converted again to return a specific value from the defuzzifier.
According to the structure of the consequent, three types of fuzzy inference systems (FIS) based on rules are distinguished: the linguistic fuzzy model or Mamdani’s model (1977), where both the antecedent and the consequent are fuzzy propositions, the fuzzy relational model or Pedrycz’ model (1984), which can be regarded as a generalization of the linguistic model with some mild modifications, and the TS model (Takagi & Sugeno, 1985), where the consequent is a function (it is evident that, in the TS fuzzy model, there is not a defuzzifier component). Figure 6 provides a diagram that shows the structure of a generic FIS based on rules.
Clustering data techniques. Above, it was already mentioned that the membership functions are defined by the expert using knowledge or using the data and algorithms that allow the construction of the functions with the fuzzy clustering data techniques.
The aim of clustering data is to divide the data into classes in such a way that the elements in a class are similar and the elements between classes are dissimilar. In fuzzy clustering data techniques, the data can belong at a certain level to more than one cluster, and a membership function can be built for each cluster.
Similar to bivalued logic, which uses graphs such as dendrograms and criteria such as the closest neighbor, maximum savings or sweep method, to make a ‘hard’ partition, multi-valued logic uses a variety of fuzzy clustering data techniques that include the c-means technique, introduced by Jim Bezdek (1981), in which data points that populate some multidimensional space are grouped into a specified number of clusters, the subtractive technique, in which data points are clustered in an optimal way according to some parameters such as the acceptance ratio or the influence range (Chiu, 1994), and finally, the adaptable method, in which the distance used to calculate the clusters is not fixed. The notion, then, is to find the membership functions that are behind the data to create the fuzzy inference system.
Adaptive Neuro-Fuzzy Inference System. An Adaptive Neuro-Fuzzy System (ANFIS) is one of the best trade-offs between neural and fuzzy systems. It integrates the best features of Fuzzy Systems, the representation of prior knowledge into a set of constraints, and Neural Networks, the adaptation-using learning algorithms, to identify the parameters of the membership functions that allow the system to ‘learn’ from training data. Designed by Jang (1993), ANFIS is applied to models to explain past data and predict future behavior (Bonissone, 2002).
Dynamical systems and social psychology. It is generally agreed that dynamical systems are a developing useful basis for studying psychology; however, what has generated dissent is the approximation of the study. Table 2 presents the main authors, who have dedicated their research to the systemic analysis of Psychology.
Thelen (Bloomington’s approach), for example, uses the concept of systems theory to define a development theory by analogy; for example, an “attractor”, which in systems theory is an ordained element of high stability surrounded by instability, in Thelen’s theory is a conduct that is defended by the individual and the behavior that will tend toward it. In contrast, Van Geert (Groningen’s approach) prefers to exploit the heuristic productivity of classical models, adapting those models according to the subject of the investigation (Castro, Sierra, & Flórez, 2012).
Lewis (2000) and Fischer (2006) share the idea that development comes from emotions, neurological functioning and psychopathology, to find pattern emergence, but Lewis’ work is based specifically on the study of stability and change points while Fischer focuses on the study of variability and instability.
Finally, there is Vallacher and Nowak (1993), whose research is to focus on social psychology instead of developmental psychology, for which the differences in the research are explained above. Their research has been to focus on motivation theory and the existence of opposite forces in human socialization processes.
Models of emotions in Artificial Intelligence. Throughout the history of Artificial Intelligence research, many models have been proposed to describe the human mind (El-Nasr et al., 2000). Fuzzy logic was one of the first tools that was exploited to develop computational emotion models to represent different aspects of emotions.
Yanaru (1997) used fuzzy inferences to define emotion while a person is reading a poem, using the 8 prototypical emotions proposed by Plutchik (1980). As a feature to highlight, Yanaru’s system used the previous emotional state to infer the new emotional state.
El-Nasr et al. (2000) proposed a fuzzy model to generate emotions in a virtual pet, to make it more credible, focusing on the behavioral and cognitive components of emotion in such a way that the pet could make its own decisions, while accounting for the goals and learning from previous experiences using reinforcement. This article shows the application of emotion fuzzy models to the HCI (Human-Computer Interaction or affective computation; Rairán, 2010), to make it easier to develop contact between humans and machines.
Mandryk and Atkins (2007) provides a method for quantifying emotional states during interaction with play technologies, using the physiological component of the emotion, mapping from physiological measures to Arousal-Valence and, then, from Arousal-Valence to five emotions. This work proves that fuzzy logic could be useful to test people’s acceptance to new products, and it also proves that fuzzy logic could measure human emotional states.
Figure 7 shows the system that explains the emotion’s estimation as a dynamic system. It was built by collecting the concept of emotion explained in Figure 1, the AV-space introduced above and the perspective of emotions as dynamic systems.
Note that only the cognitive component appears because the behavioral and physiology components are beyond the scope of the present paper. To delve into these components of emotion and its application in models, the reader can refer to El-Nasr et al. (2000) or Mandryk and Atkins (2007).
The space that is used to cluster, map and analyze the emotions with fuzzy logic is a modification of Russell’s circumplex model (Russell, 1980). In his investigations, Russell utilized three statistical techniques to distribute 28 emotion words along AV-Space: Unidimensional Scaling, Multidimensional Scaling and Ross’ direct circular technique (Ross, 1938).
To unify Russell’s investigation results, the three techniques’ results were averaged. The new space is presented in Figure 8. The 28 emotions with their angles (in degrees), using the techniques applied by Russell, and the angles that require averaging are presented in Appendix C.
Clustering and membership functions
In the beginning, the idea was to map from Arousal-Valence to the eight prototypical emotions described by Plutchik (1980); as a result, the fuzzy c-means technique was used to create eight clusters (one for each emotion). Nevertheless, the subtractive method defined the optimal number of clusters for the unified AV-Space as 5, using an acceptance ratio of .5.
To compare the subtractive and c-means results, the c-means technique was used to create five clusters; however, even when the answers were closer, there were two clusters that were far away from each other. To avoid this situation of a subtractive accounting for a group of data and c-means being different, it was decided to add another cluster for each method. Thus, the c-means was calculated to create six clusters, and the subtractive used an acceptance ratio of .4. Using these parameters, both techniques yield equal outcomes, and the center for each cluster appears in Table 3. To simplify the explanations below, each cluster is named according to an emotion.
Figure 9 shows a comparison between the fuzzy c-means for eight (x), six (triangle) and five (square) clusters and also subtractive clustering with the parameter of the acceptance ratio taking the values of .5 (diamond) and .4 (triangle). Figure 10 shows the six clusters that were found.
As mentioned in the above explanation of fuzzy clustering techniques, the notion here is to find the membership functions. Figure 11 shows the membership function for the cluster that represents ‘calm’, which was calculated using the Matlab command findcluster.
Five models were proposed for modeling emotions from Arousal and Valence: a model to determine the strongest emotion that is felt by the person (called the Strongest Emotion model and based on a model used by Mandryk and Atkins to determine emotions while playing an interactive game), a Mamdani’s model and a Takagi-Sugeno’s model (both called a rule-based FIS because of the structure of their respective rules), which calculates the angle and the intensity of the emotion and two ANFIS using a grid partition and a fuzzy clustering technique.
Strongest Emotion model: This model attempts to infer emotions from the values for Arousal and Valence, with respect to the six emotions that are defined by the clusters. In this approach, the model has two inputs (Arousal and Valence) and six outputs (Boredom, Calm, Happiness, Frustration, Excitement and Depression).
The fuzzy inference system is presented in Figure 12. This model is composed of 35 rules that were determined by the author based on knowledge and a deeper investigation, and the membership functions for the inputs and the outputs, which are the same for each input and output, are presented in Figure 13.
Mamdani’s model: This model is composed of two Fuzzy Inference Engines: the first FIS models the emotion forms Arousal and Valence to an angle in AV-Space using 48 rules. The second FIS models the intensity of the emotion from Arousal and Valence using 44 rules. Both rule bases are presented in Appendix D. The membership function for the inputs (Arousal and Valence) and the Angle output are the membership functions that are found using the clustering techniques; for the intensity, the membership functions are defined by the author because the data used to map the space has the same intensity; thus, there were no clusters to find using clustering techniques. Figure 14 shows the membership functions that were used—in the order followed by the quadrants of a plane: (i) arousal, (ii) valence, (iii) intensity, and (iv) angle.
Takagi Sugeno’s model: Takagi-Sugeno’s model is similar to Mamdani’s model (explained above), but the consequent is a linear function. Nevertheless, the FIS structure, the membership functions for the inputs and the rules are the same as those specified for Mamdani’s Model.
ANFIS: The main reason for performing Mamdani’s model and Takagi-Sugeno’s model as two FIS with two inputs and a single output and not as a single FIS with two inputs and two outputs is to allow comparisons between both models and the ANFIS models because ANFIS uses single-output FIS.
Two ANFIS are performed to model emotions: an ANFIS that uses Grid Partitioning instead of clustering techniques and the ANFIS that uses subtractive clustering with a .4 ratio of acceptance, as was done in the methods above. For both ANFIS, the system should be given training data in such a way that the system can “learn” from the data. Both ANFIS use a hybrid-learning algorithm that combines the least-squares and backpropagation gradient descent methods to identify the membership function parameters; they were designed using the Anfis option available in Matlab.
Validation data. The database to validate the model is taken using Russell’s affect grid (Russell et al., 1989), which is a tool designed to describe the cognitive component of emotions and to record judgments about single instances of affect, i.e., to record judgements about a single event. This approach is basically a quick means of assessing affect along the dimensions of the AV-Space.
The affect grid is a square net in which the center represents a neutral emotion about the situation. The horizontal dimension represents the valence of the emotion, i.e., the right side represents a positive emotion, and the left side represents a negative emotion. The vertical dimension represents the convulsion degree, in such a way that the top means arousal and the bottom relaxation.
Figure 15 shows the affect grid. People are asked to mark an X wherever they consider that their emotions are better represented. The further the X is placed from the center, the higher the level of the dimension.
The data to validate the models were taken from ten students from EAFIT’s University who were coming to take the final test. Because there were some students that marked the same place in the grid, only six data were in the validating data set; these values appear in Table 4. The first two columns are the Valence and Arousal coordinates; next are the angle and the intensity of the emotion. The last two columns show which emotions the student’s feeling is located between.
Below we present the output of the Strongest Emotion model using the validating data to compare the real values to the outcomes of the model. Additionally, we will present a comparison of the other four models using a measurement of distance.
Strongest Emotion Model: The Strongest Emotion model produces values between 0 and 1 for each emotion: the emotion that shows the largest value is the predominant emotion. In the Strongest Emotion model, the output for any validating data set would come in a n × 6 matrix, where n is the number of validating data points (in this case 6), and each column is an emotion. Table 5 is the output for the Strongest Emotion model; each numbered column is an emotion that corresponds to an output (the numbering matches the numeration in Table 3). The last column shows the emotion that predominates for each subject. The highest value is in bold, and the second highest value is in italics for each subject, i.e., for each row.
Using these results, a circular diagram can be constructed for each individual, to visualize the proportions among the emotions. Figures 16 and 17 show the results for the first individual and the average for all of the students.
Models that estimate the Angle and Intensity from the Arousal and Valence measurements: Mamdani’s model, Takagi-Sugeno’s model and both ANFIS attempt to estimate emotion from the measurements of Arousal and Valence, and as explained by the proposed models, they produce a vector with the emotion’s angle and the intensity of the emotion. Using this finding, it is possible to calculate the coordinates in AV-Space, and the distances between the estimated data and real data can be calculated to compare the methods. Table 6 presents the Euclidean distances from the estimated point to the real point for each model. The Euclidean distance is calculated as
Figure 17 shows the estimated points in AV-Space.
The system proposed in Figure 7 shows an emotion that is inferred from the cognitive component of emotion, i.e., an emotion that is inferred from an emotion. Because this paper was an introduction to show the relation between mathematical models and emotions, the models proposed above are simple; nevertheless, there are many possibilities for improving the capability of the models while adding the other two components of emotion (behavioral and physiological) and the feedback shown on Figure 7, where emotion depends on both the previous emotion and the event.
As stated above in the explanation of the structure of emotions, the main problem for structuring emotions is language because emotion words have different connotations in different countries. Russell’s results, the basis for AV-Space, which is used for mapping the emotions, are the results for his country.
After an exhaustive study, the authors determined that Russell’s results are appropriate to be applied in their culture; however, it would be an interesting experiment to create an AV-Space using the techniques proposed by Russell, to compare how people represent emotions in different countries.
Clustering and membership functions
Compared with Mandryk and Atkins’ work (Mandryk & Atkins, 2007), where five emotions are estimated from Arousal and Valence (also, Arousal and Valence are estimated from Physiological measures), it can be observed that the clusters that were established by the Fuzzy Clustering techniques are consistent with the groups that are recognized by psychologists to evaluate a play technology. The emotions that are used in their research appear in Figure 19; in contrast, in the present research, six clusters were used instead of five because using the six clusters accounts for the whole space.
The difference resides in that Mandryk and Atkins’ research involved attempting to estimate emotions to evaluate the acceptance of a play technology, and those five emotions where sufficient (Mandryk & Atkins, 2007), but in the present paper, the idea was to build a model that allows estimating emotions from any event; thus, there must be another emotion, which they did not consider when they developed their model: it would be calm because there is no relevance to whether a person is calm or not while playing.
The Strongest Emotion model is an adaptation of Mandryk and Atkins’ experiment without physiological measures, i.e., modeling emotion from Arousal and Valence values and using six emotions instead five. As already stated, the fact of adding ‘calm’ to the model allows the generalization of the model to inferred emotions from almost any singular event and not only for play technology acceptance.
Mamdani’s model, performed above, uses the membership functions that are found using clustering techniques. This approach improves its performance because of the optimal location of the centers and the form of the membership functions according to the space. However, Mamdani’s model is still weaker than the Adaptive Neuro-Fuzzy Inference Systems because of the high degree of knowledge that is needed by the deployer to develop it. Even when it is still producing good results, it is important to evaluate what combination of properties (methods for evaluating the fuzzy operators) produces better results; for the model, the properties that make the model yield better outcomes (which are the outcomes presented in the results section) are presented in Table 7.
The Takagi-Sugeno’s model produces bad results even when the rules are modified. However, both ANFIS generate fuzzy systems of the type of Sugeno, which suggests that the system is not easier to adjust to a linear function and, clearly, it is not easy to syntonize manually the parameters to a non-linear function.
Finally, the improvement that was attained by obtaining the membership functions from the clusters found is proven by comparing the two ANFIS because the angles were closer to the real angles in the second ANFIS.
Because the intensity has no significant clusters (which is why the membership functions of the intensity FIS in Mamdani’s and Takagi-Sugeno’s model were created by knowledge and not by the clustering techniques), the intensity for both ANFIS was similar, which makes the Euclidean distances offset.
In the validating data, a peculiar point appears that no model could estimate properly: the point (3.0). Additionally, this point was in the training data, which means that the ANFIS were not learning from it. The hypotheses to explain this behavior is that this point belongs to three membership functions at the same time, for each input, and thus, the model cannot determine correctly the value of this point.
The results are considered to be accurate, because even when the angles were not exactly the same as the real angles, the errors were small, and in general, the results explained the emotions of the person.
The present paper demonstrated that dynamical systems theory can be reinforced by Psychology such concepts as attention or emotions, and that specifically, the theory of emotions can be useful for mathematical computations for control, design and affective computation. Additionally, it has been proven that tools that originated in the ‘hard’ sciences, such as systems theory, can be applied to solve problems in Psychology. To reach a consensus between these theories could open unexpected possibilities in the study of the human psyche. This opportunity means that both sciences can be studied and applied together, for the benefit of both.
The possible applications of Mathematical tools in Social Psychology have been shown, and the few applications shown in the present paper confirmed the notion of the importance of using mathematical approaches to every branch of knowledge.
Finally, we developed four models based on artificial intelligence that estimate with a certain precision the emotional state of a person with respect to a single event, from a rating of Arousal and Valence. This result means that with a rating of the level of agitation and the level of goodness of the event, it was possible to define the emotion that would predominate in a person. Thus, without requiring an extensive questionnaire, it is easy to have a notion of the reaction that a specific event would cause in a person.
Babuska, R. (1998). Fuzzy modelling for control. Boston, Mass: Kluwer Academic Publishers.
Beer, R.D. (2000). Dynamical approaches to cognitive sciences. Trends in Cognitive Sciences, 4, 91-99.
Bezdek, J.C. (1981). Pattern Recognition with fuzzy objective function algorithms. New York: Plenum Press.
Bonissone, P. (2002). Adaptive neural fuzzy inference systems (ANFIS): Analysis and applications [Slides]. Class Notes. New York.
Campos, J., Campos, R., & Caplovitz, K. (1989). Emergent themes in the study of emotion development and emotion regulation. Developmental Psychology, 25(3), 394-402.
Castro, J. (2011). Sistemas dinámicos en la interacción emocional madre-hijo. Panorama, 9, 129-138.
Castro, J., Sierra, H., & Flórez, R. (2012). Una revisión de las relaciones entre los sistemas dinámicos y la psicología del desarrollo. Suma Psicológica, 19(2), 109-130.
Chiu, S. (1994). Fuzzy model identification based on cluster. Journal of Intelligent and Fuzzy Systems, 2(3), 267-278.
Correa, G.J. (2004). Aproximaciones metodológicas para la toma de decisiones apoyadas en modelos difusos (Magister Dissertation). Universidad Nacional de Colombia, sede Medellín, Medellín, Colombia.
Ekman, P., Levenson, R.W., & Friesen, W.V. (1983). Autonomic nervous system activity distinguish between emotions. Science, (221), 1208-1210.
Ekman, P. (1993). Facial expression and emotion. American Psychologist (48), 384-392.
El-Nasr, M.S., Yen, J., & Loerger, T.R. (2000). FLAME: Fuzzy Logic Adaptive Model of Emotions. Autonomous Agents and Multiagents Systems (3), 219-257.
Fischer, K. W., & Bidell, T.R. (2006). Dynamic development of action, thought and emotion. In W. Damon & R.M. Lerner (Eds.), Theoretical models of human development: handbook of child psychology (6th ed., pp. 313-399). New York: Wiley.
Frijda, N.H. (1986). The emotions. New York: Cambridge University Press.
García, M., Talavera, J.A., & González, A. (1997). Teoría de sistemas dinámicos no lineales y trastornos mentales. Revista Electronica de Psiquiatría, 1 (1).
Jang, J.S.R. (1993). ANFIS: adaptive network based fuzzy inference system. IEEE Trans. Systems, Man and Cybernetics, 23, 665-685.
Lang, P.J. (1995). The emotion probe. American Psychologist, 50(5), 372-385.
Lewis, M.D. (2000). Emotion, development and self-organization: dynamic system approaches to emotional development. New York: Cambridge University Press.
Luce, D. (1997). Several unresolved conceptual problems in mathematical psychology. Journal of Mathematical Psychology, 41, 79-87
Mamdani, E. (1977). Application of fuzzy logic to approximate reasoning using linguistic systems. Fuzzy Sets and Systems, 26, 1182-1191.
Mandryk, R.L., & Atkins, M.S. (2007). A Fuzzy physiological approach for continuously modelling emotion during interaction with play technologies. International Journal of Human-Computer Studies, 65, 329-347.
Ortony, A., Clore, G., & Collins, A. (1988). The cognitive structure of emotions. New York: Cambridge University Press.
Pedrycz, W. (1984). An identification algorithm in fuzzy relational systems. Fuzzy Sets and Systems, 13, 153-167.
Peter, C. & Herbon, A. (2006). Emotion representation and physiology assignments in digital systems. Interacting with Computers, 18, 139-170.
Plutchik, R. (1980). Emotion: A psychoevolutionary synthesis. New York: Harper Collins.
Puche, R. (2009). ¿Es la mente no lineal? Cali: Universidad del Valle.
Rairán, J.D. (2010). Control basado en emociones: una revisión. Avances en Sistemas e Informática, 7(2), 27-36.
Redorta, J., Obiols, M., & Bisquerra, R. (2006). Emoción y conflicto: aprenda a manejar las emociones. Barcelona: Paidós.
Roseman, I.J., Jose, P.E., & Spindel, M.S. (1990). Appraisal of emotion-eliciting events: testing a theory of discrete emotions. Journal of Personality and Social Psychology, 59(5), 899-915.
Ross, R.T. (1938). A statistic for circular scales. Journal of Educational Psychology, 29, 384-389.
Russell, J.A. (1979). Affective space is bipolar. Journal of Personality and Social Psychology, 37(3), 345-356.
Russell, J.A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39, 1161-1178.
Russell, J.A. & Feldman-Barret, L. (1999). Core affect, prototypical emotional episodes and other things called emotion: dissecting the elephant. Journal of Personality and Social Psychology, 76(5), 805-819.
Russell, J.A., Weiss, A., & Mendelsohn, G. (1989). Affect grid: a single item scale of pleasure and arousal. Journal of Personality and Social Psychology, 57(3), 493-502.
Ryle, G. (1949). The concept of mind. Harmondsworth: Penguin Books.
Schlosberg, H. (1952). The description of facial expressions in terms of two dimensions. Journal of Experimental Psychology, 44, 229-237.
Takagi, T. & Sugeno, M. (1985). Fuzzy identification of systems and its applications to modelling and control. IEEE Trans. Systems, Man and Cybernetics, 15(1), 116-132.
Vallacher, R. & Nowak, A. (1993). Dynamical systems in social psychology. San Diego: Academic Press.
Van Geert, P. (1994). Dynamic systems of development: Change between complexity and chaos. London: Harvester Wheatsheaf.
Van Geert, P. & Steenbeck, H. (2005). Explaining after by before: Basic aspects of a dynamic systems approach to the study of development. Developmental Review, 25, 408-442.
Velasco, J.M. (1999). Aportaciones desde la teoría de los sistemas complejos y la neurobiología en apoyo de un modelo psicodinámico. Revista Electrónica de Psiquiatría, 3(1).
Yanaru, T. (1997). An emotion processing system based on fuzzy inference and subjective observations. Information Sciences, 101, 217-247.
A review of the emotions’ discrete structure
Emotions and angles (in degrees) in AV-Space (Russel, 1980)
Rule Bases for Mamdani’s Model
Arousal-Valence to Angle:
1. If (Valence is VeryLow) and (Arousal is VeryLow) then (Angle is Depression)
2. If (Valence is VeryLow) and (Arousal is Low) then (Angle is Depression)
3. If (Valence is VeryLow) and (Arousal is MidLow) then (Angle is Depression)
4. If (Valence is VeryLow) and (Arousal is MidHigh) then (Angle is Depression)
5. If (Valence is VeryLow) and (Arousal is MidHigh) then (Angle is Frustration)
6. If (Valence is VeryLow) and (Arousal is High) then (Angle is Frustration)
7. If (Valence is VeryLow) and (Arousal is VeryHigh) then (Angle is Frustration)
8. If (Valence is Low) and (Arousal is VeryLow) then (Angle is Boredom)
9. If (Valence is Low) and (Arousal is Low) then (Angle is Boredom)
10. If (Valence is Low) and (Arousal is Low) then (Angle is Depression)
11. If (Valence is Low) and (Arousal is MidLow) then (Angle is Depression)
12. If (Valence is Low) and (Arousal is MidHigh) then (Angle is Depression)
13. If (Valence is Low) and (Arousal is MidHigh) then (Angle is Frustration)
14. If (Valence is Low) and (Arousal is High) then (Angle is Frustration)
15. If (Valence is Low) and (Arousal is VeryHigh) then (Angle is Frustration)
16. If (Valence is MidLow) and (Arousal is VeryLow) then (Angle is Boredom)
17. If (Valence is MidLow) and (Arousal is Low) then (Angle is Boredom)
18. If (Valence is MidLow) and (Arousal is MidLow) then (Angle is Boredom)
19. If (Valence is MidLow) and (Arousal is MidLow) then (Angle is Depression)
20. If (Valence is MidLow) and (Arousal is High) then (Angle is Frustration)
21. If (Valence is MidLow) and (Arousal is High) then (Angle is Excitement)
22. If (Valence is MidLow) and (Arousal is VeryHigh) then (Angle is Frustration)
23. If (Valence is MidLow) and (Arousal is VeryHigh) then (Angle is Excitement)
24. If (Valence is MidHigh) and (Arousal is VeryLow) then (Angle is Boredom)
25. If (Valence is MidHigh) and (Arousal is VeryLow) then (Angle is Calm)
26. If (Valence is MidHigh) and (Arousal is Low) then (Angle is Boredom)
27. If (Valence is MidHigh) and (Arousal is Low) then (Angle is Calm)
28. If (Valence is MidHigh) and (Arousal is MidLow) then (Angle is Boredom)
29. If (Valence is MidHigh) and (Arousal is MidLow) then (Angle is Calm)
30. If (Valence is MidHigh) and (Arousal is MidHigh) then (Angle is Happiness)
31. If (Valence is MidHigh) and (Arousal is MidHigh) then (Angle is Excitement)
32. If (Valence is MidHigh) and (Arousal is High) then (Angle is Excitement)
33. If (Valence is MidHigh) and (Arousal is VeryHigh) then (Angle is Excitement)
34. If (Valence is High) and (Arousal is VeryLow) then (Angle is Calm)
35. If (Valence is High) and (Arousal is Low) then (Angle is Calm)
36. If (Valence is High) and (Arousal is MidLow) then (Angle is Calm)
37. If (Valence is High) and (Arousal is MidHigh) then (Angle is Happiness)
38. If (Valence is High) and (Arousal is MidHigh) then (Angle is Excitement)
39. If (Valence is High) and (Arousal is High) then (Angle is Happiness)
40. If (Valence is High) and (Arousal is High) then (Angle is Excitement)
41. If (Valence is High) and (Arousal is VeryHigh) then (Angle is Excitement)
42. If (Valence is VeryHigh) and (Arousal is VeryLow) then (Angle is Calm)
43. If (Valence is VeryHigh) and (Arousal is Low) then (Angle is Calm)
44. If (Valence is VeryHigh) and (Arousal is MidLow) then (Angle is Calm)
45. If (Valence is VeryHigh) and (Arousal is MidHigh) then (Angle is Happiness)
46. If (Valence is VeryHigh) and (Arousal is High) then (Angle is Happiness)
47. If (Valence is VeryHigh) and (Arousal is VeryHigh) then (Angle is Happiness)
48. If (Valence is VeryHigh) and (Arousal is VeryHigh) then (Angle is Excitement)
Arousal-Valence to Intensity:
1. If (Valence is VeryLow) and (Arousal is VeryLow) then (Intensity is High)
2. If (Valence is VeryLow) and (Arousal is Low) then (Intensity is High)
3. If (Valence is VeryLow) and (Arousal is MidLow) then (Intensity is High)
4. If (Valence is VeryLow) and (Arousal is MidHigh) then (Intensity is High)
5. If (Valence is VeryLow) and (Arousal is High) then (Intensity is High)
6. If (Valence is VeryLow) and (Arousal is VeryHigh) then (Intensity is High)
7. If (Valence is VeryLow) and (Arousal is MidLow) then (Intensity is Middle)
8. If (Valence is Low) and (Arousal is VeryLow) then (Intensity is High)
9. If (Valence is Low) and (Arousal is Low) then (Intensity is Middle)
10. If (Valence is Low) and (Arousal is MidLow) then (Intensity is Middle)
11. If (Valence is Low) and (Arousal is MidHigh) then (Intensity is Middle)
12. If (Valence is Low) and (Arousal is High) then (Intensity is High)
13. If (Valence is Low) and (Arousal is VeryHigh) then (Intensity is High)
14. If (Valence is MidLow) and (Arousal is VeryLow) then (Intensity is High)
15. If (Valence is MidLow) and (Arousal is VeryLow) then (Intensity is Middle)
16. If (Valence is MidLow) and (Arousal is Low) then (Intensity is Middle)
17. If (Valence is MidLow) and (Arousal is MidLow) then (Intensity is Middle)
18. If (Valence is MidLow) and (Arousal is MidLow) then (Intensity is Low)
19. If (Valence is MidLow) and (Arousal is MidHigh) then (Intensity is Low)
20. If (Valence is MidLow) and (Arousal is MidHigh) then (Intensity is Middle)
21. If (Valence is MidLow) and (Arousal is High) then (Intensity is High)
22. If (Valence is MidLow) and (Arousal is High) then (Intensity is Middle)
23. If (Valence is MidLow) and (Arousal is VeryHigh) then (Intensity is High)
24. If (Valence is MidHigh) and (Arousal is VeryLow) then (Intensity is High)
25. If (Valence is MidHigh) and (Arousal is Low) then (Intensity is Middle)
26. If (Valence is MidHigh) and (Arousal is MidLow) then (Intensity is Middle)
27. If (Valence is MidHigh) and (Arousal is MidHigh) then (Intensity is Middle)
28. If (Valence is MidHigh) and (Arousal is High) then (Intensity is Middle)
29. If (Valence is MidHigh) and (Arousal is VeryHigh) then (Intensity is High)
30. If (Valence is High) and (Arousal is VeryLow) then (Intensity is High)
31. If (Valence is MidHigh) and (Arousal is High) then (Intensity is High)
32. If (Valence is High) and (Arousal is Low) then (Intensity is High)
33. If (Valence is High) and (Arousal is MidLow) then (Intensity is High)
34. If (Valence is High) and (Arousal is MidHigh) then (Intensity is High)
35. If (Valence is High) and (Arousal is High) then (Intensity is High)
36. If (Valence is High) and (Arousal is MidLow) then (Intensity is Middle)
37. If (Valence is High) and (Arousal is MidHigh) then (Intensity is Middle)
38. If (Valence is High) and (Arousal is VeryHigh) then (Intensity is High)
39. If (Valence is VeryHigh) and (Arousal is Low) then (Intensity is High)
40. If (Valence is VeryHigh) and (Arousal is VeryHigh) then (Intensity is High)
41. If (Valence is VeryHigh) and (Arousal is High) then (Intensity is High)
42. If (Valence is VeryHigh) and (Arousal is MidLow) then (Intensity is High)
43. If (Valence is VeryHigh) and (Arousal is MidHigh) then (Intensity is High)
44. If (Valence is VeryHigh) and (Arousal is VeryLow) then (Intensity is High)
Politécnico Grancolombiano, calle 57 #3-00 este, Bogotá, Colombia. E-mail: firstname.lastname@example.org
to cite this article: Mejía, S., Quintero, O.L. & Castro, J. (2016).
Dynamic Analysis of Emotions through Artificial
Intelligence. Avances en Psicología
Latinoamericana, 34(2), 205-232. doi: http://doi.org/10.12804/apl34.2.2016.02
Susana Mejía M.
Universidad EAFIT, Colombia
Olga Lucía Quintero M.
Universidad EAFIT, Colombia