Automatic Generation of Humor for Social Robots
Humor is an important feature in human communication and is an essential and ubiquitous feature of everyday life. Humor has been linked to numerous positive outcomes. The AGENTS project will leverage the power of humor to create more naturalistic and lifelike interactions with socially embodied agents, in particular, social robots. The positive effect of humor can be increased through the delivery of user-personalized humor in naturalistic settings. In AGENTS, psychological models of humor and its’ everyday functions will be of use when attempting to create a top-down approach of humor that can be modeled to match each user’s preferences, fostering more natural and positive interactions with users. The end-goal of this process will be the implementation of user personalized humoristic interactions in the context of a multi-party interaction scenario with a card game involving more than one human and more than one robot. This is expected to lead to better interaction outcomes and increase the value perception of the robot, by contributing to greater user’s task enjoyment, engagement, and positive emotions.
Ana Paiva (PI; INESC-ID, IST-UL); Patrícia Arriaga, CO-PI, Iscte-IUL)) & Louis-Philippe Morency (Carnegie Mellon University)
Raquel Oliveira & João Barreiros
Teresa Chambel (PI, FCIÊNCIAS.ID),
Patrícia Arriaga (CO-PI, ISCTE-IUL);
Thibault Langlois (FCIÊNCIAS.ID),
Manuel Monteiro da Fonseca (FCIÊNCIAS.ID),
Octavian Postolache (IT-IUL, ISCTE-IUL).
Investigate the emotional dimension of movies, to further provide support for the classification, access, navigation, and visualization of movie collections, not only including previous descriptors (genre, actors) but also emotions expressed in their content and felt by the viewers, during and after watching movies. This holds the potential to increase emotional awareness and empower future users to regulate their emotions when accessing and watching movies. To accomplish these objectives, there are four scientific challenges that we will address in the project:
1. Evaluate and select the most relevant dimensions in emotional impact and predictors of enjoyment and gratifications from movies, understand viewer’s preferences, and the importance perceived in receiving emotional information. This will allow us to identify the most relevant emotional responses to use for the evaluation of the emotional impact, to inform better and support emotional elicitation, movie classification, and access;
2. Capture the emotional impact of movies, as felt and perceived by the viewers, for classification and indexing. This will help future users to gain more awareness about the emotional impact of watching movies and allow to access movies based on these emotions;
3. Analyze and classify movie content based on the processing of three information streams: subtitles, audio, and visual video content, with a special focus on emotional information;
4. Conceive and create effective interactive access, navigation and visualization of the movies based on their content and their emotional impact on viewers, with the potential to provide insights based on the information visualized, and the users’ emotional profiles, choices, and states.
REFERENCE: 2019-1-IT01-KA202-007541-CUP G35G19000070006 YEARS: 2019-2021
Healthcare clowning is an umbrella term for a wide range of activities clowns undertake in healthcare, ranging from children’s hospitals to old folks’ homes, playing near the bedside, or accompanying medical procedures.
In the past, I have conducted several research studies on the role of hospital clowns for children (See the publication section) and also supervised a Master's dissertation and a Ph.D. thesis on this topic (See the Team Section).
In addition, I was involved in the creation of the International Network of HEALTHCARE CLOWNING RESEARCH (HCRI+), which is dedicated to the scientific study of healthcare clowning. Our network of researchers included partners from Portugal (Iscte-IUL, Minho University), Spain, Brazil, Italy, and the Netherlands).
H-CRIN+’s mission is to promote the scientific study of healthcare clowning by testing multidisciplinary theoretical approaches that contribute to a better understanding of healthcare clowning interventions. It also aims to identify the best practices in this area.
Given my prior work in this domain, I was invited to be a member of the Consortium of the Health Care Clowning Pathways project, which is a Strategic Partnership supporting innovation in the Educational and Vocational Training Sector (VET) for Health Care Clowns.
GOALS: design, test, and validate an innovative training proposal, aimed at defining the competencies requested to work as Healthcare Clown in a sharing and transferable manner at the European level; proposing the European level training standards to obtain the qualified profession as Healthcare Clown, referring to ECVET and EQF Systems.
Ana Paiva (PI, INESC-ID), Patrícia Arriaga (CO-PI, ISCTE-IUL), Francisco Melo (INESC-ID), Iolanda Leite (Department of Robotics, Perception and Learning, Royal Institute of Technology, Suécia).
Investigate the role of emotions and adaptation in interactions between a robot and a group of users, contrasting to the typical one-robot oneuser paradigm in HumanRobot Interaction (HRI). Despite the complex social challenges that longterm HRI will soon bring, so far little is known about how perception and action selection systems, typically designed for one-to-one interactions, will perform in multiparty settings. We addressed the issue of social adaptation for robots in group settings focusing on computational modeling of emotions.
Emotions play a critical role in HRI for the establishment of social interactions between one robot and one user, in particular the role of empathy. The research was developed to verify whether similar results hold when aiming for longer-term social interactions, and when the robot is in the presence of a group of people.
Octavian Adrian Postolache (PI), Gabriela Postolache ((Instituto de Medicina Molecular e Instituto de Telecomunicações), Luísa Lima, Patrícia Arriaga, André P. Catarino (Univ. Minho), António Carmo, Mguel Dias Pereira (Escola Superior de Tecnologia, Instituto Politécnico de Setúbal), Pedro Girão (Instituto Superior Técnico, Universidade de Lisboa), Raul Silva Oliveira (Faculdade de Motricidade Humana, UL), Rita Cordovil (Faculdade de Motricidade Humana, UL), Rui Miguel Madeira, Vítor M. R. Viegas (Escola Superior de Tecnologia, Instituto Politécnico de Setúbal)
Information System (IS) offers much promise to improve information management for physiotherapists, to improve clinical and administrative reporting capabilities, operational efficiency, communication among health professionals, communication with patients, data accuracy, and the capacity to support clinical research. However, the key factor for the success of healthcare information system implementation and adoption is the engagement of the end-user.
Kinect Serious Game (patient)
A tailored information system was proposed in this project, enable to non-invasively/unobtrusively measure balance and movements, and to improve the effectiveness of physiotherapy based on serious games and augmented reality. Wireless body sensors network (WSN) were developed for better balance and movements’ characterization. Usability tests of tailored virtual environments for physiotherapy were carried out, as well usability tests of information systems developed for physiotherapy practice based on implemented WSN, serious game, and augmented reality.
The system aimed to improve the effectiveness of physiotherapy and to enable measuring additional constructs to enhance more comprehensive clinical reasoning processes (i.e., motor control, determinants for behavioral change, and patient engagement).
Playing with violence: emotional desensitization, empathy and helping behaviour toward victims of violence
Patrícia Arriaga (PI, Iscte-IUl), Francisco Esteves, Maria Benedicta Monteiro (Iscte-IUL), Augusta Gaspar (Universidade Católica) & Jaime Vila (Granada University, Spain).
PLAYING with Violence
The effects of violent computer games with virtual reality on physiological arousal, cognitions, affect and behaviour
Patrícia Arriaga (PI); Francisco Esteves;
& Paula Carneiro
GAMES WITH VIRTUAL REALITY
Interindividual differences in Emotional Empathy vs. the differential “visibility” of Emotions, and its social consequences
Augusta Gaspar (PI, Univ. Católica de Lisboa), Francisco Esteves (Mid Sweden University, Östersund, Sweden); Patrícia Arriaga, Frederico Oliveira de Almada (ISPA), Miquel Llorente Espino (University Ramon Llull, Barcelona, Espanha), Susana Fonseca (ISCTE/IUL, Portugal), Olga Feliu Olleta (Mona Foundation, Barcelona)
Emotions comprise observable behaviors that can be potentially decoded by bystanders as signals of the respective emotions. Although the efficiency of such decoding has been assumed to be high for a number of universal Emotional Facial Expressions (EFE), correct appraisals of EFE depend on a number of factors, including maturation and experience, physiognomy and individual dynamics of facial action, the familiarity between actor and observer and context. Perceiver’s individual traits are neglected factors in appraisal studies of emotional facial behavior. Possibly the most critical of these receiver traits - Emotional Empathy (an embodied component of Empathy) - is finally emerging in recent studies as influential in physiological reactions to and appraisals toward at least some facial expressions of emotion such as happy and angry posed prototypical facial expressions. It is yet to be determined if the ability to appraise EFE is independent of Emotional Empathy (EE) but we believe indeed that the later could be a critical predictor of correct appraisals of various emotions, including spontaneous EFE, which have been pointed as much harder than the posed ones to correctly appraise, and of related subsequent behaviors (eg. aggressive conducts). So, in the project, we explored such predictions. To the extent of our knowledge no study has purposefully looked into the dispersion of various psychobiological measures of EE (e.g., automatic mimicry assessed by facial EMG or behavior, and arousal, assessed via skin conductance) at early ages, although a large dispersion on such EE measures (facial EMG) became evident in a study with adults. This is important because widely used measures such as Empathy questionnaires can have scores affected by social desirability.
Because not all emotions are behaviorally conspicuous and some are much more easily and efficiently decoded (e.g. Joy-playful) than others, we believe EE can be the factor that allows some individuals to overcome this difference.
By studying a wider scope of emotions we expected to identify more grades of Emotional Empathy (EE), and to be able to establish an efficient protocol for early detection of Low and High EE in nonclinical populations. We addressed an old theoretical discussion of the evolution of signal and response, and specifically “what do facial expressions express?”. We have suggested earlier that individual variation in the display of emotion is related to cost of signaling and advantages of the actor in conveying information about intentions, proximal action tendencies, and other personal and interactive relevant information, and most of all to the “interactive” nature of the given emotion. We anticipate that receivers should be more convergent in their decoding of the interactive/visible emotions, whereas other emotions should reveal larger interindividual differences. This is in line with the Behavioral Ecology view of EFE, which has been advocated both in the Emotion Psychology and Ethology literature, contending the existence of a link between emotion expression and appraisal, specifically in the case of the prototypical EFE. We aim to investigate if and to what extent EE allows highly empathic individuals to perform better overall at decoding perceived emotional stimuli, especially at overcoming the “low visibility” of certain EFE. With this approach, we contributed to the discussion of the co-evolution of emotional signals and decoding ability.
Control of cognitive and emotional processing of faces the frontal theta rhythm
Alberto Leal (PI, Centro Hospitalar Psiquiátrico de Lisboa), Patrícia Arriaga, Ricardo Lopes (CADIn), & Francisco Esteves (Mid Sweden University, Östersund, Sweden),
The visual processing of faces plays a major role in the inter-personal relationships in humans and, consequently, the brain dedicates a lot of resources to this task. Besides the contribution of multiple posterior brain cortical areas, there is functional connectivity to the frontal lobes, which is likely to be related to high-level functions such as attention, memory, emotion, error monitoring. Several studies have demonstrated significant functional covariances at a distance, which seem to implicate synchronization between the cortical rhythms of the different areas. The limbic theta rhythm has gained particular prominence as a possible synchronizing mechanism between distant areas, despite the fact that no strong causal relationship between this rhythm and the functions that are attributed to it has been demonstrated. Specifically, it has not been possible to demonstrate that the association between EEG brain rhythms and the information processing taking place in the same cortical area is a causal one, not an epiphenomenon. To give an experimental response to the last point, studied the functional connectivity between the frontal lobe and the posterior brain areas
involved in the visual processing of faces, with a special emphasis on the theta rhythm. Experimental modulation of the frontal lobe theta rhythm was performed using TMS, and the resulting effects in the processing of faces determined. The effect of placebo will be controlled by using a similar protocol but with sham TMS. We aim to characterize the role played by the frontal lobe theta rhythm in several aspects of the visual processing of faces.
Psychophysiological effects of human PHEROMONES
Francisco Esteves (PI, Mid Sweden University, Östersund, Sweden), Patrícia Arriaga, Mats Olsson (Karolinska Institutet, Sweden); Johan Lundström (Karolinska Institutet, Sweden); Pedro Barbas Albuquerque (Minho University); Maria Benedicta Monteiro; Paula Carneiro
Pheromones are airborne chemical signals produced by an individual of a species that trigger neuroendocrinal responses underlying behavior, or development in another individual of the same species. Although their existence is well documented in many animal species, the empirical evidence for their existence in humans is scarce and controversial. Nevertheless, few examples of studies showed some effects of human pheromones. For example Stern and McClintock (1998) showed that human axilatory sweat from female donors could change the menstrual cycle of women exposed to that sweat, depending on the phase of the menstrual cycle of the donors. Using a steroid that has been isolated from the human skin, androstadienone, Lundström, Gonçalves, Esteves & Olsson (2003), obtained positive mood changes in women after exposure to androstadienone. Lundström, Larsson, and Olsson (2001) also obtained a decrease in skin temperature in women exposed to androstadienone when compared to a control substance, but interestingly only in the presence of a male experimenter. Summing up, although human pheromones have been studied for some time, sound scientific proof of their effects is still needed.
Investigate the emotional effects of exposure to a putative human pheromone androstadienone using psychophysiological measures and verbal affective ratings. Several experiments were conducted to investigate the effects of androstadienone on: 1) non-verbal flirt behavior in a controlled social-interaction situation; psychophysiological measures (heart rate, skin conductance, startle reflex modulation) and mood changes.
Psychophysiology of EMOTIONS
Psychophysiology of emotions: Nonconscious learning
Francisco Esteves (PI, Mid Sweden University, Östersund, Sweden),
Patrícia Arriaga, Paula Carneiro, & Anders Flykt (Mid Sweden University, Östersund, Sweden)
On KAMA MUTA, Being Moved and Crying
Beate Seibt (University of Oslo), Thomas Schubert (University of Oslo), Alan Fiske (University of California, Los Angeles), Janis Zickfeld (Aarhus University, Denmark), Patrícia Arriaga... et al.
Watch this video by VOX in which the topic of crying (and its interplay with being moved) is presented by the main researchers working in this field
Recent publications include:
Zickfeld, J. H. et al.. (2021). Tears Evoke the Intention to Offer Social Support: A Systematic Investigation of the Interpersonal Effects of Emotional Crying Across 41 Countries. Journal of Experimental Social Psychology. https://doi.org/10.1016/j.jesp.2021.104137
Zickfeld, J. H., Arriaga, P., Santos, S. V., Schubert. T., & Seibt, B., (2020). Tears of joy, aesthetic chills, and heartwarming feelings: Physiological correlates of Kama Muta. Psychophysiology, 57 (12). https://doi.org/10.1111/psyp.13662
Zickfeld, D. H., Schubert, T. W., Seibt, B., Blomster, J. K., Arriaga, P., et al. (2019). Kama Muta: Conceptualizing and measuring the experience of being moved across 19 nations and 15 languages. Emotion, 19 (3), 402-424. https://doi.org/10.1037/emo0000450
Seibt, B., Schubert, T. W., Zickfeld, J. H., Zhu, L., Arriaga, P., Simão, C., Nussinson, R., Fiske, A. P. (2018). Kama Muta: Similar emotional responses to touching videos across the US, Norway, China, Israel, and Portugal. Journal of Cross-Cultural Psychology, 49(3) 418–435. Https://doi.org/10.1177/0022022117746240.
Arriaga, P., Murteira, C., & Oliveira, R. (2019). Adults’ responses to children’s crying after a moral transgression. The Spanish Journal of Psychology, 22, e15, 1–10. https://doi.org/10.1017/sjp.2019.21