The Impact of Background Stimuli on the Perception of Fear in Facial Expressions

Inside Out—an animated film directed by Pete Doctor— portrayed a girl named Riley and her struggle with a relocation through the interaction of five characters representing several basic emotions: joy, sadness, anger, disgust, and fear (Keltner & Ekman, 2015). Each of these characters was associated with a distinctive color (e.g., sadness’ color was blue and anger’s color was red; Keltner & Ekman, 2015). The current study examined whether a true association exists between emotion and color by exploring associations between emotions (e.g., fear) and background stimulation.

Emotion is a response to a situation, under the influence of an individual’s personality, which triggers facial expressions, physiological (sympathetic or parasympathetic) changes, and various behavior patterns (VandenBos, 2007). Ekman (1992) initially categorized the basic emotions as happy, sad, angry, surprise, fear, and disgust. Experience of a basic emotion is often communicated through facial expressions—a form of nonverbal communication created by movement of the facial muscles (VandenBos, 2007).

Fear was a focus of the current study as this emotion and accompanying facial expressions are key in social interactions (Frith, 2009). Specifically, fear is defined as the feeling present when threat or danger is imminent (VandenBos, 2007). Fear triggers emotional arousal that can change one’s physiology, for example, by activation of the sympathetic nervous system (VandenBos, 2007), thus, fearful facial expressions are important for communicating to others about potential threats (Beck, Cardini, Ladavas, & Bertini, 2015).

Ciotti (2014) explored the effect of branding and how color impacts marketing in business settings. Appropriate color choices in association with particular environments, moods, and emotions were discussed as crucial for strengthening the effects of persuasive materials (Ciotti, 2014). Ciotti (2014) discussed examples of how colors heighten certain moods; for example, green enhances calm atmospheres and brown conveys a sense of ruggedness. Background stimuli may, indeed, influence individuals in fields such as business, art (visual, performances, and music), and/or therapy. The practical applications of color’s effect on mood research is extensive.

Byrnes (1983) studied associations between different colors and selected positive (e.g., happiness, love, hope, and honesty) and negative (e.g., anger, sadness, fear, pain, and death) emotions. Children in grades 4-6 participated and were presented with positive and negative emotion words (Byrnes, 1983). They were asked to indicate which color they most associated with the word (Byrnes, 1983). Generally, bright colors were associated with positive emotions and dark colors were associated with negative emotions (Byrnes, 1983). Specifically, for example, Byrnes (1983) found the color red was highly associated with the words ‘love’ and ‘anger’ and the color black with the word ‘death.’ Byrnes (1983) replicated this procedure with adult participants and produced similar results.

Young, Elliot, Feltman, and Ambady (2013) studied angry facial expressions and their associations with color. Young et al. (2013) paired different background colors and different facial expressions. The experiment was divided into two separate but related experiments to detect accurate associations between angry facial expressions and red (Young et al., 2013). In the first experiment, angry and happy facial expressions were chosen and paired with red, green, and gray backgrounds (Young et al., 2013). After each exposure to red, green, and gray background colors, participants categorized happy or angry faces quickly and accurately (Young et al., 2013). Similarly, the second experiment contained the same outcomes except the angry and fearful facial expressions were examined and a blue background replaced green (Young et al., 2013). The results of both experiments showed angry facial expressions with red backgrounds were identified more quickly than when happiness or fear was associated with red (Young et al., 2013). These studies indicated an association between emotion and color and, specifically, between fear and the color red.

Furl, Henson, Friston, and Calder (2013) examined activation of a specific area of sensitivity in dorsal temporal and ventral temporal fusiform areas when viewing dynamic and static facial expression images including disgust, happiness, and fear. Participants rated the emotional intensity of each facial expression and Furl et al. (2013) found the ventral fusiform face area displayed higher sensitivity to fearful facial expressions presented in static and via still images.

Since previous research has shown evidence for associations between emotion and color (e.g., Byrnes, 1983; Young et al., 2013) or textures (Furl et al., 2013), the current study explored which colors (e.g., blue and black) or textures (e.g., static) might impact fear ratings of facial expressions. These backgrounds were carefully selected and tested based on previous empirical research (Byrnes, 1983; Furl et al., 2013; Young et al., 2013). Blue was selected as a background because, while not a focus of their study, Young et al. (2015) showed the color blue produced quicker reaction times with fearful facial expressions compared to red and gray. The color black was chosen as a background based on the results of the Byrnes (1983) study showing an association between the color black and fear. The static texture was chosen as a background based on Furl et al.’s (2013) research showing the ventral fusiform face area displayed sensitivity to still images containing fearful facial expressions on static backgrounds.

In the current study, images of six people (three female and three male) were selected from the Chicago face database (Ma, Correll, & Wittenbrink, 2015) conveying four different facial expressions (fearful, happy, angry, and neutral) and were used to test associations between fearful facial expressions and background color (blue, black, and static). Participants observed each facial expression with each background stimulus option and rated the amount of fear present in the facial expression on 9-point Likert scale. It was hypothesized background stimulation would influence fear ratings of fearful facial expressions viewed as still images.

Methods

The current study measured the impact of background stimuli on the perception of fear in facial expressions. Participants rated the intensity of fear in facial expressions on a 9-point Likert scale after viewing images from four different facial expression categories (fearful, angry, happy and neutral) paired with three backgrounds (static, blue, and black).

Participants

30 undergraduate students from a small liberal arts college were selected to participate in the current study using convenience sampling. Individual testing was employed with each participant. Participants were screened for color deficiencies using the Ishihara plates (Ishihara, 1972). Fourteen Ishihara plates were shown, one plate at a time (3 sec per plate), and participants were asked to report the number they saw. Participants noted their Ishihara plate number response on a half sheet of paper coded by plate number. Ishihara test results indicated all participants were free from color vision deficiencies so all proceeded to the next phase of the study. Each participant was exposed to all of the facial expression and background stimulation combinations in this within-subjects design. Participants were randomly assigned to view one of three image binders with the images randomized and counterbalanced in order. Participants received research credit for their participation.

Procedure

Participants who provided informed consent and showed no color deficiencies were randomly assigned an image binder containing 72 pages with each facial expression (fearful, angry, happy, and neutral) paired with the three background stimuli (static, blue, and black; see Appendix A). Participants were, then, provided a Likert rating sheet containing seventy-two 9-point Likert scales, were instructed about how to use the Likert scales, and that their rating for each image should be noted on this sheet. Participants had unlimited time to view each page in the image binder and rate the level of fear expressed in each. After all responses were notated, the Likert rating sheet was collected and participants were thanked for their time.

Materials

Materials included 90 consent forms. Ishihara color deficiency test (Ishihara, 1972) responses were recorded on half sheets of paper with 14 numbered spaces, one corresponding to each test plate. Alpha-numeric labels indicated image binder assignment (A, B, or C) and a participant number not associated with their identity and were included on all documents. Three image binders, each including randomized and counterbalanced orders of the four facial expressions (across gender) paired with each of the three forms of background stimulation were created. Thirty Likert rating sheets with 72 (one for each image) pre-printed and labeled Likert scales were used to obtain the participant’s rating of the amount of fear in the presented facial expression and were labeled from 1 (not fearful) to 9 (fearful).

Six people with four different facial expressions (i.e., fear, anger, happy, and neutral)— three Caucasian female faces and three Caucasian male faces—were paired with all three background stimuli (static, blue, and black; see Appendix A). Six images existed for each emotion and images were presented in color. Each image binder page was placed in ‘Samsill Non-Glare’ sheet protectors and organized into Wilson Jones round ring view binders. The faces shown were collected from the Chicago face database (Ma, Correll, & Wittenbrink, 2015). The static background image was collected from Google images and adjusted to high quality by using Reshade Image Enlarger 3.0. Two of the background stimuli, blue (color model: HSL, 170/255/128) and black (color model: HSL, 170/0/0), were created using Microsoft Office PowerPoint. The electronic device used to create the materials in the current study was an Acer (R3-71/2015) laptop.

Results

The current study examined the effect of background stimulation on ratings of fearful facial expressions viewed as still images. We hypothesized background stimulation would impact fear ratings of fearful facial expressions viewed as still images. The null hypothesis would be that of no difference in fearful ratings by background stimulation condition.

Fear scores were sorted by background stimulation (static, blue, and black) and facial expression type (fear, anger, happiness, and neutral). An Anderson-Darling test (Anderson & Darling, 1952) showed normality of variance, thus, a repeated measures ANOVA was selected to analyze fear ratings by background stimulation condition. The significance level (α) was set at .05. Finally, a paired samples t-test (α = .05) was used to validate that fearful images were rated as more fearful than neutral images.

Facial Expression Analyses

Fear ratings for each background (static = 4.68; blue = 4.38; black = 4.41) showed no significant difference (p = .103) with fearful facial expressions. For angry facial expressions, fear ratings for each background (static = 3.51; blue = 3.48; black = 3.49) also showed no significant difference (p = .981). Happy facial expressions showed a significant difference (p = .036*) in fear ratings for each background (static = 1.28; blue = 1.45; black = 1.48). The Tukey (Tukey, 1949; multi-comparison) test showed the significant difference in fearful ratings existed between static and black backgrounds. Neutral facial expressions showed no significant difference (p = .244) in fearful ratings by background (static = 2.51; blue = 2.35; black = 2.54).

Fearful vs. Neutral Fear Ratings: Validity Test

A paired samples t-test (p ≤ .0001*) was conducted on the fearful ratings of all fearful facial expression images and all neutral facial expression images from the Chicago Face Database (Ma et al., 2015) to ensure participants did see the fearful facial expressions as more fearful than the neutral facial expressions. This finding of a significant difference in the amount of fear perceived in the fearful and neutral facial expression images validates the ratings provided by Ma et al. (2015) and the existence of the intended difference between key variables of interest in the current study.

Discussion

The current study examined whether or not different backgrounds influence fear ratings of facial expressions viewed as still images. The alternative hypothesis was that background stimuli would influence fear ratings of facial expressions (i.e., fearful, angry, happy, and neutral) viewed as still images and the null hypothesis was that there would be no difference. The hypothesis focused on fearful facial expressions, but other emotions (angry, happy, and neutral) were also included.

Three types of facial expressions (fearful, angry, and neutral) did not show a significant difference in fear ratings. There is an interesting exception with happy facial expressions that we recommend be examined in more depth with future research. Based on the general pattern of findings, however, we fail to reject the null hypothesis in favor of the alternative and conclude background stimuli do not influence fear ratings of fearful facial expressions viewed as still images. Validity of the difference in fearful expression present in the images obtained through the Chicago Face Database (Ma et al., 2015) was also verified.

The results of the current study do not align with Byrnes (1983), Young et al. (2013), or Furl et al. (2013). Byrnes’ (1983) study found positive emotions were associated with bright colors and negative emotions were associated with dark colors (e.g., children associated the color black to terms such as death and fear). The current study did not support the association between the color black and fear. Young et al. (2013) focused on the strong association between red and anger, but also found blue paired with fearful facial expressions facilitated response times. The current study did not replicate this pattern of results. The study conducted by Young et al. (2013) employed response time for their dependent measure, whereas the current study used 9-point Likert scale fear rating score. Response time indicates association rather than intensity. Blue and fearful facial expressions can be quickly identified together, but are not strongly associated with Likert ratings of fear. Furl et al. (2013) focused on the neurological aspect of the fear mechanism through perceiving static fearful faces. The ventral temporal fusiform face area may still display sensitivity while perceiving fearful facial expressions with static background stimuli, but it may not be associated with the intensity level of fearful perception from an image. While Byrnes (1983), Young et al. (2013), and Furl et al. (2013) indicated a potential association of each background with fearful facial expressions, the current study indicates the intensity of association is not strong.

In retrospect, the current study would have benefitted from an extension of the dependent measure to include the ratings of other emotions presented as facial expressions (i.e., angry, happy, and neutral). At this time, conclusions are limited to an exploration of fear, but may be relevant outside this basic emotion.

The current study considered the perception of fear in facial expressions for an American population from a small liberal arts college. The particular population under consideration was limited in diversity. It is possible the results might have been different if the current study were replicated with a different population. Nisbett, Peng, Choi, and Norenzayan (2001) studied varying cultures and systems of thought between Western and Eastern cultures finding that Westerners tend to be more analytic and Easterners more holistic. There was also evidence of different naïve metaphysical and social systems across Western and Eastern cultures which could differentially impact member’s cognitive performance on various tasks (Nisbett et al., 2001). It is recommended future research examine this cultural difference as differences in perception precipitated by differences in cognition have been shown to exist (Colzato, van Beest, van den Wildenberg, Scorolli, Dorchin, Meiran, Borghi, & Hommel, 2010).

---

References

Anderson, T. W., & Darling, D. A. (1952). Asympototic theory of certain ‘goodness-of-fit’ criteria based on stochastic processes. The Annals of Mathematical Statistics, 23, 193-212.

Beck, B., Cardini, F., Làdavas, E., & Bertini, C. (2015). The enfacement illusion is not affected by negative facial expressions. Plos One, 10(8), 1-15. doi:10.1371/journal.pone.0136273

Byrnes, D. A. (1983). Color association of children. The Journal of Psychology, 113, 247-250.

Ciotti, H. (2014, August 20). Color psychology: How colors influence the mind. Retrieved from https://www.psychologytoday.com/blog/habits-not-hacks/201408/color-psychology-how-colors-influence-the-mind

Colzato, L.S., van Beest, I., van den Wildenberg, W.P., Scorolli, C., Dorchin, S., Meiran, N., Borghi, A.M., & Hommel, B. (2010). God: Do I have your attention? Cognition, 117(1), 87-94. doi:10.1016/j.cognition.2010.07.003

Ekman, P. (1992). An argument for basic emotions. Cognition and Emotion, 6(3/4), 169-200.

Frith, C. (2009). Role of facial expressions in social interactions. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1535), 3453-3458. doi:10.1098/rstb.2009.0142

Furl, N., Henson, R. N., Friston, K. J., & Calder, A. J. (2013). Top-down control of visual responses to fear by the amygdala. The Journal of Neuroscience, 33(44), 17435-17443. doi:10.1523/JNEUROSCI.2992-13.2013

Ishihara, S. (1972). Tests for colour-blindness (14 Plates ed.). Tokyo, Japan: Kanehara Shuppan.

Keltner, D., & Ekman, P. (2015, July 3). The science of ‘Inside Out.’ New York Times. Retrieved October 26, 2015 from http://www.nytimes.com/2015/07/05/opinion/sunday/the-science-of-inside-out.html?_r=0

Ma, D. S., Correll, J., & Wittenbrink, B. (2015). The Chicago face database: A free stimulus set of faces and norming data. Behavior Research Methods, 47, 1122-1135.

Nisbett, R. E., Peng, K., Choi, L., & Norenzayan, A. (2001). Culture and systems of thought: Holistic versus analytic cognition. Psychological Review, 108(2), 291-310. doi:10.1037/0033-295X.108.2.291

Tukey, J. W. (1949). The simplest signed-rank tests. Princeton University Statistical Research Group MemoReport, No. 17.

VandenBos, G. R. (2007). APA dictionary of psychology. American Psychological Association, 369.

Young, S.G., Elliot, A. J., Feltman, R., & Ambady, N. (2013). Red enhances the processing of facial expression of anger. Emotion, 13(3), 380-384. doi:10.1037/a0032471

---

Acknowledgements

Special acknowledgement goes to Steve Ellsworth, Ph.D., for advising related to the statistical procedures used in the analysis of data that originated from the current study.

This publication was made possible by a grant from the National Institute of General Medical Sciences (GM103440) from the National Institutes of Health. The contents are solely the responsibility of the authors and do not necessarily represent the official views of NIH.

---

Appendix