Friday, July 13, 2012
The McGurk Effect Weaker in Japan: Nacalianism Defeated?
The McGurk Effect (McGurk & MacDonald, 1976) refers to the tendency for visual-speech (lip movements etc) to effect the sounds that we hear. As shown in the above video, Japanese people area also susceptible to the effect but, as comparative research (Sekiyama & Tohkura, 1993; Sekiyama, 1997) has shown, Japanese and Chinese are more auditory and less visual than Americans! The Nacalian theory - that Japanese are visio-occular-centric as opposed to logo-phono-centric - is thus disproved, game, set, match? Very possibly.
Some possible explanations for the weaker McGurk effect among Japanese include
1) The Japanese are more auditory than Westerners (c.f. Ishii, Reyes, & Kitayama, 2003)
2) The Japanese move their face less when speaking due to the nature of their language (unlikely due to experiments comparing spanish and Japanese)
3) The language has more tones (Sekiyama, 1997)
4) The Japanese move their faces less during speaking for cultural reasons.
5) The Japanese watch faces less, or in other words they are less "face-aversive" (Sekiyama, 1997) for cultural reasons.
I lean towards the latter two explanations - both of which seem true - but neither bode well for Nacalian interpretations of Japanese culture. Why do the Japanese move their faces less, and look at others faces less if they are really a visually sensitive culture?
Possible reasons for the apparent lack of attention to, and lack of facial information include
1) That the Japanese are paying attention to a different part of the face - eyes - rather than mouths, as demonstrated by Yuki's research (Yuki, Maddux, & Masuda, 2007).
2) Relatedly, that the Japanese do communicate with their face but not linguistically, and that their face is reserved for displaying purely visual, not language-supplemental, information. Consider for instance the close ups of people eating food shown so often on Japanese TV, or the insets of Japanese television personalities faces showing reacting non-verbally to the image on the screen.
3) That the Japanese identify more strongly with their faces and wish to maintain an unchanging persona or mask (Watsuji, 1937) for the sake of visual self-consistency. I claim that the Japanese are into visual self-consistency, and even a visual self-enhancement of sorts, in a big way.
Ishii, K., Reyes, J. A., & Kitayama, S. (2003). Spontaneous Attention to Word Content Versus Emotional Tone. Psychological Science, 39–46. Retrieved from http://php.scripts.psu.edu/users/n/x/nxy906/COMPS/indivdualismandcollectivism/culture%20lit/to%20print/Kitayma2003threecultures.pdf
McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Retrieved from http://www.nature.com/nature/journal/v264/n5588/abs/264746a0.html
Sekiyama, K. (1997). Cultural and linguistic factors in audiovisual speech processing: The McGurk effect in Chinese subjects. Attention, Perception, & Psychophysics, 59(1), 73–80. Retrieved from http://www.springerlink.com/index/E867017U7J708970.pdf
Sekiyama, K., & Tohkura, Y. (1993). Inter-language differences in the influence of visual cues in speech perception. Journal of Phonetics. Retrieved from http://psycnet.apa.org/psycinfo/1994-12869-001
Shigeno, S. (1998). Cultural similarities and differences in the recognition of audio-visual speech stimuli. Fifth International Conference on Spoken Language Processing.
Spencer-Rodgers, J., Boucher, H. C., Mori, S. C., Lei Wang, & Kaiping Peng. (2009). The Dialectical Self-Concept: Contradiction, Change, and Holism in East Asian Cultures. Personality and Social Psychology Bulletin, 35(1), 29–44. doi:10.1177/0146167208325772
Thompson, M., & Hazan, V. (2010). The impact of visual cues and lexical knowledge on the perception of a non-native consonant contrast for Colombian adults. Retrieved from http://discovery.ucl.ac.uk/19505/
Yuki, M., Maddux, W., & Masuda, T. (2007). Are the windows to the soul the same in the East and West? Cultural differences in using the eyes and mouth as cues to recognize emotions in Japan and the United States. Journal of Experimental Social Psychology.
和辻哲郎. (1937). 面とぺルソナ. 岩波書店.
This blog represents the opinions of the author, Timothy Takemoto, and not the opinions of his employer.