Vision and Visual Arts, 2018 Multisensory Perception
and Visual Arts
JunYun Zhang
PKU
Gouache by Anne Adams entitled ‘Unravelling Boléro ’, showing emergent transmodal creativity comprising auditory to visual
transformation, 6 years before her PPA symptoms began.
视听觉跨模态的整合
Joseph‐Maurice Ravel
1875－1937
由小鼓所敲出的《Bolero》節奏，由始至終
地貫穿全首樂曲
Synesthesia 联觉(Greek, syn= together + aisthesis= perception)
A neurological phenomenon
in which stimulation of one sensory pathway leads to automatic, involuntary experiences in a second sensory pathway
联觉是一种具有神经基础的感知状态，表示一种感官刺激或认知途径会自发且非主动地引起另一种感知或认识。具有联觉的人通常被称作联觉人(synesthete)。
Types of Synesthesia
–graphemes (letters/numbers)→color –days of week/months of year →color –time units →spatial location
–sounds →color
–smells →color
–words →tastes
–vision→sounds
1/23 people have some type of synesthesia (Simner, et. al. 2005)
people with synesthesia
grapheme synethete, subject W.O.
Palmeri et al., 2002
grapheme synethete, subject W.O.
Palmeri et al., 2002
We all have synesthesia?
Bouba‐kiki effect 命名，这个是kiki bouba
K?hler, 1929
Art by synesthetes
Art by synesthetes
Wassily Kandinsky, 1866‐1944
Composition VI Wassily Kandinsky 1913
I saw all my colours in spirit, before my eyes. Wild, almost
crazy lines were sketched in front of me.
灵魂的颜色
Composition VII Wassily Kandinsky
1913
Colour is the keyboard. The eye is the hammer. The soul is
the piano with its many strings.
Vision
Carol Steen,
1996
“多年前的某天，我在做针灸治疗。
躺在那儿时，我闭上眼后看到眼前
柔和的黑色背景中，有很亮的红色
将它刺穿并在天鹅绒般的浓烈黑暗
中成形。一开始的红是一小点的强
烈颜色，然后急速的扩大。我又看
见了绿色的形状在红色中出现并在
红跟黑的领域中移动。这是我第一
次将我看到的视觉印象如实的画下
来...”
触觉引发的视觉
Clouds Rise Up Carol Steen, 2004‐05
Carol Steen, Runs Off in Front, 2003
“这幅作品基于我听到Carlos Santana的一首名为Adouma的歌曲时产生的特别多彩的光影而创作的。我看到的颜色是光的颜色，而不是颜料的颜色。当我绘制这些移动的颜色时，我一遍又一遍地播放这首歌。”
Cello Music
Marcia Smilack
Dancing To Ravel Marcia Smilack
Red Buoy 1
Marcia Smilack “当渔船在下午四点返回时，我看到红色浮标上在海面的倒影。这时候我听到一种声音，不是汽笛声，而是一种无比奇妙的声音……”