Maniscript
N200-speller using motion-onset visual response
Fei Guo, Bo Hong, Xiaorong Gao and Shangkai Gao
Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, 100084,
People’s Republic of China
E-mail: hongbo@https://www.doczj.com/doc/c017570655.html,
Abstract
Objective: This study presents a brain-computer interface (BCI) named ‘N200-speller’. Motion stimuli, such as moving bar, are displayed for inducing the motion-onset visual response.
Methods: Brief motion of chromatic visual objects embedded in the 36 onscreen virtual buttons is employed to evoke motion-onset specific N200 component. The user focuses attention on the button labeled with the letter to be communicated and performs color recognition task. The computer determines the target letter by identifying the attended row and column respectively. Support vector machine (SVM) is used in the target detection procedures of the BCI system.
Results: Eight subjects participated in this study. The characteristics and performance of the N200-speller were compared with the classical P300-speller BCI. Four of the eight subjects show a remarkably higher accuracy using the N200-speller than using the P300-speller, given the same number of trials considered.
Conclusions: Besides the advantage of low contrast and poor luminance tolerance, the proposed N200-speller can also achieve a comparable or even better performance than the P300-speller.
Significance: A novel motion-onset paradigm is proposed and assessed as an effective tool for BCI speller application.
Keywords
Brain computer interface, motion-onset visual evoked potential, N200, P300, support vector machine
1. Introduction
Brain computer interface, as a neural prosthesis, operates by allowing a subject to communicating a choice among a set of items (Wolpaw et al., 2002; Lebedev et al., 2006; Schwartz et al., 2006). For the purpose of communication, an important application is the BCI speller, such as P300-speller, by which the subject can spell out a message when successively choosing from among the alphabet via his or her neural signal. Previous studies have demostrated that the P300-based BCI, can serve as an effective speller device (Farwell and Donchin, 1988, 2000; Krusienski et al., 2006; Sellers et al., 2007). Farwell and Dochin described the P300 BCI as a system that exploited an oddball paradigm to select among a sequence of characters. The oddball paradigm was designed with randomly intensifying either a row or column of a 6 by 6 matrix that was displayed continually. The user should focus attention on one of the 36 cells of the matrix. The row and column intensification that intersect at the attended cell represent the target stimuli, which occur with a probability of 1/6 and will elicit a P300 response. Farwell and Donchin demonstrated the feasibility of this version of BCI named ‘P300-speller’. Up to the present, P300 related BCI has focused on feature extraction and classification procedures to enhance the information transfer rate and applicability of the system (Xu et al., 2004; Serby et al., 2005; Thulasidas et al., 2006; Zhang, 2008). However, one problem associated with the P300-speller is that this paradigm uses visual stimuli of intensification, which easily induces fatigue and discomfort of the BCI user, especially in the case of long time use, and thus limits the practical BCI application for clinical or home use. This problem also exists in the flash or
pattern reversal VEP based BCI. Consequently, seeking for new BCI modality has become a crucial need in high performance and user-friendly BCI system development. Based on our previous work on a visual motion-onset BCI paradigm (Guo et al., 2008), here we proposed a new type of BCI speller, which employs visual response from the dorsal pathway of the visual system and therefore allows us to use non-flash visual stimuli.
Motion-onset paradigm features the advantage of low luminance and low contrast required by the stimuli to elicit prominent motion visual evoked potentials (mVEP). Some mVEP studies indicate that a very low luminace of 0.003cd/m2 and a contrast in the range of 2% are sufficient to evoke mVEP (Kuba and Kubova, 1992; Dodt E, Kuba, 1995; Kubova et al., 2004). Motion-onset VEP is composed of three main peaks, P1, N200 and P2, in which the N200 is the most distinct response to motion onset. The motion-specific N200, with a latency around 200 ms, seems to be generated from the extrastriate temporo-occipital and associated parietal cortical areas (Kuba and Kubova, 1992; Probst, 1993; Skrandies, 1998). Employing the motion-onset VEP, Guo et al. reported a design of a novel BCI using the motion-onset paradigm, which provided a reliable way of determing the subject’s choice of visual target. In their five-class configuration, robust mVEP was evoked by using a small visual field (1.24° * 0.76°) which allows for multiple targets to be displayed in the BCI application. The accuracy of offline target detection confirmed that the motion-onset paradigm is feasible for a BCI system. In the current study, we design a visual BCI named ‘N200-speller’, which shares similar spatial and temporal configuration as the P300-speller but is based on different neurophysiological foundation. The primary difference between these two paradigms is that the N200-speller employs the motion N200 which is an exogenous component, while the P300-speller uses the P300 which is an endogenous component.
The purpose of the present study is to evaluate the motion-onset based ‘N200-speller’ as a promising BCI paradigm. Motion stimuli are embedded into the 6 rows by 6 columns onscreen alphabet buttons for user selection. When the subject gazes at the desired button and performs recognition tasks, the motion stimuli corresponding to the target button will elicit motion-onset visual evoked potentials, which are used to determine the subject’s intention of choice. The spatiotemporal patterns of the motion-onset response are investigated by the recorded EEG data from eight subjects. The motion-specific N200 component from occipito-temporal and parietal electrodes is selected as salient marker of the brain responses to the target in the N200-speller. Finally, the performance of the N200-speller is assessed in comparison with the maturely developed P300-speller. Given the same number of trials considered, four of the eight subjects achieve a higher accuracy using the N200-speller than using the P300-speller, which suggests that the proposed N200-speller can achieve a comparable or even better performance than the P300-speller.
2. Methods
2.1. Experimental setup
The visual stimuli are displayed on a 17-inch LCD monitor with 60 Hz refresh rate and 1280×1024 resolution, being viewed from a distance of 50 cm. Figure 1 shows the spatial configuration of the N200-speller paradigm while Figure 2 displays the temporal protocol of visual motion in the N200-speller.
2.1.1. Visual stimuli and task design of the N200-speller
Figure 1 gives a demonstration of the N200-speller interface, which is composed of 36 virtual buttons in the organization of 6 rows by 6 columns. In each button, a vertical bar with a height of 0.66° visual angle appears (motion onset) at the right border of a vacant rectangle and moves leftward at the velocity of 3.10 deg/s before it disappears (motion offset), forming a brief motion stimulus. The entire process of onset, motion and offset takes 140 ms. The color of the vertical bar, which might be red, green, blue, purple, yellow or brown, is randomly
designated by the stimulus software in such a manner that the colors of the six bars in the same row or the same column are different from each other. The random color is designed in order that the subject can perform color recognition task to enhance attention on the desired target. The chromatic moving bar and the rectangle inside which the bar appears compose a virtual button. A total of 36 such cells, consisting of 26 letters and 10 digits, form a virtual keyboard on a white background.
Figure 2 displays the timing protocol of the stimuli presentation in the N200-speller paradigm. The motion stimuli in each of the 36 cell buttons occur in a random order by row, or by column. A trial is defined as a complete series of motion stimuli in all the rows and columns of the keyboard. Each trial consists of 12 stimuli in a random sequence dedicated to the six rows and six columns respectively. The stimulus onset asynchrony (SOA) between two motion stimuli is 200 ms. Thus the duration of a trial is 2400 ms, and the interval between two trials is 400 ms.
During one acquisition period of 15 trials (one block), the participant is instructed to attend to a specific desired button (called ‘target’) and hold immovable sightline. The experimental task is to gaze at the target button and mentally recognize the color of the moving bar appears in the attended button. Each session consists of 6 blocks, and totally two sessions of stimuli from the N200-speller are presented to each subject.
Figure 1. Spatial configuration of the N200-speller: 26 letters and 10 digits are arranged in the six rows and six columns of the virtual keyboard. A row or column of vertical bars appears and moves leftward for 140 ms every 200 ms. The color of the vertical bar, which might be red, green, blue, purple, yellow or brown, is randomly designated in such a protocol that the colors of the six bars in the same row or the same column are different from each other.
°
Figure 2. Time course of visual motion events using SOA of 200 ms. Each block consists of 15 trials. Each trial is subdivided into twelve stimulus periods dedicated to the six rows and six columns of the virtual keyboard. In one trial the row events always occur before the column events. The row events or column events appear in a random order. Only row events (R1~R6) are shown in the figure since column events have the same temporal configuration. Take R5 as the target row for example, an ‘N200’ component should be elicited when the moving bar appears in the fifth row.
2.2. Subjects and Data recording
Eight volunteers (six male and two female, aged from 20 to 28) were recruited to participate in our studies. All subjects gave informed consent and were paid for their participation. Each subject showed normal or corrected to normal eyesight, with no history of clinical visual disease.
Subjects sat on a comfortable armchair in a shield room.The electroencephalogram (EEG) was recorded from 30 surface electrodes using a standard EEG cap (Electro-Cap International, Neuroscan, USA) following the international 10–20 system. A linked-mastoids reference was used. All electrode impedances were reduced to 5 k ? before data recording. EEG and EOG signals were sampled at 200 Hz, and band-pass filtered between 0.05 and 30 Hz (SynAmps2, Neuroscan, USA). Trigger events were acquired simultaneously. Data was collected continuously and analyzed off-line.
2.3. Data analysis
2.3.1. Analysis of evoked potentials
A total of 360 trials (15 trials /block, 12 blocks from the N200-Speller paradigm and another 12 blocks from the P300-speller paradigm) were registered from each subject. The data was digitally filtered using a bidirectional linear filter (pass band 1 to 10 Hz) that minimized DC drifts and power line interference. Single-trial EEG epochs were derived in association with each stimulus, beginning 200 ms prior to the stimulus onset and lasting for 1200 ms. Thus, each trial yielded 12 such epochs, each associated with a specific row and a specific column. Next, all epochs were baseline corrected with respect to the mean voltage over the 200 ms preceding motion onset. Finally data epochs were group averaged (target and non-target) through all trials from all subject to obtain grand average
R1
R2
R3
R4
R5
R6
results for the N200-speller and the P300-speller respectively. The latency and amplitude of the evoked potential components are defined as the following rules: the peak latency of the P300 was assessed as the time from stimulus onset to maximum peak within a 250-550 ms latency window, and the peak latency of the N200 was identified as the most negative point between 160 and 240 ms; the amplitude was measured as the activity value of the peak point. The peak latency and peak amplitude of each component was automatically calculated by the computer using the recorded EEG data.
2.3.2. Target Detection using SVM
To compare the performance of the N200-speller (and that of the P300-speller in comparison), we implemented offline target detection using Support Vector Machine. All data analyses were carried out using MATLAB (Mathworks, USA), libSVM (Chang and Lin, 2001) and EEGLAB (Delorme and Makeig, 2004)
Determining the presence or absence of the N200 features can be considered a binary classification problem. A support vector machine is designed to determine the hyperplane that maximizes the separating margin between the two classes. Each individual data set consists of two sessions, with the first session as a training set and the second session as a test set to simulate semi-online BCI applications.
Before classification, preprocessing procedures were performed to reduce the feature dimension. First, the EEG channels used for target detection were selected based on the scalp distribution of the N200 components. Generally the three-channel subset, containing P7, P3 and Pz electrodes roughly over the MT area, was selected for N200-speeller(Guo et al. 2008), while the subset of FCz, Cz and CPz was used for P300-speller according to the spatial pattern of the P300 (Krusienski et al.,2007). Then for each EEG channel in the subset, 1000 ms segment of data epoch following each stimulus-onset was extracted. To reduce the dimension of feature vector that will be used for classification, single-trial EEG epochs were filtered at 1–10 Hz and resampled at 20 Hz, yielding 9 points for the 100-500 ms period of each epoch. The resulting 9-point epochs were concatenated by channel for each stimulus, creating a single feature vector of 27 points. After preprocessing, each individual data set yielded two feature matrixes, serving as training set and test set respectively. The size of the feature matrix is 27 features * 180 stimuli.
The first feature matrix was employed to acquire the SVM model using five-fold cross validation. SVM models were trained for each subject. The training matrix contained equal number of ‘target’ vectors and ‘non-target’ vectors; amony every five ‘non-target’ vectors, one vector was randomly selected to match the number of the ‘target’ vector. The second matrix was designated as a test set to evaluate the target detection performance, using the 2-class SVM model derived from the first matrix. The method assumed that only the rows and columns containing the chosen button will elicit a detectable N200, while the others will not. Thus, the target row or column was selected with the feature vector which had the largest positive distance from the trained separating hyper-plane. Integration of the target row and target column confirms a target button in the visual interface.
As is generally the case for evoked potentials, the detection of the N200 (or P300) in the epoch following a single event is not reliable. While averaging over a few trials provides a more detectable pattern of the N200, the effectiveness of this procedure highly depends on the degree to which the target can be confirmed within a small number of trials at a relatively acceptable speed. Hence, it is necessary to determine the smallest number of trials needed to insure reliable detection performance of the BCI system. In this study we assessed the accuracy with which the target button was stimulated as a function of the number of trials considered for the detection.
3. Results
3.1. Spatio-temporal patterns of the evoked potentials in the N200-speller
The data epochs for the N200-speller were averaged over all trials registered from each of the eight subjects, for each of the electrodes recorded. Epochs associated with the target and non-target stimuli were averaged respectively. Figure 3 presents the grand average temporal waveforms and spatial characteristics of the evoked potentials elicited by the N200-speller paradigms.
Figure 3. The spatiotemporal patterns of the evoked potentials derived from the N200-speller. The red solid curve is the grand average waveform of the target stimuli; while each superimposed dash curve is the grand average waveform of each of the non-target stimuli. The upper figure and lower figure show the evoked potentials elicited by the row events and the column events respectively. The waveforms are the mean activity at electrode P7, which shows prominent N200 component; the amplitude topographic maps are plotted at 210 ms to reveal the spatial distribution of the N200.
Figure 3 display the spatiotemporal patterns derived from the N200-speller paradigm. Analysis of the grand average evoked potentials in response to target motion stimuli reveals a sharp negative deflection (motion-specific N200) peaked at 210 ms post motion-onset, followed by a broad positive deflection. The motion-specific N200 component has an unsymmetrical occipito-temporal and parietal topography, which corresponds well with
previous findings of the motion N200 (Kuba and Kubova, 1992; Probst et al., 1993; Skrandies et al., 1998). The topographic maps of the N200 component evidently reveal that the N200 amplitude differed between hemispheres. It is found that seven of the subjects show a left hemisphere dominance of N200 amplitude, while only one subject shows slightly higher amplitude in the right hemisphere. The left dominant spatial distribution is accordant with previous studies on mVEP (Kubova et al., 1990) . In addtion to the motion-specific N200, a broad positive deflection between 300~600 ms is also associated with the target stimuli, but absent for the non-target stimuli. We recognize this distinct component as Late Positive Complex (LPC), which is supposed to reflect the subject’s voluntary response to the target (Polich, 1990, 2007; Falkenstein et al., 1994; Dien et al., 2004). In our N200-speller paradigm, the experimental task of the subject is to attend a target button and voluntarily recognize the random color of the motion stimuli in that target button. This mental response of color recognition may account for the observed LPC. As also can be seen from figure 3, the waveform shows an oscillation of defection (smaller than 1.5 uv) every 200 ms, which is weaker than the N200 (around 5 uv) elicited by the target sitmuli. The detailed explanation of this 5 Hz oscillation will be discussed in section 4.1.
The comparison of responses to row and column events reflects that evoked potentials elicited by the row stimuli and the column stimuli are of the same temporal and spatial characteristics, except that the amplitudes of the N200 elicited by the column events are slightly larger. This might be attributed to the different illusion of brief motion for the row events and the column events respectively: when the motion stimuli appear in column there seems to be a more obvious vertical bar.
From observations above, the proposed N200-speller and the traditional P300-speller elicit dissimilar components with distinct spatio-temporal patterns. The evoked potentials elicited by the N200-speller are characterized by the motion-specific N200, while the neural response elicited by the P300-speller is featured as the P300 component. Especially, the components derived from the N200-speller are generally asymmetric over occipital-parietal area , while those derived from the P300-speller have clear symmetric central distribution (Farwell and Donchin, 1988, 2000).
3.2. Intra- and Inter- Subject variability
To further analyze the temporal dynamics of the N200 components, ERP-image plots (Jung et al., 2001) were created of the activity time course of these components in single trials. Plots of the P300 component derived in our data registered from the P300-speller are also exhibited for comparison. As can be seen in the upper part of figure 4, for the N200-speller, the motion-specific N200 component shows a constant motion-onset related activity, giving rise to the relatively sharp negative peak shown in the average waveform. For the P300-speller, the P300 component displays a relatively stable activity, also with larger variance of latency and amplitude across subjects. The larger variability might be attributed to concentration and attentional issues that highly affect the endogenous components like P300. Intuitively, the motion-specific N200 exhibits lower variability across trials than the P300 component.
To quantitatively measure the intra- and inter-subject variability associate with the two BCI paradigms, means and standard deviations of the latencies and amplitudes of the N200 and the P300 components were calculated on the data set registered from the N200-speller and the P300-speller respectively (Table 1). The standard deviations elucidated evidently smaller variations in the latency of the N200 than that of the P300 (10.36 ms for N200, and 60.79 ms for P300) . The lower intra- and inter-subject variability of the N200 components is a featured advantage for this BCI paradigm. Another advantage of the N200-speller over the P300-speller is that the amplitude of the motion-specific N200 (grand average value 6.53 μV) is significantly larger than the P300 amplitude (grand average value 3.68 μV), being true of each subject (ANOV A, F=8.47, P=0.01).
Figure 4. ERP-image plots of the N200 and the P300 components, showing trial-by-trial changes in response to target stimuli in the N200-speller (upper row) and the P300-speller (lower row). Single trials in ERP-image plots are shown sorted by ordinal trial order from the first subject to the eighth subject orderly in each session, and with a vertical smoothing moving window (100 trials). The color-coded amplitudes in the plots show the low variability of the N200 component activity across all trials (360*8=-2880 in total).
3.3. Target Detection Performance
According to section 2.2, the target detection process includes two steps: training for classifier model and classification with the model, using the two separate data sessions registered respectively. Individual data sets were offline trained for SVM model using the 3-trial averaged data epochs to perform five-fold cross validation. We tested different numbers of trials averaged (3, 5, and 15) to train SVM model and found that 3-trial average
500
1000
150020002500
-11
-5.5
0 5.511T r i a l I n d e x N2speller electrode P7
500 1000
150020002500
T r i a l I n d e x μV -11-5.505.5110 5001000 ms Table 1. The latencies and amplitudes of N2 and P300
Subjects N2 peak
latency (ms) N2 amplitude (μV) P300 peak latency (ms) P300 amplitude (μV)
S1xxj 210±2.31
-12.89±1.02 375±11.47 7.59±1.08 S2gj 210±5.17
-12.46±1.10 440±47.31 5.47±0.67 S3yz 200±6.08
-8.72±1.18 285±75.93 6.07±1.45 S4gf 200±4.80
-7.64±1.06 400±19.18 3.90±2.37 S5lt 190±3.49
-5.70±1.22 350±60.14 2.67±1.07 S6lb 210±4.05
-5.83±1.28 430±33.78 3.13±0.64 S7jc 220±7.15
-7.30±1.66 315±67.22 5.62±1.14 S8df 220±3.87
-5.22±0.90 445±71.18 2.39±1.72 Inter-subject
SD
10.35 2.98 60.79 1.86 Grand
Average 210 -6.53 350 3.68
was a compromise between the signal-to-noise ratio of the N200 or P300 feature and numbers of samples left to train a classifier after the average procedure. Table 2 presents the 2-class (target and nontarget) classification accuracy, as an evaluation of the training performance. For each of the eight subjects, the training accuracy of the N200-speller is higher than that of the P300-speller.
The classifier trained from the first session was employed to detect the target desired by the subject, using the EEG data in the second session. Figure 5 displays the 36-class detection accuracy of each subject, for the N200-speller (blue curves with dot) in comparison of that of the P300-speller (red curves with cross). The target detection accuracy is plotted as a function of the number of trials considered for each choice. In our sample of eight subjects, four of them show a remarkably better performance using the N200-speller, as shown in the first row the figure 5. Two of the subjects achieve higher accuracy using the P300-speller, and the other two do not show significant difference between the N200-speller and the P300-speller. Worth mentioning, the best N200-speller operator in our subjects can achieve a 100% accuracy using only 3 trials of EEG data. The target detection performance of the N200-speller strengthens the previous findings that an applicable N200-based BCI is feasible, and also comparable or even better than the intensively investigated P300-speller.
Table 2.Target Detection Accuracy
Subject N200-speller
Training Accuracy
(cross validation)
N200-speller Test Accuracy (5-trial) P300-speller Training Accuracy (cross validation)P300-speller Test Accuracy (5-trial) S1xxj 97.5±2.3
100 89.2±7.0 83.3 S2gj 90.0±6.0
72.2 88.3±6.9 83.3 S3yz 89.2±5.6
72.2 79.2±8.8 72.2 S4gf 92.5±8.0
88.9 72.5±6.3 72.2 S5lt 93.3±8.6
91.7 78.3±8.1 66.7 S6lb 83.3±4.2
55.6 70.0±14.9 72.2 S7jc 80.8±9.6
77.8 79.2±7.2 72.2 S8df 80.8±6.3
55.6 58.3±10.2 16.7 Mean 88.4±6.2 76.8 76.9±10.0 67.4
Figure 5. Performance curves of each subject, for the N200-speller (blue curve with dot) and the P300-speller (red curve with cross) respectively. The curves show the target detection accuracy as a function of the number of trials averaged for each choice. The four subjects in the first row of plots perform better using the N200-speller. The other four subjects in the second row show comparable performance or better performance using the P300-speller.
In addition to target detection accuracy, another important criterion to evaluate a BCI system is the information transfer rate (ITR), since a BCI is a communication system. The ITR measures the achievable information rate per unit time, given the decision accuracy and the time required to make a decision. The number of achievable bits per minute is given by (Wolpaw, 2000)
2221{log log (1)log []}1
P B M N P P P N ?=++?? where N is the number of possible selections, P is the accuracy probability, and M is the average number of decisions per minute.
To assess the limit on the communication speed achieved with the data recorded in the N200-speller, ITR at different levels of data averaging was calculated for each subject. Because of the limited space, only the best performance and the worst performance are illustrate in figure 5. The best operator of the N200-speller achieves an ITR of 87.81 bits/min, which owes to the high decision accuracy when using single-trial data.
Figure 6. Information Transfer Rate of the N200-speller (blue curve) and the P300-speller (red curve) respectively, for two of 0
subject xxj
Number of trials averaged
Bits/min 510 15
51015
subject df
subject xxj subject lt subject gf subject df
510 15
51015
subject yz 510155 1015A c c u r a c y A c c u r a c y Number of trials averaged %
the subjects, representing the best operator and the worst operator of the BCI system.
4. Discussion
4.1. Cause of 5 Hz oscillation in the waveform of the evoked potentials
The oscillation of 5 Hz frequency can be observed in both the target waveform and the non-target waveform in figure 3. It is partially attributed to the overlap of waveform when the grand average waveform ranges wider than the 200 ms SOA. Figure 7a displays the non-overlapped waveform extracted between 100 ms and 300 ms post motion-onset: the response to the target stimuli features a prominent N200 of about 5 uv amplitude and 210 ms peak latency; while the response to the non-target stimuli also shows a relatively weak deflection of 0.6 uv amplitude and 220 ms peak latency. Since the SOA of stimulus is 200 ms, waveform that is extracted wider than 200 ms will always be overlapped. For example, if a non-target target stimulus occurs 200 ms earlier than the target stimulus, then the N200 peak will appear at 200 ms for the target waveform and appear at 400 ms for the non-target waveform, as long as both waveforms are extracted using their respective motion-onset as 0 ms. As can be seen in figure 7b, the non-target waveform between -800 ms to 1200 ms exhibits a N200-like deflection every 200 ms, forming a 5 Hz oscillation, which can be explained by the overlap of the target N200 peak when averaged across trials to weaken its amplitude. The peak amplitudes of the oscillation at every 200 ms are determined by the probability that a non-target motion stimulus occurs n*200 ms before the target motion stimulus (n=-5, -4, -3, -2, -1, 1, 2, 3, 4, 5). The probability value can be obtained by accumulating trigger events which represent the stimulus sequence. Figure 7c reveals that the amplitude of the deflection at each 200 ms corresponds well with the calculated probability of certain stimulus asynchrony interval between target and non-target. Taken together, figure 7 explains the 5 Hz oscillation in the waveform of evoked potentials derived from N200-speller, as an unavoidable result of overlap.
A m p l i t u d e -1000-800-600-400-200 0200400600800 1000 ms N o r m a l i z e d v a l u e -800-600-400-200020040060080010001200 ms (c) 100150200 250300 ms -5 -4
-3
-2
-1
1
2
Non-overlapped waveform
at electrode P7
(a) Target Non-target A m p l i t u d e μV
Figure 7. Explanation of the 5 Hz oscillation. (a)The non-overlapped waveform of the evoked potentials derived from the N200-speller. The red solid curve is the grand average waveform of the target stimuli; while the superimposed blue dash curve is the grand average waveform of all the non-target stimuli. The evoked potentials are averaged across the ‘row’ and the ‘column’ response at electrode P7. (b)The overlapped waveform of the evoked potentials ranging from -800 ms to 1200 ms after stimulus onset. (c) Normalized histogram. The blue bar represents the amplitude of the deflection at every 200 ms from the time range between -800 ms and 1200 ms. The green bar represents the probability that a non-target motion stimulus occurs n*200 ms before the target motion stimulus (n=-5, -4, -3, -2, -1, 1, 2, 3, 4, 5).
4.2. N200 Adaptation and P300 Habituation
It has been demonstrated that the perception of visual motion in humans is susceptible to adaptation (Gopfert et al., 1984; Muller et al., 1986; Bach and Ullrich, 1994). Thus adaptation during repeated stimulation is a crucial factor in the design of motion VEP experiments. Specifically, the motion N200 component matches human motion perception in its susceptibility to motion adaptation (Hoffmann et al., 2001). According to previous studies (Courchesne, 1978; Romero and Polich, 1996), P300 has similar problem of habituation, as the P300 amplitude declines with repeated stimulus presentations. P300 habituation seems to stem from its interpretation as reflecting memory-updating (Donchin et al., 1988; Romero and Polich, 1996). Although both the N200 and the P300 components share adaptation effect, the rate of amplitude decline is determined strongly by experimental design especially task requirements (Romero and Polich, 1996). A qualitative investigation of the N200 elicited by the N200-speller and the P300 elicited by the P300-speller was made to explore the adaptation in the two BCI systems.
Figure 8 reveals that the P300 amplitude from the target stimuli decreases as a function of repeated trials, while the P300 latency slightly increases with repeated trials. As a comparison, the N200 amplitude does not decrease as fast as the P300 amplitude. Moreover, the N200 latency is well constant with repeated stimuli. These findings suggest that the N200-speller in our study is well designed to be a low-adaptation paradigm, which might be attributed to the task design of color recognition and the duty cycle range of less than 20%. Also, the low adaptation for the N200-speller may also have some relation to the paradigm design of novelty motion stimuli (the color of the motion stimuli is random), since Kremlacek et al shows that a more negative potential in the time range of 145-260 ms can be observed for novel stimuli (Kremlacek et al., 2006).
Figure 8. Time course of mean N200 and P300 amplitudes and latencies (averaged across the eight subjects) as a function of trial index. Top: The solid and cross dots show the amplitudes of the N200 and the P300 components respectively; the superimposed curves derive from exponential curve fitting. Bottom: The solid and cross dots show the latencies of the N200 and the P300 components.
4.3. Support vectors of the N200-speller and the P300-speller
In the target detection process, we used data of session 1 to train a binary SVM to distinguish between two classes of signals: the target epochs and nontarget epochs. To investigate the characteristics of the support vectors (SVs) for the N200-speller and the P300-speller, representative SVs is shown in figure 8. The SV is a feature vector of 27 points, which are normalized data points concatenated by three electrodes (P7, P3 and Pz for the N200-speller; FCz, Cz and CPz for the P300-speller). Figure 9a reveals that the target SV of the N200-speller is a representation of the motion N200 waveform, while figure 9b displays the target SV of the P300-speller as a typical P300-like shape. These results further confirm that the N200-speller and the P300-speller take advantage of different components to perform target detection.
20 40 608046
810
020 40 6080
A m p l i t u d e L a t e n c y Trial Index
Figure 9. Representative support vectormachines (SVs). Top: The target SV and nontarget SV of the N200-speller. To give an example, the two SVs are chosen from the 12 SVs for one subject. Bottom: The target SV and nontarget SV of the P300-speller. The SVs are calculated using data registered from the same subject.
4.4. Eccentricity, attention and task affects
It is demonstrated that the amplitude of the motion-specific N200 is less reduced for large eccentricities than predicted from the central magnification effect of the early pathway of visual system (Schlykowa L et al., 1993). One problem of this eccentricity insensitivity is that the subject will be more vulnerable to the peripheral non-target visual stimuli when he or she is gazing at the target. Although the motion paradigm does not take advantage of the central magnification too much, it could still amplify targe inputs through attention. O'Craven et al. found that voluntary attention modulates activation in the MT/MST region which is specialized for motion processing (O'Craven et al., 1997). Also, Torriente et al. showed that the amplitude of N200 reduced when attention was drawn away from the previously attended visual motion stimuli (Torriente et al., 1999). Addressing the attention modulation effect, cognitive tasks such as color recognition are designed in the N200-speller paradigm in order to help subject concentrate on the target stimuli to enhance the N200 amplitude. This indicates that the appropriate design of user task is crucial for an effective BCI paradigm.
39 0
1
P7P3Pz 01 FCz Cz CPz Target Nontarget
Nontarget
Target N o r m a l i z e d v a l u e Support Vectors of the P300‐speller Support Vectors of the N2‐speller N o r m a l i z e d v a l u e 15 712 18 10 14 16 2127
19 232539 15 71218 10 14 16212719
2325
Acknowledgement
The authors would like to thank J. Guo for experimental work and fruitful discussions, and A.H. Caplan for language correction. This work was partly supported by National Natural Science Foundation of China (30630022), Science and Technology Ministry of China (2006BAI03A17) and Tsinghua-Yu-Yuan Medical Sciences Fund.
References
Azizian A, Freitas AL, Parvaz MA, Squires NK. Beware misleading cues: Perceptual similarity modulates the N2/P3 complex. Psychophysiology, 2006; 43: 253–60.
Bach M, Ullrich D. Motion adaptation governs the shape of motion-evoked cortical potentials. Vision Res., 1994; 34: 1541–47.
Chang CC, Lin CJ. LIBSVM: A library for support vector machines [Online]. Software Available: https://www.doczj.com/doc/c017570655.html,.tw/~cjlin/libsvm/. 2001.
Courchesne E. Changes in P3 waves with event repetition: Long-term effects on scalp distribution and amplitude. Electroencephalogr. Clin. Neurophysiol., 1978; 45:754-66.
Delorme A, Makeig S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods, 2004; 134: 9-21.
Dien J, Spencer KM, Donchin E. Parsing the late positive complex : Mental chronometry and the ERP components that inhabit the neighborhood of the P300. Psychophysiology, 2004; 41: 665-78.
Donchin E, Coles MGH. Is the P300 component a manifestation of context updating? Behav. Brain Sci., 1988; 11: 355–72.
Dodt E, Kuba M. Simultaneously recorded retinal and cerebral potentials to windmill stimulation. Doc Ophthalmol, 1995; 89: 287-98.
Fabiani M, Gratton G, Karis D, Donchin E. Definition, identification, and reliability of the P300 component of the event-related brain potential. Advances in Psychophysiology, Ackles PK, Jennings JR, Coles MGH, Eds. New York: JAI Press, 1987: 2.
Falkenstein M, Hohnsbein J, Hoormann J. Effects of choice complexity on different subcomponents of the late positive complex of the event-related potential. Electroencephalogr. Clin. Neurophysiol., 1994; 92: 148-160.
Farwell LA, Donchin E. Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr. Clin. Neurophysiol., 1988; 70: 510-23.
Gopfert E, Muller R, Hartwig M. Effects of movement adapation on movement visual evoked potentials. Doc. Ophthal. Proc. Ser., 1984; 40: 321–24.
Guo F, Hong B, Gao XR, Gao SK. Brain-Computer Interface using Motion-onset Visual Evoked Potentials. J. Neural Eng., 2008; 5: 477-485
Halgren E, Marinkovic K, Chauvel P. Generators of the late cognitive potentials in auditory and visual oddball tasks. Electroencephalogr. Clin. Neurophysiol., 1998; 106: 156-64.
Hoffmann MB, Unsold AS, Bach M. Directional tuning of human motion adaptation as reflected by the motion VEP. Vision Res., 2001; 41: 2187-94.
Jung TP, Makeig S, Westerfield M, Townsend J, Courchesne E, Sejnowski TJ. Analysis and visualization of single-trial event-related potnetials. Hum. Brain. Mapp., 2001; 14: 166-85.
Kremlacek J, Kuba M, Chlubnova J, Kubova Z. Visual mismatch negativity elicited by magnocellular system activation. Vision Res., 2006; 46: 485–90.
Krusienski DJ, Sellers EW, McFarland DJ, Vaughan TM, Wolpaw JR. Toward enhanced P300 speller performance. J. Neurosci. Methods. 2007; 167:15-21.
Kuba M, Kubova Z. Visual evoked potentials specific for motion onset. Doc Ophthalmol, 1992; 80: 83-9.
Kubova Z, Kremlacek J, Kuba M, Chlubnova J, Sverak J. Photopic and scotopic VEPs in patients with congenital stationary night-blindness. Doc Ophthalmol, 2004; 109: 9-15.
Kubova Z, Kuba M, Hubacek J, Vit F. Properties of visual evoked potentials to onset of movement on a television screen Doc Ophthalmol, 1990; 75: 67–72.
Lebedev MA, Nicolelis MA. Brain-machine interfaces: past, present and future. Trends Neurosci., 2006; 29: 536-46.
Muller R, Gopfert E, Hartwig M. The effect of movement adaptation on human cortical potentials evoked by pattern movement. Acta Neurobiol Exp. (Warsz), 1986; 46: 293–301.
Munro GES, Dywan J, Harris GT, McKee S, Unsal A, Segalowitz SJ. Response inhibition in psychopathy: The frontal N2 and P3. Neuroscience Letters, 2007: 418: 149–53.
O’Craven K M, Rosen B R, Kwong K K, Treisman A, Savoy R L. Voluntary Attention Modulates fMRI Activity in Human MT–MST. Neuron, 1997; 18: 591–598.
Piccione F, Giorgi F, Tonin P, Priftis K, Giove S, Silvoni S, Palmas G, Beverina F. P300-based brain computer interface: Reliability and performance in healthy and paralysed participants. Clin. Neurophysio., 2006; 117: 531–37.
Probst T, Plendl H, Paulus W, Wist ER, Scherg M. Identification of the visual motion area (area V5) in the human brain by dipole source analysis. Exp Brain Res, 1993; 93: 345-51.
Polich J. P300, probability, and interstimulus interval. Psychophysiology, 1990; 27: 396-403.
Polich J. Updating P300: An integrative theory of P3a and P3b. Clinical Neurophysiology, 2007; 118:2128-48.
Romero R, Polich J. P300 Habituation from Auditory and Visual Stimuli. Physiology & Behavior, 1996; 59 (3):
517-22.
Schlykowa L, van Dijk BW, Ehrenstein WH. Motion-onset visual-evoked potentials as a function of retinal eccentricity in man. Cogn Brain Res, 1993; 1:169–174.
Schwartz AB, Cui XT, Weber DJ, Moran DW. Brain-Controlled Interface: Movement Restoration with Neural Prosthetics. Neuron, 2006; 54: 205-20.
Sellers EW, Krusienski DJ, McFarland DJ, Vaughan TM, Wolpaw JR. A P300 event-related potential brain-computer interface (BCI): The effects of matrix size and inter stimulus interval on performance. Biological Psychology, 2006; 73: 242-52
Serby H, Yom-Tov E, Inbar GF. An improved P300-Based brain computer interface. IEEE Trans. Neural Syst. Rehabil. Eng., 2005; 13(1): 89–98.
Skrandies W, Jedynak A, Kleiser R. Scalp distribution components of brain activity evoked by visual motion stimuli. Exp Brain Res, 1998; 122: 62-70.
Sutton S, Braren M, Zubin J, John ER. Evoked-potential correlates of stimulus uncertainty. Science, 1965; 150: 1187–88.
Thulasidas M, Guan C, Wu J. Robust classification of EEG signal for brain-computer interface. IEEE Trans Neural Syst Rehabil Eng., 2006; 14(1): 24-9.
Torriente I, Valdes-Sosa M, Ramirez D, Bobes M A. Visual evoked potentials related to motion-onset are modulated by attention. Vision Research, 1999; 39: 4122-4139.
Wist ER, Gross JD, Niedeggen M. Motion aftereffects with random-dot chequerboard kinematograms: relation between psychophysical and VEP measures. Perception, 1994; 23: 1155–62.
Wolpaw JR, Birbaumer N, Heetderks WJ, McFarland DJ, Peckham PH, Schalk G, Donchin E, Quatrano LA, Robinson CJ, Vaughan TM. Brain-computer interface technology: A review of the first international meeting. IEEE Trans. Rehab. Eng., 2000: 8: 164–73.
Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM. Brain-computer interfaces for communication and control. Clin. Neurophysio., 2002; 113: 767-91.
Xu N, Gao XR, Hong B, Miao XB, Gao SK. 2004 BCI competition 2003—Data set IIb: Enhancing P300 wave detection using ICA-based subspace projections for BCI applications. IEEE Trans. Biomed. Eng., 2004; 51(6): 1067–72
Zhang H, Guan C, Wang C. Asynchronous P300-based brain-computer interfaces: a computational approach with statistical models. IEEE Trans Biomed Eng., 2008; 55(6): 1754-63.