update

new/revised sections

2.1 Human brain
2.2 Neuronal Electrical Activity
2.3 Action Potential

3.1 Hypothesis

Writing

1.1 overview
1.2 motivation
1.6 training
1.7 Thesis Overview
1.8 Limitations

Moving and shaking

Ladies, gentlemen and on lookers,

Things have been going fairly smoothly the last few weeks.

1) My game written in blender will now communicate in real time with the EEG interface device via the parallel port. I'm using a library called pylpt embedded in the game to create the link and send the signal.

2) I have also successfully integrated the wii remote with my game. Excuse my french but this was a complete pain in the *#$! First, I had to find the right bluetooth adapter, because of the lax bluetooth standards the wiimote won't communicate with all bluetooth devices. Second, it would be difficult to embed the bluetooth connection with in the blender game engine, so I used an emulator called GlovePie. GlovePie uses a set of libraries to send the emulated wiiremote signal to the local host using open sound control or OSC. So, the OSC sends a packet to the local host with the wiimote status and during every pass of the game engine this packet is read.

3) THe game is working but not that the wii remote is integrated it isn't as stable. I think I need to rewrite the canned script emulating the wiimote to make it simpler.

I'm tired... time for bed!

The last four months

Geez I'm horrible about keeping up with these things.

1) I finished the game prototype. The timing is better but I have to change the stimuli to something much cooler.

2) I THOUGHT I was able to send signals from the game to the EEG interface device. This is done by sending signals via the parallel port. I am using python and in the testing environment I was using it seemed that python was switching on and off the bits. For some reason though when its connected to the EEG machine it doesn't want to trigger the events properly. Bother... I ordered another parallel cable because the one in the lab doesn't form a good connection and a device to show the state of the parallel ports. With the device attached little led lights will blink when the cable sends or receives a signal. Hopefully with these two devices I can fix this problem.

3) In talking with Dr. Keith and Jay we all think that using one keyboard might not give us the responses we want. So, I'm working on integrating the wii remote with my game prototype. Last night I was able to get the wii remote to communicate with the computer hopefully tonight or tomorrow, I will have the remote integrated with the game.

4) I've been reading a great book by Erol Basar called "Brain function and oscillations". He implicitly talks about looking at whole brain states in conjunction with Event Related Potentials. I need to start writing again soon and when I do I'll be relying heavily on this book.

Right now I am about a month behind schedule but I guess it is to be expected with such an extravagant project...

Timeline

misc. questions

ask about

1. FFT on a epoch (frame) using all 64 electrodes
2. Can the current abstract / experiment count as my thesis?
3. how should I go about looking for conferences?
1. ask dr. Keith / any other electrophyisology prof in psych dept.
   
2. ask dr. Tag and Narayan about ANN conferences
   
3. Google!
4. consult magic 8 ball?
   

Wavelet questions

1. Is my data considered 1 dimensional? All that is represented is amplitude but it is amplitude over time?
2. Should I take the fft then try to do PCA? PCA on the raw data is really noisy...
3. A wavelet transform will give me amplitude, frequency, and time but won't I still need a classifier to classify the signal?
   

Exploring: wavelet xforms and other analysis tools

• wavelet xforms
• Wavelet tutorial
  
• http://users.rowan.edu/~polikar/wavelets/wttutorial.html
• MATLAB Cohen-Daubechies-Feauveau 9/7 (CDF 9/7) wavelet transform
• http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?objectId=6449
• Paper's ordered
• http://lib.bioinfo.pl/pmid:12705422
• http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V18-4MW31YG-1&_user=10&_rdoc=1&_fmt=&_orig=search&_sort=d&view=c&_acct=C000050221&_version=1&_urlVersion=0&_userid=10&md5=b68a247d8cc31b9f06d58f774883bd03
• Wavelets plot frequency,time, and amplitude at the same time.
  
• great for non-stationary signals or signals that change over time
• good for signal pre-processing
  

• Time series outlier detection method
• exponential smoothing

Fast Fourier Transforms VS. Autoregressive Moving Model Average

Recap:

Last week we talked about how Fast Fourier Transforms are used to quickly compute the Discrete Fourier Transform of a time series thereby transforming the time series into its Frequency Domain. This week we will talk about the Autoregressive Moving Model Average and how it compares to Fast Fourier Transforms.

Why FFT or ARMA in the first place?

“Perfect feature extraction would make the job of the classifier trivial, and an omnipotent classifier would not need the help of feature extraction,”

What is the Autoregressive Moving Model Average:

Given a time series of data Xt, the autoregressive moving average model (ARMA), is used as a tool for understanding and possibly predicting future values in the time series (Wikipedia 2006). The ARMA is typically applied to time series data (Wikipedia 2006).

Autoregressive model:

ϕ
1,...ϕp are the parameters and ε is the error term

Moving average model:

w
here the θ1, ..., θq are the parameters of the model and the εt, εt-1,... are the error terms

Autoregressive moving average model:

Taking the AR model and the MA model, we get the ARMA model. The notation ARMA(p, q) refers to a model with p autoregressive terms and q moving average terms (Wikipedia 2006). This model subsumes the AR and MA models,

Autoregressive vs. Fast Fourier

• The ARMA estimate may have very sharp spectral features so one cm evaluate it on a very fine mesh near to those features, and more coarsely farther away from them

• With ARMA Good spectral estimates can be obtained from short EEG segments, where as FFT assumes the function continues to infinity on both sides.

• ARMA is preferred for prediction

• In some ARMA models Coefficients can be analyzed independent of the function.

Power approximations of a 12 Hz sine wave added to a 20 Hz sine wave sampled at 200 Hz for 0.3 seconds. Top part of the figure shows the FFT approximation, and the bottom part shows the AR model of 8th order.

Disadvantages of ARMA:

• Problem of selecting the proper model order:

  • With noisy inputs and the order being too high the result is spurious peaks

  • Too low an order yields a smooth spectra

• ARMA can be an order of magnitude slower than FFT.

  • N = the size of the signal

  • M = model order or magnitude

  • if the magnitude of M is greater then logN then ARMA is slower then FFT (NlogN)

[1] M. J. Polak, "Adaptive Logic Networks in a Brain-Computer Interface System," in Computing Science. vol. Doctor of Philosophy Edmonton, Alberta: University of Alberta, 2000, p. 122.

[2] B. Hodgson ”What is an autoregressive moving average model(ARMA)?” http://cnx.org/content/m13395/latest/

Fast Fourier Transforms

in less than 15 min!

What is 'Fast Fourier Transforms?

Fast Fourier Transform algorithms are used to compute the Discrete Fourier Transform and its inverse.

Ok, so what is 'Discrete Fourier Transform?

Discrete Fourier Transforms are used to transform one function into another that is called the Frequency domain. This makes sense because in Digital Signal processing we don't care nearly as much about the numbers as we do the FREQUENCY they are observed! Plus, the frequency is usually a less complicated number then the original input.

Still with me? Lets talk more about the Frequency Domain!

We can think of the Fourier Transform as converting the signal information to a magnitude and phase component of each frequency. Now instead of the signal input potentially being all over the place (think eye blinks or other artifacts in EEG data) the current signal in influenced by all of the previous signals and therefore cannot leave the bound of the Frequency Domain!

(Drawing time!)

Oh no, close your eyes, now its time for the math...

So this is a basic Discrete Fourier Transform. The most notable aspect is e^ -(2πi/n), this is like a log curve so every point in the signal with be plotted along this curve.

(More Drawing time!)

That wasn't so bad lets close with a list of advantages and disadvantages:

Advantages:

• Fast! Or at least faster than trying to process the un-transformed signal

Disadvantages:

• Loss of precision due to floating point numbers

• Signal segments are discrete and calculated independently. In this aspect this signal isn't very efficient

The next chapter in this exciting series:

Fast Fourier vs Autoregressive analysis

final project for pattern rec

EEGLAB workflow

• 1) read in data
• 2) change sampling rate to 256 (this makes 154 points between a [-0.3 0.3] epoch)
• 3) parse out the right hand and left hand epochs
  
• 4) take the average over all 64 electrodes (next try only working with nodes in the 'mu area')
• 5) take the fft of the average
• 6)parse the epochs
  

Things I don't know about Digital Signal Processing

This is going to be a long list:

How to learn DSP:
http://www.redcedar.com/learndsp.htm

1. hertz and their relationship to sampling rates
1. http://en.wikipedia.org/wiki/Nyquist_frequency
2. Finite Impluse Response (FIR) filter vs Infinite Impluse Response (IRR) filter
   
1. FIRFAQ: http://www.dspguru.com/info/faqs/firfaq.htm
2. IRRFAQ: http://www.dspguru.com/info/faqs/iirfaq.htm
3. Linear trends?
1. To attempt to factor out the influence of time on a trend using regressions or other statistical techniques
4. Regressions?
   
1. Regressions are used to quantify the relationship between one variable and the other variables that are thought to explain it; regressions can also identify how close and well determined the relationship is.
5. ds

Good info:

Referencing data from Earlobe:
Converting data from fixed reference (earlobe or etc.) to average reference is advocated by some researchers, particularly when the electrode montage covers nearly the whole head (as in some high−density montages).

Why filter:
Another common use for data filtering is to remove 50 Hz or 60 Hz line noise. Note: we recommend to filter continuous EEG data although epoched data can also be filtered, each epoch being filtered separately.

Removing Baseline:
Removing a mean baseline value from each epoch is useful when baseline differences between
data epochs (e.g., arising from low frequency drifts or artifacts) are not meaningful and might
otherwise dominate the data. The following window pops up automatically after the data has been epoched.


Dr. Keith's experiment

Rerun experiment, let people respond in real time, subtract stimulus time from response time, figure out overall cohearance, epoch out stim - response AND cohearance, compare

After every three response see the response after. 201 vs 203

Fast Fouier VS. Autoregressive

http://docs.google.com/Doc?id=dhrddpn6_54c69rwfc3

Fast Fourier

I gave two talks here is the first:


Fast Fourier Transforms
in less than 15 min!

What is 'Fast Fourier Transforms?

Fast Fourier Transform algorithms are used to compute the Discrete Fourier Transform and its inverse.

Ok, so what is 'Discrete Fourier Transform?

Discrete Fourier Transforms are used to transform one function into another that is called the Frequency domain. This makes sense because in Digital Signal processing we don't care nearly as much about the numbers as we do the FREQUENCY they are observed! Plus, the frequency is usually a less complicated number then the original input.

Still with me? Lets talk more about the Frequency Domain!

We can think of the Fourier Transform as converting the signal information to a magnitude and phase component of each frequency. Now instead of the signal input potentially being all over the place (think eye blinks or other artifacts in EEG data) the current signal in influenced by all of the previous signals and therefore cannot leave the bound of the Frequency Domain!

(Drawing time!)

Oh no, close your eyes, now its time for the math...


So this is a basic Discrete Fourier Transform. The most notable aspect is e^ -(2πi/n), this is like a log curve so every point in the signal with be plotted along this curve.

(More Drawing time!)

That wasn't so bad lets close with a list of advantages and disadvantages:

Advantages:

• Fast! Or at least faster than trying to process the un-transformed signal

Disadvantages:

• Loss of precision due to floating point numbers
• Signal segments are discrete and calculated independently. In this aspect this signal isn't very efficient

The next chapter in this exciting series:
Fast Fourier vs Autoregressive analysis

for pattern rec

I can look at "Inter-trial coherence " to figure out brain oscillations (state, waves)

Now I need to figure out response time for every event , what the different events are, and

adaboost papers

Adaboost papers:

found:



need to find:

1. Adaboost for improving classification of left and right hand motor imagery tasks
• http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=1499830
• ordered on ILL
  
2. A new approach in the BCI research based on fractal dimension as feature and Adaboost as classifier
• http://www.iop.org/EJ/abstract/1741-2552/1/4/004
• ordered on ILL
3. Medical and Biological Engineering and Computing: A comparison approach toward finding the best feature and classifier in cue-based BCI
• http://www.springerlink.com/content/e436280xh0u67405/
• ordered on ILL
  

Exploring EEG Lab

baseline latency range

1. split signal into EPOCHS
2. saved EPOCH file to TXT
1. -.003 before event .003 after but I don't know what EVENT
2. IF I can find out EVENT with corresponding EPOCH then ready to split into patterns and targets
3. I need to look into source coherence or just coherence.
1. This should help me figure out if alpha, beta, gamma
4. With (1,2) & (3) figured out I can solve Dr. Keith's problem
   
1. Dr. Keith's problem solved I will be able to:
   
1. Classify ERPs in my own data when I run subjects
2. Monitor my own subject's 'Attention' vs 'distraction'
1. Gamma vs Alpha waves

For tomorrow:

1. Make a schedule
2. look into coherence in EEGLAB & in General lit
3. I gave a presentation on Fast Fourier Transforms last week I need to prepare Auto Regressive VS FFTs presentation for next wednesday

Pattern Rec: Enlightenment

I've been trying to understand Pattern Rec class today and I finally got it. The example given today could have very well been my EEG data. We will know WHAT the subject's intension is by the presentation of the stimuli (left, right, up, down). We will also know WHEN the stimuli will be presented so we can assume shortly after the presentation is the EEG representation of that intention. Now the time segments of eeg (during the presentation of stimuli) will be my PATTERNS and the classification TARGETS will be from 1 - 5 (left, right, up, down and null) . With the data separated out like this I think I can run it thru any classifier.


Bill, I finally get what you were talking about. If you tell the user to make the wrong movement with a controller will the brain still move toward the stimuli. Am I right?

eeglab

http://www.sccn.ucsd.edu/eeglab/

EEGLAB: an open source toolbox for analysis of single-trial EEG ...

tinkering with eeglab today



was able to import a .edf file and export a .txt file.

• can't import whole file at once TOO LARGE
• separate file [0 100],[101 200], ...

----
try different file time segments
----

----
try .bdf tonight
----

Getting ready to

Paper: Towards adaptive classification for BCI

A Tutorial?

• Purpose of the paper:
• Non-stationarities are ubiquitous in EEG signals.
• (a) in the differences between the initial calibration measurement and the online operation of a BCI,
  
• (b) caused by changes in the subject’s brain processes during an experiment (e.g. due to fatigue, change of task involvement, etc)
• we quantify for the first time such systematic evidence of statistical differences in data recorded during offline and online sessions.
• we propose novel techniques of investigating and visualizing data distributions, which are particularly useful for the analysis of (non-)stationarities
• Our study shows that the brain signals used for control can change substantially from the offline calibration sessions to online control, and also within a single session.
• we propose several adaptive classification schemes and study their performance on
  data recorded during online experiments.
• Classification methods
• ORIG: this is the unmodified classifier trained on data from the offline scenario and serves as a baseline.
  
• REBIAS:we use the continuous output of the unmodified classifier and shift the output by an amount that would minimize the error on the labeled feedback data.
  
• RETRAIN:we use the features as chosen from the offline scenario, but retrain the LDA classifier to choose the hyper plane that minimizes the error on labeled feedback data.
  
• RECSP:we completely ignore the offline training data and perform CSP feature selection and classification training solely on the feedback data.
  
• Types and controls
• (1) all the labeled online data up to the current point (cumulative),
  
• (2) only a window over the immediate past (moving), or
  
• (3) only an initial window of data from each session(initial).
  
• We thus have C-REBIAS7, C-RETRAIN and C-RECSP, W-REBIAS, W-RETRAIN and W-RECSP, and I-REBIAS, I-RETRAIN and I-RECSP, respectively, for the three cases considered.
  
  
  

Paper: Induced sensorimortorrhythms during movement observation and motor imagery

• Interesting but not immediately relevant
  
• the visual representation of the observed action onto the motor representation of the same action“
• 9 subjects
• 
  

Paper: An efficient rhythmic component expression and weighting synthesis strategy for classifying motor imagery EEG in a brain–computer interface

• It is important to identify robust feature descriptions for BCI research, either in the time or frequency domain as derived from single-trial motor imagery (MI) EEG data
• Autoregressive models are commonly used in terms of model
  parameters without direct relation to specific brain activities
  (Anderson et al 1998, Pfurtscheller et al 1998).
• Time- domain features, such as activity, mobility and complexity, describing the properties of a signal trial have been used (Obermaier et al 2001).
  
• A direct ERD estimation calculating the energies within pieces of time windows imposed on
  EEGs at pre-selected frequency bands has been found to be a good feature in classifying right- or left-hand imagined movements (Pfurtscheller et al 1997).
  
• The primary drawback of scalp EEG is the spatial smearing
  and low signal-to-noise ratio.
• Since there are correlations among the recording electrodes, spatial filters have been used as means of accentuating localized activity and reducing diffused activity
• McFarland et al (1997) have compared the EEG classification results using different spatial filters and concluded that the common average and the Laplacian derivation yield good performance.
  
• Fifth order IIR Butterworth band-pass filters were used for
  frequency decomposition
• Two important timing cues that should be mentioned here are that at 3.75 s of a trial period,
  
• a cue (preparation cue) of one letter (L or R) appeared on the screen indicating which hand (specifically index finger) movement should be imagined;
  
• and at 5.0 s another cue (execution cue) appeared, indicating that it was time to make the requested response. The subjects were well trained to consistently respond to the cue signals within 100 ms.
  

Paper: Boosting bit rates and error detection for the classification of fast-paced motor commands based on single-trial EEG analysis

• GREAT PATTERN REC PAPER!!!
• The required discrimination of different brain states may be based on evoked poten-
  tials (like steady-state visual evoked potentials or P300) a motor output, i.e., self-paced typewriting on a computer keyboard.
• This Paper
• (a) We exploit advanced machine learning and signal processing technol-
  ogy for single-trial EEG evaluation requiring no prior sub-
  ject training.
  
• (b) We use slow pre-movement potentials as
  physiological signals, and
  
• (c) we utilize a fast-paced experimental paradigm.
  
• "Let the machine learn" so they chose any every day task
• experiment (paradim)
  
• We let our sub jects (all without neurological deficit) make a binary (left/right hand) decision coupled to a motor output, i.e., self-paced typewriting on a computer
  keyboard.
• Using multi-channel scalp EEG recordings, we
  analyze the single-trial differential potential distributions
  of the Bereitschaftspotential (BP) preceding voluntary (left
  or right hand) finger movements over the corresponding
  (right/left) primary motor cortex.
• non-oscillatory event-related potentials (ERPs) two reasons:
• (physiological) if a bci is monitoring an idling rhythm then a prerequisite is stable presence of that rhythm. This could become a problem at a fast pace because the signal might not have the time to fully recover or idle again (my words pg.2)
• (data analysis) How to classify the noisy, high-demensional EEG data? The difference between task-related brain activity and non-task-related activity might not be enough (my words pg.2)
• Robust Preprocessing, Distro of ERPs, and Classification section
• 8 subjects
• Fisher Discriminant (again)
• GREAT PATTERN REC PAPER!!!
• 
  

Paper: Classifying Single Trial EEG: Towards BCI

Classifying Single Trial EEG: Towards BCI

• single trial is difficult for signal processing and machine learning
• This study a pseudo-online evaluation of single-trial EEGs from voluntary self-paced finger movements and exploit the laterality of the left/right hand signal as one bit of information for later control
• Two key issues to start with when conceiving a BCI are
  
• (1) the definition of a behavioral context in which a subject’s brain signals will be monitored and used eventually as surrogate for a bodily, e.g., manual, input of computer commands, and
  
• (2) the choice of brain signals which are optimally capable to
  convey the subject’s intention to the computer.
• Types of experiments: two alternative force reaction VS endogeneous (natural reaction)
• a negative ›Bereitschaftspotential‹ (BP) precedes the voluntary initiation of the movement.
• A differential scalp potential distribution can be reliably demonstrated in a majority of experimental subjects with larger BP at lateral scalp positions (C3, C4) positioned over the left or right hemispherical primary motor cortex, respectively, consistenly correlating with the performing (right or left) hand
• While it is true thatbrain potentials comparable to BP are associated with an imagination of hand movements, which indeed is consistent with the assumption that the primary motor cortex is active with motor imagery, actual motor performance significantly increased these potentials
• Reason 4actually perform the typewriting finger movements, rather than to merely imagine their performance, for two reasons:
• (1)this will increase the BP signal strength optimising the signal-to-noise ratio in BCI-related single trial analyses; and
  
• (2)we propose that it is important for the subject’s task efficiency not to be engaged in an unnatural condition where, in addition to the preparation of a
  motor command, a second task, i.e., to ›veto‹ the very same movement, has to be executed.
  
• ADD SECTION: slow cortical potentials (SCP) and how they can be self-regulated in a feedback scenario
• Typical preprocessing techniques are adaptive autoregressive parameters, common spatial patterns (after band pass filtering) and band power in subject specific frequency bands. Classification is done by Fisher discriminant analysis, multi-layer neural networks or LVQ variants
• Linear methods (pg 4): Fisher Discriminant, Regular Fisher Discriminant, Sparse Fisher Discriminant, Support Vector Machine, K-Nearest Neighbor, Classification of response-aligned windows, Classificationinslidingwindows, Movementdetectionandpseudo-onlineclassification,
  
• Conclusion, High classification rate with low errors