Difference between revisions of "Predictive BCI Speller based on Motor Imagery"
(→Part 3: Project tracking) |
(→Part 4: Documents) |
||
| (21 intermediate revisions by the same user not shown) | |||
| Line 2: | Line 2: | ||
=== Project name === | === Project name === | ||
| − | + | A predictive speller for a brain-computer interface based on motor imagery | |
| − | + | ||
=== Project short description === | === Project short description === | ||
| Line 29: | Line 28: | ||
* [[User:RossellaBlatt | Rossella Blatt]] (phd student) | * [[User:RossellaBlatt | Rossella Blatt]] (phd student) | ||
| + | * [[User:RobertoTedesco | Roberto Tedesco]] (researcher) | ||
===== Students currently working on the project ===== | ===== Students currently working on the project ===== | ||
| Line 42: | Line 42: | ||
== '''Part 2: Project description''' == | == '''Part 2: Project description''' == | ||
| − | The | + | The objective of this thesis is to implement a BCI spelling application for people with severe motor impairments, adopting natural language processing techniques to improve the overall communication rate of the system. The target users of this application are subjects affected by severe motor disorders and in particular patients affected by the locked-in syndrome. Communication is performed detecting different mental states from the subject’s EEG and translating these states into input commands for a spelling application. |
| − | The | + | |
| + | The BCI paradigm adopted is motor imagery. When the subject imagines to move a certain part of the body (such as a hand or a foot) produces modifications on specific brain rhythms characteristic of the motor cortex. These modifications are detected in real-time from the EEG signal and mapped into different choices in the spelling application. This kind of communication is typically very slow and prone to errors. The EEG signal recorded from the scalp has indeed a low signal-to-noise ratio, can be affected by a number of artifacts and presents significant variability with different acquisition settings. Moreover the neurophysiological phenomena related to motor imagery manifest differently with different subjects and may also be influenced by the particular psychophysical conditions met during the task. | ||
| + | |||
| + | All these factors affect the performances of the brain-computer interface. | ||
| + | The control signal received by the spelling application is indeed characterized by low information transfer rates and low accuracy levels. This is the major problem that this thesis had to face, being the final objective of this work to obtain a BCI system suitable for effective verbal communication. The approach chosen in order to maximize the overall communication rate is two-fold: on one side there is an effort in gaining information transfer rate from the control signal, on the other side there is an effort in optimizing the way this information is employed for the purpose of verbal communication. | ||
| − | + | Dealing with the first problem, this thesis focuses on the design of novel features extracted from the EEG signal in the frequency domain and on the adoption of different machine learning techniques for feature selection and classification. Dealing with the second problem, instead, we designed an original speller interface in which redundancies in natural language are used to speed up the selection of symbols and word suggestions are provided to the user during the composition. | |
| + | |||
| + | The spelling application developed is thus composed of three main modules: the brain-computer interface, the speller interface and the language prediction module. Each module has been designed, implemented and tested specifically for the objectives of this thesis and the results have been evaluated at different levels. First we assessed the performances of the BCI module alone, both offline (with pre-recorded signals) and online (with signals acquired and classified in real-time). Then we evaluated the performances of the spelling interface with a simulator program (implemented for the purpose) and we assessed the impact of classification accuracy and language predictions on the overall system performances. Finally we tested the whole BCI spelling application online considering different subjects and different acquisition sessions. | ||
| − | The | + | The achieved results are very satisfactory and comparable with the latest works about motor-imagery BCI spellers reported in literature. With one subject we obtained high classification accuracies and the overall communication rate achieved is 3 char/min. With other two subjects, instead, we started with lower classification accuracies, but significant improvements have been obtained as the number of training sessions increased. These subjects reached a spelling speed of respectively 2 char/min and 2.7 char/min. |
| + | |||
| + | A trend of performance improvement has been observed in our tests and we believe that with more feedback sessions an even better BCI control could be obtained. | ||
| + | Further improvements may be also achieved considering EEG features in the spatial domain (for example with the method of Common Spatial Patterns) and testing other classification algorithms. Moreover alternative symbol arrangements may be investigated in order to speedup the selection process and the accuracy of language predictions could be improved by means of customized language models. | ||
| − | |||
| − | |||
| − | + | [[Image:Bci_gui.png]] | |
| + | [[Image:Bci_feedback.png]] | ||
== '''Part 3: Project tracking''' == | == '''Part 3: Project tracking''' == | ||
| Line 70: | Line 78: | ||
* 03/11/2008: thesis description document update: started a new chapter about letter prediction | * 03/11/2008: thesis description document update: started a new chapter about letter prediction | ||
* 03/11/2008: thesis description document update: updated the section on weights adaptation | * 03/11/2008: thesis description document update: updated the section on weights adaptation | ||
| − | * | + | * 05/12/2008: implemented two algorithms for weight adaptation: RLMS and Delta rule |
| − | * | + | * 10/12/2008: implemented a batch MATLAB script for the off-line analysis |
| − | * | + | * 15/12/2008: implemented a MATLAB script to simulate the on-line computation of the spectrum |
| − | * | + | * 20/12/2008: designed classes for the Speller application |
* 02/01/2009: started implementing the Speller application in BCI2000 | * 02/01/2009: started implementing the Speller application in BCI2000 | ||
| + | * 23/01/2009: first demo of the Speller controlled by keyboard and without letter prediction | ||
| + | * 05/02/2009: demo of the Speller with letter prediciton | ||
| + | * 05/02/2009: working on feature selection/extraction for the offline classification of the EEG | ||
| + | * 10/02/2009: first result data about classification accuracy inside the same session | ||
| + | * 10/02/2009: working on classification accuracy across sessions | ||
| + | * 15/02/2009: developing Speller application: formally description of the GUI behaviour using a FSM | ||
| + | * 20/02/2009: implemented XML configurability of the interface in the Speller application | ||
| + | * 22/02/2009: implemented a patch for Presage to customize prediction features | ||
| + | * 23/02/2009: generated n-gram statistics from a partition of the British National Corpus | ||
| + | * 25/02/2009: redesign of the interface structure, refactoring of the Speller application code, implementation of new Speller features | ||
| + | * 01/03/2009: working on the Speller simulator program | ||
| + | * 02/03/2009: first version of the Speller simulator without errors and with pre-calculated paths | ||
| + | * 07/03/2009: Speller simulator implemented as a stack machine, including error handling and a breadth-first path-finding algorithm | ||
| + | * 12/03/2009: formal description (with Push Down Automata) of two interfaces and simulation of their performances with different probabilities | ||
| + | * 18/03/2009: working on feature selection for EEG classification | ||
| + | * 21/03/2009: implemented a genetic algorithm for feature selection | ||
| + | * 23/03/2009: working on phase lock value | ||
| + | * 27/03/2009: finished implementing phase lock value, relevant correlations found | ||
| + | * 27/03/2009: started implementing the classification algorithm in BCI2000 | ||
| + | * 09/04/2009: finished implemented the feedback application module in BCI2000 | ||
| + | * 09/04/2009: started writing the thesis | ||
| + | * 09/04/2009: started testing the spelling application | ||
| + | * 18/06/2009: thesis document completed | ||
| + | * 21/07/2009: thesis presentation! | ||
== '''Part 4: Documents''' == | == '''Part 4: Documents''' == | ||
| − | * Thesis | + | * Thesis document: [http://www.slideshare.net/tizyweb/a-predictive-speller-for-a-braincomputer-interface-based-on-motorimagery-1752428 A predictive speller for a brain-computer interface based on motor imagery] [En] |
| + | * Presentation [http://www.slideshare.net/tizyweb/a-predictive-speller-for-a-braincomputer-interface-based-on-motorimagery A predictive speller for a brain-computer interface based on motor imagery (presentation)][En] | ||
| + | * Video [http://www.youtube.com/watch?v=R-tNE-y2QU0 A predictive speller for a brain-computer interface based on motor imagery (video)] | ||
== '''Part 5: References''' == | == '''Part 5: References''' == | ||
Latest revision as of 11:21, 22 July 2009
Contents
Part 1: Project profile
Project name
A predictive speller for a brain-computer interface based on motor imagery
Project short description
This project is aimed is to control an external device through the analysis of brain waves measured on the human scalp.
Dates
Start date: 01/05/2008
End date:
Website(s)
People involved
Project head(s)
- Matteo Matteucci (professor)
- Sbattella Licia (professor)
Other Politecnico di Milano people
- Rossella Blatt (phd student)
- Roberto Tedesco (researcher)
Students currently working on the project
- Tiziano D'Albis (master student)
Laboratory work and risk analysis
Laboratory work for this project will be mainly performed at AIRLab-IIT/Lambrate. The main activity consists in the acquisition of brain signals through an EEG amplifier for on-line or off-line processing. This is a potentially risky activity since there is an electrical instrumentation that is in direct contact with the human body. It is thus important to keep the system isolated from the power line. The EEG amplifier (as all biomedical instrumentations) is certified by the vendor to be isolated and the acquired data are transferred to the PC using an optic fiber connection . Anyhow for increased safety the PC and any other electronic device connected to the system must be disconnected from the power line. Standard safety measures described in Safety norms will be followed.
Part 2: Project description
The objective of this thesis is to implement a BCI spelling application for people with severe motor impairments, adopting natural language processing techniques to improve the overall communication rate of the system. The target users of this application are subjects affected by severe motor disorders and in particular patients affected by the locked-in syndrome. Communication is performed detecting different mental states from the subject’s EEG and translating these states into input commands for a spelling application.
The BCI paradigm adopted is motor imagery. When the subject imagines to move a certain part of the body (such as a hand or a foot) produces modifications on specific brain rhythms characteristic of the motor cortex. These modifications are detected in real-time from the EEG signal and mapped into different choices in the spelling application. This kind of communication is typically very slow and prone to errors. The EEG signal recorded from the scalp has indeed a low signal-to-noise ratio, can be affected by a number of artifacts and presents significant variability with different acquisition settings. Moreover the neurophysiological phenomena related to motor imagery manifest differently with different subjects and may also be influenced by the particular psychophysical conditions met during the task.
All these factors affect the performances of the brain-computer interface. The control signal received by the spelling application is indeed characterized by low information transfer rates and low accuracy levels. This is the major problem that this thesis had to face, being the final objective of this work to obtain a BCI system suitable for effective verbal communication. The approach chosen in order to maximize the overall communication rate is two-fold: on one side there is an effort in gaining information transfer rate from the control signal, on the other side there is an effort in optimizing the way this information is employed for the purpose of verbal communication.
Dealing with the first problem, this thesis focuses on the design of novel features extracted from the EEG signal in the frequency domain and on the adoption of different machine learning techniques for feature selection and classification. Dealing with the second problem, instead, we designed an original speller interface in which redundancies in natural language are used to speed up the selection of symbols and word suggestions are provided to the user during the composition.
The spelling application developed is thus composed of three main modules: the brain-computer interface, the speller interface and the language prediction module. Each module has been designed, implemented and tested specifically for the objectives of this thesis and the results have been evaluated at different levels. First we assessed the performances of the BCI module alone, both offline (with pre-recorded signals) and online (with signals acquired and classified in real-time). Then we evaluated the performances of the spelling interface with a simulator program (implemented for the purpose) and we assessed the impact of classification accuracy and language predictions on the overall system performances. Finally we tested the whole BCI spelling application online considering different subjects and different acquisition sessions.
The achieved results are very satisfactory and comparable with the latest works about motor-imagery BCI spellers reported in literature. With one subject we obtained high classification accuracies and the overall communication rate achieved is 3 char/min. With other two subjects, instead, we started with lower classification accuracies, but significant improvements have been obtained as the number of training sessions increased. These subjects reached a spelling speed of respectively 2 char/min and 2.7 char/min.
A trend of performance improvement has been observed in our tests and we believe that with more feedback sessions an even better BCI control could be obtained. Further improvements may be also achieved considering EEG features in the spatial domain (for example with the method of Common Spatial Patterns) and testing other classification algorithms. Moreover alternative symbol arrangements may be investigated in order to speedup the selection process and the accuracy of language predictions could be improved by means of customized language models.
Part 3: Project tracking
- 01/04/2008: started studying literature about BCI and Motor Imagery
- 01/05/2008: started acquiring data with BCI2000 - performing initial sessions
- 01/06/2008: stated performing feedback sessions with cursor movement in 1D
- 01/09/2008: first idea about the predictive BCI speller
- 10/09/2008: started studying literature about AAC and NLP
- 24/09/2008: published the first document describing the thesis
- 21/10/2008: started studying the thesis about "Soothsayer" by Matteo Vescovi
- 21/10/2008: started studying how to implement in BCI2000 the algorithm of weights adaptation for 2D cursor movement
- 24/10/2008: finished studying the thesis about "Soothsayer" by Matteo Vescovi
- 24/10/2008: added a chapter in thesis description about the weights adaptation algorithm
- 31/10/2008: thesis description document update: added a detailed description of the BCI, new appendix about R^2 calculation
- 03/11/2008: thesis description document update: started a new chapter about letter prediction
- 03/11/2008: thesis description document update: updated the section on weights adaptation
- 05/12/2008: implemented two algorithms for weight adaptation: RLMS and Delta rule
- 10/12/2008: implemented a batch MATLAB script for the off-line analysis
- 15/12/2008: implemented a MATLAB script to simulate the on-line computation of the spectrum
- 20/12/2008: designed classes for the Speller application
- 02/01/2009: started implementing the Speller application in BCI2000
- 23/01/2009: first demo of the Speller controlled by keyboard and without letter prediction
- 05/02/2009: demo of the Speller with letter prediciton
- 05/02/2009: working on feature selection/extraction for the offline classification of the EEG
- 10/02/2009: first result data about classification accuracy inside the same session
- 10/02/2009: working on classification accuracy across sessions
- 15/02/2009: developing Speller application: formally description of the GUI behaviour using a FSM
- 20/02/2009: implemented XML configurability of the interface in the Speller application
- 22/02/2009: implemented a patch for Presage to customize prediction features
- 23/02/2009: generated n-gram statistics from a partition of the British National Corpus
- 25/02/2009: redesign of the interface structure, refactoring of the Speller application code, implementation of new Speller features
- 01/03/2009: working on the Speller simulator program
- 02/03/2009: first version of the Speller simulator without errors and with pre-calculated paths
- 07/03/2009: Speller simulator implemented as a stack machine, including error handling and a breadth-first path-finding algorithm
- 12/03/2009: formal description (with Push Down Automata) of two interfaces and simulation of their performances with different probabilities
- 18/03/2009: working on feature selection for EEG classification
- 21/03/2009: implemented a genetic algorithm for feature selection
- 23/03/2009: working on phase lock value
- 27/03/2009: finished implementing phase lock value, relevant correlations found
- 27/03/2009: started implementing the classification algorithm in BCI2000
- 09/04/2009: finished implemented the feedback application module in BCI2000
- 09/04/2009: started writing the thesis
- 09/04/2009: started testing the spelling application
- 18/06/2009: thesis document completed
- 21/07/2009: thesis presentation!
Part 4: Documents
- Thesis document: A predictive speller for a brain-computer interface based on motor imagery [En]
- Presentation A predictive speller for a brain-computer interface based on motor imagery (presentation)[En]
- Video A predictive speller for a brain-computer interface based on motor imagery (video)
Part 5: References
- Control of two-dimensional movement signals by a noninvasive brain-computer interface in humans, Wolpaw J.R., McFarland J., PNAS, vol. 101, no. 51, december 2004, pages 17849-17854.
- Brain Computer interfaces for communication and control, Wolpaw J.R., Birbaumer N., McFarland D., Pfurtsheller G., Vaughan T., Clinical Neurophysiology 113, 2002, 767-791
- EEG based communication: prospects and problems, Vaughan T., Wolpaw J.R., Donchin E., IEEE transactions on rehabilitation engineering, vol. 4, no. 4, december 1996, pages 425-430.
