Difference between revisions of "Wireless Affective Devices"

Revision as of 16:17, 3 March 2009

Part 1: project profile

Project name

Wireless Affective Devices (before 2008/10/30 only Affective Device)

Project short description

This project aims the development of devices for biological signals acquisition that can be used for Affective computing applications.

Dates

Start date: 2008/01/08

Preliminary release end: 2008/08/08

End date: Unknown (WIP)

Internet site(s)

People involved

Project heads

• Matteo Matteucci
• Simone Tognetti

Other Politecnico di Milano people

• Andrea Bonarini

Students currently working on the project

• Paolo Villa

Students that have worked on the project

• Mattia Colombo (Blood Volume Pressure sensor)
• Alessia Cornaggia (Galvanic skin response sensor)
• Sala Mirko (Galvanic skin response sensor)

Laboratory work and risk analysis

Laboratory work for this project will be mainly performed at AIRLab/Lambrate. It will include electrical and electronic activity. Potentially risky activities are the following:

• Use of mechanical tools. Standard safety measures described in Safety norms will be followed.
• Use of soldering iron. Standard safety measures described in Safety norms will be followed.
• Use of high-voltage circuits. Special gloves and a current limiter will be used. Standard safety measures described in Safety norms will be followed.

Part 2: introduction (why we want do this)

”Everyone knows what an emotion is, until asked to give a definition”
                                        
                                              Beverly Fehr and James Russell

Research on Human-Computer-Interfaces has recently begun to take into account the affective state of the user. By appropriately reacting to the affective user state, interfaces could not only become more pleasant or entertaining, but also more effective or safer. It is known that affective states have bodily correlates, and thus physiological signals could provide the necessary information for acquiring the affective user state. In contrast to other sources of information about the affective user state such as speech or facial expression, most physiological signals are not under voluntary control, and thus cannot be masked up to the same extent. Furthermore, it can be presumed that the signs of affective states in physiological signals are less dependent on individual and contextual factors: this is because the signals correspond to internal physiology, largely related to the autonomous nervous and limbic systems, rather than to external expressions that can be manipulated easily. Recently, the issue of emotion regulation drew interests from several fields including neuroscience, psychology, education, computer science, etc. Possible applications are e-learningin, human computer interaction, development of communicative technologies for use by people with autism, car infotainment system, lie detection tests, cognitive workload, multimodal user interfaces MMUI (allow users to control computers using speech and gesture), etc.

Part 3: project description

The project is aimed at developing a device, connected to a PC, to acquire biological signals (such as GSR, BVP, Temperature and Head orientation) in a way that is less invasive for the subject. The term invasive is related to how much the sensors we use compromise the normal interaction between subject and machine. Through these biological signal the PC is able o understand emotion in people who wear the device.

An affective device should be weared easily, should not have cables and should acquire as much sensors as possible. We relax some constraint on the quality of signal but we add constraint on way they are acquired(i.e. the subject can put it on easily, the usual movements are not influenced..). The aim of the resulting device is not to replace the standard medical devices usually used to acquire that signals, but is to be much more suitable for industrial application in which the medical requirement are replaced by invasivity requirement.

It is composed by:

• Low power, high efficiency, battery powered design
• Wireless comunication with a PC
• Blood Volume Pressure Sensor (Preliminary release by Mattia Colombo)
• Galvanic Skin Response Sensor (Preliminary release by Sala Mirko e Alessia Cornaggia)
• Temperature Sensor
• 3 Way Accelerometer
• Possibility of further extension

GSR (or EDR or EDA)

An often misunderstood and difficult technique, GSR (also known as the electrodermal response EDR or electrodermal activity EDA) has gone through many phases of interest and rejection since the early 1900's and is very popular in psychophysiological studies since Carl Jung and his students (1907) described it as a mean to enter the “sea of the unconscious” because “every stimulus accompanied by an emotion produced a deviation of the galvanometer” directly proportional to the strength of the emotion aroused. It has been used in important research on anxiety and stress levels (Fenz & Epstein, ' 67); and it has been a part of lie detection (Raskin, ' 73). Controversy has centered around the technique, underlying mechanisms, and the meaning of the responses obtained from the skin. There has been a long history of electrodermal activity research, even if most investigators accept the phenomenon without understanding exactly what it means (Hume, ' 76). The resistance of the skin is usually large, approximately 1M; however, momentary changes in the level of the sweat gland activity causes changes in resistance (up to approximately 950K). Physiology, the GSR reflescts sweat gland activity and changes in the sympathetic nervous system and measurement variables. The activity of the sweat glands in response to sympathetic nervous stimulation ( Increased sympathetic activation ) results in an increase in the level of conductance. There ia a relationship between sympathetic activity and emotional arousal, although one cannot identify the specific emotion being elicited. Basically there are two techniques in the history of Electrodermal measurement. In one a current is passed thru the skin and the the resistance to passage is measured; in the other no current is used externally and the skin itself is the source of electrical activity. By applying a conventional 0.5 Volts across the skin and measuring changes in the corresponding conductance, the emotional state of the subject can be inferred. In general, the electrodes used are of the Ag/AgCl type which are recessed from the skin and require the use of a suitable electrode paste. It is important to note, however, that fluctuations in skin conductivity are resultant of many types of arousal. By observing only these changes, it is impossible to deduce without prior knowledge whether the subject has become happy, startled, physically active, etcetera. EDA consists of two components: tonic and phasic. The tonic component is a low frequency baseline conductivity level, which can oscillate over the course of days. The phasic component rides on top of the tonic component, exhibits more rapid fluctuations, and generally increases when a person is aroused. Problematically, each person has a different tonic conductivity, so in order to infer the arousal level of the subject, the relative changes in EDA must be analyzed over a period of time. Furthermore, skin conductance (measured in units of siemens; formerly mhos) depends on the skin path length between the two electrodes contacts, even for subjects with identical skin conductivity (measured in units of siemens/meter). It is for these reasons that it is crucial to analyze the temporal variations of the EDA signal.

BVP

The cardiovascular system is in charge to keep us alive by maintaining the vital blood flow throughout the body, providing nutrients and oxygen to our cells. The SNS controls this system by shifting the flow in response to exercise, temperature,postural and emotional changes. Blood Volume Pulse (BVP) is the most common measure of vasomotor activity because it reflects the phasic pumping of the heart, the vasodilation of vessels that changes in the amount of acral blood delivered. The Blood Volume Pulse (BVP) waveform is an important indicator of circulatory function that can be obtained noninvasively through a photoplethysmographic transducer (a PPG is often obtained by using a pulse oximeter, Shelley and Shelley, 2001) applied to the finger or the earlobe of human subjects. In particular, this signal may be useful in determining the degree of cardiovascular change undergone by a subject through an exercise session.

Principles of photoplethysmographic Technology:

The principle of photoplethysmographic (That is the same of pulse oximetry) is based on the red and infrared light absorption characteristics of oxygenated and deoxygenated hemoglobin. Oxygenated hemoglobin absorbs more infrared light and allows more red light to pass through. Deoxygenated (or reduced) hemoglobin absorbs more red light and allows more infrared light to pass through. Red light is in the 600-750 nm wavelength light band. Pulse oximetry uses a light emitter with red and infrared LEDs that shines through a reasonably translucent site with good blood flow. Opposite the emitter is a photodetector that receives the light that passes through the measuring site. There are two methods of sending light through the measuring site: transmission and reflectance. In the transmission the emitter and photodetector are opposite of each other with the measuring site in-between. The light can then pass through the site. In the reflectance method, the emitter and photodetector are next to each other on top the measuring site. The light bounces from the emitter to the detector across the site. The transmission method is the most common type used. While pulse oximeters are a commonly used medical device the PPG derived from them is rarely displayed, and is nominally only processed to determine heart rate.

Temperature

The temperatures of peripheral limbs vary as a consequence of changes in blood flow.

Previous works

Here is a preliminary release of the headset produced before 2008/08/08

If you are interested on the making of the headset please see making rubber models

* Blood Volume pressure First graph showing a signal acquired with the wireless sensor.

* Skin galvanic response (Sala Mirko e Alessia Cornaggia[1]) Galvanic skin response (GSR), also known as electrodermal response (EDR), psychogalvanic reflex (PGR), or skin conductance response (SCR), is a method of measuring the electrical resistance of the skin. There is a relationship between sympathetic activity and emotional arousal, although one cannot identify the specific emotion being elicited. The GSR is highly sensitive to emotions in some people. Fear, anger, startle response, orienting response and sexual feelings are all among the emotions which may produce similar GSR responses. Response of the skin to the passage of a small electric current. The ease with which the current flows between two points on the skin can be used to indicate stress. When a person is tense or emotional, the sweat glands become more active, increasing moisture on the skin; this allows the electric current to flow more readily. The response may also be used in relaxation training: information about the galvanic skin response is fed back aurally or visually to the subject who can, with practice, learn to increase or decrease sweating on the skin by learning to relax or tense muscles. We measure the galvanic skin response using two electrodes attached to the skin. The optimal placement is the palm of the hands, but we are experimenting other places like on the forehead. The circuit we are creating is the one below.

Part 4: project the device

Re-definition of the complete work

Sensors are an important part of an Affective Computing System because they provide information about the wearer's physical state or behavior. They can gather data in a continuous way without having to interrupt the user. To define in detail electronic specifications and requirements it's necessary a preliminary research/study (in literature and web) about state of art of sensor type used to monitor each type of biological signals we are interested about. In this way it's possible both understand problematics related to interaction between biological and electonical worlds and improve or re-desig existing prototype.

About Sensors:

BVP will be acquire by an IR transmitter-receiver measuring light reflected from skin (photoplethysmography). Particular attention will be put on position, geometry and power absorption of this stage.

GSR will be acquiret by a couple of electrodes forcing a low current (or voltage) and reading voltage (or current) through electrodes. Particular attention will be put on electrodes position and material. In addition is possible to foresee some problems in amplifier stage due to large range of skin resistance. Problem consist in chosing a correct amplification to overwork ADC full scale with a signal affected by little variation but huge possible range.

Temperature will be acquired by thermocouple or PTC/NTC. Slowly variable signal, possible problems with noise due to resolution required.

Head orientation will be acquired by an accelerometer and a gyroscope, problably integrated in the same chip. Two possibilities: analog output (need to digitalize up to six line) or digital output (like displaied in schematic block diagram).

This was the preliminary simplified schematic block diagram of the complete instrumentation setup:

NOTE:

* In schematic is not present Power Supply Module: it will be based on a rechargeable Li-ion
battery. It must satisfy requirements of high efficiency, logn life and low weight.

* ADC and Microcontroller could be unified using a microcontroller with internal ADC and
internal or external multiplexer to select lines.

* Wireless section could be implemeted in Bluetooth (instead of using a XBee module like in
preliminary release) due to it's diffusion in cellular phones, notebooks, MP3 reader...
without necessity of external modules.

It was based on a theoretical approach to any single part. Any sensing device was designed in "conventional" way founded on analog electronic (a bridge with a NTC resisistance as sensing element, discrete IR led and some photodiode around to capture the most reflefted light as possible, GSR electrodes on the gain arm of a non-inverting amplifier...). Unfortunatly a deep analysis on signal physiology, signal transduction, noise vs required accurancy, forced us to a completely different approach implementing everywhere possible a digital interface.

Sensors Design

BVP

Discrete IR leds and phototransistor, used in the first realize (see fig.), had been replaced firt with a single led and one or more photodiodes to optimize power consumption in case of non-planar sensor positioning. Subsequently an integrated Reflective Object Sensor was introduced, with benefits about space saving, transmitter-receiver matching and power consumption. To ensure light power stability, led is powered through a linear led drive LDO (Low Drop-Out). The choise of linear LDO is necessary to ensure no noise or spikes would be generated by charge pump systems or boost converters. In addition, the linear drive utilized requires only one external resitor to set led current and a decoupling capacitor, reducing part counts and space occupation. Reflective sensor is followed by an active band-pass filter followed by a VGA (Variable Gain Amplifier) digitally controlled through I2C Bus. Digitally controlled VGA allow to optimize dinamically ADC input signal amplitude directly during acquisition session, obtaining the best performance from ADC. The resulting circuit is quite compact (led drive, reflective sensor, active filter and variable amplifier), doesn't require much power and represents a good mix between analog and digital approach: analog front-end with acquisition and first conditioning, and at the chain-end a digital controlled amplifier to ensure flexibility.

GSR

GSR will be acquired by a couple of electrodes forcing a low voltage and reading current through electrodes. First designs and implementations were based on literature study: best solution appears to put GSR electrodes on the gain arm of a non-inverting amplifier (see fig.) rather than using a Wheatstone bridge due to week signal produced by this solution. Putting GSR electrodes on the gain arm produces a sort of gain modulation of the constant voltage applied between electrodes but doesn't solve problems in amplifier stage due to large range of skin resistance. Problems in chosing correct amplification to overwork ADC full scale with a signal affected by little variation but huge possible range could be reduced usign a variable resistor on gain arm (for example a digital potentiometer controlled by the microcontroller), but cannot be completely removed. In addition little signals are easily deteriorated by noise. Solution is provided considering a new point of view: measure of unknown skin resistence is extremely similar to measure unknown electrical impedence with an auto-range multimeter! Solution consist in a digital Impedance Converter: AD5934 is a high precision impedance converter system which combines an on-board frequency generator with a 12-bit, 250 kSPS, analog-to-digital converter (ADC). The frequency generator allows an external complex impedance to be excited with a known frequency; the response signal from the impedance is sampled by the on-board ADC and a discrete Fourier transform (DFT) is processed by an on-board DSP engine. The DFT algorithm returns a real (R) and imaginary (I) data-word at each output frequency. This solution not only allows to measure impedence range from 100 Ohm to 10 MOhm, but in adition allow to investigate it across a wide range of frequency, producing a noise-free digital output. The resulting circuit is extremely compact (converter chip, one setting resistor and bypass capacitors), provides a system accuracy of 0.5% and connects to microcontroller using I2C interface, ensuring an high flexibility and the possibility to modify acquisition settings after board production (through microcontroller firmware).

Temperature

First idea was to acquire temperature by a NTC connected in a Wheatstone bridge. Trying to reduce noise, bridge was powered by an oscillator circuit with a frequency of some KHerz, to translate signal out of 1/f noise region. Bridge outputs were connected to an INA amplifier wich also implements signal filtering (see fig.). Weak signal, noise and resolution required (0.1°C) put in crisis the system. In addition other unsolved problems were the optimization of thermal and mechanical contact between sensor and body and the sensor large thermic constant. Unstable thermic contact produces unreliable measures, whereas high sensor thermic constant hampers a fast sensor response (needed to detect short peak due to emotional reactions). Possible solutions to problems of weak signal and noise arrive from digital integrated temperature sensor (for example LM35 or LM74), but these don't solve the problem of thermal contact and thermal constant. In addition none of the sensors take in consideration provides the required resolution. High resolution required, noise immunity, fast response and thermal contact immunity problems had been solved using a digital optical infra red thermometer. The MLX90614 is an infra red thermometer for non contact temperature measurements based on a thermopile detector chip including signal conditioning in the same TO-39 package. Thanks to its low noise amplifier, 17-bit ADC and powerful DSP unit, a high accuracy and resolution of the thermometer is achieved. The calculated object and ambient temperatures are available in RAM of MLX90614 with 0.02°C resolution of: they are accessible by 2 wire serial SMBus compatible protocol. A special version for medical applications offers an accuracy of ±0.1ºC in a limited temperature range around the human body temperature. Small size, no external component requirements and power saving mode allow us to obtain a perfect matching between required performances and space saving.

Accelerometer

Head orientation and movements will be acquired by an accelerometer. First we think to utilize an ST Microelectronic LIS3L02AS4, due to the availability in the Lab of the evaluation board STEVAL-MKI001. It is a 3-axis ±2g/±6g linear accelerometer with 3 analog outputs, so it requires an external ADC to digitalize the three axis acceleration values. The conversion could be performed by a microcontroller with internal ADC. A fast Internet research reveled this sensor is now obsolete and out of production. The choise had been to substitute the old chip with the new one: the LIS302DL (still produced by ST Microelectronic). It's a 3-axis, ± 2g/± 8g smart digital output accelerometer with I2C/SPI digital output interface and really small package. The device is capable of measuring accelerations with an output data rate of 100 Hz or 400 Hz and the self-test capability allows the user to check the functioning of the sensor in the final application. In addition it may be configured to generate inertial wake-up/free-fall interrupt signals when a programmable acceleration threshold is crossed at least in one of the three axes. Thresholds and timing of interrupt generators are programmable by the end user on the fly. This device, providing a digital output, doesn't need signals conversion, guaranteing better noise immunity; programmable interrupts and test interface allow greater fexibility in project. In addition wery small package, only one external resistor requirement (plus baypass capacitor) reduce significantly board space requirements.

NOTE:

* In previous schematics, components phisical dimensions are not in scale.

Microcontroller

Foundamental requirements:

- High speed to ensure high data output rate even in presence of a large number af sensing elemets;

- Fast Internal ADC with multiple inputs;

- (At Least) One I2C interface;

- (At Least) One SPI interface;

- (At Least) One UART interface;

- Low-power consumption;

- Power supply 3.3V;

- Small package;

In addition microcontroller was chosen between Microchip PIC family devices, due to positive previous experiences and availability of software and programmer in the Lab.

PIC24HJ256GP206 is a 16 bit microcontroller capable of 40 MIPS speed. It provides 256KB of Flash memory, 16KB of Ram, a 10-bit 1.1Msps or 12-bit 500Ksps ADC, UART, SPI and I2C interface in a 64 pin TQFP package (10mmX10mm) powerd between 3.0V and 3.6V. Power core is provided by an internal regulator reducing parts count on the board.

The device is able to provide at the same time one I2C interface, two SPI interface and one UART interface: another UART port shares pins with an SPI port and a second I2C port shares pins with the second UART interface. Findig a device that provides at the same time, on indipendent pins, an I2C, a SPI and an UART interface reduces considerably possible choises and suggests to chose a high performance microcontroller. At the same time, high performance solutions have a larger pin counts than standard microcontroller, but this characteristic can be used to ensure further expansibility to the system. The second SPI interface can be used to connect another "intelligent" device or a mass storage device, for example a SD (Secure Digital) card to storage acquired data (see fig.).

Input pins connected to ADC and not used at the moment could be collected to a connector for further extension modules (for example an EMG or ECG peripheral or any other analog signal interesting in a particolar application). For the same reason any other pin non used at the current stage is reported to an extension connector. To guarante maximum flexibility an further update, board provides both ICD and JTAG connections (allowing on-board programming and debugging) through a couple of indipendent dedicated connectors fully compatibles with Microchip programmers and evaluation boards.

Wireless Connection

Wireless connection is based on BlueGiga WT12 module: it's a next-generation, class 2, Bluetooth 2.0+EDR (Enhanced Data Rates) module containing all the necessary elements from Bluetooth radio to antenna and a fully implemented protocol stack. By default WT12 module is equipped with powerful and easy-to-use iWRAP firmware that enables users to access Bluetooth functionality with simple ASCII commands delivered to the module over serial interface like a Bluetooth modem. The dedicated onboard microcontroller (MCU) acts as interrupt controller and event timer run the Bluetooth software stack and control the radio and host interfaces. A 16-bit reduced instruction set computer (RISC) microcontroller is used for low power consumption and efficient use of memory. WT12 module support USB, SPI and UART interface and UART interface is used in this project to connect PIC microcontroller to wireless transmitter (see fig.).

One of the most important WT12 characteristic is the high integration between all parts needed for efficent Bluetooth transmission: user interface, controller, radio circuits and antenna, all included in a low power, low size board (25.6mmx15mm) that requires a minimum number of external components. Many other bluetooth components ara available, such as LMX9830, produced by National Semiconductor. It's an extremely small device (only 6.1mm×9.1 mm), but requires a number of external components (see fig.) like oscillators, resistors, capacitors, inductors and the antenna, in addition to an high competence in radio frequency layout circuits.

Lack of experience in radio frequency circuit suggests to chose a ready-to-use module, factory optimized for best performance and easy to integrate in our board.