User talk:MaurizioGarbarino - Revision history

AndreaBonarini: /* Referenze */

2011-01-12T12:20:08Z

‎Referenze

AndreaBonarini: /* Questions */

2011-01-12T12:14:29Z

‎Questions

AndreaBonarini: /* Questions */

2011-01-12T11:41:08Z

‎Questions

MaurizioGarbarino: /* Questions */

2011-01-09T21:40:59Z

‎Questions

MaurizioGarbarino: /* Questions */

2011-01-09T21:39:28Z

‎Questions

MaurizioGarbarino at 21:07, 9 January 2011

2011-01-09T21:07:32Z

MaurizioGarbarino: New page: ---DRAFT--- In my PhD studies I have been investigating whether physiological signals can be used as means for video game adaptation. With the aim of keeping high the level of enjoyment o...

2011-01-08T12:39:28Z

New page: ---DRAFT--- In my PhD studies I have been investigating whether physiological signals can be used as means for video game adaptation. With the aim of keeping high the level of enjoyment o...

New page

---DRAFT---

In my PhD studies I have been investigating whether physiological
signals can be used as means for video game adaptation.
With the aim of keeping high the level of enjoyment of the player, the
first step was building a model for enjoyment estimation from
physiological signals.
So far, we obtained promising results on the relationship between
variation of physiological signals and variation of enjoyment during a
TORCS game session.
The next step is to study how to exploit the obtained model for the
actual dynamic adaptation of it.
During my staying at your Lab, I would like to focus on the analysis,
design and development of this second step.
Here are some key aspects that i would like to investigate:

- Which aspect of a video game can be modified to best stimulate the
player? How can these be modified at run-time to follow user
physiological signals?
Several issues have to be investigated such as the optimal time window
for adaptation that can depend on the reactivity of the game
controller and on how sensible is player response to the given change
of status in the game.

- Is it possible to formalize a general model valid for a large number
of video game genres?

- How does the adaptation by physiological signals compare to the
adaptation by performance analysis?

- Which video games could be used for the experiment?
Our previous work was based on TORCS. However, it might be interesting
to demonstrate that the model of entertainment obtained is general
enough to be successfully used with other video games as well.

---------------------------

The research questions you pose are interesting indeed. There are three main directions we could investigate
while you are here: 1) the adaptation mechanism for TORCS (as you propose) and/or 2) steps towards re-designing the models via
dissimilar experimental protocols and alternative modeling techniques and/or 3) investigations for the generalizability of the models.
We can discuss the details of these directions while you are here of course.

-----------------

Dear all, I think all the three lines are perfectly ok. If I should express my *preference* I would rank those as 2, 3, and 1.

It would be great if Maurizio could exploit the experience in game design to @itu and define how at theoretical and practical level issues of affective computing:

- how to design a game that could be adapted using biofeedback (meaning affective computing, eye tracking, image processing, whatever)?
- what is the reasonable adaptation rate with this kind of feedback (i.e., how long does it takes to have a steady state responce)? Are we really "controlling" the subject to keep her in the flow? How?
- TORCS is one example, he could proceed with that, but it would be great to define things more general and figure out if and how the protocol and the adaptivity could be verified in other games such as MsPacman or Mario or ...

All of this might require or lead to the answers we all are aiming at:

1) affective computing is really worth or performance analysis could be enough for adaptive gaming?
2) are we really measuring the user fun/engagement or just the stimuli? I mean, the heart rate is increased because of engagement or just because different movements are requested to the user?

I know this broaden even more the project proposal, but having all the ingredients on the table might help to understand the recipe ;-)
----------------------

Quello che manca per dare sostanza al lavoro finale, secondo me, è la parte di adattamento, sia analisi sulle problematiche (tempistiche, possibili parametri su cui agire ecc...), sia dei risultati su un esperimento completo.

------------------------

Colgo l'occasione per riassumerti come stanno andando le cose. L'ultima settimana ahimé l'ho passata a lavorare sulla relazione per il minore e sulla presentazione per l'ultimo esame (Microcontrollori & co. di Zappa).
Lo scorso meeting con Yannakakis abbiamo discusso sulla criticità della qualità della normalizzazione dei segnali dati in ingresso alla rete neurale nell'approcio neuroevolutionary. Infatti, dai primi test che avevo fatto, erano emersi due problemi: diverse esecuzioni dell'algoritmo di feature selection davano sottoinsiemi di features molto disomogenei e con performance che variavano dal 72% all'80%. E inoltre, alcune delle features selezionate erano sospette in quanto "ad occhio" non avremmo mai detto che sarebbero rientrate nella selezione, come ad esempio il delta tra il tempo del valore minimo e il tempo del valore massimo dell'heart rate o della temperatura che sulla carta non dovrebbero essere molto discriminanti.
Altro problema, con il numero di features che abbiamo, una rete neurale complessa ci mette molto tempo per convergere ad una soluzione ottima. L'ultimo pensiero va al fatto che le performance variano molto a seconda del fold considerato (durante la crossvalidazione). Questo è sintomo del fatto che potrebbero esserci dei cluster di player per cui un approccio di player modeling potrebe portare a migliori performance. Però su quest'ultimo aspetto non credo di avere tempo a sufficienza per investigare...

---------------------------------

@@ Line 12: / Line 12: @@
   Several issues have to be investigated such as the optimal time window for adaptation that can depend on the reactivity of the game controller and on how sensible is player response to the given change of status in the game.
-'''Andrea''' The window also depend on physiological signal dynamics, and this might be investigated also on the signals that we already have. Otherwise a specific test session has to be done (also on few people) just to be sure to search with the physiologically correct tool (and to avoid questions by reviewers with some phisiological knowledge).
+'''Andrea''' The window also depends on physiological signal dynamics, and this might be investigated also on the signals that we already have. Otherwise a specific test session has to be done (also on few people) just to be sure to search with the physiologically correct tool (and to avoid questions by reviewers with some phisiological knowledge).
 .2) How to design a game that could be adapted using biofeedback (meaning affective computing, eye tracking, image processing, whatever)?
@@ Line 19: / Line 19: @@
 Regarding question 2.2, I think it is really complementary to question 2.1. To create a game that successfully adapt its behavior, we have to know what are the parameters that can be evolved and how the player react to them. In  my opinion, these issues are more "humanistic" oriented and probably it is possible to find some discussion about it in the literature. I am not really used to theoretical assumption (maybe I would rather say that I am more used to address problems that need to be "solved"). What I mean is that I do not have a clear picture on how to proceed to answer the 2.2 question.
-'''Andrea''' One problem with feedback and game adaptation is related to the time needed to get enough information to perform adaptation. This is in turn related with physiological time response (see above) and with the frequency of stimulus change. Let's assume that a stimulus is a feature of the game that can be changed over time to adapt the game itself. For instance, in PACMan or Space Invaders can be the number of opponents, their speed or their more or less aggressive behavior. To evaluate the player's response, we have to be sure that he can physiologically perceive the stimulus (his physiology changes). Games to be tested should have features that characterize them in these terms. Before selecting  anew game, check its suitability w.r.t. the goals. Do we want to demonstrate that preference is associated to thrill, or to challenge (as in TORCS, with the opponent close by), or to speed, or to correct action choices (as in PACMan and Space Invaders)? I believe we haveto select one or two features and the relative games and make tests on them.
+'''Andrea''' One problem with feedback and game adaptation is related to the time needed to get enough information to perform adaptation. This is in turn related with physiological time response (see above) and with the frequency of stimulus change. Let's assume that a stimulus is a feature of the game that can be changed over time to adapt the game itself. For instance, in PACMan or Space Invaders can be the number of opponents, their speed or their more or less aggressive behavior. To evaluate the player's response, we have to be sure that he can physiologically perceive the stimulus (his physiology changes). Games to be tested should have features that characterize them in these terms. Before selecting a new game, check its suitability w.r.t. the goals. Do we want to demonstrate that preference is associated to thrill, or to challenge (as in TORCS, with the opponent close by), or to speed, or to correct action choices (as in PACMan and Space Invaders)? I believe we haveto select one or two features and the relative games and make tests on them.
 .1) How does the adaptation by physiological signals compare to the adaptation by performance analysis?
 .2) affective computing is really worth or performance analysis could be enough for adaptive gaming?
 I think this can be "easily" addressed once I will be back and I can meet the student that is working on performance analysis. The comparison should be straight forward and we can try to build a model that keeps the best of the two worlds. Although it is fundamental to not mix the task recognition with the enjoyment recognition. For example in our experiment, it is easy to have a misleading result regarding the button pressing activity: during the ''challenging'' race (as opposite to the other 2 ''boring'' race situation) the player has to fight with the opponent to overtake it (i.e., more button are pressed faster!). This complete analysis of physiological and performance data can be the main topic of an article. Question 3.2 can be addressed also by integrating and comparing works on other experiment on adaptation in videogames in literature.
 ) are we really measuring the user fun/engagement or just the stimuli? I mean, the heart rate is increased because of engagement or just because different movements are requested to the user?
-Probably, to answer this question, a specific control experiment is needed. In alternative, to use the data from our TORCS experiment, it might be possible to train a model that classifies the task and compare it to the model that classifies the enjoyment. Then, we could compare the performance but I am not sure that this approach can really help to answer the question. The main issue here is that often the task and the preference overlap (I mean that players often prefer the challenging race) and therefore the 2 models result to be similar.
+Probably, to answer this question, a specific control experiment is needed. In alternative, to use the data from our TORCS experiment, it might be possible to train a model that classifies the task and compares it to the model that classifies the enjoyment. Then, we could compare the performance but I am not sure that this approach can really help to answer the question. The main issue here is that often the task and the preference overlap (I mean that players often prefer the challenging race) and therefore the 2 models result to be similar.
 ------------------

@@ Line 2: / Line 2: @@
 .1) Is it possible to formalize a general model valid for a large number of video game genres?
-.2) TORCS is one example, he could proceed with that, but it would be great to define things more general and figure out if and how the protocol and the adaptivity could be verified in other games such as MsPacman or Mario or...
+.2) TORCS is one example, we could proceed with that, but it would be great to define things more general and figure out if and how the protocol and the adaptivity could be verified in other games such as MsPacman or Mario or...
-.3) Which video games could be used for the experiment? Our previous work was based on TORCS. However, it might be interesting to demonstrate that the model of entertainment obtained is general enough to be successfully used
+.3) Which video games could be used for the experiment? Our previous work was based on TORCS. However, it might be interesting to demonstrate that the obtained model of entertainment is general enough to be successfully used with other video games as well.
 with other video games as well.
 Towards this direction I'm trying to perform tests using data from two different experiments (our on TORCS and Yannakakis' one with Mazeball). I think some theoretical assumptions have to be done in order to correctly interpret the results. The answer to this question can be the main topic of an article. Testing whether the models obtained from a game can successfully predict the enjoyment on a second game (and vice-versa) could help to answer to these questions.
 .1) Which aspect of a video game can be modified to best stimulate the player? How can these be modified at run-time to follow user physiological signals?
   Several issues have to be investigated such as the optimal time window for adaptation that can depend on the reactivity of the game controller and on how sensible is player response to the given change of status in the game.
 .2) How to design a game that could be adapted using biofeedback (meaning affective computing, eye tracking, image processing, whatever)?
 This questions are really interesting but I think that a specific experiment is needed to answer to every single question. I do not think it is possible to get useful information from the data we have collected during our experiment on TORCS. Thinking about the thesis work of the biomedical guys unfortunately we did not get any reliable results. Although we do have now the full working setup (the interaction between TORCS, the physiological capture device, and the enjoyment model estimator in Matlab) for dynamic adaptation and real time enjoyment estimation. If we think to the right, and rather specific, questions it will be possible to setup some ad-hoc experiments for each questions. As usual, it is fundamental to deeply thing about what we want address *before* starting to perform the experiment :).
-Regarding the question 2.2, I thing it is really complementary to the question 2.1. To create a game that successfully adapt its behavior, we have to know what are the parameters that can be evolved and how the player react to them. In  my opinion, these issues are more "humanistic" oriented and probably it is possible to find some discussion about it in the literature. I am not really used to theoretical assumption (maybe I would rather say that I am more used to address problems that need to be "solved"). What I mean is that I do not have a clear picture on how to proceed to answer the 2.2 question.
+Regarding question 2.2, I think it is really complementary to question 2.1. To create a game that successfully adapt its behavior, we have to know what are the parameters that can be evolved and how the player react to them. In  my opinion, these issues are more "humanistic" oriented and probably it is possible to find some discussion about it in the literature. I am not really used to theoretical assumption (maybe I would rather say that I am more used to address problems that need to be "solved"). What I mean is that I do not have a clear picture on how to proceed to answer the 2.2 question.
 .1) How does the adaptation by physiological signals compare to the adaptation by performance analysis?

@@ Line 3: / Line 3: @@
 .1) Is it possible to formalize a general model valid for a large number of video game genres?
 .2) TORCS is one example, he could proceed with that, but it would be great to define things more general and figure out if and how the protocol and the adaptivity could be verified in other games such as MsPacman or Mario or...
-.3) Which video games could be used for the experiment? Our previous work was based on TORCS. However, it might be interesting to demonstrate that the model of entertainment obtained is general enough to be successfully used with other video games as well.
+.3) Which video games could be used for the experiment? Our previous work was based on TORCS. However, it might be interesting to demonstrate that the model of entertainment obtained is general enough to be successfully used
 Towards this direction I'm trying to perform tests using data from two different experiments (our on TORCS and Yannakakis' one with Mazeball). I think some theoretical assumptions have to be done in order to correctly interpret the results. The answer to this question can be the main topic of an article. Testing whether the models obtained from a game can successfully predict the enjoyment on a second game (and vice-versa) could help to answer to these questions.
@@ Line 26: / Line 27: @@
 ------------------
 Questo è il riassunto di tutto quello che ci siamo detti prima della mia partenza.
 == Deadlines ==

@@ Line 135: / Line 135: @@
 Lo scorso meeting con Yannakakis abbiamo discusso sulla criticità della qualità della normalizzazione dei segnali dati in ingresso alla rete neurale nell'approcio neuroevolutionary. Infatti, dai primi test che avevo fatto, erano emersi due problemi: diverse esecuzioni dell'algoritmo di feature selection davano sottoinsiemi di features molto disomogenei e con performance che variavano dal 72% all'80%. E inoltre, alcune delle features selezionate erano sospette in quanto "ad occhio" non avremmo mai detto che sarebbero rientrate nella selezione, come ad esempio il delta tra il tempo del valore minimo e il tempo del valore massimo dell'heart rate o della temperatura che sulla carta non dovrebbero essere molto discriminanti.
 Altro problema, con il numero di features che abbiamo, una rete neurale complessa ci mette molto tempo per convergere ad una soluzione ottima. L'ultimo pensiero va al fatto che le performance variano molto a seconda del fold considerato (durante la crossvalidazione). Questo è sintomo del fatto che potrebbero esserci dei cluster di player per cui un approccio di player modeling potrebe portare a migliori performance. Però su quest'ultimo aspetto non credo di avere tempo a sufficienza per investigare...
 ---------------------------------

@@ Line 21: / Line 21: @@
 ) are we really measuring the user fun/engagement or just the stimuli? I mean, the heart rate is increased because of engagement or just because different movements are requested to the user?
-Probably to answer this question a specific control experiment is needed. In alternative, Or, we have to think about a re
+Probably, to answer this question, a specific control experiment is needed. In alternative, to use the data from our TORCS experiment, it might be possible to train a model that classifies the task and compare it to the model that classifies the enjoyment. Then, we could compare the performance but I am not sure that this approach can really help to answer the question. The main issue here is that often the task and the preference overlap (I mean that players often prefer the challenging race) and therefore the 2 models result to be similar.
 Qua di seguito riporto le parti salienti delle mail che ci siamo scambiati, anche con Yannakakis giusto per referenza.
 In my PhD studies I have been investigating whether physiological
@@ Line 63: / Line 93: @@
 to demonstrate that the model of entertainment obtained is general
 enough to be successfully used with other video games as well.
 ---------------------------
 The research questions you pose are interesting indeed. There are three main directions we could investigate
 while you are here:
@@ Line 75: / Line 103: @@
 -----------------
 Dear all, I think all the three lines are perfectly ok. If I should express my *preference* I would rank those as 2, 3, and 1.
@@ Line 93: / Line 120: @@
 Quello che manca per dare sostanza al lavoro finale, secondo me, è la parte di adattamento, sia analisi sulle problematiche (tempistiche, possibili parametri su cui agire ecc...), sia dei risultati su un esperimento completo.
 ------------------------

@@ Line 1: / Line 1: @@
----DRAFT---
+== Questions ==
 In my PhD studies I have been investigating whether physiological
@@ Line 36: / Line 66: @@
 ---------------------------
 The research questions you pose are interesting indeed. There are three main directions we could investigate
-while you are here: 1) the adaptation mechanism for TORCS (as you propose) and/or 2) steps towards re-designing the models via
+while you are here:
-dissimilar experimental protocols and alternative modeling techniques and/or 3) investigations for the generalizability of the models.
+) the adaptation mechanism for TORCS (as you propose) and/or
 We can discuss the details of these directions while you are here of course.
 -----------------
 Dear all, I think all the three lines are perfectly ok. If I should express my *preference* I would rank those as 2, 3, and 1.
@@ Line 67: / Line 96: @@
 ------------------------
-Colgo l'occasione per riassumerti come stanno andando le cose. L'ultima settimana ahimé l'ho passata a lavorare sulla relazione per il minore e sulla presentazione per l'ultimo esame (Microcontrollori & co. di Zappa).
 Lo scorso meeting con Yannakakis abbiamo discusso sulla criticità della qualità della normalizzazione dei segnali dati in ingresso alla rete neurale nell'approcio neuroevolutionary. Infatti, dai primi test che avevo fatto, erano emersi due problemi: diverse esecuzioni dell'algoritmo di feature selection davano sottoinsiemi di features molto disomogenei e con performance che variavano dal 72% all'80%. E inoltre, alcune delle features selezionate erano sospette in quanto "ad occhio" non avremmo mai detto che sarebbero rientrate nella selezione, come ad esempio il delta tra il tempo del valore minimo e il tempo del valore massimo dell'heart rate o della temperatura che sulla carta non dovrebbero essere molto discriminanti.
 Altro problema, con il numero di features che abbiamo, una rete neurale complessa ci mette molto tempo per convergere ad una soluzione ottima. L'ultimo pensiero va al fatto che le performance variano molto a seconda del fold considerato (durante la crossvalidazione). Questo è sintomo del fatto che potrebbero esserci dei cluster di player per cui un approccio di player modeling potrebe portare a migliori performance. Però su quest'ultimo aspetto non credo di avere tempo a sufficienza per investigare...
 ---------------------------------