User talk:MaurizioGarbarino
Questions
1.1) Is it possible to formalize a general model valid for a large number of video game genres? 1.2) TORCS is one example, we could proceed with that, but it would be great to define things more general and figure out if and how the protocol and the adaptivity could be verified in other games such as MsPacman or Mario or... 1.3) Which video games could be used for the experiment? Our previous work was based on TORCS. However, it might be interesting to demonstrate that the obtained model of entertainment is general enough to be successfully used with other video games as well.
Towards this direction I'm trying to perform tests using data from two different experiments (our on TORCS and Yannakakis' one with Mazeball). I think some theoretical assumptions have to be done in order to correctly interpret the results. The answer to this question can be the main topic of an article. Testing whether the models obtained from a game can successfully predict the enjoyment on a second game (and vice-versa) could help to answer to these questions.
Andrea Possibly, different games have different aspects that affect the physiological signals (see TheModel in Simone's thesis), possibly independent from the preference or entertainment. In the selection of another game to test with, it might be important to place forecasts about the elements that might affect Phisiological signals. For instance, in TORCS there is no much thrill, and there is no way to lose the possibility to play (say to die), while in another game, unexpected things may happen, or you may have a limited number of lives (as in most of the arcades, like PACMan and SpaceInvaders). In some games you see everything, although things may change quite fast, in others, you discover the environment and face unexpected challenges. I would try to classify these aspects while you are in a Game center: do they have such models? Do they know how to characterize a game? Do they know how game features influence preference? How game features influence physiology? Try to poke around to discover this
2.1) Which aspect of a video game can be modified to best stimulate the player? How can these be modified at run-time to follow user physiological signals? Several issues have to be investigated such as the optimal time window for adaptation that can depend on the reactivity of the game controller and on how sensible is player response to the given change of status in the game.
Andrea The window also depends on physiological signal dynamics, and this might be investigated also on the signals that we already have. Otherwise a specific test session has to be done (also on few people) just to be sure to search with the physiologically correct tool (and to avoid questions by reviewers with some phisiological knowledge).
2.2) How to design a game that could be adapted using biofeedback (meaning affective computing, eye tracking, image processing, whatever)?
This questions are really interesting but I think that a specific experiment is needed to answer to every single question. I do not think it is possible to get useful information from the data we have collected during our experiment on TORCS. Thinking about the thesis work of the biomedical guys unfortunately we did not get any reliable results. Although we do have now the full working setup (the interaction between TORCS, the physiological capture device, and the enjoyment model estimator in Matlab) for dynamic adaptation and real time enjoyment estimation. If we think to the right, and rather specific, questions it will be possible to setup some ad-hoc experiments for each questions. As usual, it is fundamental to deeply thing about what we want address *before* starting to perform the experiment :).
Regarding question 2.2, I think it is really complementary to question 2.1. To create a game that successfully adapt its behavior, we have to know what are the parameters that can be evolved and how the player react to them. In my opinion, these issues are more "humanistic" oriented and probably it is possible to find some discussion about it in the literature. I am not really used to theoretical assumption (maybe I would rather say that I am more used to address problems that need to be "solved"). What I mean is that I do not have a clear picture on how to proceed to answer the 2.2 question.
Andrea One problem with feedback and game adaptation is related to the time needed to get enough information to perform adaptation. This is in turn related with physiological time response (see above) and with the frequency of stimulus change. Let's assume that a stimulus is a feature of the game that can be changed over time to adapt the game itself. For instance, in PACMan or Space Invaders can be the number of opponents, their speed or their more or less aggressive behavior. To evaluate the player's response, we have to be sure that he can physiologically perceive the stimulus (his physiology changes). Games to be tested should have features that characterize them in these terms. Before selecting a new game, check its suitability w.r.t. the goals. Do we want to demonstrate that preference is associated to thrill, or to challenge (as in TORCS, with the opponent close by), or to speed, or to correct action choices (as in PACMan and Space Invaders)? I believe we haveto select one or two features and the relative games and make tests on them.
3.1) How does the adaptation by physiological signals compare to the adaptation by performance analysis? 3.2) affective computing is really worth or performance analysis could be enough for adaptive gaming?
I think this can be "easily" addressed once I will be back and I can meet the student that is working on performance analysis. The comparison should be straight forward and we can try to build a model that keeps the best of the two worlds. Although it is fundamental to not mix the task recognition with the enjoyment recognition. For example in our experiment, it is easy to have a misleading result regarding the button pressing activity: during the challenging race (as opposite to the other 2 boring race situation) the player has to fight with the opponent to overtake it (i.e., more button are pressed faster!). This complete analysis of physiological and performance data can be the main topic of an article. Question 3.2 can be addressed also by integrating and comparing works on other experiment on adaptation in videogames in literature.
Andrea In a sense, pushing buttons (or also more steering activity) are indeed features describing the task, but their values can be used to identify what the user enjoys more (steering, or doing activity), and this is not negative per se. In this regards, if we would be able to "certify" with physiological signals which game features characterize the preferred game, we might design the game to have them present as much as possible (for instance, always leave at least an opponent close to the player in TORCS, maybe changing it to give variety, by always engaging the player).
4) are we really measuring the user fun/engagement or just the stimuli? I mean, the heart rate is increased because of engagement or just because different movements are requested to the user?
Probably, to answer this question, a specific control experiment is needed. In alternative, to use the data from our TORCS experiment, it might be possible to train a model that classifies the task and compares it to the model that classifies the enjoyment. Then, we could compare the performance but I am not sure that this approach can really help to answer the question. The main issue here is that often the task and the preference overlap (I mean that players often prefer the challenging race) and therefore the 2 models result to be similar.
Andrea Again, maybe users like to move more. If the models are coherent, it's OK. If they are not, then action may affect the physiological response without affecting emotion. It might be the case of a boring games that requires anyway a lot of activity. I do not know if activity is so much in TORCS that we can see these effects. Possibly a specific experiment has to be defined.
Questo è il riassunto di tutto quello che ci siamo detti prima della mia partenza.
Deadlines
Qua di seguito ci sono le date importanti per le 2 conferenze da non mancare. Non ho trovato altri eventi interessanti a cui poter puntare. Se ne dovessero saltare fuori li aggiungiamo qua sotto!
January 31, 2011 Tutorial and special session proposal deadline
March 15, 2011 - Paper submission deadline
August 31, 2011 - Conference starts
April 1, 2011 - Regular papers due
April 15, 2011 - Abstracts for demos due
June 1, 2011 - Acceptance notification
June 21, 2011 - Final versions due
October 9, 2011 - Conference starts
Referenze
Qua di seguito riporto le parti salienti delle mail che ci siamo scambiati, anche con Yannakakis giusto per referenza.
In my PhD studies I have been investigating whether physiological signals can be used as means for video game adaptation. With the aim of keeping high the level of enjoyment of the player, the first step was building a model for enjoyment estimation from physiological signals. So far, we obtained promising results on the relationship between variation of physiological signals and variation of enjoyment during a TORCS game session. The next step is to study how to exploit the obtained model for the actual dynamic adaptation of it. During my staying at your Lab, I would like to focus on the analysis, design and development of this second step. Here are some key aspects that i would like to investigate:
- Which aspect of a video game can be modified to best stimulate the player? How can these be modified at run-time to follow user physiological signals? Several issues have to be investigated such as the optimal time window for adaptation that can depend on the reactivity of the game controller and on how sensible is player response to the given change of status in the game.
- Is it possible to formalize a general model valid for a large number of video game genres?
- How does the adaptation by physiological signals compare to the adaptation by performance analysis?
- Which video games could be used for the experiment? Our previous work was based on TORCS. However, it might be interesting to demonstrate that the model of entertainment obtained is general enough to be successfully used with other video games as well.
The research questions you pose are interesting indeed. There are three main directions we could investigate while you are here:
1) the adaptation mechanism for TORCS (as you propose) and/or 2) steps towards re-designing the models via dissimilar experimental protocols and alternative modeling techniques and/or 3) investigations for the generalizability of the models.
We can discuss the details of these directions while you are here of course.
Dear all, I think all the three lines are perfectly ok. If I should express my *preference* I would rank those as 2, 3, and 1.
It would be great if Maurizio could exploit the experience in game design to @itu and define how at theoretical and practical level issues of affective computing:
- how to design a game that could be adapted using biofeedback (meaning affective computing, eye tracking, image processing, whatever)? - what is the reasonable adaptation rate with this kind of feedback (i.e., how long does it takes to have a steady state responce)? Are we really "controlling" the subject to keep her in the flow? How? - TORCS is one example, he could proceed with that, but it would be great to define things more general and figure out if and how the protocol and the adaptivity could be verified in other games such as MsPacman or Mario or ...
All of this might require or lead to the answers we all are aiming at:
1) affective computing is really worth or performance analysis could be enough for adaptive gaming? 2) are we really measuring the user fun/engagement or just the stimuli? I mean, the heart rate is increased because of engagement or just because different movements are requested to the user?
I know this broaden even more the project proposal, but having all the ingredients on the table might help to understand the recipe ;-)
Quello che manca per dare sostanza al lavoro finale, secondo me, è la parte di adattamento, sia analisi sulle problematiche (tempistiche, possibili parametri su cui agire ecc...), sia dei risultati su un esperimento completo.
Il primo obiettivo che abbiamo stabilito è l'integrazione dell'algoritmo che hanno presentato a CIG per la stima della funzione di preferenza. Al posto del metodo lineare che abbiamo usato noi, loro usano una rete neurale appresa tramite un algoritmo genetico. Fatto questo, proverò ad applicare il nostro metodo lineare ai loro dati per poi vedere come si comporta il modello appreso su un esperimento quando viene applicato ad un altro set di dati derivanti da un'altra tipologia di esperimento.
Lo scorso meeting con Yannakakis abbiamo discusso sulla criticità della qualità della normalizzazione dei segnali dati in ingresso alla rete neurale nell'approcio neuroevolutionary. Infatti, dai primi test che avevo fatto, erano emersi due problemi: diverse esecuzioni dell'algoritmo di feature selection davano sottoinsiemi di features molto disomogenei e con performance che variavano dal 72% all'80%. E inoltre, alcune delle features selezionate erano sospette in quanto "ad occhio" non avremmo mai detto che sarebbero rientrate nella selezione, come ad esempio il delta tra il tempo del valore minimo e il tempo del valore massimo dell'heart rate o della temperatura che sulla carta non dovrebbero essere molto discriminanti.
Altro problema, con il numero di features che abbiamo, una rete neurale complessa ci mette molto tempo per convergere ad una soluzione ottima. L'ultimo pensiero va al fatto che le performance variano molto a seconda del fold considerato (durante la crossvalidazione). Questo è sintomo del fatto che potrebbero esserci dei cluster di player per cui un approccio di player modeling potrebe portare a migliori performance. Però su quest'ultimo aspetto non credo di avere tempo a sufficienza per investigare...
Andrea Mi sembra che, rispetto ai piani citati sopra nelle mail scambiate, siamo fini a ridurre un po' tanto. Cerca almeno di cogliere dalle loro competenze gli aspetti citati nei miei commenti sopra. Le cose dette nelle references qui sopra non sono poi cosi' scollegate da quanto riportato all'inizio. Ci sarebbe da lavorarci. prova a fare uno schedule di cose da fare e a metterlo qui sotto.
