AIRWiki - User contributions [en]

Development an Artificial Intelligence System solving the MS Pac-Man videogame

2008-07-18T14:33:26Z

SandroBaranzini: /* '''Part 2: project description''' */

== '''Part 1: project profile''' ==

=== Project name ===

Development an Artificial Intelligence System solving the videogame MS Pac-Man

=== Project short description ===

The project is about the use of the neuro evolutionary algorithm NEAT in order to create one complex neural net able to solve the MS PacMan game. The algorithm is apply and modify in the way to accelerate the learning process an reach our goals, so we have to find a Neural Net explore the maze, escape from the ghosts, eat all the pill in the maze and finally maximise the score.

=== Dates ===
Start date: 2008/01/07

End date: 2008/09/30

=== Website(s) ===

http://cswww.essex.ac.uk/staff/sml/pacman/PacManContest.html

http://anji.sourceforge.net/

=== People involved ===

===== Project head(s) =====

P.L.Lanzi - [[User:PierLucaLanzi]]

D.Loiacono - [[User:DanieleLoiacono]]

===== Other Politecnico di Milano people =====

===== Students currently working on the project =====

Sandro Baranzini - [[User:SandroBaranzini]]

===== Students who worked on the project in the past =====

===== External personnel: =====

=== Laboratory work and risk analysis ===

== '''Part 2: project description''' ==
[Temporary Version]

''' >>> INTRODUCTION '''

This thesis borns from the challenge presented in the WCCI (Wolrd Congress on Computational Intelligence) 2008 placed in Hong Kong in June 2008, in particular from the MSPacMan Competition.
PacMan is a Namco arcade game, born in the 1980, early became most famous and loved by the worldwide audience. The main difference between the original PacMan and the Competition's version MSPacMan is the non deterministic character of Ghost’s movement that bring the MSPacMan version more interesting in the Artificial Intelligence science than the original one.
So this Thesis work is to find an AI formalism able to play the MSPacMan, controlling the agent in order to escape from Ghosts, living and score as much as he can. The main difficulties are about the non deterministic character of the Ghosts, the complexity of the maze, presenting during the level in three stages (see next figure), and the long time simulation needed by whatever Machine Learning algorithm.

[[Image:MSPacMan.jpg]]

Initially we decide to using the neuro evolutionary algorithm calling NEAT created by Kenneth O. Stanley and implemented in C++ in its first version. Many other version was born soon, in particular ANJI, the version we use, a good implemented and documented version written in Java. The main idea of this Algorithm is to merge the benefits of the Neural Nets, with Genetic Algorithm’s great evolutional feature. Through epoch Generations, Neural Nets Populations evolve, thanks to Crossover, Mutation and Speciations, guided by a Fitness function that measure the single Neural Net goodness for our goal.

''' >>> SOFTWARE AVAILABLE '''

We have three software system have to integrated themselves and cooperate during long time Simulation. So we got the game MSPacMan, the Microsoft version included in the “Return Of Arcade – Anniversary Edition“ pack, the competition’s software that capture the screen image, elaborated it pixel for pixel and reassume all the information in a java Class creating the game state; it was necessary modified it in order to “insert coin”, and “start game” automatically, respectively pressing the F2 and F3 key. Finally we have the ANJI source code that we had integrated with competition’s software in the way each Neural Net can play the game, allowing us to measure its fitness.

''' >>> EARLY ATTEMPTS '''

First of all we have to choose the inputs of the Nets; we have many variables in our problem every stored in the Game State Class. So we start the simulations assumed that the MSPAcMan is a classical Predator-Prey problem. So we choose, like problem inputs, the position of the agent, the positions of ghosts, the wall of the maze in all of four cardinal point centered in the agent, the last four inputs are the direction of ghosts. In these early simulations we measure the Fitness like agent’s lifetime counter.

[[Image:Sim1Results.jpg]]

Results of these early attempts was no satisfactory, see the figure above with the graphical representation of moving average of fitness during 40 generations, and shows us that the net needs more generations to evolve and the information about the closest pill by the PacMan in order to give agent the capacity to explore the maze in the direction of remaining pills. So we did, and doing again today, other simulation with less input, a local representation of the agent, of ghosts and of the closest pill; in addition we measure the Fitness like the game score. We get better results, you can see in next figure that the process of learning able to increase the score about 1000 points, that confirm us that’s the right direction to solve the problem ….

[[Image:ScoreGen.jpg]]

Development an Artificial Intelligence System solving the MS Pac-Man videogame

2008-07-17T17:03:00Z

SandroBaranzini: /* '''Part 2: project description''' */

== '''Part 1: project profile''' ==

=== Project name ===

Development an Artificial Intelligence System solving the videogame MS Pac-Man

=== Project short description ===

The project is about the use of the neuro evolutionary algorithm NEAT in order to create one complex neural net able to solve the MS PacMan game. The algorithm is apply and modify in the way to accelerate the learning process an reach our goals, so we have to find a Neural Net explore the maze, escape from the ghosts, eat all the pill in the maze and finally maximise the score.

=== Dates ===
Start date: 2008/01/07

End date: 2008/09/30

=== Website(s) ===

http://cswww.essex.ac.uk/staff/sml/pacman/PacManContest.html

http://anji.sourceforge.net/

=== People involved ===

===== Project head(s) =====

P.L.Lanzi - [[User:PierLucaLanzi]]

D.Loiacono - [[User:DanieleLoiacono]]

===== Other Politecnico di Milano people =====

===== Students currently working on the project =====

Sandro Baranzini - [[User:SandroBaranzini]]

===== Students who worked on the project in the past =====

===== External personnel: =====

=== Laboratory work and risk analysis ===

== '''Part 2: project description''' ==
[Temporary Version]

''' >>> INTRODUCTION '''

This thesis borns from the challenge presented in the WCCI (Wolrd Congress on Computational Intelligence) 2008 placed in Hong Kong in June 2008, in particular from the MSPacMan Competition.
PacMan is a Namco arcade game, born in the 1980, early became most famous and loved by the worldwide audience. The main difference beetween the original PacMan and the Competition's version MSPacMan is the non deterministic character of Ghost’s movement that bring the MSPacMan version more interesting in the Artificial Intelligence science than the original one.
So this Thesis work is to find an AI formalism able to play the MSPacMan, controlling the agent in order to escape from Ghosts, living and score as much as he can. The main difficulties are about the non deterministic character of the Ghosts, the complexity of the maze, presenting during the level in three stages (see next figure), and the long time simulation needed by whatever Machine Learning algorithm.

[[Image:MSPacMan.jpg]]

Initially we decide to using the neuro evolutionary algorithm calling NEAT created by Kenneth O. Stanley and implemented in C++ in its first version. Many other version was born soon, in particular ANJI, the version we use, a good implemented and documented version written in Java. The main idea of this Algorithm is to merge the benefits of the Neural Nets, with Genetic Algorithm’s great evolutional feature. Through epoch Generations, Neural Nets Populations evolve, thanks to Crossover, Mutation and Speciations, guided by a Fitness function that measure the single Neural Net goodness for our goal.

''' >>> SOFTWARE AVAILABLE '''

We have three software system have to integrated themselves and cooperate during long time Simulation. So we got the game MSPacMan, the Microsoft version included in the “Return Of Arcade – Anniversary Edition“ pack, the competition’s software that capture the screen image, elaborated it pixel for pixel and reassume all the information in a java Class creating the game state; it was necessary modified it in order to “insert coin”, and “start game” automatically, respectively pressing the F2 and F3 key. Finally we have the ANJI source code that we had integrated with competition’s software in the way each Neural Net can play the game, allowing us to measure its fitness.

''' >>> EARLY ATTEMPTS '''

First of all we have to choose the inputs of the Nets; we have many variables in our problem every stored in the Game State Class. So we start the simulations assumed that the MSPAcMan is a classical Predator-Prey problem. So we choose, like problem inputs, the position of the agent, the positions of ghosts, the wall of the maze in all of four cardinal point centered in the agent, the last four inputs are the direction of ghosts. In these early simulations we measure the Fitness like agent’s lifetime counter.

[[Image:Sim1Results.jpg]]

Results of these early attempts was no satisfactory, see the figure above with the graphical representation of moving average of fitness during 40 generations, and shows us that the net needs more generations to evolve and the information about the closest pill by the PacMan in order to give agent the capacity to explore the maze in the direction of remaining pills. So we did, and doing again today, other simulation with less input, a local representation of the agent, of ghosts and of the closest pill; in addition we measure the Fitness like the game score. We get better results, you can see in next figure that the process of learning able to increase the score about 1000 points, that confirm us that’s the right direction to solve the problem ….

[[Image:ScoreGen.jpg]]

User:SandroBaranzini

2008-07-17T17:02:35Z

SandroBaranzini:

Sandro Baranzini
born in Milan, Italy on February 16, 1983

Graduated on September 25, 2005
in Software Engineering with valutation of 108/110
at Politecnico di Milano

Ended the exams in Specialistic Software Engineering (Robotic and Artificial Intelligence field) I'm constantly working on my thesis about Neuro Evolutional Algorithm applied at MSPacMan videogame that I hope to finish in October 2008.

Actually I'm working in a company development and provides Computerized Numerical Controls for milling (and other kind) machines. In particular I'm a Firmware programmer specialized on Motion Control Algorithm.

Development an Artificial Intelligence System solving the MS Pac-Man videogame

2008-07-17T16:29:49Z

SandroBaranzini: /* '''Part 2: project description''' */

== '''Part 1: project profile''' ==

=== Project name ===

Development an Artificial Intelligence System solving the videogame MS Pac-Man

=== Project short description ===

The project is about the use of the neuro evolutionary algorithm NEAT in order to create one complex neural net able to solve the MS PacMan game. The algorithm is apply and modify in the way to accelerate the learning process an reach our goals, so we have to find a Neural Net explore the maze, escape from the ghosts, eat all the pill in the maze and finally maximise the score.

=== Dates ===
Start date: 2008/01/07

End date: 2008/09/30

=== Website(s) ===

http://cswww.essex.ac.uk/staff/sml/pacman/PacManContest.html

http://anji.sourceforge.net/

=== People involved ===

===== Project head(s) =====

P.L.Lanzi - [[User:PierLucaLanzi]]

D.Loiacono - [[User:DanieleLoiacono]]

===== Other Politecnico di Milano people =====

===== Students currently working on the project =====

Sandro Baranzini - [[User:SandroBaranzini]]

===== Students who worked on the project in the past =====

===== External personnel: =====

=== Laboratory work and risk analysis ===

== '''Part 2: project description''' ==
[Temporary Version]

''' >>> INTRODUCTION '''

This thesis born from the challenge presented in the WCCI (Wolrd Congress on Computational Intelligence) 2008 placed in Hong Kong in June 2008, in particular from the MSPacMan Competition.
PacMan is a Namco arcade game, born in the 1980, early became most famous and loved by the worldwide audience. The main difference beetween the original PacMan and the Competition's version MSPacMan is the non deterministic character of Ghost’s movement that bring the MSPacMan version more interesting in the Artificial Intelligence science than the original one.
So this Thesis work is to find an AI formalism able to play the MSPacMan, controlling the agent in order to escape from Ghosts, living and score as much as he can. The main difficulties are about the non deterministic character of the Ghosts, the complexity of the maze, presenting during the level in three stages (see next figure), and the long time simulation needed by whatever Machine Learning algorithm.

[[Image:MSPacMan.jpg]]

Initially we decide to using the neuro evolutionary algorithm calling NEAT created by Kenneth O. Stanley and implemented in C++ in its first version. Many other version was born soon, in particular ANJI, the version we use, a good implemented and documented version written in Java. The main idea of this Algorithm is to merge the benefits of the Neural Nets, with Genetic Algorithm’s great evolutional feature. Through epoch Generations, Neural Nets Populations evolve, thanks to Crossover, Mutation and Speciations, guided by a Fitness function that measure the single Neural Net goodness for our goal.

''' >>> SOFTWARE AVAILABLE '''

We have three software system have to integrated themselves and cooperate during long time Simulation. So we got the game MSPacMan, the Microsoft version included in the “Return Of Arcade – Anniversary Edition“ pack, the competition’s software that capture the screen image, elaborated it pixel for pixel and reassume all the information in a java Class creating the game state; it was necessary modified it in order to “insert coin”, and “start game” automatically, respectively pressing the F2 and F3 key. Finally we have the ANJI source code that we had integrated with competition’s software in the way each Neural Net can play the game, allowing us to measure its fitness.

''' >>> EARLY ATTEMPTS '''

First of all we have to choose the inputs of the Nets; we have many variables in our problem every stored in the Game State Class. So we start the simulations assumed that the MSPAcMan is a classical Predator-Prey problem. So we choose, like problem inputs, the position of the agent, the positions of ghosts, the wall of the maze in all of four cardinal point centered in the agent, the last four inputs are the direction of ghosts. In these early simulations we measure the Fitness like agent’s lifetime counter.

[[Image:Sim1Results.jpg]]

Results of these early attempts was no satisfactory, see the figure above with the graphical representation of moving average of fitness during 40 generations, and shows us that the net needs more generations to evolve and the information about the closest pill by the PacMan in order to give agent the capacity to explore the maze in the direction of remaining pills. So we did, and doing again today, other simulation with less input, a local representation of the agent, of ghosts and of the closest pill; in addition we measure the Fitness like the game score. We get better results, you can see in next figure that the process of learning able to increase the score about 1000 points, that confirm us that’s the right direction to solve the problem ….

[[Image:ScoreGen.jpg]]

Development an Artificial Intelligence System solving the MS Pac-Man videogame

2008-07-17T16:27:17Z

SandroBaranzini: /* '''Part 2: project description''' */

== '''Part 1: project profile''' ==

=== Project name ===

Development an Artificial Intelligence System solving the videogame MS Pac-Man

=== Project short description ===

The project is about the use of the neuro evolutionary algorithm NEAT in order to create one complex neural net able to solve the MS PacMan game. The algorithm is apply and modify in the way to accelerate the learning process an reach our goals, so we have to find a Neural Net explore the maze, escape from the ghosts, eat all the pill in the maze and finally maximise the score.

=== Dates ===
Start date: 2008/01/07

End date: 2008/09/30

=== Website(s) ===

http://cswww.essex.ac.uk/staff/sml/pacman/PacManContest.html

http://anji.sourceforge.net/

=== People involved ===

===== Project head(s) =====

P.L.Lanzi - [[User:PierLucaLanzi]]

D.Loiacono - [[User:DanieleLoiacono]]

===== Other Politecnico di Milano people =====

===== Students currently working on the project =====

Sandro Baranzini - [[User:SandroBaranzini]]

===== Students who worked on the project in the past =====

===== External personnel: =====

=== Laboratory work and risk analysis ===

== '''Part 2: project description''' ==
[Temporary Version]

'''%%% INTRODUCTION %%%'''

This thesis born from the challenge presented in the WCCI (Wolrd Congress on Computational Intelligence) 2008 placed in Hong Kong in June 2008, in particular from the MSPacMan Competition.
PacMan is a Namco arcade game, born in the 1980, early became most famous and loved by the worldwide audience. The main difference beetween the original PacMan and the Competition's version MSPacMan is the non deterministic character of Ghost’s movement that bring the MSPacMan version more interesting in the Artificial Intelligence science than the original one.
So this Thesis work is to find an AI formalism able to play the MSPacMan, controlling the agent in order to escape from Ghosts, living and score as much as he can. The main difficulties are about the non deterministic character of the Ghosts, the complexity of the maze, presenting during the level in three stages (see next figure), and the long time simulation needed by whatever Machine Learning algorithm.

[[Image:MSPacMan.jpg]]

Initially we decide to using the neuro evolutionary algorithm calling NEAT created by Kenneth O. Stanley and implemented in C++ in its first version. Many other version was born soon, in particular ANJI, the version we use, a good implemented and documented version written in Java. The main idea of this Algorithm is to merge the benefits of the Neural Nets, with Genetic Algorithm’s great evolutional feature. Through epoch Generations, Neural Nets Populations evolve, thanks to Crossover, Mutation and Speciations, guided by a Fitness function that measure the single Neural Net goodness for our goal.

'''%%% SOFTWARE AVAILABLE %%%'''

We have three software system have to integrated themselves and cooperate during long time Simulation. So we got the game MSPacMan, the Microsoft version included in the “Return Of Arcade – Anniversary Edition“ pack, the competition’s software that capture the screen image, elaborated it pixel for pixel and reassume all the information in a java Class creating the game state; it was necessary modified it in order to “insert coin”, and “start game” automatically, respectively pressing the F2 and F3 key. Finally we have the ANJI source code that we had integrated with competition’s software in the way each Neural Net can play the game, allowing us to measure its fitness.

'''%%% EARLY ATTEMPTS %%%'''

First of all we have to choose the inputs of the Nets; we have many variables in our problem every stored in the Game State Class. So we start the simulations assumed that the MSPAcMan is a classical Predator-Prey problem. So we choose, like problem inputs, the position of the agent, the positions of ghosts, the wall of the maze in all of four cardinal point centered in the agent, the last four inputs are the direction of ghosts. In these early simulations we measure the Fitness like agent’s lifetime counter.

[[Image:Sim1Results.jpg]]

Results of these early attempts was no satisfactory, see the figure above with the graphical representation of moving average of fitness during 40 generations, and shows us that the net needs more generations to evolve and the information about the closest pill by the PacMan in order to give agent the capacity to explore the maze in the direction of remaining pills. So we did, and doing again today, other simulation with less input, a local representation of the agent, of ghosts and of the closest pill; in addition we measure the Fitness like the game score. We get better results, show in next figure, that confirm us that’s the right direction to solve the problem ….

[[Image:ScoreGen.jpg]]

File:ScoreGen.jpg

2008-07-17T16:18:48Z

SandroBaranzini: Score as fitness during 200 generations

Score as fitness during 200 generations

Development an Artificial Intelligence System solving the MS Pac-Man videogame

2008-07-17T16:18:04Z

SandroBaranzini: /* '''Part 2: project description''' */

== '''Part 1: project profile''' ==

=== Project name ===

Development an Artificial Intelligence System solving the videogame MS Pac-Man

=== Project short description ===

The project is about the use of the neuro evolutionary algorithm NEAT in order to create one complex neural net able to solve the MS PacMan game. The algorithm is apply and modify in the way to accelerate the learning process an reach our goals, so we have to find a Neural Net explore the maze, escape from the ghosts, eat all the pill in the maze and finally maximise the score.

=== Dates ===
Start date: 2008/01/07

End date: 2008/09/30

=== Website(s) ===

http://cswww.essex.ac.uk/staff/sml/pacman/PacManContest.html

http://anji.sourceforge.net/

=== People involved ===

===== Project head(s) =====

P.L.Lanzi - [[User:PierLucaLanzi]]

D.Loiacono - [[User:DanieleLoiacono]]

===== Other Politecnico di Milano people =====

===== Students currently working on the project =====

Sandro Baranzini - [[User:SandroBaranzini]]

===== Students who worked on the project in the past =====

===== External personnel: =====

=== Laboratory work and risk analysis ===

== '''Part 2: project description''' ==
[Temporary Version]

'''%%% INTRODUCTION %%%'''

This thesis born from the challenge presented in the WCCI (Wolrd Congress on Computational Intelligence) 2008 placed in Hong Kong in June 2008, in particular from the MSPacMan Competition.
PacMan is a Namco arcade game, born in the 1980, early became most famous and loved by the worldwide audience. The main difference beetween the original PacMan and the Competition's version MSPacMan is the non deterministic character of Ghost’s movement that bring the MSPacMan version more interesting in the Artificial Intelligence science than the original one.
So this Thesis work is to find an AI formalism able to play the MSPacMan, controlling the agent in order to escape from Ghosts, living and score as much as he can. The main difficulties are about the non deterministic character of the Ghosts, the complexity of the maze, presenting during the level in three stages (see next figure), and the long time simulation needed by whatever Machine Learning algorithm.

[[Image:MSPacMan.jpg]]

Initially we decide to using the neuro evolutionary algorithm calling NEAT created by Kenneth O. Stanley and implemented in C++ in its first version. Many other version was born soon, in particular ANJI, the version we use, a good implemented and documented version written in Java. The main idea of this Algorithm is to merge the benefits of the Neural Nets, with Genetic Algorithm’s great evolutional feature. Through epoch Generations, Neural Nets Populations evolve, thanks to Crossover, Mutation and Speciations, guided by a Fitness function that measure the single Neural Net goodness for our goal.

'''%%% SOFTWARE AVAILABLE %%%'''

We have three software system have to integrated themselves and cooperate during long time Simulation. So we got the game MSPacMan, the Microsoft version included in the “Return Of Arcade – Anniversary Edition“ pack, the competition’s software that capture the screen image, elaborated it pixel for pixel and reassume all the information in a java Class creating the game state; it was necessary modified it in order to “insert coin”, and “start game” automatically, respectively pressing the F2 and F3 key. Finally we have the ANJI source code that we had integrated with competition’s software in the way each Neural Net can play the game, allowing us to measure its fitness.

'''%%% EARLY ATTEMPTS %%%'''

First of all we have to choose the inputs of the Nets; we have many variables in our problem every stored in the Game State Class. So we start the simulations assumed that the MSPAcMan is a classical Predator-Prey problem. So we choose, like problem inputs, the position of the agent, the positions of ghosts, the wall of the maze in all of four cardinal point centered in the agent, the last four inputs are the direction of ghosts. In these early simulations we measure the Fitness like agent’s lifetime counter.

[[Image:Sim1Results.jpg]]

Results of these early attempts was no satisfactory, see the figure above with the graphical representation of the fitness during 20 generations, and shows us that the net needs the information about the closest pill by the PacMan in order to give agent the capacity to explore the maze in the direction of remaining pills. So we did, and doing again today, other simulation with less input, a local representation of the agent, of ghosts and of the closest pill; in addition we measure the Fitness like the game score. We get better results, show in next figure, that confirm us that’s the right direction to solve the problem ….

[[Image:ScoreGen.jpg]]

Development an Artificial Intelligence System solving the MS Pac-Man videogame

2008-07-17T16:15:05Z

SandroBaranzini: /* '''Part 2: project description''' */

== '''Part 1: project profile''' ==

=== Project name ===

Development an Artificial Intelligence System solving the videogame MS Pac-Man

=== Project short description ===

The project is about the use of the neuro evolutionary algorithm NEAT in order to create one complex neural net able to solve the MS PacMan game. The algorithm is apply and modify in the way to accelerate the learning process an reach our goals, so we have to find a Neural Net explore the maze, escape from the ghosts, eat all the pill in the maze and finally maximise the score.

=== Dates ===
Start date: 2008/01/07

End date: 2008/09/30

=== Website(s) ===

http://cswww.essex.ac.uk/staff/sml/pacman/PacManContest.html

http://anji.sourceforge.net/

=== People involved ===

===== Project head(s) =====

P.L.Lanzi - [[User:PierLucaLanzi]]

D.Loiacono - [[User:DanieleLoiacono]]

===== Other Politecnico di Milano people =====

===== Students currently working on the project =====

Sandro Baranzini - [[User:SandroBaranzini]]

===== Students who worked on the project in the past =====

===== External personnel: =====

=== Laboratory work and risk analysis ===

== '''Part 2: project description''' ==
[Temporary Version]

'''%%% INTRODUCTION %%%'''

This thesis born from the challenge presented in the WCCI (Wolrd Congress on Computational Intelligence) 2008 placed in Hong Kong in June 2008, in particular from the MSPacMan Competition.
PacMan is a Namco arcade game, born in the 1980, early became most famous and loved by the worldwide audience. The main difference beetween the original PacMan and the Competition's version MSPacMan is the non deterministic character of Ghost’s movement that bring the MSPacMan version more interesting in the Artificial Intelligence science than the original one.
So this Thesis work is to find an AI formalism able to play the MSPacMan, controlling the agent in order to escape from Ghosts, living and score as much as he can. The main difficulties are about the non deterministic character of the Ghosts, the complexity of the maze, presenting during the level in three stages (see next figure), and the long time simulation needed by whatever Machine Learning algorithm.

[[Image:MSPacMan.jpg]]

Initially we decide to using the neuro evolutionary algorithm calling NEAT created by Kenneth O. Stanley and implemented in C++ in its first version. Many other version was born soon, in particular ANJI, the version we use, a good implemented and documented version written in Java. The main idea of this Algorithm is to merge the benefits of the Neural Nets, with Genetic Algorithm’s great evolutional feature. Through epoch Generations, Neural Nets Populations evolve, thanks to Crossover, Mutation and Speciations, guided by a Fitness function that measure the single Neural Net goodness for our goal.

'''%%% SOFTWARE AVAILABLE %%%'''

We have three software system have to integrated themselves and cooperate during long time Simulation. So we got the game MSPacMan, the Microsoft version included in the “Return Of Arcade – Anniversary Edition“ pack, the competition’s software that capture the screen image, elaborated it pixel for pixel and reassume all the information in a java Class creating the game state; it was necessary modified it in order to “insert coin”, and “start game” automatically, respectively pressing the F2 and F3 key. Finally we have the ANJI source code that we had integrated with competition’s software in the way each Neural Net can play the game, allowing us to measure its fitness.

'''%%% EARLY ATTEMPTS %%%'''

First of all we have to choose the inputs of the Nets; we have many variables in our problem every stored in the Game State Class. So we start the simulations assumed that the MSPAcMan is a classical Predator-Prey problem. So we choose, like problem inputs, the position of the agent, the positions of ghosts, the wall of the maze in all of four cardinal point centered in the agent, the last four inputs are the direction of ghosts. In these early simulations we measure the Fitness like agent’s lifetime counter.

[[Image:Sim1Results.jpg]]

Results of these early attempts was no satisfactory, see the figure above with the graphical representation of the fitness during 20 generations, and shows us that the net needs the information about the closest pill by the PacMan in order to give agent the capacity to explore the maze in the direction of remaining pills. So we did, and doing again today, other simulation with less input, a local representation of the agent, of ghosts and of the closest pill; in addition we measure the Fitness like the game score. We get better results, show in next figure, that confirm us that’s the right direction to solve the problem ….

File:Sim1Results.jpg

2008-07-17T16:13:48Z

SandroBaranzini: Lifetime counter of PacMan, used as fitness, during 20 generation

Lifetime counter of PacMan, used as fitness, during 20 generation

Development an Artificial Intelligence System solving the MS Pac-Man videogame

2008-07-17T16:12:17Z

SandroBaranzini: /* '''Part 2: project description''' */

== '''Part 1: project profile''' ==

=== Project name ===

Development an Artificial Intelligence System solving the videogame MS Pac-Man

=== Project short description ===

The project is about the use of the neuro evolutionary algorithm NEAT in order to create one complex neural net able to solve the MS PacMan game. The algorithm is apply and modify in the way to accelerate the learning process an reach our goals, so we have to find a Neural Net explore the maze, escape from the ghosts, eat all the pill in the maze and finally maximise the score.

=== Dates ===
Start date: 2008/01/07

End date: 2008/09/30

=== Website(s) ===

http://cswww.essex.ac.uk/staff/sml/pacman/PacManContest.html

http://anji.sourceforge.net/

=== People involved ===

===== Project head(s) =====

P.L.Lanzi - [[User:PierLucaLanzi]]

D.Loiacono - [[User:DanieleLoiacono]]

===== Other Politecnico di Milano people =====

===== Students currently working on the project =====

Sandro Baranzini - [[User:SandroBaranzini]]

===== Students who worked on the project in the past =====

===== External personnel: =====

=== Laboratory work and risk analysis ===

== '''Part 2: project description''' ==
[Temporary Version]

'''%%% INTRODUCTION %%%'''

This thesis born from the challenge presented in the WCCI (Wolrd Congress on Computational Intelligence) 2008 placed in Hong Kong in June 2008, in particular from the MSPacMan Competition.
PacMan is a Namco arcade game, born in the 1980, early became most famous and loved by the worldwide audience. The main difference beetween the original PacMan and the Competition's version MSPacMan is the non deterministic character of Ghost’s movement that bring the MSPacMan version more interesting in the Artificial Intelligence science than the original one.
So this Thesis work is to find an AI formalism able to play the MSPacMan, controlling the agent in order to escape from Ghosts, living and score as much as he can. The main difficulties are about the non deterministic character of the Ghosts, the complexity of the maze, presenting during the level in three stages (see next figure), and the long time simulation needed by whatever Machine Learning algorithm.

[[Image:MSPacMan.jpg]]

Initially we decide to using the neuro evolutionary algorithm calling NEAT created by Kenneth O. Stanley and implemented in C++ in its first version. Many other version was born soon, in particular ANJI, the version we use, a good implemented and documented version written in Java. The main idea of this Algorithm is to merge the benefits of the Neural Nets, with Genetic Algorithm’s great evolutional feature. Through epoch Generations, Neural Nets Populations evolve, thanks to Crossover, Mutation and Speciations, guided by a Fitness function that measure the single Neural Net goodness for our goal.

'''%%% SOFTWARE AVAILABLE %%%'''

We have three software system have to integrated themselves and cooperate during long time Simulation. So we got the game MSPacMan, the Microsoft version included in the “Return Of Arcade – Anniversary Edition“ pack, the competition’s software that capture the screen image, elaborated it pixel for pixel and reassume all the information in a java Class creating the game state; it was necessary modified it in order to “insert coin”, and “start game” automatically, respectively pressing the F2 and F3 key. Finally we have the ANJI source code that we had integrated with competition’s software in the way each Neural Net can play the game, allowing us to measure its fitness.

'''%%% EARLY ATTEMPTS %%%'''

First of all we have to choose the inputs of the Nets; we have many variables in our problem every stored in the Game State Class. So we start the simulations assumed that the MSPAcMan is a classical Predator-Prey problem. So we choose, like problem inputs, the position of the agent, the positions of ghosts, the wall of the maze in all of four cardinal point centered in the agent, the last four inputs are the direction of ghosts. In these early simulations we measure the Fitness like agent’s lifetime counter.

[[Image:Sim1Results.jpg]]

Results of these early attempts was no satisfactory, see the Figure above with the graphical representation of the fitness during 20 generations, and shows us that the net needs the information about the closest pill by the PacMan in order to give agent the capacity to explore the maze in the direction of remaining pills. So we did, and doing again today, other simulation with less input, a local representation of the agent, of ghosts and of the closest pill; in addition we measure the Fitness like the game score. We get better results, show in Figure 3, that confirm us that’s the right direction to solve the problem ….

Development an Artificial Intelligence System solving the MS Pac-Man videogame

2008-07-17T16:10:52Z

SandroBaranzini: /* '''Part 2: project description''' */

== '''Part 1: project profile''' ==

=== Project name ===

Development an Artificial Intelligence System solving the videogame MS Pac-Man

=== Project short description ===

The project is about the use of the neuro evolutionary algorithm NEAT in order to create one complex neural net able to solve the MS PacMan game. The algorithm is apply and modify in the way to accelerate the learning process an reach our goals, so we have to find a Neural Net explore the maze, escape from the ghosts, eat all the pill in the maze and finally maximise the score.

=== Dates ===
Start date: 2008/01/07

End date: 2008/09/30

=== Website(s) ===

http://cswww.essex.ac.uk/staff/sml/pacman/PacManContest.html

http://anji.sourceforge.net/

=== People involved ===

===== Project head(s) =====

P.L.Lanzi - [[User:PierLucaLanzi]]

D.Loiacono - [[User:DanieleLoiacono]]

===== Other Politecnico di Milano people =====

===== Students currently working on the project =====

Sandro Baranzini - [[User:SandroBaranzini]]

===== Students who worked on the project in the past =====

===== External personnel: =====

=== Laboratory work and risk analysis ===

== '''Part 2: project description''' ==
[Temporary Version]

'''%%% INTRODUCTION %%%'''

This thesis born from the challenge presented in the WCCI (Wolrd Congress on Computational Intelligence) 2008 placed in Hong Kong in June 2008, in particular from the MSPacMan Competition.
PacMan is a Namco arcade game, born in the 1980, early became most famous and loved by the worldwide audience. The main difference beetween the original PacMan and the Competition's version MSPacMan is the non deterministic character of Ghost’s movement that bring the MSPacMan version more interesting in the Artificial Intelligence science than the original one.
So this Thesis work is to find an AI formalism able to play the MSPacMan, controlling the agent in order to escape from Ghosts, living and score as much as he can. The main difficulties are about the non deterministic character of the Ghosts, the complexity of the maze, presenting during the level in three stages (see next figure), and the long time simulation needed by whatever Machine Learning algorithm.

[[Image:MSPacMan.jpg]]

Initially we decide to using the neuro evolutionary algorithm calling NEAT created by Kenneth O. Stanley and implemented in C++ in its first version. Many other version was born soon, in particular ANJI, the version we use, a good implemented and documented version written in Java. The main idea of this Algorithm is to merge the benefits of the Neural Nets, with Genetic Algorithm’s great evolutional feature. Through epoch Generations, Neural Nets Populations evolve, thanks to Crossover, Mutation and Speciations, guided by a Fitness function that measure the single Neural Net goodness for our goal.

'''%%% SOFTWARE AVAILABLE %%%'''

We have three software system have to integrated themselves and cooperate during long time Simulation. So we got the game MSPacMan, the Microsoft version included in the “Return Of Arcade – Anniversary Edition“ pack, the competition’s software that capture the screen image, elaborated it pixel for pixel and reassume all the information in a java Class creating the game state; it was necessary modified it in order to “insert coin”, and “start game” automatically, respectively pressing the F2 and F3 key. Finally we have the ANJI source code that we had integrated with competition’s software in the way each Neural Net can play the game, allowing us to measure its fitness.

'''%%% EARLY ATTEMPTS %%%'''

First of all we have to choose the inputs of the Nets; we have many variables in our problem every stored in the Game State Class. So we start the simulations assumed that the MSPAcMan is a classical Predator-Prey problem. So we choose, like problem inputs, the position of the agent, the positions of ghosts, the wall of the maze in all of four cardinal point centered in the agent, the last four inputs are the direction of ghosts. In these early simulations we measure the Fitness like agent’s lifetime counter.
Results of these early attempts was no satisfactory, see the Figure 2 with the graphical representation of the fitness during 20 generations, and shows us that the net needs the information about the closest pill by the PacMan in order to give agent the capacity to explore the maze in the direction of remaining pills. So we did, and doing again today, other simulation with less input, a local representation of the agent, of ghosts and of the closest pill; in addition we measure the Fitness like the game score. We get better results, show in Figure 3, that confirm us that’s the right direction to solve the problem ….

File:MSPacMan.jpg

2008-07-17T16:07:49Z

SandroBaranzini: Different Stages of MSPacMan during levels

Different Stages of MSPacMan during levels

Development an Artificial Intelligence System solving the MS Pac-Man videogame

2008-07-17T16:01:44Z

SandroBaranzini:

== '''Part 1: project profile''' ==

=== Project name ===

Development an Artificial Intelligence System solving the videogame MS Pac-Man

=== Project short description ===

The project is about the use of the neuro evolutionary algorithm NEAT in order to create one complex neural net able to solve the MS PacMan game. The algorithm is apply and modify in the way to accelerate the learning process an reach our goals, so we have to find a Neural Net explore the maze, escape from the ghosts, eat all the pill in the maze and finally maximise the score.

=== Dates ===
Start date: 2008/01/07

End date: 2008/09/30

=== Website(s) ===

http://cswww.essex.ac.uk/staff/sml/pacman/PacManContest.html

http://anji.sourceforge.net/

=== People involved ===

===== Project head(s) =====

P.L.Lanzi - [[User:PierLucaLanzi]]

D.Loiacono - [[User:DanieleLoiacono]]

===== Other Politecnico di Milano people =====

===== Students currently working on the project =====

Sandro Baranzini - [[User:SandroBaranzini]]

===== Students who worked on the project in the past =====

===== External personnel: =====

=== Laboratory work and risk analysis ===

== '''Part 2: project description''' ==
[Temporary Version]

'''%%% INTRODUCTION %%%'''

This thesis born from the challenge presented in the WCCI (Wolrd Congress on Computational Intelligence) 2008 placed in Hong Kong in June 2008, in particular from the MSPacMan Competition.
PacMan is a Namco arcade game, born in the 1980, early became most famous and loved by the worldwide audience. The main difference beetween the original PacMan and the Competition's version MSPacMan is the non deterministic character of Ghost’s movement that bring the MSPacMan version more interesting in the Artificial Intelligence science than the original one.
So this Thesis work is to find an AI formalism able to play the MSPacMan, controlling the agent in order to escape from Ghosts, living and score as much as he can. The main difficulties are about the non deterministic character of the Ghosts, the complexity of the maze, presenting during the level in three forms (see Figure1), and the long time simulation needed by whatever Machine Learning algorithm.
Initially we decide to using the neuro evolutionary algorithm calling NEAT created by Kenneth O. Stanley and implemented in C++ in its first version. Many other version was born soon, in particular ANJI, the version we use, a good implemented and documented version written in Java. The main idea of this Algorithm is to merge the benefits of the Neural Nets, with Genetic Algorithm’s great evolutional feature. Through epoch Generations, Neural Nets Populations evolve, thanks to Crossover, Mutation and Speciations, guided by a Fitness function that measure the single Neural Net goodness for our goal.

'''%%% SOFTWARE AVAILABLE %%%'''

We have three software system have to integrated themselves and cooperate during long time Simulation. So we got the game MSPacMan, the Microsoft version included in the “Return Of Arcade – Anniversary Edition“ pack, the competition’s software that capture the screen image, elaborated it pixel for pixel and reassume all the information in a java Class creating the game state; it was necessary modified it in order to “insert coin”, and “start game” automatically, respectively pressing the F2 and F3 key. Finally we have the ANJI source code that we had integrated with competition’s software in the way each Neural Net can play the game, allowing us to measure its fitness.

'''%%% EARLY ATTEMPTS %%%'''

First of all we have to choose the inputs of the Nets; we have many variables in our problem every stored in the Game State Class. So we start the simulations assumed that the MSPAcMan is a classical Predator-Prey problem. So we choose, like problem inputs, the position of the agent, the positions of ghosts, the wall of the maze in all of four cardinal point centered in the agent, the last four inputs are the direction of ghosts. In these early simulations we measure the Fitness like agent’s lifetime counter.
Results of these early attempts was no satisfactory, see the Figure 2 with the graphical representation of the fitness during 20 generations, and shows us that the net needs the information about the closest pill by the PacMan in order to give agent the capacity to explore the maze in the direction of remaining pills. So we did, and doing again today, other simulation with less input, a local representation of the agent, of ghosts and of the closest pill; in addition we measure the Fitness like the game score. We get better results, show in Figure 3, that confirm us that’s the right direction to solve the problem ….

Projects

2008-07-17T15:54:12Z

SandroBaranzini: /* Machine Learning */

''This page is a repository of links to the pages describing the '''projects''' we are currently working on at AIRLab.
See the list of our finished projects on the [[Finished Projects]] page.''

== Ongoing projects ==
''by research area (areas are defined in the [[Main Page]]); for each project a name and a link to its AIRWiki page is given''

==== [[Agents, Multiagent Systems, Agencies]] ====
----

* [[Multiagent cooperation|Multiagent cooperating system]]

* [[Planning in Ambient Intelligence scenarios| Planning in Ambient Intelligence scenarios]]

==== [[BioSignal Analysis]] ====
----
====== [[Affective Computing]] ======

* [[Relatioship between Cognition and Emotion in Rehabilitation Robotics]]
* [[Driving companions]]
* [[Emotion from Interaction]]
* [[Affective Devices]]

====== [[Brain-Computer Interface]] ======

* [[Online P300 and ErrP recognition with BCI2000]]
* [[BCI based on Motor Imagery]]
* [[Graphical user interface for an autonomous wheelchair]]
* [[Mu and beta rhythm-based BCI]]

====== [[Automatic Detection Of Sleep Stages]] ======

* [[Sleep Staging with HMM]]

====== [[Analysis of the Olfactory Signal]] ======

* [[Lung Cancer Detection by an Electronic Nose]]
* [[HE-KNOWS - An electronic nose]]

==== [[Computer Vision and Image Analysis]] ====
----

* [[Automated extraction of laser streaks and range profiles]]

* [[Data collection for mutual calibration|Data collection for laser-rangefinder and camera calibration]]

* [[Image retargeting by k-seam removal]]

* [[Particle filter for object tracking]]

* [[Template based paper like reconstruction when the edges are straight]]

* [[Wii Remote headtracking and active projector]]

* [[Vision module for the Milan Robocup Team]]

* [[Long Exposure Images for Resource-constrained video surveillance]]

* [[NonPhotorealistic rendering of speed lines]].

* [[Restoration of blurred objects using cues from the alpha matte]]

* [[Analyzing Traffic Speed From a Single Night Image - Light Streaks Detection]]

* [[Plate detection algorithm]]

* [[A vision-based 3D input device for space curves]]

==== [[Machine Learning]] ====
----
* [[Adaptive Reinforcement Learning Multiagent Coordination in Real-Time Computer Games|Adaptive Reinforcement Learning Multiagent Coordination in Real-Time Computer Games]]

* [[B-Smart Behaviour Sequence Modeler and Recognition tool|B-Smart Behaviour Sequence Modeler and Recognition tool]]

* [[Player modeling in TORCS exploiting SVMs and GPUs parallelism|Player modeling in TORCS exploiting SVMs and GPUs parallelism]]

* [[Development an Artificial Intelligence System solving the MS Pac-Man videogame |Development an Artificial Intelligence System solving the MS Pac-Man videogame ]]

==== [[Ontologies and Semantic Web]] ====
----
* [[Extending a wiki with semantic templates]]
* [[GeoOntology|Geographic ontology for a semantic wiki]]

==== [[Philosophy of Artificial Intelligence]] ====
----
==== [[Robotics]] ====
----
===== [[Robot development]] =====
* [[LURCH - The autonomous wheelchair]]

* [[Balancing robots: Tilty, TiltOne]]

* [[ Brake Padal Implementing on a Golf Cart ]]

===== [[Benchmarking]] =====

* [[Rawseeds|RAWSEEDS]]

===== [[Bio Robotics]] =====

* [[PoliManus]]

* [[ZOIDBERG - An autonomous bio-inspired RoboFish]]

* [[Styx The 6 Whegs Robot]]

* [[PolyGlove: a body-based haptic interface]]

* [[ULISSE]]

* [[PEKeB: a PiezoElectric KeyBoard]]

* [[Anthropomorphic Robotic Wrist]]

===== [[Robogames]] =====

* [[ROBOWII]]
----

== Note for students ==

If you are a student and there isn't a '''page describing your project''', this is because YOU have the task of creating it and populating it with (meaningful) content. If you are a student and there IS a page describing your project, you have the task to complete that page with (useful and comprehensive) information about your own contribution to the project. Be aware that the quality of your work (or lack of it) on the AIRWiki will be evaluated by the Teachers and will influence your grades.

Instructions to add a new project or to add content to an existing project page are available at [[Projects - HOWTO]].

User:SandroBaranzini

2008-07-14T16:45:22Z

SandroBaranzini:

Sandro Baranzini
nato a Milano il 16 febbraio 1983

Laureato il 28 settembre 2005
in Ingegneria Informatica col punteggio di 108/110
presso il Politecnico di Milano

Terminati gli esami del corso di Laurea Specialistica in Ingegneria Informatica
sto svolgendo una tesi sugli algoritmi neuro evolutivi applicati al videogioco MSPacMan
che spero di consegnare ad Ottobre 2008.

Attualmente lavoro presso la TPA spa di Sesto San Giovanni, azienda fornitrice di controlli numerici,
nella divisione Ricerca & Sviluppo come firmwarista e, più in particolare, come progettista di Motion Control.