Matheus Portela

Undergraduate Thesis (Portuguese)

Seleção de Comportamentos em Múltiplos Agentes Autônomos com Aprendizagem por Reforço em Ambientes Estocásticos

Behavior selection for multiple autonomous agents with reinforcement learning in stochastic environments
University of Brasília
Date: December 4, 2015
Download: PDF
Resumo: Agentes inteligentes agem baseados nas suas medições sensoriais a fim de alcançar seus objetivos. Em ambientes dinâmicos, como sistemas multiagentes, agentes devem adaptar seus processos de seleção de ações de acordo com o estado do sistema mutável, uma vez que comportamentos anteriormente considerados adequados podem tornar-se sub-ótimos. Tal problema é ainda maior se o ambiente é estocástico, forçando os agentes a lidarem com incertezas. Esse trabalho propõe um algoritmo de aprendizado por reforço para sistemas multiagentes estocásticos, utilizando programação bayesiana para estimação de estados e Q-learning com aproximação de funções para prover aos agentes a capacidade de aprender a selecionar os comportamentos mais adequados. Experimentos indicam resultados positivos para a abordagem, onde agentes aprenderam a cooperar, de forma autônoma, em um jogo eletrônico estocástico multiagente.

Peer-Reviewed Papers

State Estimation and Reinforcement Learning for Behavior Selection in Stochastic Multiagent Systems

Authors:
Matheus Vieira Portela - University of Brasilia, Brasilia, Brazil
Guilherme Novaes Ramos - Department of Computer Science, University of Brasilia, Brasilia, Brazil
Published at: XIV Brazilian Symposium of Digital Games and Entertainment - 2015 - Proceedings of SBGames 2015 - Computing Track - Short Papers
Publisher: SBC - Brazilian Computing Society
Publication date: November 11-13, 2015
Download: PDF
Abstract: Intelligent agents can act based on sensor measurements in order to fulfill their goals. In dynamic systems, agents must adapt its behavior selection processes to reflect the changing system state since behaviors that previously were considered the best choice may become sub-optimal. Multiple agents that co-exist in the environment is one example of such a dynamic system. The problem is even greater when stochastic systems are considered, since the states the agents are actually in are unknown. This work proposes a learning algorithm for stochastic multiagent systems, in which Bayesian programming is used for state estimation and Q-learning provides learning capabilities to the agents. An experimental setup using electronic games is described to evaluate the effectiveness of this approach.

Gaze Enhanced Speech Recognition for Truly Hands-Free and Efficient Text Input During HCI

Authors:
Matheus Vieira Portela - University of Brasilia, Brasilia, Brazil
David Rozado - CSIRO, QLD, Brisbane, Australia
Published at: OzCHI ‘14 Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: the Future of Design
Publisher: ACM http://dl.acm.org/citation.cfm?id=2686679
Publication date: December 2, 2014
Download: PDF
Abstract: The performance of current speech recognition algorithms is well below that of human speech recognition, with high number of misrecognized words in quiet environments and degrading even further in noisy ones. Therefore, hands-free interaction remains a deeply frustrating experience. In this work, we present an innovative form of correcting misrecognized words during a speech recognition task by using gaze tracking technology in a multimodal approach. We propose to employ the user’s gaze to point at misrecognized words and select appropriate alternatives. We compare the performance of this multimodal approach with traditional modalities of correcting words: usage of mouse and keyboard and usage of voice alone. The results of the user study show that whereas the proposed system is not as fast as using mouse and keyboard for correction, gaze enhanced correction significantly outperforms voice alone correction and is preferred by the users, offering a truly hands-free means of interaction.

Non Peer-Reviewed Papers

Bayesian Programming and Reinforcement Learning in Stochastic Multiagent Systems

Authors:
Matheus Vieira Portela - University of Brasilia, Brasilia, Brazil
Guilherme Novaes Ramos - Department of Computer Science, University of Brasilia, Brasilia, Brazil
Published at: Graduate School Workshop at the Department of Computer Science - University of Brasilia
Publication date: October 16-17, 2015
Download: PDF
Abstract: Intelligent agents act based on sensor measurements in order to fulfill their goals. When the environment is dynamic, such as a multiagent system, agents must adapt its actions selection processes to reflect the ever changing system state since behaviors that previously were considered the best choice may becomes sub-optimal. The problem is even greater when stochasticity is taken into account, since the environment true state is unknown to the agents. This work proposes a learning algorithm for stochastic multiagent systems, in which Bayesian programming is used for state estimation and Q-learning with function approximation provides learning capabilities so as agents can select the appropriate behaviors. An experimental setup to evaluate the effectiveness of this approach using electronic games is described, as well as the preliminary results.