Results of the CoG 2019 Level Generation Competition
Our fourth AIBIRDS Level Generation competition was held in London on August 21st as part of CoG 2019. Participants in the competition had to submit a level generation program that creates game levels for Angry Birds that satisfy several constraints, such as the number of pigs or blocks types. During the competition, 50 levels were generated for each participant, of which four were then randomly selected. These selected levels were then evaluated by a panel of judges based on the categories of fun, creativity/aesthetic and difficulty balance. The total scores across all three categories determined the overall winner.
This year we had 5 entrants:
- Naive (baseline generator)
- Funny Quotes ft. Dominoes (Yuxuan Jiang, Ryota Ishii, Tomohiro Harada, Ruck Thawonmas from Ritsumeikan University, Japan)
- Fitra (Febri Abdullah, Pujana Paliyawan, Ruck Thawonmas, Tomohiro Harada from Ritsumeikan University, Japan; Fitra Bachtiar from Brawijaya University, Indonesia)
- BamBird (Thilo Martens from Bamburg University, Germany)
- MelodyGen (Ceren Kocaogullar from Koc University, Istanbul, Turkey)
A team of nine independent judges evaluated all selected levels and reported the results back to us in terms of scores between 0 and 10 for each generator. The identity of the generators and which levels belonged to which generator was kept a secret until after the judging scores were all aggregated and the ranking was finalised.
Generators were ranked based on the sum of the scores given by all judges.
Fun:
1. MelodyGen 43
2. Fitra 35
3. Naive 28
Creativity / Aesthetic:
1. Fitra 45
2. MelodyGen 42
3. Funny Quotes 32
Difficulty Balance:
1. MelodyGen 41
2. Funny Quotes 32
3. Fitra 28
Overall:
1. MelodyGen 126
2. Fitra 108
3. Funny Quotes 89
MelodyGen by Ceren Kocaogullar achieved the highest overall score, as well as coming top in both the Fun and Difficulty Balance categories, making it the clear winner of the AIBIRDS 2019 Level Generation Competition! Fitra came in second place overall, followed by Funny Quotes ft. Dominoes in third place.
The winning generator from 2017 and 2018 (Iratus Aves) did not participate this year, but the source code for this generator is now freely available at: https://github.com/stepmat/IratusAves for future competition entrants to build on if they wish.
One major benefit of having level generators available, is that it is now possible to create large numbers of training and test levels for developing improved AI agents for the main AIBIRDS competition. Particularly agents based on Deep Reinforcement Learning, a technique that has taken much of AI by storm and is very successful for many other games, will benefit from a large number of available levels. We hope all participants will make their generators available, so people can use them to improve their AI agents.
This was our fourth level generation competition. We thank all participants for submitting sophisticated generators and all judges for contributing to their evaluation. We hope to continue our competition next year and encourage all interested teams to participate in this exciting challenge.
You can find the results of the 2018 competition here, the results of the 2017 competition here, and the results of the 2016 competition here