2018 Competition Results

This year we had 9 participants, 4 improved versions of some of the best agents of the past few years, the defending champion and 4 new agents. We had improved versions of AngryHex and IHSEV, the two teams with the most experience who participated in all competitions since 2013. IHSEV was the finalist in 2016 and 2017, AngryHex made it to the semi finals in the past. PlanA+, the first agent who scored over 1,000,000 points in our benchmarks and a semi-finalist in 2017 also participated again. BamBirds, the winner from 2016 and still the only agent that plans ahead, participated again with a much improved version. As every year, the defending champion participated with an unmodified version. In 2017, Eagle's Wings was the surprise winner, developed by Tian Jian Wang, an undergrad student from the University of Waterloo. 

The new agents this year were Eagle's Wings SIM, an improved version of last years winner that also includes simulation, a strategy very successful with IHSEV. MetaBirds from Ben Gurion University is a meta agent that is composed of several open source agents from past years together with a strategy of which agent to select for which level. We have also experimented with our own meta agent and even published a paper about it. It showed superior performance, but our meta agent was computationally too demanding and couldn't run on the competition laptops. We were excited to see how Metabirds would perform. Finally, we had two Deep Learning agents, DQ-Birds from Charles University and MYTBirds from Ritsumeikan University. Deep Learning is extremely successful with other games, most notably Go and Atari games, but hasn't shown good performance in the AIBIRDS competition yet. MetaBirds, DQ-Birds and MYTBirds each wrote a paper about their agents and presented it before their agents were running. All three sounded very impressive, so we were excited how they would perform. 

Altogether we had an excellent line-up that promised a lot of excitement. It happened quite a few times in the past that newcomers won the competition, so anything was possible. What was new this year was that we had some representatives from Rovio's Stockholm office attend the competition. 

Since we had 9 participants and can run 4 agents at a time, we decided to have 3 quarter finals with 3 agents each. At every round, the agents have 30 minutes to solve 8 new levels. For every match, all agents can see the current high scores per level for all agents of the same match. and can use this information to select which level to solve next. Levels can be solved again in any order until the time is up.

The first quarter final was dominated by IHSEV, just like in previous years. Their simulation approach is particularly good for levels that are not so difficult, as used in earlier competition rounds.We were surprised by the performance of AngryHex, who have done quite well in the past, but not this time.

The second quarter final was exciting as the two past champions were competing against each other. Last years winner Eagle's Wings showed a very strong performance and easily won. BamBirds, the winner from 2016, did very well, but couldn't keep up with Eagle's Wings. DQ-Birds showed a respectable performance, particularly compared to last years Deep Learning agent, and did better overall than AngryHex. Since only the best 4 agents of the quarter finals moved to the semi finals, it was already clear that both DQ-Birds and AngryHex were eliminated. 

Before the third quarter final, only Eagle's Wings was safe, the other three could still be eliminated. This was particularly exciting as the improved version of Eagle's Wing was still playing, also MetaBirds who we expected to perform very well, and MYTBirds whose paper presentation was impressive and was certainly good for a surprise. Eagle's Wings SIM ended up winning the last quarter final and eliminated PlanA+ from the semi finals. The other two agents didn't qualify. 

The four agents who qualified for the semi finals were the two Eagle's Wings, Bambirds and the serial semi finalist IHSEV. Before the competition we agreed with Tian that should both of his agents qualify for the finals only the improved 2018 version would be used. But fortunately, we didn't have to use this rule as BamBirds showed an incredible performance in the semi finals. They solved all levels and were far ahead of the others. Surprisingly, Eagle's Wings was ahead of the supposedly improved version Eagle's Wings SIM and joins BamBirds into the Grand Final. IHSEV ended last and didn't qualify for the Grand Final for the first time since 2015.

With the stellar semi final performance, BamBirds was the clear favourite in the Grand Final. But as it happened so often in the past, the grand final was a completely different match and performance in previous rounds did not translate to automatic success. Eagle's Wings began extremely well, solving a bunch of levels. BamBirds started catching up, but had difficulties solving some of the levels. Now something very surprising happened: BamBirds stopped trying to solve the levels it hadn't solved yet and instead solved all the levels again it had solved already. We couldn't believe it. That way, BamBirds was of course unable to catch up with Eagle's Wings as it would eventually run out of time. We looked at the output BamBirds was producing to see if we can understand this strange strategy. And there it was: It turned out that BamBirds had a rule that if a level cannot be solved in three attempts, the level would not be played again! Diedrich Wolter, the BamBirds team lead who was attending the competition was furious, wondering who of his team of 12 students was responsible. We all realised that this was the end for BamBirds and that Eagle's Wings would win again, particularly since it showed a very impressive performance, solving all levels and even improving scores as it solved levels again. 

Therefore: Eagle's Wing is the old and new AIBIRDS Champion!!! 

Congratulations to Tian Jian Wang from Zazzle Inc. who was able to defend his title. Very well done! 

 

That was the end of a very exciting competition. Despite many improvements across a number of agents and some promising new agents, the defending champion won again. The four semi-finalists qualified for the Man vs Machine Competition the next day.

The next competition will be part of IJCAI 2019 in Macau and we hope to see many improved agents and many new agents there. Angry Birds remains a very challenging problem for AI. We are still waiting to see an exceptionally good deep learning agent, or any other agent that can come close to human performance. We encourage and challenge all members of the AI community to take on this problem and to develop AI that can successfully deal with a physical environment. We hope to see you at IJCAI 2019! 

Jochen, Matthew, Peng and Gary. 

 

 

All scores of the competition can be found in the following tables:   

Grand Final     Quarter Final 1  
         
1. Eagle's Wing 2017 455,150   1. IHSEV 398,480
2. BamBirds 213,590   2. PlanA+ 272,060
      3. AngryHex 145,240
         
Semi Final        
      Quarter Final 2  
1. BamBirds 427,420      
2. Eagle's Wing 2017 281,560   1. Eagle's Wing 2017 405,260
3. Eagle's Wing 2018 243,490   2. BamBirds 290,020
4. IHSEV 195,980   3. DQ-Birds 185,860
         
         
Quarter Final Ranking   Quarter Final 3  
         
1. Eagle'sWing 2017 405,260   1. Eagle's Wing 2018 323,050
2. IHSEV 398,480   2. MetaBirds 163,920
3. Eagle's Wing 2018 323,050   3. MYTBirds 49,900
4. BamBirds 290,020      
5. PlanA+ 272,060      
6. DQ-Birds 185,869      
7. MetaBirds 163,920      
8. AngryHex 145,240      
9. MYTBirds 49,900