AI Birds.org Angry Birds AI Competition

2014 Benchmarks

In order to allow participants to easily compare the performance of their agents with participants of the 2013 and 2014 competitions, we benchmarked all agents that participated in the 2013 and 2014 Angry Birds AI competition. We ran them on the first 21 levels of the freely available Poached Eggs levels. These are the default levels when using the Chrome version of Angry Birds. Each agent had a time limit of 3 minutes per level on average, that is, a total time of 63 minutes for the 21 levels. Agents could spend the total time in any way they wanted. We used the same settings of the agents as in the competition. The following table gives an overview of the performance of all thirty agents (2014 agents in bold):

It is interesting to see that 17 agents hold at least one high score for the 21 levels. But it is clear that solving more levels is better than having high scores. The score required in order to get a three star rating is listed as well and is taken from the angrybirdsnest website. In only 1 of the 21 levels, none of the agents received three stars, in level 14. 

Surprisingly, the clear winner of the 2014 competition is not the best agent in the benchmarks and does not hold an individual high score, but is overall better than any agent of the 2013 competition. The best agent in the benchmarks and the only agent who ever received a total score of more than 1,000,000 is PlanA+ who was eliminated in the semi finals by the tiniest possible margin. AngryBER who beat PlanA+ in the semi finals also does not have an individual high score. 

One major difference between the 2013 and the 2014 agents is that in 2014 the provided vision system detects the real shape of objects and also detects ground and hills. Without detecting those, some of the Poached Eggs levels above level 21 cannot be solved. Because this is now possible, we also benchmarked the 2014 agents on the second set of 21 Poached Eggs levels. We give the result in the following table: 

The top three agents of the competition are clearly the best agents on these levels, which are much more difficult than the first 21 levels. 6 of the agents have individual high scores and for 10 levels at least one of the agents has a three star score. From level 2-14 on, none of the agents has a three star score. Level 2-20 hasn't been solved by any agent and seems to be the most difficult level so far for AI agents. 

When considering all 42 levels, the three best agents of the 2014 competition are also the three top performers in the benchmarks, with team DataLab Birds well ahead: