2012 Benchmarks
In order to allow participants to easily compare the performance of their agents with participants of the 2012 challenge, we benchmarked the eight agents that participated in the classic track of the 2012 Angry Birds AI challenge. We ran them on the first 21 levels of the freely available Poached Eggs levels. These are the default levels when using the Chrome version of Angry Birds. Since none of the agents solved any level again during the competition, only the first successful attempt for each of the 21 levels was considered. Each agent had a time limit of 30 minutes per level. If an agent did not solve a level after 30 minutes, it proceeded to the next level. The following table gives an overview of the performance of the eight agents that participated in the classic track:
Level | ABC-AI-UNICAL | ABC-IS-UNICAL | CU | Dan | FEI2 | Naive | RedBacks | Zonino | High score | 3 stars |
1 | 27550 | 30490 | 27090 | 28890 | 0 | 29330 | 33340 | 29080 | 33340 | 32000 |
2 | 52420 | 34600 | 52590 | 42960 | 60880 | 43440 | 44080 | 52560 | 60880 | 60000 |
3 | 33460 | 41070 | 42880 | 32880 | 21070 | 41350 | 23480 | 40800 | 42880 | 41000 |
4 | 18690 | 27990 | 36960 | 27710 | 0 | 28090 | 38160 | 19200 | 38160 | 28000 |
5 | 36280 | 62780 | 52740 | 55560 | 66050 | 54350 | 67630 | 63230 | 67630 | 64000 |
6 | 17870 | 17500 | 19270 | 23630 | 32550 | 24510 | 24530 | 17900 | 32550 | 35000 |
7 | 22510 | 20560 | 31240 | 26240 | 0 | 23620 | 0 | 24020 | 31240 | 45000 |
8 | 47400 | 40440 | 47780 | 47180 | 34190 | 48270 | 47920 | 46820 | 48270 | 50000 |
9 | 35600 | 42500 | 51570 | 43710 | 0 | 31440 | 33410 | 36720 | 51570 | 50000 |
10 | 41530 | 43970 | 51820 | 37130 | 0 | 53390 | 58530 | 51240 | 58530 | 55000 |
11 | 60420 | 53250 | 57900 | 39010 | 0 | 57280 | 42950 | 57610 | 60420 | 54000 |
12 | 63440 | 63440 | 46530 | 46800 | 0 | 56980 | 52810 | 54930 | 63440 | 45000 |
13 | 36560 | 36640 | 21030 | 36940 | 0 | 26530 | 32480 | 32950 | 36940 | 47000 |
14 | 65790 | 60110 | 54180 | 52470 | 0 | 46010 | 0 | 55640 | 65790 | 70000 |
15 | 46760 | 45800 | 44740 | 48200 | 0 | 42540 | 33570 | 39050 | 48200 | 41000 |
16 | 55110 | 50730 | 54370 | 43980 | 0 | 50890 | 66180 | 60140 | 66180 | 64000 |
17 | 41590 | 48950 | 48090 | 34350 | 0 | 39090 | 49140 | 55150 | 55150 | 53000 |
18 | 55170 | 51240 | 57320 | 38860 | 0 | 40060 | 38630 | 47070 | 57320 | 48000 |
19 | 27590 | 33340 | 23040 | 38720 | 0 | 27950 | 29510 | 34960 | 38720 | 35000 |
20 | 40410 | 44900 | 49480 | 35370 | 0 | 37580 | 38900 | 51440 | 51440 | 50000 |
21 | 0 | 0 | 71740 | 0 | 0 | 57010 | 54460 | 0 | 71740 | 75000 |
Total | 826150 | 850300 | 942360 | 780590 | 214740 | 859710 | 809710 | 870510 | 1080390 | 1042000 |
Time | 2h4m | 1h48m | 1h30m | 2h22m | 9h03m | 1h12m | 2h33m | 1h25m |
It is interesting to see that all participating agents hold at least one high score for the 21 levels. Agent Redback and Agent CU have the most high scores, and Agent CU has by far the highest overall score. The overall time used by the agents is also listed. The Naive Agent is the fastest and Zonino agent the second fastest. The score required in order to get a three star rating is listed in the last column and is taken from the angrybirdsnest website. In 6 of the 21 levels, none of the agents received three stars.
Please note: The new 2013 Naive Agent behaves differently than the 2012 Naive Agent listed here, as we improved the vision component and the trajectory planning component. But this does not make the new Naive Agent better than the old one. You can check this for yourself.