Page 1 of 2

Qualification Dual-Round Rules

PostPosted: Sat Jul 20, 2013 4:02 am
by Christian
We'd like some clarification on the qualification that goes in round 1 and 2.

1) Does the agent know if it is playing in round 1 or 2 to be able to try something different in another round, e.g. from a command line parameter?
I'm aware that you can get some comparison scores via the interface, but that doesn't seem sufficient to identify where this comes from.

2) How exactly is the score calculated?
Let's say we score X in round 1 and Y in round 2.
As we read the rules, this would mean that the score for this level will be:
X + max(X, Y)
i.e. the sum of rounds 1 and 2, with the fact that you can only improve in round 2.
Is this correct?

Re: Qualification Dual-Round Rules

PostPosted: Sat Jul 20, 2013 4:11 am
by Jochen
Hi Christian,

we are releasing the updated protocol withing the next 24h. It will include a modified initialisation command which tells you which round we are in and how many levels this round has. It is your task to implement your agent so that it knows that Q2 are the same levels as Q1 but that you get different high scores.

Regarding the scoring, the total qualification score is max(X,Y), so only your highest score per level over round 1 and 2 counts.

I hope that makes sense.

Jochen

Re: Qualification Dual-Round Rules

PostPosted: Mon Jul 22, 2013 1:17 am
by Christian
OK, so in short:
The idea for the qualification is that the resulting behavior will be: We play for 3 minutes/level, but can get information about other's scores after 1.5 minutes/level.

Re: Qualification Dual-Round Rules

PostPosted: Mon Jul 22, 2013 1:56 am
by Jochen
Yes, but it is not necessarily 2x 1.5 minutes per level on average. That's the minimum, you might get more time.

Re: Qualification Dual-Round Rules

PostPosted: Thu Aug 01, 2013 10:42 pm
by kaliaragorn
So, just in order to be sure.. According to the docs

Get My Score
[...]
The server will return a fixed length bytes array with every four slots indicates a best score of the corresponding level. Scores of unsolved levels are zero.


w.r.t. this function, we understand that:

1. at the very beginning of the 1st qualification round the returned array contains only zeros.
2. at the very beginning of the 2nd qualification round the returned array contains our agent's best scores achieved in the 1st qualification round.

Is that correct?
thank you in advandce.
f

Re: Qualification Dual-Round Rules

PostPosted: Fri Aug 02, 2013 12:45 am
by Jochen
Hi,

at the beginning we will run the sample agent on all Q1 levels. So during all of Q1, you will get the scores of the sample agent.

However, since the sample agent just keeps trying to solve a level, we will do some modifications to the sample agent so that if one level cannot be solved, the sample agent will move to the next level after some time.

During all of Q2, you will get the highest scores from Q1.

Regards,
Jochen

Re: Qualification Dual-Round Rules

PostPosted: Fri Aug 02, 2013 12:50 am
by Jochen
By the way, the reason we do that is so that we do not disadvantage the agents that start first.
Otherwise the last agent gets all high scores while the first agent doesn't.

During the group stages and in the knock out matches, you will always get live high scores as all relevant agents run in parallel.

I realise it causes some confusion, but I hope it makes sense and makes it fair to everyone...

Regards,
Jochen

Re: Qualification Dual-Round Rules

PostPosted: Fri Aug 02, 2013 3:49 am
by kaliaragorn
Thank you for your prompt reply. However, we were actually asking about My Score :mrgreen:
f

Jochen wrote:Hi,

at the beginning we will run the sample agent on all Q1 levels. So during all of Q1, you will get the scores of the sample agent.

However, since the sample agent just keeps trying to solve a level, we will do some modifications to the sample agent so that if one level cannot be solved, the sample agent will move to the next level after some time.

During all of Q2, you will get the highest scores from Q1.

Regards,
Jochen

Re: Qualification Dual-Round Rules

PostPosted: Fri Aug 02, 2013 11:48 am
by Jochen
Oh, sorry.

My score always gives you the current best score you have for all levels.
At the start of Q2 it will show you your high scores of Q1, you are right.

Jochen

Re: Qualification Dual-Round Rules

PostPosted: Fri Aug 02, 2013 8:30 pm
by kaliaragorn
Ok, this is what we wanted to know.. thank you! :) :
f

Jochen wrote:Oh, sorry.

My score always gives you the current best score you have for all levels.
At the start of Q2 it will show you your high scores of Q1, you are right.

Jochen