Qualification Dual-Round Rules

[Christian]

We'd like some clarification on the qualification that goes in round 1 and 2.

1) Does the agent know if it is playing in round 1 or 2 to be able to try something different in another round, e.g. from a command line parameter?
I'm aware that you can get some comparison scores via the interface, but that doesn't seem sufficient to identify where this comes from.

2) How exactly is the score calculated?
Let's say we score X in round 1 and Y in round 2.
As we read the rules, this would mean that the score for this level will be:
X + max(X, Y)
i.e. the sum of rounds 1 and 2, with the fact that you can only improve in round 2.
Is this correct?

[Jochen]

Hi Christian,

we are releasing the updated protocol withing the next 24h. It will include a modified initialisation command which tells you which round we are in and how many levels this round has. It is your task to implement your agent so that it knows that Q2 are the same levels as Q1 but that you get different high scores.

Regarding the scoring, the total qualification score is max(X,Y), so only your highest score per level over round 1 and 2 counts.

I hope that makes sense.

Jochen

[Christian]

OK, so in short:
The idea for the qualification is that the resulting behavior will be: We play for 3 minutes/level, but can get information about other's scores after 1.5 minutes/level.

[Jochen]

Yes, but it is not necessarily 2x 1.5 minutes per level on average. That's the minimum, you might get more time.

[kaliaragorn]

So, just in order to be sure.. According to the docs

Get My Score
[...]
The server will return a fixed length bytes array with every four slots indicates a best score of the corresponding level. Scores of unsolved levels are zero.

w.r.t. this function, we understand that:

1. at the very beginning of the 1st qualification round the returned array contains only zeros.
2. at the very beginning of the 2nd qualification round the returned array contains our agent's best scores achieved in the 1st qualification round.

Is that correct?
thank you in advandce.
f

[Jochen]

Hi,

at the beginning we will run the sample agent on all Q1 levels. So during all of Q1, you will get the scores of the sample agent.

However, since the sample agent just keeps trying to solve a level, we will do some modifications to the sample agent so that if one level cannot be solved, the sample agent will move to the next level after some time.

During all of Q2, you will get the highest scores from Q1.

Regards,
Jochen

[Jochen]

By the way, the reason we do that is so that we do not disadvantage the agents that start first.
Otherwise the last agent gets all high scores while the first agent doesn't.

During the group stages and in the knock out matches, you will always get live high scores as all relevant agents run in parallel.

I realise it causes some confusion, but I hope it makes sense and makes it fair to everyone...

Regards,
Jochen

[kaliaragorn]

Thank you for your prompt reply. However, we were actually asking about My Score
f

Hi,

at the beginning we will run the sample agent on all Q1 levels. So during all of Q1, you will get the scores of the sample agent.

However, since the sample agent just keeps trying to solve a level, we will do some modifications to the sample agent so that if one level cannot be solved, the sample agent will move to the next level after some time.

During all of Q2, you will get the highest scores from Q1.

Regards,
Jochen

[Jochen]

Oh, sorry.

My score always gives you the current best score you have for all levels.
At the start of Q2 it will show you your high scores of Q1, you are right.

Jochen

[kaliaragorn]

Ok, this is what we wanted to know.. thank you! :
f

Oh, sorry.

My score always gives you the current best score you have for all levels.
At the start of Q2 it will show you your high scores of Q1, you are right.

Jochen