The prediction results of the test dataset can be submitted at any time during the challenge. The result of the test set is a txt file with the following fields:
Predict the click (finish+like)-through probability on each item of test dataset. We use AUC (area under ROC curve) as our challenge metric.
In our competition, one must output a score for the probability of clicking (finish+like) each item. Then AUC is calculated as follows:
where T is a varying threshold parameter which is used to calculate the TPR and FPR, given by
The leaderboard is ranked according to the AUC. The higher the AUC, the higher the ranking.
Short Video Understanding Challenge