PASCAL Agnostic Learning
vs.
Prior Knowledge
IJCNN07

The challenge is now over. But it remains open for post-challenge submissions!


IMPORTANT: Entries made since February 1st 2007 might be using validation data, now available for training.

the good

Submitted by reference

A combination of reference models, selected via 100-fold validation (which provides a good unbiased estimator of performance).

Models Chosen:

ADA - xent quadratic lssvm : BER = 0.187897

GINA - xent cubic lssvm : BER = 0.052681

HIVA - press quadratic lssvm : BER = 0.244383

NOVA - xent linear lssvm : BER = 0.048260

SYLVA - serate quadratic lssvm : BER = 0.007641

N.B. sadly "the good" are unlikely to win this time because the models including ARD kernels were not included - they just take too long to perform the validation :-( It will be interesting to see how the HIVA, NOVA and SYLVA selections compare though. My vote goes with "the ugly"!

Dataset Balanced Error Area Under Curve  
Train Valid Test Train Valid Test
ada 0.1682 0.2127 0.1876 0.9165 0.8742 0.8984 agnostic
gina 0 0.0285 0.0471 1 0.9956 0.99 agnostic
hiva 0.0212 0.2535 0.2733 0.9972 0.7253 0.772 agnostic
nova 0.0004 0.044 0.0465 1 0.9968 0.99 agnostic
sylva 0.002 0.0053 0.0078 0.9999 0.999 0.9988 agnostic
Overall 0.0384 0.1088 0.1125 0.9827 0.9182 0.9299 agnostic