PASCAL Agnostic Learning
vs.
Prior Knowledge
IJCNN07

The challenge is now over. But it remains open for post-challenge submissions!


IMPORTANT: Entries made since February 1st 2007 might be using validation data, now available for training.

SNB(CMA) +10K F(2D) t

Submitted by Marc B oulle

SNB(CMA) +10K F(2D) t
Same method as at the IJCNN 2006 Performance Prediction Challenge.

Mainly: Naive Bayes classifier, with variable selection and model averaging
Paper: http://clopinet.com/isabelle/Projects/modelselect/Papers/Boulle_paper_IJCNN06.pdf
Software: http://www.francetelecom.com/en/group/rd/offer/software/applications/providers/khiops.html

Performance guess:
ada: 18.1
gina: 9.3
hiva: 29.7
nova: 10.4
sylva: 0.7

Dataset Balanced Error Area Under Curve  
Train Valid Test Train Valid Test
ada 0.1689 0.2111 0.1732 0.9173 0.8842 0.9166 agnostic
gina 0.0462 0.0726 0.0872 0.9897 0.9792 0.9675 agnostic
hiva 0.2439 0.3041 0.3066 0.8356 0.7573 0.7472 agnostic
nova 0.0477 0.114 0.0774 0.99 0.9402 0.9727 agnostic
sylva 0.0036 0.0098 0.0066 0.9995 0.9994 0.9992 agnostic
Overall 0.102 0.1423 0.1302 0.9464 0.912 0.9206 agnostic

This entry is a complete agnostic learning entry.