PASCAL Agnostic Learning
vs.
Prior Knowledge
IJCNN07

The challenge is now over. But it remains open for post-challenge submissions!


IMPORTANT: Entries made since February 1st 2007 might be using validation data, now available for training.

SNB(CMA) +10k F(3D) t

Submitted by Marc B oulle

SNB(CMA) +10k F(3D) t
Same method as at the IJCNN 2006 Performance Prediction Challenge.

Mainly: Naive Bayes classifier, with variable selection and model averaging
Paper: http://clopinet.com/isabelle/Projects/modelselect/Papers/Boulle_paper_IJCNN06.pdf
Software: http://www.francetelecom.com/en/group/rd/offer/software/applications/providers/khiops.html

Performance guess:
ada: 18.2
gina: 9.4
hiva: 29.4
nova: 9.2
sylva: 0.8

Dataset Balanced Error Area Under Curve  
Train Valid Test Train Valid Test
ada 0.1652 0.2191 0.1693 0.9199 0.8841 0.9174 agnostic
gina 0.0394 0.0632 0.0848 0.9927 0.9838 0.9699 agnostic
hiva 0.2267 0.3627 0.3046 0.8458 0.732 0.7472 agnostic
nova 0.0492 0.12 0.0852 0.9893 0.9314 0.9722 agnostic
sylva 0.0029 0.0098 0.0072 0.9995 0.9994 0.9992 agnostic
Overall 0.0967 0.155 0.1302 0.9495 0.9061 0.9212 agnostic

This entry is a complete agnostic learning entry.