PASCAL Agnostic Learning
vs.
Prior Knowledge
IJCNN07

The challenge is now over. But it remains open for post-challenge submissions!


IMPORTANT: Entries made since February 1st 2007 might be using validation data, now available for training.

SNB(CMA) t

Submitted by Marc B oulle

SNB(CMA) t
Same method as at the IJCNN 2006 Performance Prediction Challenge.

Mainly: Naive Bayes classifier, with variable selection and model averaging
Paper: http://clopinet.com/isabelle/Projects/modelselect/Papers/Boulle_paper_IJCNN06.pdf
Software: http://www.francetelecom.com/en/group/rd/offer/software/applications/providers/khiops.html

Performance guess:
ada: 19.0
gina: 13.7
hiva: 28.9
nova: 9.2
sylva: 0.7

Dataset Balanced Error Area Under Curve  
Train Valid Test Train Valid Test
ada 0.1799 0.2334 0.1743 0.9077 0.8704 0.9101 agnostic
gina 0.1099 0.1204 0.1259 0.956 0.9485 0.9452 agnostic
hiva 0.2537 0.3471 0.3245 0.8121 0.7297 0.733 agnostic
nova 0.0466 0.112 0.0717 0.99 0.9376 0.9741 agnostic
sylva 0.0041 0.0102 0.0077 0.9993 0.9994 0.9991 agnostic
Overall 0.1188 0.1646 0.1408 0.933 0.8971 0.9123 agnostic

This entry is a complete agnostic learning entry.