The NN benchmarking resources page at 
http://wwwipd.ira.uka.de/~prechelt/NIPS_bench.html was created after a NIPS
1995 workshop on NN benchmarking. The page contains pointers to various
papers on proper benchmarking methodology and to various sources of
datasets. 

Benchmark studies require some familiarity with the statistical design and
analysis of experiments. There are many textbooks on this subject, of which
Cohen (1995) will probably be of particular interest to researchers in
neural nets and machine learning (see also the review of Cohen's book by Ron
Kohavi in the International Journal of Neural Systems, which can be found
on-line at http://robotics.stanford.edu/users/ronnyk/ronnyk-bib.html). 

Reference: 

   Cohen, P.R. (1995), Empirical Methods for Artificial Intelligence,
   Cambridge, MA: The MIT Press. 

Some parts © 2009 Advameg, Inc.