Top Document: comp.ai.neural-nets FAQ, Part 3 of 7: Generalization Previous Document: What is Bayesian Learning? Next Document: How many hidden layers should I use? See reader questions & answers on this topic! - Help others by sharing your knowledge Methods for combining networks are a subject of active research. Many different methods with different purposes have been proposed. The properties and relationships of these methods are just beginning to be understood. Some methods, such as boosting, are remedies for underfitting. Other methods, such as bagging, are mainly remedies for overfitting or instability. Bayesian learning naturally leads to model averaging (Hoeting et al., 1999). A good general reference is the book edited by Sharkey (1999), especially the article by Drucker (1999). Regarding the effects of bagging and weight decay used together, see Taniguchi and Tresp (1997). Here is a list of terms used for various methods of combining models, mostly taken from Christoph M. Friedrich's web page (see below): o Adaboost o ADDEMUP o arcing: adaptive recombination of classifiers o bagging: bootstrap aggregation o bag-stacking: bagging plus stacking o boosting o cascading o combination of classifiers o committees of networks o consensus theory o cragging: cross aggregation (like k-fold cross validation) o dagging: disjoint-sample aggregation o dag-stacking: dagging plus stacking o divide and conquer classifiers o ensembles o hagging: half-sample aggregation o mixture of experts o multiple classifier systems: o multi-stage and multi-level classifiers o OLC: optimal linear combination o pandemonium of reflective agents o sieving algorithms o stacking: feeding outputs of several models (and possibly the the original inputs) into a second-level model o voting URLs: o Christoph M. Friedrich's web page, "Combinations of Classifiers and Regressors Bibliography and Guide to Internet Resources" at http://www.tussy.uni-wh.de/~chris/ensemble/ensemble.html o Tirthankar RayChaudhuri's web page on combining estimators at http://www-comp.mpce.mq.edu.au/~tirthank/combest.html o Robert E. Schapire's boosting page at http://www.research.att.com/~schapire/boost.html o http://www.boosting.org/ References: Clemen, Robert T. (1989), "Combining forecasts: A review and annotated bibliography", International Journal of Forecasting, Vol 5, pp 559-584. Drucker, H. (1999), "Boosting using neural networks," in Sharkey (1999), pp. 51-78. Hoeting, J. A., Madigan, D., Raftery, A.E., and Volinsky, C.T. (1999) "Bayesian Model Averaging: A Tutorial (with discussion)," Statistical Science, 14:4, 382-417. Corrected version available at http://www.stat.washington.edu/www/research/online/hoeting1999.pdf Sharkey, A.J.C. (1999), Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems, London: Springer. Taniguchi, M., and Tresp, V. (1997), "Averaging regularized estimators," Neural Computation, 9, 1163-1178. User Contributions:Top Document: comp.ai.neural-nets FAQ, Part 3 of 7: Generalization Previous Document: What is Bayesian Learning? Next Document: How many hidden layers should I use? Part1 - Part2 - Part3 - Part4 - Part5 - Part6 - Part7 - Single Page [ Usenet FAQs | Web FAQs | Documents | RFC Index ] Send corrections/additions to the FAQ Maintainer: saswss@unx.sas.com (Warren Sarle)
Last Update March 27 2014 @ 02:11 PM
|
Comment about this article, ask questions, or add new information about this topic: