LOGIN TO YOUR ACCOUNT

Username
Password
Remember Me
Or use your Academic/Social account:

CREATE AN ACCOUNT

Or use your Academic/Social account:

Congratulations!

You have just completed your registration at OpenAire.

Before you can login to the site, you will need to activate your account. An e-mail will be sent to you with the proper instructions.

Important!

Please note that this site is currently undergoing Beta testing.
Any new content you create is not guaranteed to be present to the final version of the site upon release.

Thank you for your patience,
OpenAire Dev Team.

Close This Message

CREATE AN ACCOUNT

Name:
Username:
Password:
Verify Password:
E-mail:
Verify E-mail:
*All Fields Are Required.
Please Verify You Are Human:
fbtwitterlinkedinvimeoflicker grey 14rssslideshare1
Vadera, S
Publisher: ACM
Languages: English
Types: Article
Subjects: other, QA75
This article presents a new decision tree learning algorithm called CSNL that induces Cost-Sensitive Non-Linear decision trees. The algorithm is based on the hypothesis that nonlinear decision nodes provide a better basis than axis-parallel decision nodes and utilizes discriminant analysis to construct nonlinear decision trees that take account of costs of misclassification.\ud \ud The performance of the algorithm is evaluated by applying it to seventeen datasets and the results are compared with those obtained by two well known cost-sensitive algorithms, ICET and MetaCost, which generate multiple trees to obtain some of the best results to date. The results show that CSNL performs at least as well, if not better than these algorithms, in more than twelve of the datasets and is considerably faster. The use of bagging with CSNL further enhances its performance showing the significant benefits of using nonlinear decision nodes.\ud \ud \ud The performance of the algorithm is evaluated by applying it to seventeen data sets and the results are \ud compared with those obtained by two well known cost-sensitive algorithms, ICET and MetaCost, which generate multiple trees to obtain some of the best results to date.\ud The results show that CSNL performs at least as well, if not better than these algorithms, in more than twelve of the data sets and is considerably faster. \ud The use of bagging with CSNL further enhances its performance showing the significant benefits of using non-linear decision nodes.
  • The results below are discovered through our pilot algorithms. Let us know how we are doing!

    • Abe, N., Zadrozny, B., and Langford, J. 2004. An iterative method for multi-class costsensitive learning. In Proc. of KDD, Seattle, Washington. 3-11.
    • Allwein, E. L., Schapire, R. E., and Singer, Y. 2000. Reducing multiclass to binary: A unifying approach for margin classifiers. In Proc. 17th Int. Conf. on Machine Learning. San Francisco: Morgan Kaufmann, 9-16.
    • Althoff, K., Auriol, E., Barletta, R., and Manago, M. 1995. A review of industrial casebased reasoning tools. Oxford: AI Intelligence.
    • Bauer, E. and Kohavi, R. 1999. An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learning 36, 1-2, 105-139.
    • Bennett, K. P. 1999. On mathmatical programming methods and support vector machines. In Advances in Kernel Methods - Support Vector Machines, A. Schoelkopf, C. Burges, and A. Smola, Eds. Cambridge, MA:MIT Press, Chapter 19, 307-326.
    • Berry, M. and Linoff, G. 2004. Data Mining Techniques, 2nd ed. New York : Wiley.
    • Blake, C. and Merz, C. 1998. UCI Repository of Machine Learning Databases. Irvine, CA: University of California, Department of Information and Computer Science, USA, available at //www.ics.uci.edu/ mlearn/ MLRepository.html.
    • Bradford, J., Kunz, C., Kohavi, R., Brunk, C., and Brodley, C. 1998. Pruning decision trees with misclassification costs. In Proc. of the 10th European Conf. on Machine Learning, Chemnitz, Germany, Lecture Notes in Computer Science, No 1398. Heidelberg:Springer Verlag, 131-136.
    • Breiman, L. 1996. Bagging predictors. Machine Learning 24, 2, 123-140.
    • Breiman, L., Friedman, J. H., Olsen, R. A., and Stone, C. J. 1984. Classification and Regression Trees. Belmont: Wadsworth.
    • Breslow, L. and Aha, D. 1997a. Comparing tree-simplification procedures. In Proc. of the 6th Int. Workshop on Artificial Intelligence and Statistics, Ft. Lauderdale, Florida. 67-74.
    • Breslow, L. and Aha, D. 1997b. Simplifying decision trees: A survey. Knowledge Engineering Review 12, 1-40.
    • Brown, G. 2009. Feature selection by filters: a unifying perspective. In Proc. of the 5th UK Symposium on Knowledge Discovery and Data Mining, S. Vadera, Ed. University of Salford, UK, 34-43.
    • Dash, M. and Liu, H. 1997. Feature selection for classification. Intelligent Data Analysis 1, 3, 131-156.
    • Domingos, P. 1999. MetaCost: A general method for making classifiers cost-sensitive. In Proc. of the 5th Int. Conf. on Knowledge Discovery and Data Mining. 155-164.
    • Draper, B., Brodley, C. E., and Utgoff, P. E. 1994. Goal-directed classification using linear machine decision trees. IEEE Transactions on Pattern Analysis and Machine Intelligence 16, 9, 888-893.
    • Drummond, C. and Holte, R. 2006. Cost curves: An improved method for visualizing classifier performance. Machine Learning 65, 1, 95-130.
    • Elkan, C. 2001. The foundations of cost-sensitive learning. In Proc. of the 17th Int. Joint Conf. on Artificial Intelligence. San Francisco : Morgan Kaufmann, 973-978.
    • Esmeir, S. and Markovitch, S. 2008. Anytime induction of low-cost, low-error classifiers: a sampling-based approach. Journal of Artificial Intelligence Research (JAIR) 33, 1-31.
    • Esposito, F., Malerba, D., and Semeraro, G. 1997. A compartive analysis of methods for pruning decision trees. IEEE Transactions on Pattern Analysis and Machine Intelligence 19, 5, 476-491.
    • Fan, W., Stolfo, S., Zhang, J., and Chan, P. 1999. AdaCost: Misclassification cost-sensitive boosting. In Proc. of the 16th Int. Conf. on Machine Learning. San Francisco : Morgan Kaufmann, 97-105.
    • Fisher, R. 1936. The use of multiple measurements in taxonomic problems. Annals of Eugenics 8, 179-188.
    • Frank, E. and Witten, I. 1998. Reduced-error pruning with significance tests, available at http://citeseer.ist.psu.edu/frank98reducederror.html.
    • Grefenstette, J. 1986. Optimization of control parameters for genetic algorithms. IEEE Transactions on Systems, Man, and Cybernetics 16, 122-128.
    • Hong, S. J. 1997. Use of contextual information for feature ranking and discretization. IEEE Transactions on Knowledge and Data Engineering 9, 718-730.
    • Ji, S. and Carin, L. 2007. Cost-sensitive feature acquisition and classification. Pattern Recognition 40, 5, 1474-1485.
    • Johnson, R. and Wichern, D. 1998. Applied Multivariate Statistical Analysis, 4th ed. Englewood Cliffs:Prentice Hall.
    • Kanani, P. and Melville, P. 2008. Prediction-time active feature-value acquisition for customer targeting. In Proc. of the Workshop on Cost Sensitive Learning, NIPS 2008, available at http://www.cs.iastate.edu/ oksayakh/csl/accepted papers/kanani.pdf.
    • Knoll, U., Nakhaeizadeh, G., and Tausend, B. 1994. Cost-sensitive pruning of decision trees. In Proc. of the 8th European Conf. on Machine Learning. Vol. 2. Berlin:Springer-Verlag, 383- 386.
    • Kohavi, R. and John, G. H. 1997. Wrappers for feature subset selection. Artificial Intelligence 97, 1-2, 273-324.
    • Ling, C., Sheng, V., and Yang, G. 2006. Test strategies for cost-sensitive decision trees. IEEE Transactions on Knowledge and Data Engineering 18, 8, 1055-1067.
    • Margineantu, D. 2001. Methods for cost-sensitive learning. Ph.D. thesis, School of Electrical Engineering and Computer Science, Oregan State University, Corvallis, USA.
    • Martin, A., Doddington, G., Kamm, T., Ordowski, M., and Przybocki, M. 1997. The DET curve in assessment of detection task performance. In Proc. of Eurospeech '97. Int. Speech Communications Association, Bonn, Germany, Rhodes, Greece, 1895-1898.
    • Masnadi-Shirazi, H. and Vasconcelos, N. 2007. Asymmetric boosting. In Proc. of 24th Int. Conf. on Machine Learning. 609-616.
    • Masnadi-Shirazi, H. and Vasconcelos, N. 2008. On the design of loss functions for classification: theory, robustness to outliers, and SavageBoost. In Proc. of Neural Information Processing Systems, D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, Eds. 1049-1056.
    • Merler, S., Furlanello, C., Larcher, B., and Sboner, A. 2003. Automatic model selection in cost-sensitive boosting. Information Fusion 4, 3-10.
    • Mingers, J. 1989. An empirical comparison of pruning methods for decision tree induction. Machine Learning 4, 227-243.
    • Murthy, S., Kasif, S., and Salzberg, S. 1994. A system for induction of oblique decision trees. Journal of Artificial Intelligence Research 2, 1-32.
    • Nilsson, N. 1965. Learning Machines. New York : McGraw-Hill.
    • Nun~ez, M. 1991. The use of background knowledge in decision tree induction. Machine Learning 6, 231-250.
    • Pazzani, M., Merz, C., Murphy, P., Ali, K., Hurne, T., and Brunk, C. 1994. Reducing misclassification costs: Knowledge-intensive approaches to learning from noisy data. In Proc. of the 11th Int. Conf. on Machine Learning. San Francisco: Morgan Kaufmann, 217-225.
    • Pazzani, M. J. 2000. Knowledge discovery from data? IEEE Intelligent Systems 15, 2, 10-13.
    • Provost, F. J. and Buchanan, B. G. 1995. Inductive policy: The pragmatics of bias selection. Machine Learning 20, 35-61.
    • Provost, F. J., Fawcett, T., and Kohavi, R. 1998. The case against accuracy estimation for comparing induction algorithms. In Proc. of the 15th Int. Conf. on Machine Learning. 445-553.
    • Quinlan, J. R. 1987. Simplifying decision trees. Int. Journal of Man-Machine Studies 27, 221-234.
    • Quinlan, J. R. 1993. C4.5 : Programs for Machine Learning. California:Morgan Kauffman.
    • Savage, L. 1971. The elicitation of personal probabilities and expectations. Journal of the American Statistical Association 66, 783-801.
    • Swets, J. 1964. Signal Detection and Recognition by Human Observers. John Wiley & Sons Inc.
    • Tan, M. 1993. Cost-sensitive learning of classification knowledge and its applications in robotics. Machine Learning 13, 7-33.
    • Ting, K. 2000. A comparative study of cost-sensitive boosting algorithms. In Proc. of the 17th Int. Conf. on Machine Learning. San Francisco : Morgan Kaufmann, 983-990.
    • Ting, K. and Zheng, Z. 1998. Boosting trees for cost-sensitive classifications. In Proc. of the 10th European Conf. on Machine Learning. 190-195.
    • Turney, P. 1995. Cost sensitive classification: Empirical evaluation of a hybrid genetic decision tree induction algorithm. Journal of Artificial Intelligence Research 2, 369-409.
    • Turney, P. 2000. Types of cost in inductive concept learning. In Proc. of the Workshop on Cost-Sensitive Learning, 7th Int. Conf. on Machine Learning. 15-21.
    • Vadera, S. 2005a. Inducing Cost-Sensitive Nonlinear Decision Trees, Technical Report. School of Computing, Science and Engineering, University of Salford.
    • Vadera, S. 2005b. Inducing safer oblique trees without costs. The International Journal of Knowledge Engineering and Neural Networks 22, 4, 206-221.
    • Vadera, S. and Ventura, D. 2001. A comparison of cost-sensitive decision tree learning algorithms. In Proc. of the 2nd European Conf. on Intelligent Management Systems in Operations. Operational Research Society, Birmingham, UK, 79-86.
    • W.Buntine and Niblett, T. 1992. A further comparison of splitting rules for decision-tree induction. Machine Learning 8, 75-85.
    • Zadrozny, B., Langford, J., and Abe, N. 2003. Cost-sensitive learning by cost-proportionate example weighting. In Proc. of the 3rd IEEE International Conference on Data Mining. 435- 442.
    • Zhu, X., Wu, X., Khoshgoftaar, T., and Shi, Y. 2007. An empirical study of noise impact on cost-sensitive learning. In Proc. of 20th Int.Joint Conf. on Artificial Intelligence. 1168-1174.
  • No related research data.
  • No similar publications.

Share - Bookmark

Cite this article