Classification rule based on Bayesian naive Bayes models with feature selections bias corrected

Longhai Li, Department of Mathematics and Statistics, University of Saskatchewan

Description

This R package is used to predict the binary response based on high dimensional binary features with Bayesian naive Bayes models. The software also accepts real values but they will be converted into binary by thresholding at the medians estimated from the data. A small number of features can be selected based on the correlations with the response. The bias due to this selection can be corrected.

A short-cut function for doing cross-validation with the classifier is also provided.

The software is most suitable for analyzing the data with very high dimension, for example the diagnosis of cancer based on the gene expression data.

Copyright

Permission is granted for anyone to copy, use, modify, or distribute these programs and accompanying documents for any purpose, provided this copyright notice is retained and prominently displayed, and note is made of any changes made to these programs. These programs and documents are distributed without any warranty, express or implied. As the programs were written for research purposes only, they have not been tested to the degree that would be advisable in any important application. All use of these programs is entirely at the user's own risk.

If you found that this software is useful in your work, please do not hesitate to cite the software or the papers below.

Documentations and Source Packages

  1. predbayescor_1.1-4, released 10/04/2008, Documentation: predbayescor_1.1-4.pdf, source package: predbayescor_1.1-4.tar.gz
  2. predbayescor_1.1-3, released 21/02/2008, Documentation: predbayescor_1.1-3.pdf, source package: predbayescor_1.1-3.tar.gz

References

  • Li, L., Zhang, J., and Neal, R. M. (2007), A Method for Avoiding Bias from Feature Selection with Application to Naive Bayes Classification Models: abstract

  • Li, L. (2007), Bayesian Classification and Regression with High Dimensional Features: abstract

    Instruction of Installing an R package

    This is an R add-on package. R is a free software environment for statistical computing and graphics, available from http://www.r-project.org/. You need to install R and add this package.

    Install

    Method 1: Install from source

    R instruction for installing from source package can be found by typing ?INSTALL in R console. Simply, you need to do the following:

    Download the add-on R package, say mypkg, and type the following command in Unix console to install it to /my/own/R-packages/:

    $ R CMD INSTALL mypkg -l /my/own/R-packages/

    Method 2: Install from CRAN directly

    Type the following command in R console to install it to /my/own/R-packages/ directly from CRAN:

    > install.packages("mypkg", lib="/my/own/R-packages/")

    Load the library

    Type the following command in R console to load the package

    > library("mypkg", lib.loc="/my/own/R-packages/")

    Instruction of using R

    1. Official R manuals
    2. A short R tutorial with examples (PDF, 19 pages) by myself

    Examples of classification with Colon gene expression data

    The original real-valued colon data of R format: colon.rda. The binary colon data of R format: colon.bin.rda. There are 62 patients (40 vs 22) and 2000 genes. They can be loaded into R workspace by using "load" function:

    > load("colon.bin.rda")

    Test how well the above method with leave-one-out crossvalidation:

    >cv.bayes(colon.bin,T,62,4,0.4,30,0.8,5,30,T,40)

    Results:
    The result of above R command is shown by cv-colon-result. The error rate of above analysis is 0.0967742, i.e. 6 out 62 cases were misclassified. This is the lowest error rate for Colon data to my knowledge. We selected only 4 features out of 2000 for each iteration in cross-validation. Our method is also very fast, taking totally 103 secs for 62 folds crossvalidation, which includes also the time for feature selection. One more thing, our method is also pretty simple conceptually.

    Click here for the older version (1.0) of this software.
    Back to Longhai Li's home page, or
    Back to Longhai Li's software packages