Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Sparse Bernoulli Naive Bayes #23939

Copy link
Copy link
Open
Open
Copy link
@carlosg-m

Description

@carlosg-m
Issue body actions

Describe the workflow you want to enable

The idea seems simple enough as described in this paper: https://arxiv.org/pdf/1905.09884.pdf

Due to its linear complexity, naive Bayes classification remains an attractive su-
pervised learning method, especially in very large-scale settings. We propose a
sparse version of naive Bayes, which can be used for feature selection. This leads
to a combinatorial maximum-likelihood problem, for which we provide an exact
solution in the case of binary data, or a bound in the multinomial case. We prove
that our bound becomes tight as the marginal contribution of additional features
decreases. Both binary and multinomial sparse models are solvable in time almost
linear in problem size, representing a very small extra relative cost compared to
the classical naive Bayes. Numerical experiments on text data show that the naive
Bayes feature selection method is as statistically effective as state-of-the-art feature
selection methods such as recursive feature elimination, l1-penalized logistic re-
gression and LASSO, while being orders of magnitude faster. For a large data set,
having more than with 1.6 million training points and about 12 million features,
and with a non-optimized CPU implementation, our sparse naive Bayes model can
be trained in less than 15 seconds.

Describe your proposed solution

Add a regularization parameter to the current implementation of BernoulliNB that allows to control sparsity of the solution similar to regularization strength or "C" in LogisticRegression with a L1 penalty.

Make the necessary adjustments to the algorithm.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      Morty Proxy This is a proxified and sanitized view of the page, visit original site.