In the previous post, we discussed the Naive Bayes Probabilistic classifier. In this post, we learn Linear classifiers. Firstly, Linear means a straight line.

Fundamental Concept in Linear Probabilistic Classifer

In Linear classifiers, we consider weights of each feature.

As we discussed in the previous post (Naive Bayes classifier), features are created from the the input documents during the machine learning model training.

Introduction Linear Classifiers

 Linear classifiers use weights of each feature in the input documents and try to find a linear line. Linear models, it does consider each feature together and their weightage while identifying the linear line.

In reality, there will not be any line (straight line) among the English words considering various words within a document and those words coming together in real life is impractical.

So, Linear Models try to identify the difference between the “ideal” Linear Line among the words and the “actual” Linear Line that came up after considering all the words in the input documents.

Linear models will find the difference between the two lines (“ideal” line vs. “actual” line). By changing the weights of each word in the actual data, Linear Models try to have minimum difference value between the two lines (“ideal” line vs. “actual” line). So, one of the main jobs of Linear Models is minimizing the difference between the (“ideal” line vs. the “actual” line).

Machine Learning models developed based on Linear Classifier is for predicting the category of a new text. 

Above means, consider 1 million documents, you train them and classify them as Entertainment Politics Sports or Science Technology Family Nature, and more. Now you give a new text to the model, it should be able to classify as one or more of the possible values Entertainment Politics Sports Science Technology Family Nature, and more.

Linear classifier definition:

score (or probability) of a particular cclassification is based on a linear combination of features and
their weights

Categories of Linear Classifiers

Binary Classifiers – only two categories or classes to predict for the given new input text

Multi-Class Classifiers – predict two or more categories or classes to predict for the given new input text

What activities to do in Linear Classifiers

As mentioned above, Linear means straight line and classifiers one of the jobs is to create such a line from the training text documents. This line has a minimalistic distance from its “ideal” straight line. For this, Linear Classifiers use weights of each word available in the training text documents.

So, Linear Classifiers should find “weight” of each word that has importance in the overall linear line for the given training text documents.

Types of Classifiers

Perceptron

Large-margin (support vector machine)

Logistic regression