In all the previous posts we discussed foundational concepts of NLP and Language Models. In this post, we introduce Classifiers in Language Models.

Classifiers Definition in Simple Terms

In simple terms, a classifier is a “collection of rules” you write using Python or R or Java, or any other language. These rules are for how you want to categorize your input data. These rules are the main of your Machine Learning process in your algorithm and it is known as a classifier.

Support Vector Machines (SVM), Naïve Bayes, or a Neural Network are examples of classifiers. 

Another definition is, Classification is categorizing input data or a document into one or more classes like a Sports document, Education document, Politics data, etc.

What are Classifiers in NLP?

For the provided input text or large text, Classifiers divided the input data into two or more categories or classes.

At a very high level, “Input pdf/word document -> convert to large text -> split into paragraphs -> split into sentences -> split into independent words (one-gram) OR a combination of words (n-grams) -> pass the split text into a function (this function is classifier) -> store the output of this function which could be a class (sentiment value OR Parts of Speech kind OR business term relevancy of the split text)

The last phrase in the above sentence is important i.e. what is meant by Classifier class.

Each of the split text before passing to the function (as mentioned above i.e. classifier) can be considered a feature.

So, selecting what a feature should be (i.e. one-gram or n-gram) is important. Some combinations of words may not be useful in the context of your NLP business application. It is very important to consider or select features that are both informative and independent.

Difference Between Classifier and a Model:

Model = training data passed through a classifier

Classifier = set of rules to categorize your input data

What Rules do You Write In A Classifier?

If you consider a large document, it contains pages, paragraphs, sentences, phrases, and words. Combine one or more words, it might provide some context or meaning. For example, consider this sentence “Michael Jordan, Sachin Tendulkar, Tiger Woods, Steffi Graf all are great players and had a great career”. In this sentence, you can find names, adjectives, and something about them. The steps or rules to categorize the above sentence based on the concepts (names or adjectives) the sentence has is called a classifier.

So possible concepts you can consider in a sentence are – places, numbers, names, and more. These are called “features” in a sentence or a document.

So, based on the “features” in a document we can classify the document as sports or politics, or entertainment.

How do you write Rules in a Classifier?

Below is one possible approach for writing Rules in a classifier:

Step 1: Have a list of all possible names, places, numbers, popular phrases, popular sentiment phrases, and more.

Step 2: For the given input document, divide them into sentences or words or a combination of words

Step 3: Scan the above division (sentences, words, combination of words) across what you have in Step 1 above

Step 4: Find the count of occurrences of those (sentences, words, combination of words) across what you have in Step 1 above

Step 5: Now based on your requirement weigh them (places vs names vs numbers or more categories). Which category you like to give more importance 

Step 6: Now you have a classifier with the above as the rules of a classifier

So if we consider

Y = { possible categories in a document} ;

X = { input data split into sentences or combination of words or phrases and then classified as names or places or numbers or anything else} = { Xnames, Xplaces, Xnumbers, ….. Xn)

Then

X -> Classifier -> produces Y

 

Related Topics:

1. Natural Language Processing Primer
2. NLP Learnings 02 – After Primer – NLP Models & Evaluating NLP Models
3. NLP Learnings 03 – What is Linguistic in NLP and Linguistic Categories or Linguistic Levels
4. NLP Learnings 04 – Sentence Linguistic Analysis – Words As First Step
5. NLP Learnings 05 – Sentence Linguistic Analysis – Pragmatics Analysis
6. NLP Learnings 06 – What can we do with A Sentence in NLP Tasks
7. NLP Learnings 07 – Introducing NLP Language Models
8. NLP Learnings 08 – Language Models – Probability Types
9. NLP Learnings 09 – Language Models – Measuring Text Based On Probability
10. NLP Learnings 10 – Language Models – Defining Word Boundaries
11. NLP Learnings 11 – Language Models – Your Business Text Data and Their Words Representation
12. NLP Learnings 12 – Language Models – Regularization Techniques And Your Business Text Data
13. NLP Learnings 13 – Language Models – Smoothing Regularization Techniques And Your Business Text Data
14. NLP Learnings 14 – Language Models – NLP Key Terms and Concepts
15. NLP Learnings 15 – Language Models – Classifiers Introduction
16. NLP Learnings 16 – Language Models – Classifiers – What are Probabilistic Classifiers
17. NLP Learnings 17 – Language Models – Classifiers – Simple Classifiers Introduction
18. NLP Learnings 18 – Language Models – Classifiers – Simple Classifiers – Linear Probabilistic Classifier Introduction
19. NLP Learnings 19 – Language Models – Classifiers – Evaluating Classifiers – Precision Recall F-Score Confusion Matrix
20. NLP Learnings 20 – Language Models – Classifiers – Represent Words In A Document – Choosing and Representing Features In The Right Way
21. NLP Learnings 21 – Language Models – Word Embeddings
22. NLP Learnings 22 – Language Models – Sentence and Document Embeddings
23. NLP Learnings 23 – Language Models – Sentence and Document Embeddings – Which one to consider