In the previous posts, we discussed various foundational topics related to NLP and Language Models. In this topic, we discuss NLP key terms and concepts.

With all the knowledge we obtained through previous posts and from this post,  we should be able to learn important concepts of NLP and Language Models.

NLP Key Terms and Concepts

Corpus

corpus means a collection of texts

Tokenization

In any NLP task, generally, Tokenization is the first step the NLP engineer does. For the given input text, splitting into smaller text, as small as words or characters are defined as Tokenization.

Stop Words

Specific words in an “input” sentence or large text or corpus are removed OR filtered out before NLP processing the “input”, since those words in the “input” text do not contribute to the overall NLP processing purpose.

Bag of Words (BoW)

For the given “input” text, Baig of Words’ main objective is to count the number of occurrences of words within the “input” text.

The bag of words does not give importance OR omits grammar and also the word order in the “input” text.

NOTE:

There are many other key terms and concepts which are discussed. in previous posts and in the future posts.