NLP Learnings 08 – Language Models – Probability Types

AI & ML & Blockchain, Blog, Machine Learning

In the previous post, we discussed various uses or applications of NLP Language Models. Probability is one of the applications. For a given sentence or text or corpus,

Probability = Number of occurrences of a word OR phrase / Total Number of words or phrases

Language Models – Probability Types

In the context of Language Models, there are two types of Prabilities, (a) Joint Probability (b) Conditional Probability

Join Probability

P(X=x, Y=y) where X and Y are independent

Conditional Probability

P(X=x | Y=y) here given Y as a condition what is the probability of X

Formula for conditional probability is, P(X | Y ) = P(X, Y) / P(Y)

In conditional probability of Language Models, we will be moving left to right of sentence movement.

Meaning, in the sentence “This is a great post”, we will be analyzing starting from the first word i.e. from the left word toward the end

Related Topics:

  1. Natural Language Processing Primer
  2. NLP Learnings 02 – After Primer – NLP Models & Evaluating NLP Models
  3. NLP Learnings 03 – What is Linguistic in NLP and Linguistic Categories or Linguistic Levels
  4. NLP Learnings 04 – Sentence Linguistic Analysis – Words As First Step
  5. NLP Learnings 05 – Sentence Linguistic Analysis – Pragmatics Analysis
  6. NLP Learnings 06 – What can we do with A Sentence in NLP Tasks
  7. NLP Learnings 07 – Introducing NLP Language Models
  8. NLP Learnings 08 – Language Models – Probability Types
  9. NLP Learnings 09 – Language Models – Measuring Text Based On Probability
  10. NLP Learnings 10 – Language Models – Defining Word Boundaries
  11. NLP Learnings 11 – Language Models – Your Business Text Data and Their Words Representation
  12. NLP Learnings 12 – Language Models – Regularization Techniques And Your Business Text Data
  13. NLP Learnings 13 – Language Models – Smoothing Regularization Techniques And Your Business Text Data
  14. NLP Learnings 14 – Language Models – NLP Key Terms and Concepts
  15. NLP Learnings 15 – Language Models – Classifiers Introduction
  16. NLP Learnings 16 – Language Models – Classifiers – What are Probabilistic Classifiers
  17. NLP Learnings 17 – Language Models – Classifiers – Simple Classifiers Introduction
  18. NLP Learnings 18 – Language Models – Classifiers – Simple Classifiers – Linear Probabilistic Classifier Introduction
  19. NLP Learnings 19 – Language Models – Classifiers – Evaluating Classifiers – Precision Recall F-Score Confusion Matrix
  20. NLP Learnings 20 – Language Models – Classifiers – Represent Words In A Document – Choosing and Representing Features In The Right Way
  21. NLP Learnings 21 – Language Models – Word Embeddings
  22. NLP Learnings 22 – Language Models – Sentence and Document Embeddings
  23. NLP Learnings 23 – Language Models – Sentence and Document Embeddings – Which one to consider
Post Views: 509