In the previous post, we learned what are Sentence and Document Embeddings. In this short post, we learn which embedding (sentence or document) to consider.
Sentence Embedding vs Document Embedding – Which one to consider
| Sentence Embedding | Document Embedding |
|
If you need detailed information on a large text, then consider words and sentence embeddings. |
If you are more interested in higher-level of information on a large text, then consider document embeddings. high-level means the classification of a document as a whole. Classification of a document could be Entertainment, Politics, Sports, and more. the higher level also means the TOPIC classification of a document. |
| Consider word and sentence embeddings if you are looking for a lower level of detailed information like emotions at the tech sentence level, the subject of a political scandal or a sentence, the subject of a large news article, and more. | While analyzing the documents, remove some of the details obtained from the sentences. |
| You can consider the first “N” paragraphs of a document instead of considering all the text in the document. | |
|
Through the TF-IDF technique, you can determine a. words that are more relevant to the topic(s) of document b. most discriminatory words of document c. words that are more/less likely to appear in a document |
Related Topics:
- Natural Language Processing Primer
- NLP Learnings 02 – After Primer – NLP Models & Evaluating NLP Models
- NLP Learnings 03 – What is Linguistic in NLP and Linguistic Categories or Linguistic Levels
- NLP Learnings 04 – Sentence Linguistic Analysis – Words As First Step
- NLP Learnings 05 – Sentence Linguistic Analysis – Pragmatics Analysis
- NLP Learnings 06 – What can we do with A Sentence in NLP Tasks
- NLP Learnings 07 – Introducing NLP Language Models
- NLP Learnings 08 – Language Models – Probability Types
- NLP Learnings 09 – Language Models – Measuring Text Based On Probability
- NLP Learnings 10 – Language Models – Defining Word Boundaries
- NLP Learnings 11 – Language Models – Your Business Text Data and Their Words Representation
- NLP Learnings 12 – Language Models – Regularization Techniques And Your Business Text Data
- NLP Learnings 13 – Language Models – Smoothing Regularization Techniques And Your Business Text Data
- NLP Learnings 14 – Language Models – NLP Key Terms and Concepts
- NLP Learnings 15 – Language Models – Classifiers Introduction
- NLP Learnings 16 – Language Models – Classifiers – What are Probabilistic Classifiers
- NLP Learnings 17 – Language Models – Classifiers – Simple Classifiers Introduction
- NLP Learnings 18 – Language Models – Classifiers – Simple Classifiers – Linear Probabilistic Classifier Introduction
- NLP Learnings 19 – Language Models – Classifiers – Evaluating Classifiers – Precision Recall F-Score Confusion Matrix
- NLP Learnings 20 – Language Models – Classifiers – Represent Words In A Document – Choosing and Representing Features In The Right Way
- NLP Learnings 21 – Language Models – Word Embeddings
- NLP Learnings 22 – Language Models – Sentence and Document Embeddings
- NLP Learnings 23 – Language Models – Sentence and Document Embeddings – Which one to consider