KwickClips Artificial Intelligence · 42 sec · free
Why does 'the' score zero?
Short answer. It is in every document, so log(1) = 0.
What is the TF-IDF formula?
TF x log(N/DF).
TF-IDF, N = 3
| Word | DF | IDF |
|---|---|---|
| the | 3 | log(1) = 0 |
| cricket | 1 | log(3) = 0.477 |
Remember
| TF x log(N/DF) |
| Rare words score high |
Why does the word the score zero in TF-IDF? TF-IDF is TF times log of N by DF. The is in all three documents, so zero. Cricket is rare, so higher. Remember. TF times log N by DF. Rare words score high.
This clip is from the full lesson: Text Processing: Normalisation, Tokenisation, Bag of Words and TF-IDF — 8 minutes, with the tables, the quick answers and the whole lesson in text.
Useful for: CBSE Class 10 Artificial Intelligence (417), CBSE Class 10 Artificial Intelligence (417), CBSE Class 10 Artificial Intelligence (417)
More KwickClips from this lesson
What comes after sentence segmentation?46 sec
What is the stem of 'studies'?42 sec
What is the first step?38 secDisclaimer. KwickAcademy is free study material for general learning and revision. Parts of it, including the voice-over, are produced with the help of AI tools and may contain errors; if you spot one, please tell us and we will correct it. Syllabus, marks and exam details follow the latest official board publications available to us, and boards can change them at any time, so always confirm against your board's official website and your school. Using this material does not guarantee any marks or result. Board names and trademarks belong to their owners; Kwickprep is not affiliated with or endorsed by any examination board. We never ask for passwords, OTPs or ID numbers. Your progress is saved only in this browser. Full disclaimer · Privacy