What is tokenization used for in natural language processing (NLP)?
- ATo encrypt text data
- BTo compress text files
- CTo break text into smaller units for processing (correct answer)
- DTo translate text between languages
Reveal answer & explanationHide answer
The correct answer is C. Option C: To break text into smaller units for processing.