What This Document Is
This document represents Session Four of INFO 256: Applied Natural Language Processing at UC Berkeley. It’s a lecture-based exploration of fundamental resources used in the field – specifically, how to leverage existing text collections and linguistic data for NLP tasks. The session delves into the practical aspects of working with both raw text and text that has been enhanced with linguistic information. It’s designed to build a strong foundation for more advanced topics covered later in the course.
Why This Document Matters
This session is crucial for students aiming to build real-world NLP applications. Understanding how to source, prepare, and utilize text data is a core skill for any NLP practitioner. It’s particularly valuable for those new to corpus linguistics or seeking to expand their knowledge of available resources. Reviewing this material will be beneficial before undertaking projects that require data acquisition and pre-processing, and serves as a reference point throughout the course.
Topics Covered
* Text Corpora: Definition, characteristics, and sources.
* Lexical Resources: Exploring different types of dictionaries and databases for linguistic information.
* Corpus Creation: Considerations for building your own text collections.
* Text Annotation: The process of adding linguistic information to text.
* Specific Corpora: Overview of resources like the Gutenberg Corpus and web-based text collections.
* Resource Discovery: Utilizing tools and platforms for finding language resources.
What This Document Provides
* An overview of commonly used text corpora available through the NLTK library.
* Information on accessing and exploring these corpora programmatically.
* Introductions to key lexical resources such as WordNet, VerbNet, and FrameNet.
* Guidance on locating additional language resources through organizations like the Open Language Archives Community (OLAC).
* A foundational understanding of the role of annotation in NLP.
