ENC2045 Computational Linguistics¶
These lecture notes are still work-in-progress. There will always be last-minute changes to the files and notebooks. Please ALWAYS download the most recent version of the notebooks before the class of each week.
Computational Linguistics (CL) is now a very active sub-discipline in applied linguistics. Its main focus is on the computational text analytics, which is essentially about leveraging computational tools, techniques, and algorithms to process and understand natural language data (in spoken or textual formats). Therefore, this course aims to introduce useful strategies and common workflows that have been widely adopted by data scientists to extract useful insights from natural language data. In this course, we will focus on textual data.
A selective collection of potential topics may include:
A Pipeline for Natural Language Processing
Parsing and Chunking
Issues for Chinese Language Processing (Word Segmentation)
Machine Learning Basics
Feature Engineering and Text Vectorization
Traditional Machine Learning
Classification Models (Naive Bayes, SVM, Logistic Regression)
Common Computational Tasks
Tex Clustering and Topic Modeling
Deep Learning NLP
Neural Language Model
Explainable Artificial Intelligence and Computational Linguistics
This course is extremely hands-on and will guide the students through classic examples of many task-oriented implementations via in-class theme-based tutorial sessions. The main coding language used in this course is Python. We will make extensive use of the language. It is assumed that you know or will quickly learn how to code in Python. In fact, this course assumes that every enrolled student has working knowledge of Python. (If you are not sure whether you fulfill the prerequisite, please contact the instructor first.)
A test on Python Basics will be conducted on the second week of the class to ensure that every enrolled student fulfills the prerequisite. (To be more specific, you are assumed to have already had working knowledge of all the concepts included in the book, Lean Python: Learn Just Enough Python to Build Useful Tools). Those who fail on the Python basics test are NOT advised to take this course.
Please note that this course is designed specifically for linguistics majors in humanities. For computer science majors, this course will not feature a thorough description of the mathematical operations behind the algorithms. We focus more on the practical implementation.
All the course materials will be available on the course website ENC2045. You may need a password to access the course materials/data. If you are an officially enrolled student, please ask the instructor for the passcode.
We will also have a Moodle course site for assignment submission.
Please read the FAQ of the course website before course registration.
The course materials are mainly based on the following readings.
Also, there are many other useful reference books, which are listed as follows in terms of three categories:
In addition to books, there are many wonderful on-line resources, esp. professional blogs, providing useful tutorials and intuitive understanding of many complex ideas in NLP and AI development. Among them, here is a list of my favorites:
YouTube Channels ¶
We assume that you have created and set up your python environment as follows:
Install the python with Anaconda
Create a conda environment named
Run the notebooks provided in the course materials in this self-defined conda environment
We will use
jupyter notebookfor python scripting in this course. All the assignments have to be submitted in the format of jupyter notebooks. (See Jupyter Notebook installation documentation).
We assume you have created a conda virtual environment, named,
python-notes, for all the scripting, and you are able to run the notebooks in the conda environment kernel in Jupyter.
You can also run the notebooks directly in Google Colab, or alternatively download the
.ipynbnotebook files onto your hard drive or Google Drive for further changes.
When you run the installation of the python modules and other packages in the terminal, please remember to change the arguments of the parameters when you follow the instructions provided online.
For example, when you create the conda environment:
conda create --name XXX
You need to specify the conda environment name
XXX on your own.
Similarly, when you add the self-defined conda environment to the notebook kernel list:
python -m ipykernel install --user --name=XXX
You need to specify the conda environment name
There are several important things here:
You need to install the relevant modules AFTER you activate the conda environment in the terminal.
You need to add the kernel name with
python -m ipykernel install --user --name=XXXwithin the conda enviroment as well.
In other words, you need to install the module
ipykernelin the target conda environment as well.
After a few trial-and-errors, I think the best environment setting is that you only add the kernel name (conda environment) to
ipykernelwithin the conda environment. Do not add the conda environment again in your base python environment.
What’s even better is to install
jupyterin your conda environment (
python-notes) and run your notebook from this
For Windows users, it is highly recommended to run the installation of python-related modules in
Anaconda Prompt instead of
Because some of the window’s users are suffering from the problem that the notebook cannot find the correct path of your
python-notes environment. Could you please try the following?
We assume you run the following steps in Anaconda Powershell Prompt.
Create the new conda environment,
python-notes(if you have done this, ignore)
$ conda create --name python-notes python=3.7
Activate the newly-created
$ conda activate python-notes
python-notesenvironment, install the module
$ conda install ipykernel
Deactivate the conda environment
$ conda deactivate
Install the module
nb_conda_kernelsin your Anaconda base python (This is crucial to the use of self-defined conda environments in notebook.)
$ conda install nb_conda_kernels
Then check if the system detects your
$ python -m nb_conda_kernels list
Ideally, you should see your
python-notes popping up in the results like:
If you see the new conda environment name, then it should work. Next, initiate your jupyter notebook
$ jupyter notebook
Now you should be able to see you conda environment
python-notesin your notebook, but the kernel name would be something like
python [conda evn: python-notes].
Not all modules used in this course come with the default installation of the python environment. Please remember to install these packages in use if you get a module-not-found type of errors.
The following is a list of packages we will use (non-exhaustive):
Please google these packages for more details on their installation.
Coding Assignments Reminders¶
You have to submit your assignments via Moodle.
Please name your files in the following format:
Please always submit both the Jupyter notebook file and its HTML version.
Paul Gerrard. Lean Python: Learn Just Enough Python to Build Useful Tools. Apress, 2016.
Wes McKinney. Python for data analysis: Data wrangling with Pandas, NumPy, and IPython. " O'Reilly Media, Inc.", 2012.
Aurélien Géron. Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow: Concepts, tools, and techniques to build intelligent systems. O'Reilly Media, 2019.
Dipanjan Sarkar. Text analytics with Python: a practitioner's guide to natural language processing. Apress, 2019.
Steven Bird, Ewan Klein, and Edward Loper. Natural language processing with Python: analyzing text with the natural language toolkit. " O'Reilly Media, Inc.", 2009.
Sowmya Vajjala, Bodhisattwa Majumder, Anuj Gupta, and Harshit Surana. Practical Natural Language Processing: A Comprehensive Guide to Building Real-World NLP Systems. O'Reilly Media, 2020.
Jacob Perkins. Python 3 text processing with NLTK 3 cookbook. Packt Publishing Ltd, 2014.
Bhargav Srinivasa-Desikan. Natural Language Processing and Computational Linguistics: A practical guide to text analysis with Python, Gensim, spaCy, and Keras. Packt Publishing Ltd, 2018.
Chollet Francois. Deep learning with Python. Manning Publications Company, 2017.