Lectures: Tues 14-16 in Room -1.05, Building C7.3
Tutorials: Thurs 14-16 in Room -1.05, Building C7.3
Begin: Tues 19.04.2022
Exam: Tues, July 19, 2022 @ 14-16
The course will use Microsoft Teams: CLICK HERE. All course materials, links, downloads, and tutorial assignments will be distributed (and submitted) there.
** Course Organisation for Summer 2022 **
- NOTE: The introductory lecture on April 19 @ 14:15 will take place on-line in Teams.
- After this initial meeting, the course is planned to take place in presence. Lectures will not be streamed or recorded.
- Tutorial sheet will be posted in Teams, after each lecture: please get as far as you can before the Tutorial
- Tutorials usually take place on Thursday at 14:15, and you are expected to participate in all sessions
- Tutorials will provide assistance in progressing with the tutorials, as well as addition Q&A regarding lecture material
- Completed tutorial sheets should be submitted in Teams by midnight, before the next lecture.
Topic: This course will examine neurocomputational (or connectionist) models of human language processing. We will start from biological neurons, and show how their processing behaviour can be modelled mathematically. The resulting artificial neurons will then be wired together to form artificial neural networks, and we will discuss how such networks can be applied to build neurocomputational models of language learning and language processing. It will be shown that such models effectively all share the same computational principles, and that any differences in their behaviour is driven by differences in the representations that they process and construct. Near the end of the course, we will use the accumulated knowledge to construct a psychologically plausible neurocomputational model of incremental (word-by-word) language comprehension that constructs a rich utterance representation beyond a simple syntactic derivation or semantic formula.
The Basics:
- Modelling neural information processing (Connectionism)
- Two-layer neural networks and their properties (The Perceptron)
- Multi-layer neural networks: Towards internal representations (Multi-layer Perceptrons)
- Neural information encoding: Localist versus Distributed schemes (Representations)
- Modelling the acquisition of the English past-tense and reading aloud
- Processing sequences: Simple Recurrent Networks (SRNs)
- Modelling the acquisition of hierarchical syntactic knowledge
- Richer representations for sentence understanding
- Neurobiological plausibility of connectionism
Requirements: While the course does not involve programming, students should be comfortable with basic concepts of linear algebra, as well as with basic linux command line use and file editing/manipulation.
Software: The tutorial assignments will use the Mesh neural network simulator (developed by Harm Brouwer). This can be run on the coli servers, or your can compile it on your own computer.
Literature:
P. McLeod, K. Plunkett and E. T. Rolls (1998). Introduction to Connectionist Modelling of Cognitive Processes. Oxford University Press. Chapters: 1-5, 7, 9.
K. Plunkett and J. Elman (1997). Exercises in rethinking innateness: A Handbook for Connectionist Simulations. MIT Press. Chapters: 1-8, 11, 12.
J. Elman (1990). Finding Structure in Time. Cognitive Science, 14: 179-211.
J. Elman (1991). Distributed representations, simple recurrent networks, and grammatical structure. Machine Learning, 7: 195-225.
Additional readings:
N. Chater and M. Christiansen (1999). Connectionism and natural language processing. Chapter 8 of Garrod and Pickering (eds.): Language Processing. Psychology Press.
M. Christiansen and N. Chater (1999). Connectionist Natural Language Processing: The State of the Art. Cognitive Science, 23(4): 417-437.
J. Elman et al. (1996). Chapter 2: Why Connectionism? In: Rethinking Innateness. MIT Press.
J. Elman (1993). Learning and development in neural networks: The importance of starting small. Cognition, 48: 71-99.
M. Seidenberg and M. MacDonald (1999). A Probabilistic Constraints Approach to Language Acquisition and Processing. Cognitive Science, 23(4): 569-588.
M. Steedman (1999). Connectionist Sentence Processing in Perspective. Cognitive Science, 23(4): 615-634.