Syllabus for Q750:
Recent Advances in
Connectionist Models
(listed as "Neural Networks as
Models of Cognition")
Spring 1996, Section 1972, Tu & Th
2:30-3:45, PY 230
Instructor: Prof.
John K. Kruschke
Office: 855-3192, PY 336 by appt.
E-mail: kruschke@indiana.edu
Contents:
Course Description and Goals:
This course selectively surveys some recent applications of
connectionist models to cognitive phenomena. The emphasis is on
models that attempt to address detailed empirical data. We will not
survey "generic" algorithms or architectures that are not directed at
specific cognitive phenomena and empirical data. The survey will be
accomplished by reading and discussing various journal articles. Each
student will lead (or co-lead) discussion of selected articles.
Each student will also program, from scratch, a connectionist model
applied to cognitive phenomena of his or her choice. The model could
be a novel architecture or an existing model applied to new phenomena.
The last few weeks of the course will be devoted to presentations of
these projects.
The course has three main goals. First, students should gain
understanding of a variety of connectionist models applied to
empirical data. Second, students should gain experience applying a
connectionist model to specific cognitive phenomena. Third, students
should gain a better understanding of the role of models in explaining
empirical phenomena.
Prerequisites:
- You must have strong background knowledge in connectionist models.
This can be achieved by previously completing the course Q550. If you
are self-taught or have taken some other introductory course in
connectionist models, please discuss the matter with Prof. Kruschke.
- You must be adept at computer programming. We will not have time
to discuss programming methods much in class, so you are expected to
be able to program independently. The preferred programming language
is C, but any language is permitted. For all languages, the code
turned in with your final project must be thoroughly commented.
Schedule:
The schedule below is approximate, and may flex and bend as the course
progresses. The order of topics will probably remain the same, but
the exact dates for each topic might change. There might be changes to
the readings as the course progresses. Full reference information for
each reading is provided at the end of the syllabus.
Schedule of Project Due Dates:
A brief
project proposal is due on the Thursday of the fourth week of class.
The proposal must specify (1) the empirical phenomena to be addressed,
(2) the type of model architecture you intend to use, (3) the
programming language you will use, and (4) which computer you will
use. The final write-up of your project is due on the Thursday of the
15th week of class, except for those people who present their projects
on that day. For these people, the final write-up is due the next day,
Friday. There is no final exam.
- Schedule of Readings:
-
- Week 1: Introduction and scheduling of presentations.
-
- Week 2: Meta-Theory.
- McCloskey, M. (1991).
Networks and theories: The place of connectionism in cognitive
science.
- Seidenberg, M. S. (1993a). Connectionist models and
cognitive theory.
- Week 3: Category Learning.
- Kruschke, J. K.
(1992). ALCOVE: An exemplar-based connectionist model of category
learning.
- Kruschke, J. K. (1996). Base rates in category
learning.
- Week 4: Development.
- Karmiloff-Smith, A.
(1992). Nature, nurture and PDP: Preposterous Developmental
Postulates?
- Plunkett, K., Sinha, C., Moller, M. F. & Strandsby,
O. (1992). Symbol grounding or the emergence of symbols? Vocabulary
growth in children and a connectionist net.
- Week 5: Sequence Learning.
- Cleeremans, A. &
McClelland, J. L. (1991). Learning the structure of event sequences.
- Dienes, Z. (1992). Connectionist and memory-array models of
artificial grammar learning.
- Week 6: Similarity.
- Goldstone, R. L. (1994).
Similarity, interactive activation, and mapping.
- Goldstone, R. L.
& Medin, D. L. (1994). Time course of comparison.
- Week 7: Strength of Processing.
- Cohen, J,
D., Servan-Schreiber, D. & McClelland, J. L. (1992). A parallel
distributed processing approach to automaticity.
- Cohen, J. D. &
Servan-Schreiber, D. (1992). Context, cortex, and dopamine: A
connectionist approach to behavior and biology in schizophrenia.
- Week 8: Memory.
-
- Chappell, M. & Humphreys,
M. S. (1994). An auto-associative neural network for sparse
representations: Analysis and application to models of recognition and
cued recall.
- McClelland, J. L., McNaughton, B. L. & O'Reilly, R.
C. (1995). Why there are complementary learning systems in the
hippocampus and neocortex: Insights from the successes and failures of
connectionist models of learning and memory.
- Week 9: Neural Dynamics of Vision.
- Grossberg, S. (1994). 3-D vision and figure-ground separation by
visual cortex.
- Week 10: Word Recognition.
- Seidenberg, M.
S., Plaut, D. C., Petersen, A. S., McClelland, J. L. et al. (1994).
Nonword pronunciation and models of word recognition.
- Seidenberg,
M. S. (1993b). A connectionist modeling approach to word recognition
and dyslexia.
- Week 11: Lesions.
- Hinton, G. E. & Shallice,
T. (1991). Lesioning an attractor network: Investigations of acquired
dyslexia.
- Humphreys, G. W.; Freeman, T. A.; Muller, H. J. (1992).
Lesioning a connectionist model of visual search: Selective effects on
distractor grouping.
- Week 12: Reprise of Meta-Theory.
- Slezak,
Latimer et al. (1994) Symposium on Connectionist Models and
Psychology: The Rationale for Psychologists Using (Connectionist)
Models
- with reference to previously read papers by McCloskey,
M. (1991) and Seidenberg, M. S. (1993a).
- Weeks 13-15: Presentations of Student Projects.
Grading Method:
Grades will be assigned on the basis of (a) quality in leading
discussion of articles, (b) thoughtful participation in discussion
even when not formally leading discussion, and (c) quality of the
modeling project. For leading discussion, quality is judged by
accuracy of content and by clarity of presentation. For the modeling
project, quality is judged by many criteria, including but not limited
to the following: Appropriateness of the model mechanisms to the
phenomena being modeled; breadth and detail of empirical phenomena
addressed; clarity of presenting the model and the program code;
thoughtfulness in discussion of how the model helps explain the
cognitive phenomena, based on successes and/or failures of the model.
Computers:
There will be two main uses of computers in this course. First, we
will use the World Wide Web and a Newsgroup (ac.cogs.q750) for posting
announcements and discussion. Our class web page is
http://www.indiana.edu/~jkkteach/Q750/q750.html
^ ^
upper case lower case
Second, you will be using a computer to program your own connectionist
models from scratch. You may use any computer that is convenient for
you. Connectionist models are typically "computationally intensive,"
so you will want to locate a fast machine. If you have access in a
lab to a Unix machine or a fast PC or Mac, that would be fine.
Otherwise, there are various UCS Unix machines on which you can get
accounts. For information regarding how to get accounts, see the Web
page ... and get additional information from the UCS Knowledge Base at
http://kb.indiana.edu/ .
Required Readings:
We will read the articles listed below. A package of the collected
readings is available at the Indiana Memorial Union (IMU) Bookstore.
There might be changes to the readings as the course progresses.
- Chappell, M. & Humphreys, M. S. (1994).
- An auto-associative
neural network for sparse representations: Analysis and application to
models of recognition and cued recall.
Psychological
Review, 101, 103-128.
- Cleeremans, A. & McClelland, J. L. (1991).
- Learning the
structure of event sequences.
Journal of Experimental
Psychology: General, 120, 235-253.
- Cohen, J. D. & Servan-Schreiber, D. (1992).
- Context, cortex,
and dopamine: A connectionist approach to behavior and biology in
schizophrenia.
Psychological Review,
99, 45-77.
- Cohen, J, D., Servan-Schreiber, D. & McClelland, J. L. (1992).
- A parallel distributed processing approach to automaticity.
American Journal of Psychology, 105,
239-269.
- Dienes, Z. (1992).
- Connectionist and memory-array models of
artificial grammar learning.
Cognitive Science,
16, 41-79.
- Goldstone, R. L. (1994).
- Similarity, interactive activation,
and mapping.
Journal of Experimental Psychology: Learning,
Memory & Cognition, 20, 3-28.
- Goldstone, R. L. & Medin, D. L. (1994).
- Time course of
comparison
Journal of Experimental Psychology: Learning,
Memory & Cognition, 20, 29-50.
- Grossberg, S. (1994).
- 3-D vision and figure-ground separation
by visual cortex.
Perception & Psychophysics,
55, 48-120.
- Hinton, G. E. & Shallice, T. (1991).
- Lesioning an attractor
network: Investigations of acquired dyslexia.
Psychological
Review, 98, 74-95.
- Humphreys, G. W.; Freeman, T. A.; Muller, H. J. (1992).
- Lesioning a connectionist model of visual search: Selective
effects on distractor grouping.
Canadian Journal of
Psychology, 46, 417-460.
- Karmiloff-Smith, A. (1992)
- Nature, nurture and PDP:
Preposterous Developmental Postulates?
Connection
Science, 4, 253-269.
- Kruschke, J. K. (1992).
- ALCOVE: An exemplar-based
connectionist model of category learning.
Psychological
Review, 99, 22-44.
- Kruschke, J. K. (1996).
- Base rates in category learning.
Journal of Experimental Psychology: Learning, Memory and
Cognition, 22, 1-24.
- McClelland, J. L., McNaughton, B. L. & O'Reilly, R. C. (1995).
- Why there are complementary learning systems in the hippocampus
and neocortex: Insights from the successes and failures of
connectionist models of learning and memory.
Psychological
Review, 102, 419-457.
- McCloskey, M. (1991).
- Networks and theories: The place of
connectionism in cognitive science.
Psychological
Science, 2, 387-395.
- Plunkett, K., Sinha, C., Moller, M. F. & Strandsby, O. (1992).
- Symbol grounding or the emergence of symbols? Vocabulary growth
in children and a connectionist net.
Connection
Science, 4, 293-312.
- Seidenberg, M. S. (1993a).
- Connectionist models and cognitive
theory.
Psychological Science, 4,
228-235.
- Seidenberg, M. S. (1993b).
- A connectionist modeling approach
to word recognition and dyslexia.
Psychological Science,
4, 299-304.
- Seidenberg, M. S., Plaut, D. C., Petersen, A. S., McClelland, J.
L. et al. (1994).
- Nonword pronunciation and models of word
recognition.
Journal of Experimental Psychology: Human
Perception and Performance, 20, 1177-1196.
- Slezak, P., Latimer, C., Coltheart, M., Andrews, S., Oliphant, G.,
Bakker, P., Heath, R. A. & Watson, E. (1994).
- Symposium on
Connectionist Models and Psychology: The Rationale for Psychologists
Using (Connectionist) Models, January 1994, University of
Queensland, Australia.
http://www.cs.indiana.edu/Noetica/OpenForum.html
Disclaimer: This syllabus
is meant to be suggestive, not absolute. Any and all of the
information on this syllabus is subject to change at any time,
including grading policies, office hours, etc. Changes will be
announced in class.