Laboratory for Brain and Machine Intelligence

Laboratory for Brain and Machine Intelligence @ KAIST

Laboratory for brain and machine intelligence, KAIST

Advanced Intelligence (2019 Spring BiS400C)


[Important notice] For those who previously attended my other graduate courses, it is not recommended to sign up for this one. The previous course, Frontiers in brain-inspired AI (BiS800), is now split into the two courses :

  • Advanced Intelligence (undergraduate course) covers the first half, and

  • Brain-inspired AI (graduate course) focuses on the rest: reinforcement learning theory, algorithms, and neuroscience.


Summary: A blend of machine learning with neuroscience

The aim of this course is to understand how machine learning algorithms and brains learn to make sense of the world, which is formulated as the inverse problem:

Ax=y.

The course provides a general introduction to linear models, neural networks, and deep learning, neuroscience of deep learning, and basics of reinforcement learning. Students are expected not only to understand how algorithms and brains work, but, more importantly, why they work.

Lecture Room: #207 (E16-1 YBS Bldg.)
Time: Monday and Wednesday 13:00-14:30
Instructor: Sang Wan Lee (sangwan@kaist.ac.kr, #516 E16-1)
Office Hours: Tuesday and Thursday 11:00 – 11:55
TAs: TBA
Credit:  3 units (3:0:0).
Prerequisite: Linear algebra and probability (or equivalent)
Assessment: Attendance (40%), mid-term exam (40%), term projects (20%)

Textbook: Lecture materials (70%) + a few chapters of the followings (30%):
[T1] John Shawe-Taylor,‎ Nello Cristianini, Kernel Methods for Pattern Analysis.
[T2] I. Goodfellow, et al. Deep learning, MIT press.


Lecture slides, announcements, and etc.

KAIST KLMS


Lecture schedule

  • Introduction

  • Matrix algebra essentials : Ax=b
    Singular value decomposition
    Quadratic forms
    Inequalities and maximization lemma

  • Linear models
    Least squares estimators
    Gradient-based optimization
    Component analysis

  • Neural networks (I) : basics
    Feedforward neural networks
    Statistical learning theory
    Regularization
    Optimal neural networks (support vector machines)

  • Neural networks (II) : kernel
    Kernel methods
    Optimal neural networks (kernel support vector machines)

  • Neural networks (III) : hands-on session

  • — Midterm exam —

  • Deep learning (I) : algorithms
    Boltzmann machines
    Convolutional neural networks
    Training, sparsity, information bottleneck theory

  • Deep learning (II) : advanced topics
    Generative adversarial networks
    Recurrent neural networks

  • Deep learning (III) : hands-on session

  • Neuroscience of deep learning (I) : basics
    Bottom-up cortical information processing
    Top-down cortical information processing

  • Neuroscience of deep learning (II) : advanced topics
    Selectivity and dimensionality of cortical activity
    Optimization in cortical networks

  • Reinforcement learning : basics*
    Markov decision process
    Model-free prediction and control
    (*This is a prerequisite for the graduate course, Brain-inspired AI (BCE772).)

  • Reinforcement learning : hands-on session

  • — Final exam week (Term project report due) —