From Networks to Causal Models in Artificial Intelligence (updated September 1st, 2023)

Instructor: Ole Jakob Mengshoel

+++++ News flash +++++

Welcome to the Course TDT41!

Note: In 2023, the first meeting is Tuesday 5.9. (5th of September), 10:15am-12noon.  Workshops will be every second Tuesday in that time slot.   Workshops will be online.  While the official program will only start at 10:15am, I encourage students to join a few minutes early to ensure that their audio and video work properly. 

If you are registered for the course, you have been or will be added to a Teams team called Causal_2023_Fall. As much activity as possible will take place there. Please check that you are part of this team as soon as possible.  If you are NOT there in Causal_2023_Fall, as of Monday 4.9. (4th of September) at 3pm, please send an email to the instructor.  Further, if you may be interested in taking the course but have questions or similar, feel free to contact the instructor as soon as possible by email: ole.j.mengshoel at ntnu.no.

+++++

The collection, storage, and analysis of massive and diverse data is now a well-established fact of life.  This dramatic increase in data handling capability is due to technological advances in sensors (including the Internet of Things), smartphones, communication infrastructures, computing hardware, as well software frameworks and libraries. 

Data also plays a key role in artificial intelligence (AI) and machine learning (ML). However, the disciplines of AI and ML are as much – if not more – about models as about data.  Indeed, popular and well-established methods in AI and ML focus on network models such as neural networks and Bayesian networks (and probabilistic graphical models more broadly).  There are, for example, recent impressive advances in deep learning with neural networks. 

Another important recent development is the increased emphasis on causality in AI.   Clearly, concepts like causality or cause-and-effect are not new and have also been treated – at least to some extent – in disciplines such as econometrics, epidemiology, philosophy, psychology, and statistics. That being said, it appears that the AI perspective brings something new to the study of causality. In this course, students will acquire a deeper understanding of causality along with its application from an AI perspective.  Connections will be made to other models and methods within AI as well as to other disciplines. We will structure the course around the recent book “The Book of Why: The New Science of Cause and Effect” by Judea Pearl and Dana Mackenzie.  One of the book’s authors, the 2011 Turing Award winner Judea Pearl, is one of the key advocates of causality in AI.  In addition to “The Book of Why,” closely related articles and papers will be studied and discussed in the course.

Learning Goals

Knowledge: The students will obtain a deeper insight into research questions, concepts, and techniques related to the notion of cause-and-effect in AI, including relationships to other methods and models used in AI. Knowledge will be acquired through studies of theory as well as application examples.

Competence: The students should be able to put the subject into a broader societal, scientific, technical, and business context as a result of taking the course.

Learning Activities

This course will be workshop-based. The syllabus is structured according to “The Book of Why”, and that book is used in the course along with other materials including a collection of scientific articles and papers. Students will be required to read book chapters and any additional literature prior to the workshops. The workshop will be interactive sessions, with short presentations by students along with open discussions. 

An exception to the format described in the paragraph above is the very first workshop.  The first workshop, which will be more of a seminar, will be led by the instructor and will introduce the topics as well as the structure of the course.

Mandatory Activities

Participation in workshops (or seminars) as well participation in activities that the course scores will be based upon (see under Evaluation below) are mandatory activities. The first workshop is in late-August or early-September.  Please see below for dates. 

Note: In 2023, the first meeting is Tuesday 5.9. (5th of September), 10:15am-12noon.  Workshops will be every second Tuesday in that time slot.   Workshops will be online.  While the official program will only start at 10:15am, I encourage students to join a few minutes early to make ensure that their audio and video work properly. 

Evaluation

Workshops: 30% of final mark.  

• Students must participate in all six workshops that make up the course (see below for details)
• Students need to prepare for and actively participate in each workshop
• Each student needs to lead certain discussions, such as discussions of papers and other topics, in some of the workshops 
• When a student is not leading the discussion, it is still expected that the student actively participates

Take-home essay, test or oral exam: 70% of final mark.  The exact nature of this will depend on the number of students participating.  The date for this is at the end of the semester, in late-November or early-December, after the final workshop.  The date is tentatively set to Tuesday November 21, 2023.    

Syllabus

• Workshop 1, Tuesday 5.9., 10:15-12: Introduction “Mind over Data,” Chapter 3 “From Evidence to Causes: Reverend Bayes Meets Mr. Holmes,” plus structure and overview of course
• Workshop 2, Tuesday 19.9., 10:15-12: Chapter 1 “The Ladder of Causation” and Chapter 2 “From Buccaneers to Guinea Pigs:  The Genesis of Causal Inference” plus related literature
• Workshop 3, Tuesday 3.10., 10:15-12: Chapter 4: “Confounding and Deconfounding: Or, Slaying the Lurking Variable” and Chapter 6 “Paradoxes Galore!” plus related literature
• Workshop 4, Tuesday 17.10., 10:15-12: Chapter 5 “The Smoke-Filled Debate: Clearing the Air” and Chapter 7 “Beyond Adjustment: The Conquest of Mount Intervention” plus related literature
• Workshop 5, Tuesday 31.10., 10:15-12: Chapter 8 “Counterfactuals: Mining Worlds that Could Have Been” and Chapter 9 “Mediation: The Search for a Mechanism” (up to but excluding "Mediation in Linear Wonderland") plus related literature
• Workshop 6, Tuesday 14.11., 10:15-12: Chapter 9 “Mediation: The Search for a Mechanism” (from and including "Mediation in Linear Wonderland")  and Chapter 10 “Big Data, Artificial Intelligence, and the Big Questions” plus related literature

The chapter titles above refer to chapters in “The Book of Why: The New Science of Cause and Effect” by Judea Pearl and Dana Mackenzie.   The “related literature” will be detailed during and early on in the course, and will in part depend on the backgrounds, interests, and number of students taking the course.   Thus there may be some minor adjustments to the course, relative to the above schedule, for Fall 2023. 

Recommended Prerequisites

The course targets 4th year MS students in the CS department with a specialization in AI, but should be accessible to other students with a similar background.  Recommended prerequisites are: 

• University mathematics
• Fundamentals in AI, corresponding to NTNU courses TDT-4136 (Introduction to AI) and TDT-4171 (AI Methods)
• Programming and machine learning knowledge will be useful

If you're not familiar with the NTNU courses TDT-4136 and TDT-4171, the courses cover "many of" the chapters in Russel and Norvig's textbook "Artificial Intelligence: A Modern Apprach" up to and including Chapter 21 "Reinforcement Learning."  However, some of the more advanced topics from that textbook are skipped in TDT-4136 and TDT-4171. 

Communication

We will mainly use Microsoft Teams for communication and course materials.  If you don't have Teams access (yet) or have a more personal concern to bring up, please contact the instructor by email. 

If you are registered for the course, you will receive an email with further details about course participation by 16:00 on Monday right before the start of the course. If you do not receive such details at that time despite being registered for the course, or may be interested in taking the course but have not registered (yet), please contact the instructor as soon as possible by email: ole.j.mengshoel at ntnu.no.