This course will cover recent progress in AI/ML in applications, architectures, and
systems. The course will be self-contained as much as possible. If you are unsure about your background, please send the instructor an email inquiry with your background.

0-3 credits
ABCF Grading

No required course materials

Topics (subject to change):
• Overview of recent AI/ML applications
• ChatGPT overview
• Techniques behind ChatGPT: transformer
• Systems behind ChatGPT: GPU clusters, accelerators
• Algorithms behind ChatGPT: reinforcement learning
• Other applications based on ChatGPT
• Survey of competitive models vs transformer
• Survey of key systems development
• Survey of algorithmic innovations
• Sustainable AI and AI for sustainability
• Other topics: responsible AI, secure AI, edge AI (depends on time)

Learning outcomes:
• Understand the current trend in recent AI/ML applications
• Understand the basic concepts and popular applications of ChatGPT
• Understand the challenges and opportunities in the development of next generation AI/ML
applications
• Conduct a course project on related topics

 

*************

AMS 691.03:   Fundamentals of Reinforcement Learning

Deep understanding of reinforcement learning (RL) is essential for machine learning researchers, data scientists and practicing engineers working in areas such as artificial intelligence, machine learning, data/network science, natural language processing, computer vision, among others. RL has found its applications in our everyday life such as AlphaGo, AlphaFold, autonomous driving, healthcare, etc. This course will provide an introduction to the field of RL, and emphasize on hands-on experiences. Students are expected to become well versed in key ideas and techniques for RL through a combination of lectures, written and coding assignments. Students will advance their understanding and the field of RL through a course project. The topics that will be covered (time permitting) include but not limited to
• Markov Decision Processes (MDPs);
• Value Functions;
• Policy Iteration and Value Iteration;
• Monte Carlo Methods;
• Temporal Difference (TD) Learning;
• SARSA and Q-Learning;
• TD(𝜆);
• (Linear) Function Approximation;
• Policy Gradient Algorithms;
• Other topics (e.g., Multi-Agent RL, RL Theory; Deep RL).

Prerequisites
• Calculus and Linear Algebra (You should be comfortable taking derivatives and understanding matrix vector operations and notation.)
• Basic Probability and Statistics (You should know basics of probabilities, mean, standard deviation, etc)
• Python: All programming in the assignments and the project will be in Python (e.g., using numpy and Tensorflow). There will be roughly two programming problems in the assignments. You are expected to be efficient in Python or eager to learn it by yourself. This course will NOT teach programming.
• We will be formulating cost functions, taking derivatives and performing optimization with gradient descent.
• Have heard of Markov decision process and RL before in an AI or ML course, but we will quickly cover the basics.

ABCF grading
3 credits

No course materials required.