프로그램
최승진
IntellicodeDeep Learning 101: Optimization and Training
10:00 am - 12:00 pm 2:00 pm - 4:00 pmAbstract
This tutorial aims to provide a comprehensive introduction to optimization and training techniques for beginners in the field of deep learning. Divided into two parts, the tutorial begins with an exploration of fundamental concepts in gradient descent and its convergence behavior in convex optimization. Part I delves into the evolution of optimization algorithms, transitioning from traditional gradient descent to more sophisticated methods like stochastic gradient descent (SGD), momentum, AdaGrad, RMSProp, and Adam. In Part II, the tutorial shifts its focus to various training methods essential for building deep neural networks. It introduces dropout, a regularization technique that combats overfitting, and batch normalization, a method designed to stabilize and expedite training. The tutorial also explores transfer learning via fine-tuning, an approach to adapt pre-trained models to new tasks. Finally, it introduces the lottery ticket hypothesis, uncovering the secrets behind finding sparse and trainable neural networks with matching the test accuracy of the original network.
Bio
- 2022-현재 연구소장, Intellicode
- 2019-2021 CTO, BARO AI & 상임고문, BARO AI Academy
- 2001-2019 POSTECH 컴퓨터공학과 교수
- 2019-2021 정보과학회 인공지능소사이어티 회장
- 2018 삼성전자 종합기술원 자문교수
- 2017-2018 삼성리서치 AI센터 자문교수
- 2016-2017 신한카드 빅데이터센터 자문교수
- 2014-2016 정보과학회 머신러닝연구회 초대위원장
노영균
한양대학교/고등과학원
Methods for Information Estimation and its Application to Machine Learning
10:00 am - 12:00 pmAbstract
In this lecture, we review how we can take advantage of using information theory. As a theory, it provides referential and comprehensive concepts for practical applications. I will explain the theoretical motivations and justifications for many machine learning algorithms. The explanations will include various methods for feature selection, regularization and in particular, recent advances in the estimation of information contents for various applications.
Bio
- 2023-현재 한양대학교 인공지능학과 주임교수
- 2021-현재 한양대학교 컴퓨터소프트웨어학부 부교수
- 2021-현재 한국고등과학원 겸직교수
- 2022-현재 미국 메이오클리닉 로체스터 Research Collaborator Appointment
- 2018-현재 일본 RIKEN-AIP 방문과학자
- 2019-2020 한국고등과학원 Associate Member
- 2019-2020 한양대학교 컴퓨터소프트웨어학부 조교수
신현정
아주대학교Loss Functions and Performance Metrics
2:00 pm - 4:00 pmAbstract
The choice of loss functions and performance metrics is a fundamental aspect of machine learning and deep learning, playing a crucial role in model training and evaluation. This presentation will explore commonly used loss functions and performance metrics, evaluating their strengths and limitations. The goal is to offer insights into various loss and performance measurements, helping guide the selection of suitable ones for specific tasks.
Bio
- 2006-현재 아주대학교 산업공학과/인공지능학과/융합시스템공학과/과학기술정책학과/의료인공지능학과 교수
- 2006 서울대학교 의과대학 연구교수
- 2005-2006 Friedrich-Mierscher-Lab, Max-Planck-Institute(독일) 수석연구원
- 2004-2005 Max-Planck-Institute for Biological Cybernetics(독일) 연구원
- 2000-2005 서울대학교 공과대학 (산업공학/데이터마이닝) 공학박사
- 2007-현재 한국 BI 데이터마이닝 학회 등기이사
- 2013-현재 대한 의료정보학회 정보의학인증의 운영진 및 강사
- 2014-현재 서울대 의과대학 유전체/임상/정보분석 전문가과정 강사
- 2011-현재 국민건강보험 심사평가원 검사 심의위원회 부위원장
- 2017-현재 한국정보과학회 인공지능소사이어티 교육부 회장
- 2020-현재 한국연구재단 책임연구위원(CRB)
윤철희
KAISTAn Introduction to Mathematical Theory of Deep Learning
10:00 am - 12:00 pmAbstract
Over the last decade, deep learning has redefined the state-of-the-art in many application areas of machine learning and artificial intelligence. However, the success of deep learning has left many theoretical questions that still remain elusive today. This tutorial aims to provide a crash course on foundations and recent research results in the mathematical theory of deep learning. The following three central questions are covered. 1) Approximation: What functions can deep neural networks represent and approximate? 2) Optimization: Why can gradient-based optimization methods train deep neural networks to global optimality? 3) Generalization: Why can deep neural networks interpolate all training data points, and generalize to unseen data at the same time?
Bio
- 2022-현재 KAIST 김재철AI대학원 조교수
- 2016-2021 Ph.D. in Electrical Engineering & Computer Science, MIT
- 2014-2016 M.S. in Electrical Engineering, Stanford University
- 2007-2014 B.S. in Electrical Engineering, KAIST
김은솔
한양대학교Understanding Transformers and the recent trends
2:00 pm - 4:00 pmAbstract
In this lecture, we aim to understand the structure of the Transformer, which is widely used in many large-scale foundation models. Starting with Recurrent Neural Networks, we'll explain the background leading to the proposal of the attention mechanism and delve into the structure and learning principles of the Transformer. Particularly, we will explain the relationship between attention mechanisms and max-margin token selection through an understanding of the nonconvex optimization dynamics of the soft attention method. Additionally, we will introduce recent methods proposed to overcome the limitations of attention methods from a sequential learning perspective. Specifically, we will present the latest approaches (Mamba, Hyena, S4, etc.) proposed based on the state space machine method to address the long-range dependency problem.
Bio
- 2021-현재 한양대학교 컴퓨터소프트웨어학부 조교수
- 2018-2021 카카오브레인 연구원, 팀장
- 2018 서울대학교 컴퓨터공학부 공학박사 University