Kernel Embedding

10:00 am - 12:00 pm

Abstract

In classical kernel methods, kernel embedding can be viewed as a generalization of the individual data-point feature mapping as done in support vector machines and other kernel methods. Recently, it has meanwhile been extended to the embedding of distributions into infinite-dimensional feature spaces that can preserve all of the statistical features of arbitrary distributions, while allowing one to compare and manipulate distributions using Hilbert space operations such as inner products, distances, projections, linear transformations, and spectral analysis. We begin with a brief introduction to a reproducing kernel Hilbert space (RKHS) in which the key arsenal of kernel methods. Then positive definite kernels which forms the backbone of this research area are explained followed by introduction to various kernels proposed for various data types which are vectors, strings, graphs, images, time series, sequences, manifolds, and other structured objects. Then we gear up to simple embedding techniques such as kPCA, kCCA, kICA, etc. Next, we discuss the embedding of distributions which enables us to apply RKHS methods a wide range of applications such as sample testing and domain adaptation. Lastly, we discuss relationships between kernel embedding and other related areas.

Bio

• 2017-현재 한국정보과학회 기계학습연구회 회장
• 2017-현재 한국정보과학회 인공지능소사이어티 교육부 회장
• 2011-현재 국민건강보험 심사평가원 검사 심의위원회 부위원장
• 2014-현재 서울대 의과대학 유전체/임상/정보분석 전문가과정 강사
• 2013-현재 대한 의료정보학회 정보의학인증의 운영진 및 강사
• 2007-현재 한국 BI 데이터마이닝 학회 등기이사
• 2006-현재 아주대학교 산업공학과/인공지능학과/융합시스템공학과 교수
• 2000-2005 서울대학교 공과대학 (산업공학/데이터마이닝) 공학박사
• 2004-2005 Max-Planck-Institute for Biological Cybernetics(독일) 연구원
• 2005-2006 Friedrich-Mierscher-Lab, Max-Planck-Institute(독일) 수석연구원
• 2006 서울대학교 의과대학 연구교수

Random Projections and Fourier Features

2:00 pm - 4:00 pm

Abstract

Random projection is a probabilistic dimensionality reduction method where a high-dimensional data point is projected onto a column space of a random matrix to construct a low-dimensional representation. What is good about this is that the projection matrix does not have to be learned from data, unlike most of dimensionality reduction methods such as principal component analysis or Fisher’s discriminant analysis. In other words, random projection is a data-independent method. What is even awesome characteristics of random projection is that it guarantees an epsilon-distortion distance preserving embedding into a lower-dimensional space, which is well justified by the celebrating Johnson-Lindenstrauss (JL) lemma. It has been extensively studied in machine learning and theoretical computer science. In the first half of this tutorial, I will begin with the linear projection and will review the random projection and JL lemma. The second half of the talk will be on nonlinear random projection. Of particular interests are random Fourier features which scales up kernel machines. A successful example is the ‘random kitchen sinks’ which scales up the kernel regression as well as deep neural networks, via randomized feature map instead of optimizing feature representations. If time allowed, I will also mention a method of speeding up random kitchen sinks, which is known as ‘fastfood’.

Bio

• 2022-현재 연구소장, Intellicode
• 2019-2021 CTO, BARO AI & 상임고문, BARO AI Academy
• 2001-2019 POSTECH 컴퓨터공학과 교수
• 2019-2021 정보과학회 인공지능소사이어티 회장
• 2018 삼성전자 종합기술원 자문교수
• 2017-2018 삼성리서치 AI센터 자문교수
• 2016-2017 신한카드 빅데이터센터 자문교수
• 2014-2016 정보과학회 머신러닝연구회 초대위원장
• 1997-2001 충북대학교 전기전자공학부 조교수
• 1997 Frontier Researcher, RIKEN, Japan
• 1996 Ph.D. University of Notre Dame, Indiana, USA

Methods for Information Estimation and its Application to Machine Learning

10:00 am - 12:00 pm

Abstract

In this lecture, we review how we can take advantage of using information theory. As a theory, it provides referential and comprehensive concepts for practical applications. I will explain the theoretical motivations and justifications for many machine learning algorithms. The explanations will include various methods for feature selection, regularization and in particular, recent advances in the estimation of information contents for various applications.

Bio

• 2021-현재 한양대학교 컴퓨터소프트웨어학부 부교수
• 2021-현재 한국고등과학원 겸직교수
• 2022-현재 미국 메이오클리닉 로체스터 Research Collaborator Appointment
• 2018-현재 일본 RIKEN-AIP 방문과학자
• 2019-2020 한국고등과학원 Associate Member
• 2019-2020 한양대학교 컴퓨터소프트웨어학부 조교수

Graph Learning for Personalized Medicine and Drug Discovery

2:00 pm - 4:00 pm

Abstract

In this talk, we survey recent developments in graph-level learning. Traditional and widely investigated topics on graph learning are mostly on node-level and edge-/link prediction in networks, e.g., in social networks. Graph learning has become more important in scientific and medical domains, which is the topic of this tutorial lecture. Graph learning in scientific and medical domains is fundamentally different from traditional graph learning in two respects. First, in these domains, an individual, e.g., a patient and a chemical compound, is a graph where an individual is a whole graph and nodes in each graph are features of an individual such as atoms or genes. Note that in social networks, individuals are nodes. Second, graph learning in these domains is to discover distinguishing features of patient groups or chemical compound sets and classify them in terms of annotated characteristics such as cancer metastasis or toxicity, thus mining on a number of graphs. This tutorial is to survey recent developments in this topic. There are a number of research questions in this topic. First, graph construction is not straightforward and construction of graphs from data requires thoughtful strategies. Second, there are usually small number of samples in these domains, thus graph augmentations is another important research question. Third, in the medical domains, graph learning needs to require decoding of complex interactions of features, e.g., genes. Fourth, in the chemical domain, graphs vary significantly in terms of graph size, and graph mining in the chemical domain requires to optimize quite a number of tasks simultaneously, thus multi-task learning with graphs of varying sizes. Lastly, in the drug repositioning task, it is necessary to deal with heterogeneous networks where nodes are of completely different entities such as genes, diseases, and drug compounds. Mining associations in these heterogeneous networks require sophisticated computational strategies. In this tutorial, we will define research questions and survey recent developments in each of research questions with limitations of current technologies and future directions.

Bio - 김선

• 2022-현재 목암생명과학연구소 소장
• 2011-현재 서울대학교 컴퓨터공학부 교수
• 2011-2021 서울대학교 생물정보연구소 소장

Bio - 이상선

• 2020-현재 서울대학교 컴퓨터연구소 박사후연구원
• 2014-2020 서울대학교 컴퓨터공학부 박사
• 2010-2013 서울대학교 컴퓨터공학부 학사

Self-Supervised Learning

10:00 am - 12:00 pm

Abstract

본 강연에서는 대규모 모델의 학습의 기반이 되는 자기감독학습 (Self-supervised learning)의 기초와 응용에 대해 살펴본다. 우선 자기감독학습의 폭발적 성장을 이끈 초기의 기본 모델에 대해 다루고, 다형식(Multimodal data) 데이터 기반 모델과 상업적으로 큰 성공을 거둔 방법론에 대해 논의한다. 마지막으로 현재의 연구 경향과 미래의 발전 방향에 대해 함께 생각해본다.

Bio

• 2018.03-현재 CEO, 리플에이아이
• 2015.02-현재 부교수, 서울대학교 컴퓨터공학부
• 2015-2019.02 Disney Research, Postdoctoral Researcher
• 2013-2019 카네기멜론대학교 Computer Science Department 박사
• 2001-2006 연구원, 한국과학기술연구원 지능로봇연구센터

Deep Neural Net Approaches for Natural Language Processing

2:00 pm - 4:00 pm

Abstract

본 발표는 최근의 자연어처리 연구에 있어서 심층신경망을 이용한 여러 연구접근에 대해서 설명하고자 한다. 도입에서는 자연어처리의 여러 응용분야 및 다루어야 할 단위 task 즉 문장분류, 품사태깅, 개체명태깅, 구문분석, 동시참조처리, 자동요약등의 문제에 대해 설명하고 왜 이러한 문제들이 기술적으로 어려운지 애매성 및 여러 예시를 들어 설명한다. 이후 이러한 task를 처리하기 위한 심층학습의 기본원리와 심층기계학습을 이용한 시퀀스처리에 대해 설명한다. 이는 향후 자연어에 있어 단어열을 처리하는 기본 원리가 된다. 자연어처리에서 가장 다루기 힘든 어휘 단계의 애매성을 처리하기 위한 워드임베딩 기본 원리를 설명하고 이를 확장한 문맥 기반 워드 임베딩에 대해 기본 원리와 응용을 설명한다. 이러한 문맥 기반 워드 임베딩을 만들기 위한 가장 각광을 받는 모델로 트랜스포머 구조와 응용을 설명하고 이를 이용한 버트 및 GPT 사전학습 구조를 다룬다. 이후 기계번역등 자연어처리의 여러 시퀀스태깅의 기본 문제 및 대표적 응용들인 자연어 기계독해와 자연어 챗봇 대화 시스템에 대해 하나의 심층신경망 모델로 전체 학습 가능한 모델에 대해 설명하고 본 발표를 요약하고자 한다.

Bio

• 1991.09-현재 Professor, POSTECH
• 2015.10-2019.12 Senior Vice President, Samsung Research Samsung Electronics (Director of AI Center)
• 2013.09-2014.08 Technical Advisor, Samsung Electronics
• 2000.09-2001.08 Visiting researcher, CSLI, Stanford University
• 2000.03-2001.08 CTO, DiQuest