High-Performance Computing Environment with Cooperation Between Supercomputer and Cloud

Toshihiro Kotani, Yusuke Gotoh

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

Due to the recent popularization of machine learning, such a deep reinforcement learning as AlphaGO has advanced to analyze large-scale data and is attracting great attention. In deep reinforcement learning, users evaluate many functions in large-scale computer environments, including supercomputer and cloud systems. Cloud services can provide computer resources based on the scale of the computer environment desired by users. On the other hand, in conventional large-scale computer environment that only consists of CPUs or GPUs, the processing time greatly increases according to the scale of the calculation processing. In this paper, we propose a high-performance computing environment for deep reinforcement learning that links supercomputer and cloud systems. Our proposed system can construct a high-performance computing environment based on the scale of the computing process by the cooperation of the supercomputing and cloud systems with short physical distance and short network distance. In our evaluation of deep reinforcement learning using our proposed system, we confirmed that computer resources can be effectively used by allocating suitable processing for the supercomputer and the cloud according to the usage situations of the CPU, the GPU, and the memory.

Original languageEnglish
Title of host publicationLecture Notes in Networks and Systems
PublisherSpringer
Pages433-443
Number of pages11
DOIs
Publication statusPublished - Jan 1 2020

Publication series

NameLecture Notes in Networks and Systems
Volume96
ISSN (Print)2367-3370
ISSN (Electronic)2367-3389

Fingerprint

Supercomputers
Reinforcement learning
Program processors
Processing
Learning systems
Computer systems
Data storage equipment
Graphics processing unit

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Signal Processing
  • Computer Networks and Communications

Cite this

Kotani, T., & Gotoh, Y. (2020). High-Performance Computing Environment with Cooperation Between Supercomputer and Cloud. In Lecture Notes in Networks and Systems (pp. 433-443). (Lecture Notes in Networks and Systems; Vol. 96). Springer. https://doi.org/10.1007/978-3-030-33509-0_40

High-Performance Computing Environment with Cooperation Between Supercomputer and Cloud. / Kotani, Toshihiro; Gotoh, Yusuke.

Lecture Notes in Networks and Systems. Springer, 2020. p. 433-443 (Lecture Notes in Networks and Systems; Vol. 96).

Research output: Chapter in Book/Report/Conference proceedingChapter

Kotani, T & Gotoh, Y 2020, High-Performance Computing Environment with Cooperation Between Supercomputer and Cloud. in Lecture Notes in Networks and Systems. Lecture Notes in Networks and Systems, vol. 96, Springer, pp. 433-443. https://doi.org/10.1007/978-3-030-33509-0_40
Kotani T, Gotoh Y. High-Performance Computing Environment with Cooperation Between Supercomputer and Cloud. In Lecture Notes in Networks and Systems. Springer. 2020. p. 433-443. (Lecture Notes in Networks and Systems). https://doi.org/10.1007/978-3-030-33509-0_40
Kotani, Toshihiro ; Gotoh, Yusuke. / High-Performance Computing Environment with Cooperation Between Supercomputer and Cloud. Lecture Notes in Networks and Systems. Springer, 2020. pp. 433-443 (Lecture Notes in Networks and Systems).
@inbook{fc1701468222424e871363eabd5b87cf,
title = "High-Performance Computing Environment with Cooperation Between Supercomputer and Cloud",
abstract = "Due to the recent popularization of machine learning, such a deep reinforcement learning as AlphaGO has advanced to analyze large-scale data and is attracting great attention. In deep reinforcement learning, users evaluate many functions in large-scale computer environments, including supercomputer and cloud systems. Cloud services can provide computer resources based on the scale of the computer environment desired by users. On the other hand, in conventional large-scale computer environment that only consists of CPUs or GPUs, the processing time greatly increases according to the scale of the calculation processing. In this paper, we propose a high-performance computing environment for deep reinforcement learning that links supercomputer and cloud systems. Our proposed system can construct a high-performance computing environment based on the scale of the computing process by the cooperation of the supercomputing and cloud systems with short physical distance and short network distance. In our evaluation of deep reinforcement learning using our proposed system, we confirmed that computer resources can be effectively used by allocating suitable processing for the supercomputer and the cloud according to the usage situations of the CPU, the GPU, and the memory.",
author = "Toshihiro Kotani and Yusuke Gotoh",
year = "2020",
month = "1",
day = "1",
doi = "10.1007/978-3-030-33509-0_40",
language = "English",
series = "Lecture Notes in Networks and Systems",
publisher = "Springer",
pages = "433--443",
booktitle = "Lecture Notes in Networks and Systems",

}

TY - CHAP

T1 - High-Performance Computing Environment with Cooperation Between Supercomputer and Cloud

AU - Kotani, Toshihiro

AU - Gotoh, Yusuke

PY - 2020/1/1

Y1 - 2020/1/1

N2 - Due to the recent popularization of machine learning, such a deep reinforcement learning as AlphaGO has advanced to analyze large-scale data and is attracting great attention. In deep reinforcement learning, users evaluate many functions in large-scale computer environments, including supercomputer and cloud systems. Cloud services can provide computer resources based on the scale of the computer environment desired by users. On the other hand, in conventional large-scale computer environment that only consists of CPUs or GPUs, the processing time greatly increases according to the scale of the calculation processing. In this paper, we propose a high-performance computing environment for deep reinforcement learning that links supercomputer and cloud systems. Our proposed system can construct a high-performance computing environment based on the scale of the computing process by the cooperation of the supercomputing and cloud systems with short physical distance and short network distance. In our evaluation of deep reinforcement learning using our proposed system, we confirmed that computer resources can be effectively used by allocating suitable processing for the supercomputer and the cloud according to the usage situations of the CPU, the GPU, and the memory.

AB - Due to the recent popularization of machine learning, such a deep reinforcement learning as AlphaGO has advanced to analyze large-scale data and is attracting great attention. In deep reinforcement learning, users evaluate many functions in large-scale computer environments, including supercomputer and cloud systems. Cloud services can provide computer resources based on the scale of the computer environment desired by users. On the other hand, in conventional large-scale computer environment that only consists of CPUs or GPUs, the processing time greatly increases according to the scale of the calculation processing. In this paper, we propose a high-performance computing environment for deep reinforcement learning that links supercomputer and cloud systems. Our proposed system can construct a high-performance computing environment based on the scale of the computing process by the cooperation of the supercomputing and cloud systems with short physical distance and short network distance. In our evaluation of deep reinforcement learning using our proposed system, we confirmed that computer resources can be effectively used by allocating suitable processing for the supercomputer and the cloud according to the usage situations of the CPU, the GPU, and the memory.

UR - http://www.scopus.com/inward/record.url?scp=85074686601&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85074686601&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-33509-0_40

DO - 10.1007/978-3-030-33509-0_40

M3 - Chapter

AN - SCOPUS:85074686601

T3 - Lecture Notes in Networks and Systems

SP - 433

EP - 443

BT - Lecture Notes in Networks and Systems

PB - Springer

ER -