High-Performance Computing Environment with Cooperation Between Supercomputer and Cloud

Toshihiro Kotani, Yusuke Gotoh

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

Due to the recent popularization of machine learning, such a deep reinforcement learning as AlphaGO has advanced to analyze large-scale data and is attracting great attention. In deep reinforcement learning, users evaluate many functions in large-scale computer environments, including supercomputer and cloud systems. Cloud services can provide computer resources based on the scale of the computer environment desired by users. On the other hand, in conventional large-scale computer environment that only consists of CPUs or GPUs, the processing time greatly increases according to the scale of the calculation processing. In this paper, we propose a high-performance computing environment for deep reinforcement learning that links supercomputer and cloud systems. Our proposed system can construct a high-performance computing environment based on the scale of the computing process by the cooperation of the supercomputing and cloud systems with short physical distance and short network distance. In our evaluation of deep reinforcement learning using our proposed system, we confirmed that computer resources can be effectively used by allocating suitable processing for the supercomputer and the cloud according to the usage situations of the CPU, the GPU, and the memory.

Original languageEnglish
Title of host publicationLecture Notes in Networks and Systems
PublisherSpringer
Pages433-443
Number of pages11
DOIs
Publication statusPublished - Jan 1 2020

Publication series

NameLecture Notes in Networks and Systems
Volume96
ISSN (Print)2367-3370
ISSN (Electronic)2367-3389

    Fingerprint

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Signal Processing
  • Computer Networks and Communications

Cite this

Kotani, T., & Gotoh, Y. (2020). High-Performance Computing Environment with Cooperation Between Supercomputer and Cloud. In Lecture Notes in Networks and Systems (pp. 433-443). (Lecture Notes in Networks and Systems; Vol. 96). Springer. https://doi.org/10.1007/978-3-030-33509-0_40