Please use this identifier to cite or link to this item:
Title: LEARNAE: Distributed and Resilient Deep Neural Network Training for Heterogeneous Peer to Peer Topologies
Authors: Nikolaidis, Spyridon
Refanidis, Ioannis
Type: Conference Paper
Subjects: FRASCATI::Natural sciences::Computer and information sciences
Keywords: Decentralized neural network training
Distributed asynchronous stochastic gradient decent
Model averaging
Peer-to-Peer topologies
Distributed Ledger Technology
Issue Date: 15-May-2019
Publisher: Springer Nature Switzerland AG 2019
Source: Proceedings of the 20th International Conference on Engineering Applications of Neural Networks (EANN-2019)
First Page: 286
Last Page: 298
Part of Series: Communications in Computer and Information Science book series (CCIS, volume 1000)
Part of Series: Communications in Computer and Information Science book series (CCIS, volume 1000)
Abstract: Learnae is a framework proposal for decentralized training of Deep Neural Networks (DNN). The main priority of Learnae is to maintain a fully distributed architecture, where no participant has any kind of coordinating role. This solid peer-to-peer concept covers all aspects: Underlying network protocols, data acquiring/distribution and model training. The result is a resilient DNN training system with no single point of failure. Learnae focuses on use cases where infrastructure heterogeneity and network unreliability result to an always changing environment of commodity-hardware nodes. In order to achieve this level of decentralization, new technologies had to be utilized. The main pillars of this implementation are the ongoing projects of IPFS and IOTA. IPFS is a platform for a purely decentralized filesystem, where each node contributes local data storage. IOTA aims to be the networking infrastructure of the upcoming IoT reality. On top of these, we propose a management algorithm for training a DNN model collaboratively, by optimal exchange of data and model weights, always using distribution-friendly gossip protocols.
ISBN: 978-3-030-20256-9
Electronic ISBN: 978-3-030-20257-6
Other Identifiers: 10.1007/978-3-030-20257-6_24
Appears in Collections:Department of Applied Informatics

Files in This Item:
File Description SizeFormat 
LEARNAE_FirstPaper.pdfCAMERA READY514,15 kBAdobe PDFView/Open

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.