Please use this identifier to cite or link to this item:
https://ruomo.lib.uom.gr/handle/7000/720
Title: | LEARNAE: Distributed and Resilient Deep Neural Network Training for Heterogeneous Peer to Peer Topologies |
Authors: | Nikolaidis, Spyridon Refanidis, Ioannis |
Type: | Conference Paper |
Subjects: | FRASCATI::Natural sciences::Computer and information sciences |
Keywords: | Decentralized neural network training Distributed asynchronous stochastic gradient decent Model averaging Peer-to-Peer topologies Distributed Ledger Technology IPFS IOTA |
Issue Date: | 15-May-2019 |
Publisher: | Springer Nature Switzerland AG 2019 |
Source: | Proceedings of the 20th International Conference on Engineering Applications of Neural Networks (EANN-2019) |
First Page: | 286 |
Last Page: | 298 |
Part of Series: | Communications in Computer and Information Science book series (CCIS, volume 1000) |
Part of Series: | Communications in Computer and Information Science book series (CCIS, volume 1000) |
Abstract: | Learnae is a framework proposal for decentralized training of Deep Neural Networks (DNN). The main priority of Learnae is to maintain a fully distributed architecture, where no participant has any kind of coordinating role. This solid peer-to-peer concept covers all aspects: Underlying network protocols, data acquiring/distribution and model training. The result is a resilient DNN training system with no single point of failure. Learnae focuses on use cases where infrastructure heterogeneity and network unreliability result to an always changing environment of commodity-hardware nodes. In order to achieve this level of decentralization, new technologies had to be utilized. The main pillars of this implementation are the ongoing projects of IPFS and IOTA. IPFS is a platform for a purely decentralized filesystem, where each node contributes local data storage. IOTA aims to be the networking infrastructure of the upcoming IoT reality. On top of these, we propose a management algorithm for training a DNN model collaboratively, by optimal exchange of data and model weights, always using distribution-friendly gossip protocols. |
URI: | https://doi.org/10.1007/978-3-030-20257-6_24 https://ruomo.lib.uom.gr/handle/7000/720 |
ISBN: | 978-3-030-20256-9 |
Electronic ISBN: | 978-3-030-20257-6 |
Other Identifiers: | 10.1007/978-3-030-20257-6_24 |
Appears in Collections: | Department of Applied Informatics |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
LEARNAE_FirstPaper.pdf | CAMERA READY | 514,15 kB | Adobe PDF | View/Open |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.