Consolidating incentivization in distributed neural network training via decentralized autonomous organization

Abstract

Big data has reignited research interest in machine learning. Massive quantities of data are being generated regularly as a consequence of the development in the Internet, social networks, and online sensors. Particularly deep neural networks benefited greatly from this unprecedented data availability. Large models with millions of parameters are becoming common, and big data has been proved to be essential for their effective training. The scientific community has come up with a number of methods to create more accurate models, but most of these methods require high-performance infrastructure. There is also the issue of privacy, since anyone using leased processing power from a remote data center is putting their data in the hands of a third party. Studies on decentralized and non-binding methods among individuals with commodity hardware are scarce, though. Our work on LEARNAE seeks to respond to this challenge by creating a totally distributed and fault-tolerant framework of artificial neural network training. In our recent work, we demonstrated a method for incentivizing peers to participate to collaborative process, even if they are not interested in the neural network produced. For this, LEARNAE included a subsystem that rewards participants proportionately to their contribution using digital assets. In this article we add another important piece to the puzzle: A decentralized mechanism to mitigate the effect of bad actors, such as nodes that attempt to exploit LEARNAE’s network power without following the established rewarding rules. This is achieved by a novel reward mechanism, which takes into account the overall contribution of each node to the entire swarm. The network collaboratively builds a contribution profile for every participant, and the final rewards are dictated by these profiles. Taking for granted that the majority of the peers are benevolent, the whole process is tamper-proof, since it is implemented on blockchain and thus is protected by distributed consensus. All codebase is structured as a decentralized autonomous organization, which allows LEARNAE to embed new features like digital asset locking, proposal submitting, and voting.