XLCNN: A Transformer Model for Malware Detection

Abstract

The present research describes a Transformer-based neural network model that was developed in order to detect malicious software. We believe that the scientific community should take advantage of the contribution of Transformer models in the field of cybersecurity and go beyond the limits set by the classic natural language processing. For this purpose a new and more sophisticated algorithm was created based on the methodology used by the XLNet neural network which was proposed by the Google AI Brain Team. The proposed XLCNN model detects malicious code with a higher success rate than its predecessor. The method of detecting malware is based on the extraction and analysis of metadata contained in Windows executable files. From our carried out experiments, it was found that the size and architecture of the feed-forward neural network in combination with our proposed tokenizer, is one of the most important factors of XLCNN for classification problems. To justify the concept of XLCNN as an effective approach to detecting malware, the effectiveness and efficiency of the algorithm was measured for a finite number of epochs and compared to other Transformer models such as XLNet, BERT and Transformer-XL using exactly the same inputs. Using our proposed network has proven to be not only a reliable way for security researchers to detect malware, but also an effective and highly accurate method that offers high accuracy rate of 98.88%.