Botnets and machine learning: a story of “hide and seek”

One way to categorise botnets is by the technology they adopt for their command and control (C&C) mechanism. In terms of C&C, the architecture of a botnet can be either centralised (Figure 1) or decentralised (Figure 2). In the first category, the bots communicate with one or more servers using the client-server model. The first generation of centralised botnets used IRC channels to communicate with the C&C server. However, due to the single-point-of-failure nature of centralised architectures, the criminals started developing botnets that were based on peer-to-peer (P2P) communications, overcoming the problem of the previous generation of botnets. Indeed, P2P botnets, having the advantage of resilience and robustness, formed an even greater threat to organisations, but they also have two major drawbacks. To begin with, their maintenance is very difficult because of the complexity of their deployment and development and, secondly, since there is no longer a central C&C server, the herder might not have full control of the botnet any more.

The solution adopted by malware authors was to return to the centralised architecture model. However, they did not use the IRC protocol for the communications between the herder and the bots; the HTTP protocol was used instead. The advantage and strength of this solution is that the HTTP protocol is commonly used by legitimate, non-malicious web applications and services. So, the attackers are able to embed their traffic in non-malicious, legitimate HTTP traffic and hide C&C commands among normal network activities. This gives HTTP-based botnets their great advantage which is their ability to stay “under the radar” and perform their nefarious operations undetected.

Many researchers have dedicated their efforts to the study and analysis of HTTP botnets and finding accurate ways to detect them. A large number of researchers approach the problem by employing behaviour-based detection techniques, since the traditional signature-based systems are often easily bypassed by new generations of malware. More specifically, the analysis of network traffic and its characteristics (not necessarily the packets’ payload) can provide very insightful information as to whether a network flow or packet is benign or if it is part of a botnet’s C&C mechanism, even in cases where traffic is encrypted. Examples of traffic characteristics that could prove useful are the flow duration, the total number of packets exchanged in a flow, the length of the first packet in a flow and the median of payload bytes per packet.

Machine learning plays a key role in this approach, as behaviour-based botnet detection systems are usually built using a classification model that is trained on a dataset with specified features (set of network characteristics in our case). This classification model is able to identify efficiently and accurately malware-generated traffic when certain behaviour patterns are met. Apart from classification, more machine learning tools (e.g. feature extraction) could be used in order to make our system as accurate and fast as possible. In general, novel attacks deployed by newer or more advanced versions of existing malware can be prevented using this approach, as this detection system is not based on malware signatures.

Unsurprisingly, attackers started looking for ways and techniques that would allow them to overcome detection systems’ progress and bypass behaviour-based detection. Adversarial machine learning is an emerging technique that, among others, could target and evade security systems that utilise machine learning for dealing with malicious activities. Typically, its functionality is based on taking advantage of classifier’s weaknesses. For example, there might be a space of instances (i.e. flows/packets) that the classifier might not be able to describe well, so instances that belong to that space will be misclassified. Another kind of attack that can be performed against systems based on machine learning is when adversaries attempt to attack the training phase of classification; that is, they try to inject adversarial training data to the classification model. This eventually leads to a model that labels malicious instances incorrectly as non-malicious, thus increasing the number of false negatives and leaving the system vulnerable.

Obfuscation techniques used by attackers should also be taken into consideration when implementing detection systems based on behaviour. More specifically, attackers might attempt to convert the value of certain attributes and characteristics of network traffic flows that are indicative of malicious activity, into values that are typical and normal for non-malicious flows, thereby evading security measures. Therefore, if the obfuscated features are used by the classification system, the malicious flows will have a greater chance of bypassing the detection system.

To conclude, a best practice for organisations in terms of security is to always be up-to-date with the current trends in the cyber threat landscape as it is a field that changes constantly and radically. Machine learning has proven to be an extremely powerful ally in the battle against certain kinds of malware and it currently seems to be the ideal method for keeping up with the evolution of threats, both in terms of detection accuracy and efficiency. Of course, behaviour-based systems have the drawback of false positives, but the benefits of this approach are more than enough to ignore that disadvantage. However, when employing behaviour-based systems, organisations should not overlook the complexity and difficulty of building such systems and the caveats that come with this solution, some of them mentioned above (i.e. adversarial machine learning, obfuscation of features). Technical expertise, along with patience and the ability to gain insight, are probably the most important values professionals and organisations should be equipped with, in order to successfully deploy and manage such complex systems that will help them adjust to today’s threat landscape and continue operating in a secure environment.