Complex network attack detection method based on cross-host abnormal behavior recognition

Description

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of international PCT application serial no. PCT/CN2023/099376, filed on Jun. 9, 2023, which claims priority benefit of China patent application No. 202310423444.5 filed on Apr. 20, 2023. The entirety of each of the above-mentioned patent applications is hereby incorporated by reference herein and made a part of this specification.

TECHNICAL FIELD

The present disclosure relates to the technical field of network security and deep learning, and particularly relates to a complex network attack detection method based on cross-host abnormal behavior recognition.

BACKGROUND

Complex network attacks are becoming increasingly prominent in an era of modern networks. The complex network attacks are persistent network attacks made through various means. An attacker first uses various techniques to penetrate a host in a target network, and then achieves lateral movement from the host to another host for internal reconnaissance and data theft. These features of the complex network attacks make it difficult to deal with them by use of traditional single-point network attack methods.

According to recent studies, a provenance graph is a most effective tool for detecting complex network attacks. The provenance graph is an association graph constructed based on kernel log data of an operating system, its nodes represent system entities (such as processes and files), and its edges represent system events (such as process reading files). The provenance graph is capable to capture control flow and data flow between the system entities well, and better reflects the contextual correlation and causal relationship between the system events, so that a better effect is achieved in detecting long-term, multi-step complex network attacks. In a further aspect, with rapid development of deep learning, a deep learning technology, particularly a graph neural network technology capable of effectively processing provenance graphs, is gradually applied to detecting complex network attacks.

However, there still exist deficiencies in the existing complex network attack detection technology that integrates provenance graphs and deep learning: first, the prior art still focuses on detecting single attack events, and cannot perform correlation analysis on events of attacks on a plurality of hosts. Second, due to lack of correlation analysis of attack events in the prior art, the false alarm rate is very high. Third, the prior art relies on a large number of labeled training samples.

SUMMARY

To overcome defects in the prior art, the present disclosure provides a complex network attack detection method based on cross-host abnormal behavior recognition.

The technical solution of the present disclosure is as follows:

• • a complex network attack detection method based on cross-host abnormal behavior recognition, including the following steps:
  • 1) intra-host abnormal behavior detection: firstly collecting kernel log data of a host to construct an intra-host association graph, then employing a graph convolution network (GCN) to extract features, and finally using an autoencoder to detect abnormal nodes in the intra-host association graph;
  • 2) inter-host abnormal behavior detection: firstly constructing an inter-host interaction graph by using network flow log data, then training a prediction model based on a sequence of the inter-host interaction graph, and finally detecting abnormal edges in the inter-host interaction graph according to prediction differences; and
  • 3) cross-host network attack detection: firstly constructing a cross-host anomaly association graph according to intra-host and inter-host anomaly detection results, then propagating anomaly scores by employing a PageRank algorithm, and finally confirming an attacked host based on the anomaly scores.

Further, in the step 1), detailed steps of the intra-host abnormal behavior detection are as follows:

• • 1.1) constructing a benign intra-host association graph: for each host H_k in a target network, using an operating system kernel log collection tool to collect system log data generated during normal operation of the host, and constructing an association graph nIAG_k, where nodes of nIAG_k represent system entities, and edges thereof represent interaction events that occur between the system entities;
  • 1-2) extracting initial features of the nodes: an initial feature vector x_i of each node v_i in nIAG_k is [α_i1, α_i2, . . . , α_iN], where α_ij is the number of edges of a j^th type among a set of edges formed by connecting the node v_i to its one-hop neighbor nodes in nIAG_k;
  • 1-3) extracting semantic features of the nodes: using a graph convolutional neural network to process nIAG_k, and updating the feature vector of each node v_i in nIAG_k, to obtain a semantic feature vector y_i of each node v_i;
  • 1-4) autoencoder training: first, collecting semantic feature vectors of all “process” nodes in nIAG_k of all hosts; then, inputting these semantic feature vectors into an autoencoder for training; two multi-layer perceptrons of the autoencoder serve as an encoder and a decoder respectively, where the encoder encodes the inputted semantic feature vector y_i into a low-dimensional implicit vector, and the decoder decodes the implicit vector into a reconstruction vector z_i with the same dimension as the inputted semantic feature vector; an objective of the training is to minimize differences between the inputted semantic feature vector y_i and the outputted reconstruction vector z_i;
  • 1-5) intra-host anomaly detection: first, collecting real-time system log data of each host H_k in the target network and constructing an association graph rIAG_k; then, extracting the semantic feature vector of each node in rIAG_k according to the methods in the steps 1-2) and 1-3); finally, inputting the semantic feature vector y_i of each “process” node v_i in rIAG_k into the autoencoder trained in the step 1-4), to obtain the outputted reconstruction vector z_i; when a difference between y_i and z_i is greater than a specified threshold, the node v_i is deemed to be an abnormal node; and
  • 1-6) intra-host anomaly scoring: first, calculating an anomaly score of each abnormal node v_i according to the formula (1), and then calculating the anomaly score of the host H_k according to the formula (2), where B(H_k) represents a set of abnormal nodes of the host H_k.

s ⁡ ( v i ) =  y i - z i  F 2 ( 1 ) s ⁡ ( H k ) = max v i ∈ B ⁡ ( H k ) s ⁡ ( v i ) ( 2 )

Further, in the step 2), detailed steps of the inter-host abnormal behavior detection are as follows:

• • 2-1) constructing a sequence of inter-host interaction graphs: using a network traffic capture tool to collect log data of network interactions between all hosts in the target network during normal operation of the hosts; a node of an inter-host interaction graph IRG_t at each discrete time slot t is a host, and an edge represents an interaction between different hosts at the time slots t-1 and t; a sequence of inter-host interaction graphs at continuous time slots can be defined as IRGS=<IRG₁, IRG₂, . . . , IRG_T>;
  • 2-2) constructing a training set: using a sliding window with a window size of w and a step size of 1 to segment a historical sequence of an inter-host interaction graph to obtain a large number of training samples and further form a training set SS; each training sample S_t, in the SS is defined as (D_t, L_t), where D_t=<IRG_t-w, IRG_t-w+1, . . . , IRG_t-1> is a data part of S_t, and L_t=IRG_t is a label part of S_t;
  • 2-3) training an inter-host interaction graph prediction model: training a prediction model based on the training set SS, and predicting the next inter-host interaction graph according to w continuous inter-host interaction graphs; it is assumed that that S_t, is an inputted sample, a structure of the prediction model consists of the following four layers:
  • an input layer: it is assumed that the system's network structure (i.e., the hosts in the network) remains unchanged, so the node sets of different inter-host interaction graphs are the same, and therefore only an adjacent matrix of an edge set of an inter-host interaction graph is inputted, that is, <IRA_t-w, IRA_t-w+1, . . . , IRA_t-1> is inputted, where IRA_k is an adjacent matrix of an edge set of IRG_k;
  • a convolutional layer: each IRA_k is inputted into a convolutional neural network to generate a feature matrix CM_k;
  • a circulation layer: t continuous feature matrices <CM_t-w, CM_t-w+1, . . . , CM_t-1> are inputted into an LSTM to generate a feature matrix RM;
  • an output layer: RM is inputted into a fully connected neural network to generate an adjacent matrix pIRA_t at a predicted time slot t; and
  • 2-4) inter-host anomaly detection: first, following the step 2-1) to generate w inter-host interaction graphs D_t=<IRG_t-w, IRG_t-w+1, . . . , IRG_t-1> before the current time slot t; then, inputting D_t into the prediction model trained in the step 2-3) to predict and obtain an adjacent matrix pIRA_t, at the time slot t; finally, comparing pIRA_t, with a real adjacent matrix IRA_t, at the time slot t; when pIRA_t[i][j]=0, while IRA_t[i][j]=1, which means that the model predicts that there should be no interaction between hosts H_i and H_j at the time slot t, but in fact there is an interaction between them, the interaction between the hosts H_i and H_j at the time slot t is deemed to be abnormal.

Further, in the step 3), detailed steps of cross-host complex network attack detection are as follows:

• • 3-1) constructing a cross-host anomaly association graph: constructing a cross-host anomaly association graph as follows according to a given time slot t, an intra-host data collection time span w, and a set (HS={H₁, H₂, . . . , H_M}) of hosts in the target network:
  • 3-1-1) following the step 1-1) to collect system log data of each host in HS generated between time slots t-w and t, and constructing a real-time intra-host association graph rIAG_k for each host H_k;
  • 3-1-2) based on the real-time intra-host association graph rIAG_k of each host H_k, detecting abnormal nodes in rIAG_k according to the step 1-5), and performing anomaly scoring for each host H_k according to the step 1-6);
  • 3-1-3) following the step 2-1) to collect the inter-host interaction graphs generated between the time slots t-w and t, and forming a sequence D_t=<IRG_t-w, IRG_t-w+1, . . . , IRG_t-1> of the inter-host interaction graphs;
  • 3-1-4) following the step 2-4) to detect any inter-host abnormal interaction at the time slot t; and
  • 3-1-5) constructing a cross-host anomaly association graph CG, where each node p_k represents a host H_k; when an interaction between the hosts H_i and H_j at the time slot t is abnormal, there exists an edge between nodes p_i and p_j;
  • 3-2) propagating the anomaly scores: using the PageRank algorithm to iteratively update a weight of each node in CG, with detailed steps as follows:
  • 3-2-1) setting an initial weight PR(p_k) of each node p_k in CG as an anomaly score s_k of H_k;
  • 3-2-2) updating the weight of each node p_k in CG according to the formula (3); in the formula (3), d is a damping coefficient, N_CG is the number of nodes in CG, I(p_k) is a set of nodes corresponding to all incoming edges of the node p_k, and L(p_i) is an outgoing degree of the node p_i;

PR ⁡ ( p k ) = 1 - d N C ⁢ G + d ( ∑ p i ∈ I ⁡ ( p k ) PR ⁡ ( p i ) L ⁡ ( p i ) ) ( 3 )

• • 3-2-3) when weight variation of any node after this iteration is less than a specified threshold, or the number of iterations reaches a specified upper limit, stopping the iteration, otherwise jumping to the step 3-2-2) to restart the iteration; and
  • 3-3) detecting an attacked host: confirming a host represented by a node in CG whose weight is higher than the specified threshold as the attacked host, and confirming a pair of attacked hosts connected by an edge as lateral movement targets of an attacker.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow chart of a method of the present disclosure.

FIG. 2 is a diagram of an example in which an initial feature of a node in an intra-host association graph of the present disclosure is extracted.

FIG. 3 is an example diagram of an inter-host interaction graph sequence of the present disclosure.

FIG. 4 is a network structure diagram of an inter-host interaction graph prediction model of the present disclosure.

FIG. 5 is a benign intra-host association graph of Embodiment 1 of the present disclosure.

FIG. 6 is a diagram of an inter-host interaction graph sequence of Embodiment 1 of the present disclosure.

FIG. 7 is a real-time detection input diagram of Embodiment 1 of the present disclosure.

FIG. 8 is a cross-host anomaly association diagram of Embodiment 1 of the present disclosure.

DETAILED DESCRIPTION OF THE EMBODIMENTS

The present disclosure will be further described below with reference to the accompanying drawings.

With reference to FIG. 1, a complex network attack detection method based on cross-host abnormal behavior recognition, includes the following steps:

• • 1) intra-host abnormal behavior detection: firstly collecting kernel log data of a host to construct an intra-host association graph, then employing a graph convolution network (GCN) to extract features, and finally using an autoencoder to detect abnormal nodes in the intra-host association graph;
  • detailed steps of the intra-host abnormal behavior detection are as follows:
  • 1-1) constructing a benign intra-host association graph: for each host H_k in a target network, using an operating system kernel log collection tool (such as Event Tracing for Windows (ETW) and Auditd of the Linux Auditing System) to collect system log data generated during normal operation of the host, and constructing an association graph nIAG_k (known as a normal intra-host association graph), where nodes of nIAG_k represent system entities (such as processes and files), and edges thereof represent interaction events (such as process reading files) that occur between the system entities;
  • 1-2) extracting initial features of the nodes: an initial feature vector x_i of each node v_i in nIAG_k is [α_i1, α_i2, . . . , α_iN], where α_ij is the number of edges of a j^th type among a set of edges formed by connecting the node v_i to its one-hop neighbor nodes in nIAG_k (a total of N types of edges); FIG. 2 shows an example, in which it is assumed that when there are 8 types of edges in total, and there are 6 edges formed by connecting a central node v₀ with its one-hop neighbor nodes (the number on an edge is a type number), the initial feature vector of v₀ is [2, 1, 1, 1, 0, 0, 0, 1];
  • 1-3) extracting semantic features of the nodes: using a graph convolutional neural network to process nIAG_k, and updating the feature vector of each node v_i in nIAG_k, to obtain a semantic feature vector y_i of each node v_i;
  • 1-4) autoencoder training: first, collecting semantic feature vectors of all “process” nodes in nIAG_k of all hosts; then, inputting these semantic feature vectors into an autoencoder for training; two multi-layer perceptrons of the autoencoder serve as an encoder and a decoder respectively, where the encoder encodes the inputted semantic feature vector y_i into a low-dimensional implicit vector, and the decoder decodes the implicit vector into a reconstruction vector z_i with the same dimension as the inputted semantic feature vector; an objective of the training is to minimize differences between the inputted semantic feature vector y_i and the outputted reconstruction vector z_i;
  • 1-5) intra-host anomaly detection: first, collecting real-time system log data of each host H_k in the target network and constructing an association graph rIAG_k (known as a real-time intra-host association graph); then, extracting the semantic feature vector of each node in rIAG_k according to the methods in the steps 1-2) and 1-3); finally, inputting the semantic feature vector y_i of each “process” node v_i in rIAG_k into the autoencoder trained in the step 1-4), to obtain the outputted reconstruction vector z_i; when a difference between y; and z_i is greater than a specified threshold, the node v; is deemed to be an abnormal node; and
  • 1-6) intra-host anomaly scoring: first, calculating an anomaly score of each abnormal node v_i according to the formula (1), and then calculating the anomaly score of the host H_k according to the formula (2), where B(H_k) represents a set of abnormal nodes of the host H_k;

s ⁡ ( v i ) =  y i - z i  F 2 ( 1 ) s ⁡ ( H k ) = max v i ∈ B ⁡ ( H k ) s ⁡ ( v i ) ( 2 )

• • 2) inter-host abnormal behavior detection: firstly constructing an inter-host interaction graph by using network flow log data, then training a prediction model based on a sequence of the inter-host interaction graph, and finally detecting abnormal edges in the inter-host interaction graph according to prediction differences; and
  • detailed steps of the inter-host abnormal behavior detection are as follows:
  • 2-1) constructing a sequence of inter-host interaction graphs: using a network traffic capture tool to collect log data of network interactions between all hosts in the target network during normal operation of the hosts; a node of an inter-host interaction graph IRG_t at each discrete time slot t is a host, and an edge represents an interaction between different hosts at the time slots t-1 and t; a sequence of inter-host interaction graphs at continuous time slots can be defined as IRGS=<IRG₁, IRG₂, . . . , IRG_T>; FIG. 3 shows an example, in which at a time slot t₁, a host of a user A accesses a host as a domain controller for authentication, and at a time slot t₂, the host of the user A accesses a host as a file server; it can be seen that node sets of the inter-host interaction graphs at different time slots are the same, but edge sets are different;
  • 2-2) constructing a training set: using a sliding window with a window size of w and a step size of 1 to segment a historical sequence of an inter-host interaction graph to obtain a large number of training samples and further form a training set SS; each training sample S_t in the SS is defined as (D_t, L_t), where D_t=<IRG_t-w, IRG_t-w+1, . . . , IRG_t-1> is a data part of S_t, and L_t, =IRG_t is a label part of S_t;
  • 2-3) training an inter-host interaction graph prediction model: training a prediction model based on the training set SS, and predicting the next inter-host interaction graph according to w continuous inter-host interaction graphs;
  • it is assumed that that S_t, is an inputted sample, a structure of the prediction model consists of the following four layers, as shown in FIG. 4:
  • an input layer: it is assumed that the system's network structure (i.e., the hosts in the network) remains unchanged, so the node sets of different inter-host interaction graphs are the same, and therefore only an adjacent matrix of an edge set of an inter-host interaction graph is inputted, that is, <IRA_t-w, IRA_t-w+1, . . . , IRA_t-1> is inputted, where IRA_k is an adjacent matrix of an edge set of IRG_k;
  • a convolutional layer: each IRA_k is inputted into a convolutional neural network to generate a feature matrix CM_k;
  • a circulation layer: t continuous feature matrices <CM_t-w, CM_t-w+1, . . . , CM_t-1> are inputted into an LSTM to generate a feature matrix RM;
  • an output layer: RM is inputted into a fully connected neural network to generate an adjacent matrix pIRA_t, at a predicted time slot t; and
  • 2-4) inter-host anomaly detection: first, following the step 2-1) to generate w inter-host interaction graphs D_t=<IRG_t-w, IRG_t-w+1, . . . , IRG_t-1> before the current time slot t; then, inputting D_t into the prediction model trained in the step 2-3) to predict and obtain an adjacent matrix pIRA_t, at the time slot t; finally, comparing pIRA_t, with a real adjacent matrix IRA_t, at the time slot t; when pIRA_t[i][j]=0, while IRA_t[i][j]=1, which means that the model predicts that there should be no interaction between hosts H_i and H_j at the time slot t, but in fact there is an interaction between them, the interaction between the hosts H_i and H_j at the time slot t is deemed to be abnormal;
  • 3) cross-host complex network attack detection: firstly constructing a cross-host anomaly association graph according to intra-host and inter-host anomaly detection results, then propagating anomaly scores by employing a PageRank algorithm, and finally confirming an attacked host based on the anomaly scores;
  • detailed steps of cross-host complex network attack detection are as follows:
  • 3-1) constructing a cross-host anomaly association graph: constructing a cross-host anomaly association graph as follows according to a given time slot t, an intra-host data collection time span w, and a set (HS={H₁, H₂, . . . , H_M}) of hosts in the target network:
  • 3-1-1) following the step 1-1) to collect system log data of each host in HS generated between time slots t-w and t, and constructing a real-time intra-host association graph rIAG_k for each host H_k;
  • 3-1-2) based on the real-time intra-host association graph rIAG_k of each host H_k, detecting abnormal nodes in rIAG_k according to the step 1-5), and performing anomaly scoring for each host H_k according to the step 1-6) (an anomaly score of H_k is recorded as s_k);
  • 3-1-3) following the step 2-1) to collect the inter-host interaction graphs generated between the time slots t-w and t, and forming a sequence D_t=<IRG_t-w, IRG_t-w+1, . . . , IRG_t-1> of the inter-host interaction graphs;
  • 3-1-4) following the step 2-4) to detect any inter-host abnormal interaction at the time slot t; and
  • 3-1-5) constructing a cross-host anomaly association graph CG, where each node p_k represents a host H_k; when an interaction between the hosts H_i and H_j at the time slot t is abnormal, there exists an edge between nodes p_i and p_j;
  • 3-2) propagating the anomaly scores: using the PageRank algorithm to iteratively update a weight of each node in CG, with detailed steps as follows:
  • 3-2-1) setting an initial weight PR(p_k) of each node p_k in CG as an anomaly score s_k of H_k;
  • 3-2-2) updating the weight of each node p_k in CG according to the formula (3); in the formula (3), d is a damping coefficient, N_CG is the number of nodes in CG, I(p_k) is a set of nodes corresponding to all incoming edges of the node p_k, and L(p_i) is an outgoing degree of the node p_i;

PR ⁡ ( p k ) = 1 - d N C ⁢ G + d ( ∑ p i ∈ I ⁡ ( p k ) PR ⁡ ( p i ) L ⁡ ( p i ) ) ( 3 )

• • 3-2-3) when weight variation of any node after this iteration is less than a specified threshold, or the number of iterations reaches a specified upper limit, stopping the iteration, otherwise jumping to the step 3-2-2) to restart the iteration; and
  • 3-3) detecting an attacked host: confirming a host represented by a node in CG whose weight is higher than the specified threshold as the attacked host, and confirming a pair of attacked hosts connected by an edge as lateral movement targets of an attacker.

Embodiment 1

A given small network system includes a plurality of hosts and a domain controller, where a system kernel log collection tool is installed in each host, and a network traffic capture tool is installed in the domain controller. On this basis, this embodiment includes two parts, i.e., model training and real-time detection, where the model training further includes two parts: intra-host abnormal behavior detection model training and inter-host abnormal behavior detection model training.

1. Intra-Host Abnormal Behavior Detection Model Training

• • (1) The system kernel log collection tool is used to collect log data of each host generated in the normal working process, and a benign intra-host association graph is constructed according to the step 1-1). As shown in FIG. 5, an example of a benign intra-host association graph is given.
  • (2) According to the step 1-2), initial features of nodes in the benign intra-host association graph are extracted, and when there are 7 types of edges [read, open, write, accept, create, execute, and send] in the benign intra-host association graph, the initial feature of a winword.exe node is expressed as [3, 1, 2, 0, 0, 0, 0]; then, feature vectors of all nodes are updated according to the step 1-3); finally, according to the step 1-4), feature vectors of all nodes are inputted to train an autoencoder M₁.
    2. Inter-Host Abnormal Behavior Detection Model Training
  • (1) The network traffic capture tool is used to collect data of interactions between hosts in the network system, and an inter-host interaction graph sequence is constructed according to the step 2-1). As shown in FIG. 6, an example of an inter-host interaction graph sequence is given (in the figure, each edge represents an inter-host interaction during that time period).
  • (2) A training set is constructed according to the step 2-2), and then an inter-host interaction graph prediction model M₂ is trained according to the step 2-3).
    3. Real-Time Detection
  • (1) The intra-host association graph of each host and the inter-host interaction graph sequence of the network system are collected in a time range [t-w, t) for real-time detection, where t is a given current time slot; as shown in FIG. 7, an example of input for real-time detection is given.
  • (2) A model M₁ is used to detect intra-host anomalies according to the step 1-5), and anomaly scores are calculated according to the step 1-6); the model M₂ is used to detect inter-host anomalies according to the step 2-3). When the detected abnormal interactions are defined as e₄, e₅, and e₆, and anomaly scores of a host 3, a host 4, and a host 5 are 98, 6, and 13 respectively, a cross-host anomaly association graph (as shown in FIG. 8) is constructed according to the step 3-1-5). Then, final anomaly scores of the host 3, the host 4, and the host 5 are calculated according to the step 3-2). When the final anomaly scores are 102, 18, and 21 respectively, it can be finally determined that the host 3 is an attacked host.

Compared with the prior art, the present disclosure has the following advantages:

• • (1) through cross-host combined network attack detection, intra-host malicious operation events and inter-host lateral movement events are mutually verified, so that the false alarm rate can be effectively reduced; and
  • (2) self-supervised deep learning technology is adopted, without need of any actual network attack training samples.