CN102968591B - Malicious-software characteristic clustering analysis method and system based on behavior segment sharing - Google Patents

Abstract

The invention discloses a malicious-software characteristic clustering analysis method and system based on behavior segment sharing. The malicious-software characteristic clustering analysis method comprises the following steps of: distributing acquisition and analysis nodes, establishing a distribution type hash table module, segmenting behaviors of an acquired malicious software sample into behavior segments, and storing the behavior segments to a hash table; and carrying out statistics on the global characteristics of the segments, returning the global characteristics to the acquisition and analysis nodes which construct a characteristic vector of the malicious software sample, carrying out clustering, and extracting the characteristics and the attributes of the clustering as a comprehensive analysis result and inputting the result. The system comprises a plurality of acquisition and analysis nodes which comprise a behavior-segment segmenting module, the distribution type hash table module, a behavior-segment synergy sharing module, a malicious-software sample representation module, a malicious-software sample local clustering module and a malicious-software sample local analysis module. The invention has the advantages of high analysis accuracy, strong analysis performance and good expandability.

Description

The characteristic of malware clustering method that Behavior-based control fragment is shared and system

Technical field

The present invention relates to computer network security technology field, be specifically related to a kind of collection analysis node for being distributed in network various places how to realize high efficiency interactive information thus can to local Malware sample carry out accurate cluster and analysis Behavior-based control fragment share characteristic of malware clustering method and system.

Background technology

Threaten the definition of term in report according to national Internet emergency center internet security, Malware refers in unwarranted situation, installs, performs the program reaching improper object in infosystem.Malware mainly comprises: 1) Trojan Horse (Trojan Horse), to steal userspersonal information, or even remote control user computing machine is the Malware of main target.2) bot program, for building the Malware of attack in force platform.According to the communication protocol used, bot program can be further divided into: IRC (Internet Relay Chat) bot program, HTTP (Hypertext Transfer Protocol) bot program, P2P (peer-to-peer) bot program etc.3) worm, refers to energy self-replacation and wide-scale distribution, to take the Malware that System and Network resource is fundamental purpose.4) virus, is propagated by infect computers file, to destroy or to distort user data, affects infosystem and normally operate to fundamental purpose Malware.

The examination and analysb of Malware is just becoming more and more difficult, is mainly manifested in following three aspects.1) Malware enormous amount and exponentially level increase, Symantec Corporation's a series of network security threats report (Symantec Internet SecurityThreat Report) points out current Malware enormous amount and exponentially level increases, Symantec Corporation found 400,000,000 new Malware samples altogether in 2011, average every day 1,100,000.How Malware sample huge so correctly identifies to malware detection system, sort out, describe Malware is brought huge challenge.2) behavior of Malware presents stronger diversity, by message encryption, conversion route of transmission, the technology such as polymorphic, the different schedule of samples of same Malware reveal different behaviors, are difficult to carry out correct effectively analysis to the Malware sample observed.3) sample of Malware spatially extensively distribution and have very high disguise, therefore the number of samples of same Malware that can observe of single LAN (Local Area Network) or enterprise network is very limited.Due to the diversity of Malware behavior, when number of samples is limited, cannot obtain the essential characteristic of Malware, precision of analysis cannot ensure.Therefore malware analysis system generally adopts distribution collection mode to cover abundant Malware sample.The researchist of Symantec Corporation network security top-level meeting SP'11 (IEEE Symposium on Securityand Privacy 2011) write short essay (Benchmarking Computer Security Using WINE) point out the said firm at global deployment 24 extremely evil meaning software collection nodes.

Current research personnel propose many malware analysis methods, but wherein the overwhelming majority is all centralized analytical approach.Patent " a kind of Malware behavioural characteristic extracting method " (patent No.: 200910237422.X) proposes first to run Malware at hardware simulator, and the instruction sequence of Malware is obtained by adding a dis-assembling engine at hardware simulator translation layer, then the method for dynamic tainting is adopted to build control dependency graph and the data dependence graph of Malware, finally carry out similarity cluster according to dependency graph, and be that every class extracts behavioural characteristic.Scholar Ulrich Bayer etc. to publish thesis " Scalable international top-level meeting NDSS 2009 (16th Annual Network & Distributed System Security Symposium) of network security, Behavior-Based Malware Clustering ", the execution track (instruction sequence etc.) that first adopt binary code analysis techniques obtain Malware the same as patent " a kind of Malware behavioural characteristic extracting method ", then improve analysis speed to improve precision of analysis simultaneously, execution trajectory table is shown as resource object by author, to the abstract representation of the dependence between the operation of resource object and operation, reduce the attribute dimensions of Malware, limitedly eliminate the impact of Malware owing to adopting polymorphic technology to bring in behavior details, last author carries out cluster analysis based on abstract representation to Malware.Except analyzing based on Malware Host behavior, research member also proposed the method that more flow Network Based is analyzed Malware.Scholar Zhichun Li etc. publish thesis " Hamsa:Fast Signature Generation for Zero-day Polymorphic Worms withProvable Attack Resilience " international top-level meeting SP'06 (IEEE Symposium on Security andPrivacy 2006) of network security, adopt the common field between prefix array extraction same class Malware sample, then build Malware traffic characteristic based on common field.Scholar Roberto Perdisci etc. publish thesis " Behavioral Clustering of HTTP-Based Malware and Signature Generation Using MaliciousNetwork Traces " on international top-level meeting NSDI'10 (USENIX Symposium on Networked System Design and Implementation 2010) of network system, for the Malware based on http communication, URL Malware being accessed web is divided into four parts by structure: requesting method (get, post etc.), requests for page address, required parameter title, required parameter value, give the weights that different piece is different, based on URL this structural similarity cluster Malware and extract the URL feature of Malware sample in such for each class.

But along with Malware sample exponentially level constantly increases, collective analysis system needs to process increasing Malware sample, cluster analysis simultaneously and feature extraction are all the tasks that computing cost is large, and therefore centralized malware analysis faces serious Calculation bottleneck problem.Many developers adopt different approaches to solve the Calculation bottleneck problem of malware analysis for this reason.Patent " method and system of the network behavior of detection of malicious the software sample " (patent No.: 201010107195.1) propose first to run Malware and obtain its network service behavior, and communications command is extracted from network service behavior, these communications commands of last test follow the trail of other behavior of Malware, analyze resources costs, reduction computing cost to reach to reduce.Scholar Jiyong Jang publishes thesis " BitShred:Feature Hashing Malware for Scalable Triage and Semantic Analysis " at network security top-level meeting CCS'11 (18th ACM Conference on Computer and Communications Security), propose to adopt attribute Hash (feature hashing) to reduce the dimension of Malware greatly, reduce the computing cost of cluster analysis, improve the extensibility of malware analysis.But the Calculation bottleneck problem of integrated system limitedly can only be extenuated by the method reducing Malware attribute dimension and minimizing amount of analysis, fundamentally can not solve the Calculation bottleneck problem that it analyzes Malware.

In sum, for network malware detection, current single-point acquiring analytical technology exists adopts the low defect of Malware Finite Samples, precision of analysis and validity; And distribution collection the technology of collective analysis, exist and calculate and communication performance bottleneck.And two kinds of malware analysis technology all rely on human expert, can not support intelligent automatic analysis.

Distributed hashtable (DHT, Distributed Hash Table) is generally used for Distributed Storage and retrieval, has the characteristic such as decentralization, reliable and good extensibility.In DHT network, each node is responsible for the storage of a route processing among a small circle and segment data object (data).The ultimate principle of DHT each data resource Object table is shown as a data resource object directory entry (Key, Node) right, Key is called key word, be the cryptographic hash of resource object descriptor (as resource name, resource number, resource content etc.), Node is the descriptor (as IP address, title etc.) of the node of this resource object of actual storage.All data resource object indexing entry (namely all (Key, Node) to) composition a resource object index Hash table, as long as the Key value (key value) of input target resource object, just can find the address of node or other descriptors that store this resource object from this resource object index Hash table.The major function of DHT is by this resource index Hash table distribution, be divided into a lot of section fritter (local Hash table), then according to specific rule, each block local Hash table one of being assigned in system is participated on node, make each node be responsible for the one piece of local Hash table safeguarded wherein.

Summary of the invention

The technical problem to be solved in the present invention is to provide the characteristic of malware clustering method and system that a kind of precision of analysis is high, analytical performance is strong, extensibility is good Behavior-based control fragment shares.

In order to solve the problems of the technologies described above, the technical solution used in the present invention is:

The characteristic of malware clustering method that Behavior-based control fragment is shared, implementation step is as follows:

1) the collection analysis node of geographic position dispersion is arranged in a network respectively, each collection analysis node is responsible for collection and the analysis of Malware sample in a slice network area, sets up the distributed hashtable module for building distributed hashtable in collection analysis node;

2) behavior of the Malware sample of collection is divided into multiple behavior segment by described collection analysis node;

3) described collection analysis node obtains the local statistical property of behavior segment, behavior segment and local statistical property thereof are shared to distributed hashtable module, by distributed hashtable module, behavior segment and local statistical property thereof is stored to distributed hashtable; The global property statistics of different behavior segment is responsible for by different collection analysis nodes, described collection analysis node respectively by all local statistical property statistics of the described behavior segment stored in distributed hashtable obtain the global property of responsible behavior segment, and return the behavior segment set with global property to each collection analysis node issuing described behavior segment;

4) the described collection analysis node received with the behavior segment set of global property calculates the weights of each behavior segment in behavior segment set based on the global property of behavior segment, and builds the eigen vector of described Malware sample according to the weights of each behavior segment in Malware sample;

5) described collection analysis node carries out cluster according to the distance between the eigen vector of Malware sample to each Malware sample and obtains cluster set, analyze each cluster in cluster set, extract the feature of this cluster and attribute and export as the Comprehensive analysis results of described Malware.

Further improvement as the present invention is based on the characteristic of malware clustering method that behavior segment is shared:

Described step 2) detailed step as follows;

2.1) behavior of Malware sample is considered as the behavior sequence that order performs, from the behavior sequence that described order performs, selects the Continuous behavior subsequence of regular length as splitting the behavior segment obtained; Or set up behavior dependency graph according to the dependence between the behavior service data of Malware sample, from described behavior dependency graph, select the behavior obtaining fixed vertices number to rely on subgraph as splitting the behavior segment obtained;

2.2) adopt heuristic rule housing choice behavior fragment, obtain the behavior segment set comprising many group fixed qty behavior segments;

2.3) through type (1) calculates the local characteristics weights of each behavior segment;

$w\left(t_i\right)=\log H_i\log \frac{1}{F_i}---\left(1\right)$

In formula (1), F _ifor behavior segment t _ithe frequency occurred in normal procedure behavior, H _ifor comprising the ratio of behavior fragment in Malware sample; W (t _i) be behavior segment t _ilocal characteristics weights; t _ibe i-th behavior segment;

2.4) the local characteristics weights of Behavior-based control fragment build the local characteristics vector of Malware sample, the dimension of local characteristics vector is the arrangement of behavior segment according to a definite sequence, the value of certain dimension of local characteristics vector of a certain Malware sample is determined by following two kinds of situations: if having the behavior of the behavior segment representated by this dimension in the behavior of Malware sample, then described value equals the local characteristics weights of behavior fragment; Otherwise then described value equals zero;

2.5) through type (2) calculates the similarity of each group behavior segment and original Malware sample in behavior segment set, the one group of behavior segment representatively behavior segment selecting similarity maximum from described behavior segment set; Meanwhile, if select number of times to reach preset times time and the similarity that behavior segment set exists one group of behavior segment and original Malware sample is more than or equal to 0.8, then stop selection and by described behavior segment representatively behavior segment;

$\mathrm{similar}(\mathrm{rep},\mathrm{orig})=\underset{(\mathrm{gram}\_i\∈\mathrm{rep})}{\mathrm{\Σ}}w(\mathrm{gram}\_i)/\underset{(\mathrm{gram}\_j\∈\mathrm{gram}\_\mathrm{set})}{\mathrm{\Σ}}w(\mathrm{gram}\_j)---\left(2\right)$

In formula (2), gram_set represents all characteristic sets that original sample is possible, rep is one group of behavior segment in behavior segment set, orig is original Malware sample, similar (rep, orig) the Similarity measures result between one group of behavior segment rep in the set of expression behavior segment and original Malware sample orig, gram_i is that in rep, sequence number is the behavior segment of i, gram_j is that in gram_set, sequence number is the behavior segment of j, the local characteristics weight computing function that w (gram_i) is behavior segment gram_i, the local characteristics weight computing function that w (gram_j) is behavior segment gram_j,

2.6) export and describedly represent behavior segment.

Described step 3) detailed step as follows:

3.1) described collection analysis node calls the key value that distributed hashtable module obtains described behavior segment;

3.2) in described this collection analysis of collection analysis node statistics node all Malwares sample, behavior comprises the local statistical property of the sample number of described behavior segment;

3.2) described behavior segment and local statistical property thereof are encapsulated as distributed hashtable message by described collection analysis node, then described key value and distributed hashtable message are sent to distributed hashtable module, and the collection analysis node address of behavior segment issued in record; Described distributed hashtable module searches the collection analysis node of this key value responsible according to key value, and is stored to the collection analysis node being responsible for this key value by distributed hashtable Message routing;

3.3) different collection analysis nodes is responsible for the global property statistics of different behavior segment, described collection analysis node returns the distributed hashtable message of be responsible for behavior segment respectively according to collection analysis node address and distributed hashtable module, statistics is included in distributed hashtable the Malware number of samples of the same behavior fragment comprised from different acquisition analysis node, issue the collection analysis number of nodes of behavior fragment at interior global property, behavior segment and global property thereof are assembled into return messages and return to the collection analysis node address issuing behavior segment, the collection analysis node that described collection analysis node address is corresponding receives the behavior segment set with global property that each behavior segment and global property thereof obtain.

Described step 4) detailed step as follows:

4.1) the described collection analysis node received with the behavior segment set of global property calculates the weights of each behavior segment in described behavior segment set according to formula (3);

$V_i=(N_i+\log S_i)\log \frac{1}{F_i}---\left(3\right)$

In formula (3), N _i, S _i, F _ithe global property of constituting action fragment, N _ifor sharing the collection analysis nodes of behavior fragment in whole network to distributed hashtable module, S _ifor there is the Malware sample number of behavior fragment behavior, F _ifor the frequency that behavior segment occurs in normal procedure, V _ifor the weights of behavior segment;

4.2) described collection analysis node builds the eigen vector of described Malware sample according to the weights of each behavior segment in Malware sample; The dimension of described eigen vector is the arrangement of behavior segment set according to a definite sequence of collection analysis node this locality, the value of the eigen vector dimension of any Malware sample is determined by following two kinds of situations: if having the behavior of the behavior segment representated by this dimension in the sample behavior of Malware sample, then value equals the weights of behavior fragment; Otherwise value equals zero.

Described step 5) detailed step as follows:

5.1) distance between the eigen vector of any two Malware samples of described collection analysis node calculate, the less similarity then between two Malware samples of distance is larger;

5.2) described collection analysis node is according to the distance between the eigen vector of Malware sample, adopts and does not need in advance to know that the clustering algorithm of clusters number carries out cluster to each Malware sample and obtains cluster set;

5.3) each cluster in cluster set described in described collection analysis node traverses, extract the feature of joint act as this cluster of all Malware samples in this cluster, add up the network attribute of all Malware samples in this cluster and the local behavior property attribute as cluster simultaneously, the feature of described cluster and the attribute analysis result as described Malware is exported.

The characteristic of malware cluster analysis system that the present invention also provides a kind of Behavior-based control fragment to share, comprise multiple collection analysis node for gathering Malware and analyzing, the geographic position dispersion in a network of described collection analysis node, each collection analysis node is responsible for collection and the analysis of Malware sample in a slice network area, and described collection analysis node comprises:

Behavior segment segmentation module, the behavior for the Malware sample by collection is divided into multiple behavior segment;

Distributed hashtable module, for building distributed hashtable;

Behavior segment coordination sharing module, for obtaining the local statistical property of behavior segment, behavior segment and local statistical property thereof are shared to distributed hashtable module by behavior segment coordination sharing module, by distributed hashtable module, behavior segment and local statistical property thereof are stored to distributed hashtable, the global property of behavior segment is obtained by all local statistical property statistics of the described behavior segment stored in distributed hashtable, and return the behavior segment set with global property to each collection analysis node issuing described behavior segment, the global property statistics of the different behavior segment of behavior segment coordination sharing module in charge of different collection analysis nodes,

Malware sample characterization module, global property for Behavior-based control fragment after receiving the behavior segment set with global property calculates the weights of each behavior segment in behavior segment set, and builds the eigen vector of described Malware sample according to the weights of each behavior segment in Malware sample;

The local cluster module of Malware sample, obtains cluster set for carrying out cluster according to the distance between the eigen vector of Malware sample to each Malware sample;

Malware sample local analytics module, for analyzing each cluster in cluster set, extracting the feature of this cluster and attribute and exporting as the Comprehensive analysis results of described Malware.

Further improvement as the present invention is based on the characteristic of malware cluster analysis system that behavior segment is shared:

Described behavior segment segmentation module is that Continuous behavior subsequence segmentation module or behavior rely on dividing sub-picture module; The behavior of Malware sample is considered as the behavior sequence that order performs by described Continuous behavior subsequence segmentation module, selects the Continuous behavior subsequence of regular length as splitting the behavior segment obtained from the behavior sequence that described order performs; Rely on dividing sub-picture module and set up behavior dependency graph according to the dependence between the behavior service data of Malware sample, from described behavior dependency graph, select the behavior obtaining fixed vertices number to rely on subgraph as splitting the behavior segment obtained; Described behavior segment segmentation module also selects module that behavior segment is distributed to behavior segment coordination sharing module by behavior segment, and described behavior segment selects module to comprise:

Behavior segment chooser module, comprises the behavior segment set of many group fixed qty behavior segments for adopting heuristic rule housing choice behavior fragment to obtain;

Behavior segment representativeness checking submodule, the local characteristics weights of each behavior segment are calculated for through type (1), the local characteristics weights of Behavior-based control fragment build the local characteristics vector of Malware sample, the dimension of local characteristics vector is the arrangement of behavior segment according to a definite sequence, the value of certain dimension of local characteristics vector of a certain Malware sample is determined by following two kinds of situations: if having the behavior of the behavior segment representated by this dimension in the behavior of Malware sample, then described value equals the local characteristics weights of behavior fragment; Otherwise then described value equals zero; Through type (2) calculates the similarity of each group behavior segment and original Malware sample in behavior segment set, the one group of behavior segment representatively behavior segment selecting similarity maximum from described behavior segment set; Meanwhile, if select number of times to reach preset times time and the similarity that behavior segment set exists one group of behavior segment and original Malware sample is more than or equal to 0.8, then stop selection and by described behavior segment representatively behavior segment; And the described behavior segment that represents is distributed to behavior segment coordination sharing module;

$w\left(t_i\right)=\log H_i\log \frac{1}{F_i}---\left(1\right)$

In formula (1), F _ifor behavior segment t _ithe frequency occurred in normal procedure behavior, H _ifor comprising the ratio of behavior fragment in Malware sample; W (t _i) be behavior segment t _ilocal characteristics weights; t _ibe i-th behavior segment;

$\mathrm{similar}(\mathrm{rep},\mathrm{orig})=\underset{(\mathrm{gram}\_i\∈\mathrm{rep})}{\mathrm{\Σ}}w(\mathrm{gram}\_i)/\underset{(\mathrm{gram}\_j\∈\mathrm{gram}\_\mathrm{set})}{\mathrm{\Σ}}w(\mathrm{gram}\_j)---\left(2\right)$

In formula (2), gram_set represents all characteristic sets that original sample is possible, rep is one group of behavior segment in behavior segment set, orig is original Malware sample, similar (rep, orig) the Similarity measures result between one group of behavior segment rep in the set of expression behavior segment and original Malware sample orig, gram_i is that in rep, sequence number is the behavior segment of i, gram_j is that in gram_set, sequence number is the behavior segment of j, the local characteristics weight computing function that w (gram_i) is behavior segment gram_i, the local characteristics weight computing function that w (gram_j) is behavior segment gram_j.

Described behavior segment coordination sharing module comprises:

Behavior segment issues submodule, the key value of described behavior segment is obtained for calling distributed hashtable module, behavior in this collection analysis node all Malwares sample of adding up comprises the local statistical property of the sample number of described behavior segment, described behavior segment and local statistical property thereof are encapsulated as distributed hashtable message, then by described key value and distributed hashtable news release to distributed hashtable module;

Behavior segment receives submodule, for receiving the distributed hashtable message that distributed hashtable module returns and the collection analysis node address recording issue behavior segment;

Behavior segment statistics submodule, for after receiving the behavior segment set with global property according to the distributed hashtable message that collection analysis node address and distributed hashtable module return, statistics is included in distributed hashtable the Malware number of samples of the same behavior fragment comprised from different acquisition analysis node, issues the global property of the collection analysis number of nodes of behavior fragment, and the behavior segment statistics submodule of different collection analysis nodes is responsible for the global property statistics of different behavior segment;

Behavior segment global property returns submodule, returns to for behavior segment and global property thereof being assembled into return messages the collection analysis node address issuing behavior segment;

Behavior segment global property receives submodule, behavior segment global property for receiving other collection analysis nodes returns each behavior segment and the global property thereof of submodule transmission, obtains the behavior segment set with global property comprising the behavior segment that this collection analysis node is issued;

Described distributed hashtable module comprises:

Behavior segment key word mapping submodule, carries out for utilizing the behavior segment of hash function to input the key value of cryptographic hash as behavior segment that Hash calculation is fixed length;

Key word route submodule, for being stored into according to the key value of input the Hash routing table Message routing comprising behavior segment and local statistical property thereof the collection analysis node of this key value responsible.

Malware sample characterization module comprises:

Behavior segment weight computing submodule, for calculating the weights of each behavior segment according to formula (3);

Malware sample properties vector characterizes submodule, for building the eigen vector of described Malware sample according to the weights of each behavior segment in Malware sample; The dimension of described eigen vector is the arrangement of behavior segment set according to a definite sequence of collection analysis node this locality, the value of the eigen vector dimension of any Malware sample is determined by following two kinds of situations: if having the behavior of the behavior segment representated by this dimension in the sample behavior of Malware sample, then value equals the weights of behavior fragment; Otherwise value equals zero;

$V_i=(N_i+\log S_i)\log \frac{1}{F_i}---\left(3\right)$

In formula (3), N _i, S _i, F _ithe global property of constituting action fragment, N _ifor sharing the collection analysis nodes of behavior fragment in whole network to distributed hashtable module, S _ifor there is the Malware sample number of behavior fragment behavior, F _ifor the frequency that behavior segment occurs in normal procedure, V _ifor the weights of behavior segment.

The local cluster module of described Malware sample comprises:

Eigen vector distance calculating sub module, for calculate any two Malware samples eigen vector between distance, the less similarity then between two Malware samples of distance is larger;

The local cluster submodule of Malware sample, for according to the distance between the eigen vector of Malware sample, adopts and does not need in advance to know that the clustering algorithm of clusters number carries out cluster to each Malware sample and obtains cluster set;

Each cluster in the cluster set that the local cluster submodule of Malware sample described in described Malware sample local analytics module walks exports, extract the feature of joint act as this cluster of all Malware samples in this cluster, add up the network attribute of all Malware samples in this cluster and the local behavior property attribute as cluster simultaneously, the feature of described cluster and the attribute analysis result as described Malware is exported.

The present invention is based on behavior segment share characteristic of malware clustering method there is following advantage: due to Malware extensive dispersiveness and extremely disguise, the quantity of information that single-point malware analysis system obtains is few, and Malware uses encryption, various attacks approach, polymorphic technology etc. simultaneously, the sample of same Malware is made to have different behaviors, be difficult to the intrinsic propesties therefrom extracting Malware when Malware sample is few, cause precision of analysis low; And on the other hand, Malware enormous amount exponentially level increase, be submitted to distributed capture centralized processing system that centralized servers carries out focusing on and exist by unified for all samples collected and calculate and communication performance bottleneck problem.Although the different samples of same Malware have different behaviors, because similar, the communication of same Malware due to function are tied identical and causes behavior to have similarity, thus part behavior segment is identical.The present invention is based on this characteristic of Malware, propose Behavior-based control fragment share and obtain the method for total system Malware behavior global view, for solving these defects of above-mentioned prior art, the present invention disposes Malware and the behavior thereof that collection analysis node (can WSO disposed in the past Malware acquisition node) is responsible for gathering specified network region in network various places, these collection analysis nodes obtain the global view of system-wide Malware behavior by sharing behavior segment, then based on this global view, collection analysis node disjoint accurate analysis is carried out to the Malware that this locality collects, the distributed hashtable module composition distributed hashtable storage system of Malware acquisition node of the present invention, therefore behavior segment collection is had concurrently, share, store, analyze in one, form the distributed capture analytic system of Malware, both the few and precision of analysis problem caused of single-point malware analysis system acquisition information amount had been solved, solve again distributed capture collective analysis system due to Malware huge amount and exponential growth and the calculating caused and communication performance bottleneck problem, there is precision of analysis high, analytical performance is strong, the advantage that extensibility is good, and taken into account accuracy and the extensibility of malware analysis simultaneously, have a good application prospect.

The present invention is based on characteristic of malware cluster analysis system that behavior segment shares is the present invention is based on system corresponding to characteristic of malware clustering method that behavior segment shares, there is the technique effect identical with the present invention is based on characteristic of malware clustering method that behavior segment shares, not repeating them here.

Accompanying drawing explanation

Fig. 1 is the basic procedure schematic diagram of embodiment of the present invention method.

Fig. 2 is the structural representation of the nerve of a covering shared towards behavior segment that in the embodiment of the present invention, collection analysis node is formed.

Fig. 3 is the segmentation principle schematic of Continuous behavior subsequence in the embodiment of the present invention.

Fig. 4 is the segmentation principle schematic that in the embodiment of the present invention, behavior relies on subgraph.

Fig. 5 is the framed structure schematic diagram of embodiment of the present invention system.

Fig. 6 is the framed structure schematic diagram that in the embodiment of the present invention, behavior segment selects module.

Fig. 7 is the workflow schematic diagram that in the embodiment of the present invention, behavior segment selects module.

Fig. 8 is the framed structure schematic diagram of behavior segment coordination sharing module in the embodiment of the present invention.

Fig. 9 is the framed structure schematic diagram of distributed hashtable module in the embodiment of the present invention.

Figure 10 is the workflow schematic diagram of behavior segment coordination sharing module in the embodiment of the present invention.

Figure 11 is the framed structure schematic diagram of Malware sample characterization module in the embodiment of the present invention.

Figure 12 is the workflow schematic diagram of Malware sample characterization module in the embodiment of the present invention.

Embodiment

As shown in Figure 1, the implementation step of characteristic of malware clustering method shared of the present embodiment Behavior-based control fragment is as follows:

1) the collection analysis node of geographic position dispersion is arranged in a network respectively, each collection analysis node is responsible for collection and the analysis of Malware sample in a slice network area, sets up the distributed hashtable module for building distributed hashtable in collection analysis node;

2) behavior of the Malware sample of collection is divided into multiple behavior segment by collection analysis node;

3) collection analysis node obtains the local statistical property of behavior segment, behavior segment and local statistical property thereof are shared to distributed hashtable module, by distributed hashtable module, behavior segment and local statistical property thereof is stored to distributed hashtable; The global property statistics of different behavior segment is responsible for by different collection analysis nodes, collection analysis node obtains the global property of the responsible behavior segment of institute respectively by all local statistical property statistics of the behavior segment stored in distributed hashtable, and returns the behavior segment set with global property to collection analysis node of each issue behavior segment;

4) the collection analysis node received with the behavior segment set of global property calculates the weights of each behavior segment in behavior segment set based on the global property of behavior segment, and builds the eigen vector of Malware sample according to the weights of each behavior segment in Malware sample;

5) collection analysis node carries out cluster according to the distance between the eigen vector of Malware sample to each Malware sample and obtains cluster set, analyze each cluster in cluster set, extract the feature of this cluster and attribute and export as the Comprehensive analysis results of Malware.

The present embodiment is at the collection analysis node of network deploy multiple address locations dispersion, Malware behavior is divided into behavior segment by collection analysis node, realized on the basis of Malware distributed capture by collection analysis node, realize efficient Malware sample information by distributed hashtable module to share, the overall perception of the behavior of Malware can be realized to load balancing with very little network service, each collection analysis node can carry out Malware cluster analysis more accurately in this locality based on global property vector automatically, the analysis task of Malware is distributed to each collection analysis node to perform, thus build the automatic malware analysis scheme of a super distributed, communication and the computing cost of dispersion analysis can be realized, keep correctness and the validity of collective analysis, there is precision of analysis high, analytical performance is strong, the advantage that extensibility is good, can either detect and be extracted in the simple form Malware that behavior in communication process does not change, but also all kinds of employing message encryption can be tackled, conversion route of transmission, the polymorphic Malware waiting complex technology.

Each collection analysis node of the present embodiment is responsible for the analysis of Malware sample behavior in a slice network area, form one by one towards the nerve of a covering that behavior segment is shared, the system-wide global property of behavior segment is obtained for the behavior segment of its this locality of nodes sharing of system, and then the approximate global view reaching acquisition Malware sample; The present embodiment sharing by the overall situation simultaneously, the different Malware samples of same Malware can be obtained, different Malware sample has different behaviors, but due to the similarity of behavior, its part behavior segment is identical, the present embodiment is just based on this characteristic of Malware, propose Behavior-based control fragment share and obtain the method for total system Malware behavior global view, greatly can improve the accuracy of malware analysis, this is one of key point of the present embodiment realization.The nerve of a covering shared towards behavior segment that the present embodiment is made up of behavior segment coordination sharing module and the distributed hashtable module of all collection analysis nodes, the behavior segment coordination sharing module of each collection analysis node issues the behavior segment set of this sensor selection problem according to the mode of DHT by the correlation function that calls distributed hashtable module, the concrete grammar issuing each behavior segment in behavior set of segments is: with the behavior fragment for resource name, with behavior segment and local statistical property thereof for content, hash function is adopted to carry out to behavior segment (resource name) the key word key that Hash obtains behavior fragment, the destination node being responsible for this key word key is arrived based on DHT issue behavior fragment and local statistical information (resource content) thereof, destination node is receiving and is storing behavior fragment and local statistical information thereof, and the source node address of behavior fragment issued in record, according to the characteristic of DHT, all identical behavior segment from different node all can route and be stored in same destination node, and therefore destination node has the global information of the fragment of behavior in distributed malware analysis system.Destination node obtains the global property of behavior segment from the local statistical information of not source node by accumulative identical behavior segment; Behavior segment and global property thereof are returned to the source node set of all issue behavior fragments by destination node; Through above-mentioned issue return course, each collection analysis node obtains the global property of the behavior segment that himself is selected.

As shown in Figure 2, the present embodiment comprises some collection analysis nodes, each collection analysis node is responsible for the Malware sample and the behavior thereof that gather certain network area, form between collection analysis node and collection analysis node one " nerve of a covering towards behavior segment is shared ", Malware behavior is divided into behavior segment set by each collection analysis node in " nerve of a covering towards behavior segment is shared " respectively, then share respective local behavior set of segments by " nerve of a covering towards behavior segment is shared " and obtain the global property of behavior segment in whole " nerve of a covering towards behavior segment is shared ", reach the object of the Malware sample behavior view obtaining whole " nerve of a covering towards behavior segment is shared ", each collection analysis node last is under this system-wide Malware behavior view basis, can realize carrying out local analytics accurately to Malware sample local separately.

The present embodiment builds distributed malware analysis method on the basis of maturation, easily extensible, reliably distributed hashtable (DHT, Distributed Hash Table) module.In the present embodiment, behavior segment coordination sharing module is located in collection analysis node, and collection analysis node is simultaneously also as the memory node of distributed hashtable module.In addition behavior segment coordination sharing module also can adopt independent computer node to realize, the memory node of distributed hashtable module also can carry out completely independent with collection analysis node or part is independent, its implementation method is identical with the present embodiment, does not repeat them here.

In the present embodiment, step 2) detailed step as follows;

2.1) behavior of Malware sample is considered as the behavior sequence that order performs, from the behavior sequence that order performs, selects the Continuous behavior subsequence of regular length as splitting the behavior segment obtained; Or setting up behavior dependency graph according to the dependence between the behavior service data of Malware sample, in subordinate act dependency graph, selecting the behavior obtaining fixed vertices number to rely on subgraph as splitting the behavior segment obtained;

2.2) adopt heuristic rule housing choice behavior fragment, obtain the behavior segment set comprising many group fixed qty behavior segments;

2.3) through type (1) calculates the local characteristics weights of each behavior segment;

$w\left(t_i\right)=\log H_i\log \frac{1}{F_i}---\left(1\right)$

In formula (1), F _ifor behavior segment t _ithe frequency occurred in normal procedure behavior, H _ifor comprising the ratio of behavior fragment in Malware sample; W (t _i) be behavior segment t _ilocal characteristics weights; t _ibe i-th behavior segment;

2.4) the local characteristics weights of Behavior-based control fragment build the local characteristics vector of Malware sample, the dimension of local characteristics vector is the arrangement of behavior segment according to a definite sequence, the value of certain dimension of local characteristics vector of a certain Malware sample is determined by following two kinds of situations: if having the behavior of the behavior segment representated by this dimension in the behavior of Malware sample, then value equals the local characteristics weights of behavior fragment; Otherwise then value equals zero;

2.5) through type (2) calculates the similarity of each group behavior segment and original Malware sample in behavior segment set, the one group of behavior segment representatively behavior segment selecting similarity maximum in subordinate act set of segments; Meanwhile, if select number of times to reach preset times and the similarity that behavior segment set exists one group of behavior segment and original Malware sample is more than or equal to 0.8, then stop selection and by behavior segment representatively behavior segment;

$\mathrm{similar}(\mathrm{rep},\mathrm{orig})=\underset{(\mathrm{gram}\_i\∈\mathrm{rep})}{\mathrm{\Σ}}w(\mathrm{gram}\_i)/\underset{(\mathrm{gram}\_j\∈\mathrm{gram}\_\mathrm{set})}{\mathrm{\Σ}}w(\mathrm{gram}\_j)---\left(2\right)$

In formula (2), gram_set represents all characteristic sets that original sample is possible, rep is one group of behavior segment in behavior segment set, orig is original Malware sample, gram_i is that in rep, sequence number is the behavior segment of i, gram_j is that in gram_set, sequence number is the behavior segment of j, the local characteristics weight computing function that w () is behavior segment;

2.6) output represents behavior segment.

As shown in Figure 3, suppose that certain Malware sample performs behavior in order: under assigned catalogue, 1) create the CreateFile behavior copying the object file of Malware, 2) registry entries reading nominal key in registration table obtains the ReadRegistry behavior in the source file path of copying Malware, 3) by the CopyFile behavior of the Malware file copy under this source file path to new establishment file, 4) new process runs the file just copied RunFile behavior in executable file mode is then created, 5) the DestroyFile behavior of the Malware file under source file path is deleted, then 5 behaviors of certain Malware sample are considered as the behavior subsequence (CreateFile that order performs, ReadRegistry, CopyFile, RunFile, DestroyFile), step 2) behavior of Malware sample is divided into behavior segment and is the behavior subsequence performed from order and selects the Continuous behavior subsequence of regular length as splitting the behavior segment obtained, in the present embodiment, regular length is 3, this behavior sequence be made up of 5 behaviors obtains the Continuous behavior subsequence (1:CreateFile that 3 length are 3 after over-segmentation, ReadRegistry, CopyFile, 2:ReadRegistry, CopyFile, RunFile, 3:CopyFile, RunFile, DestroyFile).

As shown in Figure 4, if the dependence between subordinate act service data instead of from perform order consideration behavior between relation, behavior dependency graph between 5 behaviors of then this Malware sample is as shown in Fig. 4 upper part, such as, CopyFile and DestroyFile depends on the file path that ReadRegistry exports.In behavior dependency graph situation, behavior segment is any connected subgraph that vertex number is fixed, the dependency graph on 5 summits be made up of these 5 behaviors obtains the behavior subgraph that 4 length (number of vertex) are 2 after over-segmentation, and namely 4 length is behavior segment (1:CreateFile, CopyFile of 2; 2:ReadRegistry, CopyFile; 3, CopyFile, RunFile; 4, ReadRegistry, DestroyFile).Behavior dependency graph more can express the behavior of Malware accurately, Malware in order to complete a certain specific function can change arbitrarily do not have rely on behavior between execution sequence, when causing the different samples of same Malware to complete appointed function, there is different behavior sequences; And behavior dependency graph pays close attention to the dependence between behavior service data, dependence can not arbitrarily change, and generally has identical behavior dependency graph when therefore the different samples of same Malware complete appointed function.Input regardless of behavior subsequence segmentation module is behavior sequence or behavior dependency graph, and the embodiment of the present invention can effectively be run.Which kind of mode is the embodiment of the present invention specifically adopt represent, and the deployed environment of system and obtainable information type are depended in the behavior of Malware sample.

In the present embodiment, step 3) detailed step as follows:

3.1) collection analysis node calls the key value that distributed hashtable module obtains behavior segment;

3.2) in this collection analysis of collection analysis node statistics node all Malwares sample, behavior comprises the local statistical property of the sample number of behavior segment;

3.2) behavior segment and local statistical property thereof are encapsulated as distributed hashtable message by collection analysis node, then key value and distributed hashtable message are sent to distributed hashtable module, and the collection analysis node address of behavior segment issued in record; Distributed hashtable module searches the collection analysis node of this key value responsible according to key value, and is stored to the collection analysis node being responsible for this key value by distributed hashtable Message routing;

3.3) different collection analysis nodes is responsible for the global property statistics of different behavior segment, collection analysis node returns the distributed hashtable message of be responsible for behavior segment respectively according to collection analysis node address and distributed hashtable module, statistics is included in distributed hashtable the Malware number of samples of the same behavior fragment comprised from different acquisition analysis node, issue the collection analysis number of nodes of behavior fragment at interior global property, behavior segment and global property thereof are assembled into return messages and return to the collection analysis node address issuing behavior segment, the collection analysis node that collection analysis node address is corresponding receives the behavior segment set with global property that each behavior segment and global property thereof obtain.

In the present embodiment, step 4) detailed step as follows:

4.1) the collection analysis node received with the behavior segment set of global property calculates the weights of each behavior segment in behavior segment set according to formula (3);

$V_i=(N_i+\log S_i)\log \frac{1}{F_i}---\left(3\right)$

In formula (3), N _i, S _i, F _ithe global property of constituting action fragment, N _ifor sharing the collection analysis nodes of behavior fragment in whole network to distributed hashtable module, S _ifor there is the Malware sample number of behavior fragment behavior, F _ifor the frequency that behavior segment occurs in normal procedure, V _ifor the weights of behavior segment;

4.2) collection analysis node builds the eigen vector of Malware sample according to the weights of each behavior segment in Malware sample; The dimension of eigen vector is the arrangement of behavior segment set according to a definite sequence of collection analysis node this locality, the value of the eigen vector dimension of any Malware sample is determined by following two kinds of situations: if having the behavior of the behavior segment representated by this dimension in the sample behavior of Malware sample, then value equals the weights of behavior fragment; Otherwise value equals zero.

In the present embodiment, step 5) detailed step as follows:

5.1) distance between the eigen vector of any two Malware samples of collection analysis node calculate, the less similarity then between two Malware samples of distance is larger;

5.2) collection analysis node is according to the distance between the eigen vector of Malware sample, adopts and does not need in advance to know that the clustering algorithm of clusters number carries out cluster to each Malware sample and obtains cluster set;

5.3) each cluster in the set of collection analysis node traverses cluster, extract the feature of joint act as this cluster of all Malware samples in this cluster, add up the network attribute of all Malware samples in this cluster and the local behavior property attribute as cluster simultaneously, the feature of cluster and the attribute analysis result as Malware is exported.

The characteristic of malware cluster analysis system that the present embodiment Behavior-based control fragment is shared comprises multiple collection analysis node for gathering Malware and analyzing, the geographic position dispersion in a network of collection analysis node, each collection analysis node is responsible for collection and the analysis of Malware sample in a slice network area, as shown in Figure 5, collection analysis node comprises:

Behavior segment segmentation module, the behavior for the Malware sample by collection is divided into multiple behavior segment;

Distributed hashtable module, for building distributed hashtable;

Behavior segment coordination sharing module, for obtaining the local statistical property of behavior segment, behavior segment and local statistical property thereof are shared to distributed hashtable module by behavior segment coordination sharing module, by distributed hashtable module, behavior segment and local statistical property thereof are stored to distributed hashtable, the global property of behavior segment is obtained by all local statistical property statistics of the behavior segment stored in distributed hashtable, and return the behavior segment set with global property to each collection analysis node issuing behavior segment, the global property statistics of the different behavior segment of behavior segment coordination sharing module in charge of different collection analysis nodes,

Malware sample characterization module, global property for Behavior-based control fragment after receiving the behavior segment set with global property calculates the weights of each behavior segment in behavior segment set, and builds the eigen vector of Malware sample according to the weights of each behavior segment in Malware sample;

The local cluster module of Malware sample, obtains cluster set for carrying out cluster according to the distance between the eigen vector of Malware sample to each Malware sample;

Malware sample local analytics module, for analyzing each cluster in cluster set, extracting the feature of this cluster and attribute and exporting as the Comprehensive analysis results of Malware.

In the present embodiment, sheet behavior segment segmentation module is that Continuous behavior subsequence segmentation module or behavior rely on dividing sub-picture module; The behavior of Malware sample is considered as the behavior sequence that order performs by Continuous behavior subsequence segmentation module, selects the Continuous behavior subsequence of regular length as splitting the behavior segment obtained from the behavior sequence that order performs; Relying on dividing sub-picture module and set up behavior dependency graph according to the dependence between the behavior service data of Malware sample, in subordinate act dependency graph, selecting the behavior obtaining fixed vertices number to rely on subgraph as splitting the behavior segment obtained.

As shown in Figure 5, the behavior segment segmentation module of the present embodiment also selects module that behavior segment is distributed to behavior segment coordination sharing module by behavior segment.As shown in Figure 6, behavior segment selects module to comprise:

Behavior segment chooser module, comprise the behavior segment set of many group fixed qty behavior segments for adopting heuristic rule housing choice behavior fragment to obtain, wherein the concrete number of housing choice behavior fragment is determined by the concrete network environment of disposing and experience;

Behavior segment representativeness checking submodule, the local characteristics weights of each behavior segment are calculated for through type (1), the local characteristics weights of Behavior-based control fragment build the local characteristics vector of Malware sample, the dimension of local characteristics vector is the arrangement of behavior segment according to a definite sequence, the value of certain dimension of local characteristics vector of a certain Malware sample is determined by following two kinds of situations: if having the behavior of the behavior segment representated by this dimension in the behavior of Malware sample, then value equals the local characteristics weights of behavior fragment; Otherwise then value equals zero; Through type (2) calculates the similarity of each group behavior segment and original Malware sample in behavior segment set, the one group of behavior segment representatively behavior segment selecting similarity maximum in subordinate act set of segments; Meanwhile, if select number of times to reach preset times time and the similarity that behavior segment set exists one group of behavior segment and original Malware sample is more than or equal to 0.8, then stop selection and by behavior segment representatively behavior segment; And be distributed to behavior segment coordination sharing module by representing behavior segment;

$w\left(t_i\right)=\log H_i\log \frac{1}{F_i}---\left(1\right)$

In formula (1), F _ifor behavior segment t _ithe frequency occurred in normal procedure behavior, H _ifor comprising the ratio of behavior fragment in Malware sample; W (t _i) be behavior segment t _ilocal characteristics weights; t _ibe i-th behavior segment;

$\mathrm{similar}(\mathrm{rep},\mathrm{orig})=\underset{(\mathrm{gram}\_i\∈\mathrm{rep})}{\mathrm{\Σ}}w(\mathrm{gram}\_i)/\underset{(\mathrm{gram}\_j\∈\mathrm{gram}\_\mathrm{set})}{\mathrm{\Σ}}w(\mathrm{gram}\_j)---\left(2\right)$

In formula (2), gram_set represents all characteristic sets that original sample is possible, rep is one group of behavior segment in behavior segment set, orig is original Malware sample, gram_i is that in rep, sequence number is the behavior segment of i, gram_j is that in gram_set, sequence number is the behavior segment of j, the local characteristics weight computing function that w () is behavior segment.

Behavior segment selects module with the behavior segment set of behavior segment segmentation module output for input, under the prerequisite as far as possible keeping similarity between Malware sample, select the representative of behavior segment as Malware behavior of setting number, these are represented behavior segment set by the behavior segment composition selected.Behavior segment selects the module heuristic selection of main consummatory behavior fragment and behavior segment representativeness checking two functions, corresponding behavior Piece Selection submodule and behavior segment representativeness checking submodule two functional modules, when behavior segment representativeness checking submodule calculates behavior segment chooser model choice behavior segment set expression Malware sample out, similarity between sample and the similarity system design between original (during with all behavior segments as attribute representation's sample) sample, if otherness is between the two approximate minimum, then show that these behavior segment set selected to represent Malware characteristic, these set of segments are called and represent behavior segment set.Behavior segment chooser module, comprises the behavior segment set of many group fixed qty behavior segments for adopting heuristic rule housing choice behavior fragment to obtain.

As shown in Figure 7, behavior segment representativeness checking submodule calculate with these behavior segment set chosen as attribute representation's Malware sample time, similarity between sample and the similarity system design between original (during with all behavior segments as attribute representation's sample) sample, if otherness is between the two minimum (or approximate minimum), then show that these behavior segment set selected to represent Malware characteristic, these set of segments are called and represent behavior segment set.The principle that behavior segment representativeness checking submodule calculates the Similarity measures between Malware sample is as follows: the sample number that 1) behavior segment weights and behavior in local Malware sample comprise behavior fragment behavior is directly proportional, and is inversely proportional to the frequency that occurs in normal procedure behavior; 2) then the weights of Behavior-based control fragment build the eigen vector of Malware sample, the dimension of eigen vector is the arrangement of behavior segment according to a definite sequence, the value of the eigen vector dimension of a certain Malware sample is determined by following two kinds of situations: the behavior in this sample behavior with the behavior segment representated by this dimension, and value equals the weights of behavior fragment; Otherwise this value equals zero; 3) with the similarity between the similarity representative sample between eigen vector.How effectively to choose the research that there is comparative maturity of representativeness in machine learning field of behavior segment and checking behavior segment, the present embodiment only specify that the target that behavior segment is selected to reach and the principle that Similarity measures should be followed, and other various algorithms meeting these requirements also can be adopted in addition to carry out the representativeness checking of the selection of specific implementation behavior segment and behavior segment.

As shown in Figure 8 and Figure 9, in the present embodiment, behavior segment coordination sharing module comprises:

Behavior segment issues submodule, the key value of behavior segment is obtained for calling distributed hashtable module, behavior in this collection analysis node all Malwares sample of adding up comprises the local statistical property of the sample number of behavior segment, behavior segment and local statistical property thereof are encapsulated as distributed hashtable message, then by key value and distributed hashtable news release to distributed hashtable module;

Behavior segment receives submodule, for receiving the distributed hashtable message that distributed hashtable module returns and the collection analysis node address recording issue behavior segment;

Behavior segment statistics submodule, for after receiving the behavior segment set with global property according to the distributed hashtable message that collection analysis node address and distributed hashtable module return, statistics is included in distributed hashtable the Malware number of samples of the same behavior fragment comprised from different acquisition analysis node, issues the global property of the collection analysis number of nodes of behavior fragment, and the behavior segment statistics submodule of different collection analysis nodes is responsible for the global property statistics of different behavior segment;

Behavior segment global property returns submodule, returns to for behavior segment and global property thereof being assembled into return messages the collection analysis node address issuing behavior segment;

Behavior segment global property receives submodule, behavior segment global property for receiving other collection analysis nodes returns each behavior segment and the global property thereof of submodule transmission, obtains the behavior segment set with global property comprising the behavior segment that this collection analysis node is issued;

Distributed hashtable module comprises:

Behavior segment key word mapping submodule, carries out for utilizing the behavior segment of hash function to input the key value of cryptographic hash as behavior segment that Hash calculation is fixed length;

Key word route submodule, for being stored into according to the key value of input the Hash routing table Message routing comprising behavior segment and local statistical property thereof the collection analysis node of this key value responsible.

Distributed hashtable module comprises behavior segment key word and maps and key word route two functional units, complete respectively by behavior segment by being mapped as the value in keyword space after Hash calculation, and behavior segment and relevant local statistical property are routed to the node of this key value responsible according to key value.Behavior segment coordination sharing module construction is on distributed hashtable module, call distributed hashtable module and issue local representative behavior segment set, the identical behavior segment realized from different acquisition analysis node is brought to same Nodes, different behavior segments is brought to different collection analysis Nodes, realizes load balancing, dispersion, the extendible identical behavior segment gathered from different acquisition analysis node; Then in the gather round of behavior segment, the statistics of global property is carried out to behavior segment; Finally behavior segment and global property thereof are returned to the set of node having issued behavior fragment.Net result is the global property of representative behavior segment at all standing net that each collection analysis node obtains the Malware sample himself collected.

The major function of distributed hashtable module is by behavior segment by being mapped as the value in keyword space after Hash calculation, then according to this key value, behavior segment and relevant local statistical property is routed to the node of this key value responsible.Comprise behavior segment key word mapping submodule and key word route submodule two submodules.Behavior segment key word mapping submodule is called by behavior segment coordination sharing module, behavior segment coordination sharing module provides behavior segment as input for fragment key word mapping submodule, fragment key word mapping submodule utilizes the hash functions such as SHA-1 to do to behavior segment the cryptographic hash that Hash calculation is fixed length, namely the value of behavior segment in the keyword space of DHT (distributed hashtable), finally returns to behavior segment coordination sharing module by the key value of behavior segment.It is the input of routed keyword as key word route submodule that behavior segment and local statistical property thereof are assembled into the key word that message and behavior segment key word mapping submodule return by behavior segment coordination sharing module, and key word route submodule is that route is according to the memory node (collection analysis node) message of input being routed to (comprising behavior segment and local statistical property thereof) this key word responsible with key word.

As shown in Figure 10, main four steps of the interaction flow of distributed hashtable module and behavior segment coordination sharing module are as follows: 1.. and behavior segment issues representative behavior segment set that submodule selects module to export with behavior segment for input, does following process for each behavior segment in behavior segment set: first to map as the fragment key word that distributed hashtable module is called in input using behavior fragment and obtain key value corresponding to behavior fragment; Then behavior in all Malware samples of this node of adding up contains the local statistical properties such as the sample number of behavior fragment behavior; Finally by the behavior fragment and local statistical property be assembled into DHT message, and map the key value obtained forwards this information to this routed keyword responsible node as the key word routing function calling distributed hashtable module for routed keyword using key word.2.. behavior segment receives submodule and receives the behavior segment information that distributed hashtable module is routed to this node, it is the expansion that traditional DHT route messages receives that behavior segment receives submodule, except traditional DHT route messages receiving function, go back the source node address that recording messages issues this message.3.. every a time period, receive route messages (comprising behavior segment and local statistical property thereof) and message source publisher node address that submodule receives for input with behavior segment, add up from global properties such as the number of samples (the local sample number that each publisher node accumulative comprises behavior fragment behavior obtains) comprising behavior fragment in total system of the same behavior fragment of different node and the source node numbers of adding up issue behavior fragment.4.. behavior segment global property returns submodule using the source node address set of behavior segment obtained in the previous step, corresponding global property and issue behavior fragment as input, the global property of behavior segment and correspondence is assembled into return messages, then direct each address these return messages sent in source node address set.The behavior segment coordination sharing module of each collection analysis node in the present embodiment and distributed hashtable module cooperative constitute a nerve of a covering shared towards behavior segment, the representative behavior segment set of himself is issued by source node, be responsible for the global property of destination node statistics from the not behavior segment the same with behavior fragment of homology publisher node of the corresponding key value of behavior segment, then behavior segment and global property are returned to source publisher node, last each node all obtains the representative behavior segment of the Malware sample himself collected at system-wide global property.The present embodiment is by carrying out statistics global property to different collection analysis nodes different behavior fragment allocation, and after statistics, return to the source collection analysis node issuing behavior segment, namely the distributed arithmetic of statistics global property is achieved, make full use of the computational resource of whole collection analysis node, communication when can reduce statistics global property and Calculation bottleneck, raising the present embodiment are analyzing the combination property of Malware.

Behavior segment coordination sharing module construction is on distributed hashtable module, call distributed hashtable module and issue local representative behavior segment set, the identical behavior segment realized from different node is brought to same Nodes, different behavior segments is brought to different Nodes, realizes load balancing, dispersion, the extendible identical behavior segment gathered from different node; Then in the gather round (the collection analysis node of corresponding behavior segment) of behavior segment, behavior segment is carried out to the statistics of global property; Finally behavior segment and global property thereof are returned to the set of node having issued behavior fragment; Net result is that each node obtains the representative behavior segment of the Malware sample himself collected at system-wide global property.Behavior segment coordination sharing module comprises: behavior segment issues submodule, behavior segment receives submodule, behavior segment statistics submodule, behavior segment global property return submodule and behavior segment global property reception submodule.Behavior segment is issued submodule and is selected the representative behavior segment set of module output for input with behavior segment, following process is done for each behavior segment in behavior segment set: first obtain key value corresponding to behavior fragment using behavior fragment as the fragment key word mapping submodule that distributed hashtable module is called in input, then behavior in all Malware samples of this node of adding up contains the local statistical properties such as the sample number of behavior fragment behavior, finally using the behavior fragment and the key value that returns of the local statistical property fragment key word mapping submodule that is assembled into DHT message Hash table module in a distributed manner as routed keyword is as the input of the key word route submodule of distributed hashtable module, the node forwarding this information to this routed keyword responsible is completed by the key word route submodule of distributed hashtable module.Behavior segment receives the expansion that submodule is traditional DHT route messages receiver module, and this submodule goes back the source node address that recording messages issues this message except the route messages receiving function of traditional DHT route messages receiver module.Behavior segment statistics submodule receives route messages (comprising behavior segment and local statistical property thereof) and message source publisher node address that submodule receives for input with behavior segment, adds up from global properties such as the number of samples (the local sample number that each publisher node accumulative comprises behavior fragment behavior obtains) comprising behavior fragment in total system of the same behavior fragment of different node and the source node numbers of adding up issue behavior fragment.Behavior segment global property returns the source node address set of behavior segment, corresponding global property and the issue behavior fragment that submodule exports using behavior segment statistics submodule as input, the global property of behavior segment and correspondence is assembled into return messages, then direct each address these return messages sent in source node address set.It is a simple service end socket interface that behavior segment global property receives submodule, receives other node and sends to the return messages of this service end socket interface (comprising behavior segment and corresponding global property).The behavior segment coordination sharing module of each node in distributed malware analysis system and distributed hashtable module cooperative constitute a nerve of a covering shared towards behavior segment, the representative behavior segment set of himself is issued by source node, be responsible for the global property of destination node statistics from the not behavior segment the same with behavior fragment of homology publisher node of the corresponding key value of behavior segment, then behavior segment and global property are returned to source publisher node, finally this node all obtains the representative behavior segment of the Malware sample himself collected at system-wide global property.

As is illustrated by figs. 11 and 12, in the present embodiment, Malware sample characterization module comprises:

Behavior segment weight computing submodule, for calculating the weights of each behavior segment according to formula (3);

Malware sample properties vector characterizes submodule, for building the eigen vector of Malware sample according to the weights of each behavior segment in Malware sample; The dimension of eigen vector is the arrangement of behavior segment set according to a definite sequence of collection analysis node this locality, the value of the eigen vector dimension of any Malware sample is determined by following two kinds of situations: if having the behavior of the behavior segment representated by this dimension in the sample behavior of Malware sample, then value equals the weights of behavior fragment; Otherwise value equals zero.

Suppose that the global property of behavior segment Ti is as follows: the nodes sharing behavior fragment is N _i, the sample number with behavior fragment behavior is S _i.Then behavior segment weight computing submodule calculates the global property weights V of behavior segment according to formula (3) _i:

$V_i=(N_i+\log S_i)\log \frac{1}{F_i}---\left(3\right)$

In formula (3), N _i, S _i, F _ithe global property of constituting action fragment, N _ifor sharing the collection analysis nodes of behavior fragment in whole network to distributed hashtable module, S _ifor there is the Malware sample number of behavior fragment behavior, F _ifor the frequency that behavior segment occurs in normal procedure, V _ifor the weights of behavior segment.This formula meets 3 principles of fragment global property weights, namely shares nodes N _ilarger, weights are higher; There is the sample number S of behavior fragment _imore, weights are higher; The frequency F occurred in normal procedure behavior _ilower, weights are higher.

The global property of behavior segment weight computing submodule Behavior-based control fragment calculates the global property weights of behavior fragment, and fragment global property weights meet following principle: 1) higher by the behavior segment global property weights that more multinode is shared; 2) sample number comprising behavior fragment behavior in distributed malware analysis system is more, and behavior segment global property weights are higher; 3) frequency occurred in normal procedure behavior is lower, and behavior segment global property weights are higher.

In the present embodiment, the local cluster module of Malware sample comprises:

Eigen vector distance calculating sub module, for calculate any two Malware samples eigen vector between distance, the less similarity then between two Malware samples of distance is larger;

The local cluster submodule of Malware sample, for according to the distance between the eigen vector of Malware sample, adopts and does not need in advance to know that the clustering algorithm of clusters number carries out cluster to each Malware sample and obtains cluster set;

Each cluster in the cluster set that the local cluster submodule of Malware sample local analytics module walks Malware sample exports, extract the feature of joint act as this cluster of all Malware samples in this cluster, add up the network attribute of all Malware samples in this cluster and the local behavior property attribute as cluster simultaneously, the feature of cluster and the attribute analysis result as Malware is exported.

The above is only the preferred embodiment of the present invention, protection scope of the present invention be not only confined to above-described embodiment, and all technical schemes belonged under thinking of the present invention all belong to protection scope of the present invention.It should be pointed out that for those skilled in the art, some improvements and modifications without departing from the principles of the present invention, these improvements and modifications also should be considered as protection scope of the present invention.

Claims (10)

1. the characteristic of malware clustering method that Behavior-based control fragment is shared, is characterized in that implementation step is as follows:

1) the collection analysis node of geographic position dispersion is arranged in a network respectively, each collection analysis node is responsible for collection and the analysis of Malware sample in a slice network area, sets up the distributed hashtable module for building distributed hashtable in collection analysis node;

2) behavior of the Malware sample of collection is divided into multiple behavior segment by described collection analysis node;

3) described collection analysis node obtains the local statistical property of behavior segment, behavior segment and local statistical property thereof are shared to distributed hashtable module, by distributed hashtable module, behavior segment and local statistical property thereof is stored to distributed hashtable; The global property statistics of different behavior segment is responsible for by different collection analysis nodes, described collection analysis node respectively by all local statistical property statistics of the described behavior segment stored in distributed hashtable obtain the global property of responsible behavior segment, and return the behavior segment set with global property to each collection analysis node issuing described behavior segment;

4) after receiving the behavior segment set with global property, described collection analysis node calculates the weights of each behavior segment in behavior segment set based on the global property of behavior segment, and builds the eigen vector of described Malware sample according to the weights of each behavior segment in Malware sample;

5) described collection analysis node carries out cluster according to the distance between the eigen vector of Malware sample to each Malware sample and obtains cluster set, analyze each cluster in cluster set, extract the feature of this cluster and attribute and export as the Comprehensive analysis results of described Malware.

2. Behavior-based control fragment according to claim 1 share characteristic of malware clustering method, it is characterized in that: described step 2) detailed step as follows;

2.1) behavior of Malware sample is considered as the behavior sequence that order performs, from the behavior sequence that described order performs, selects the Continuous behavior subsequence of regular length as splitting the behavior segment obtained; Or set up behavior dependency graph according to the dependence between the behavior service data of Malware sample, from described behavior dependency graph, select the behavior obtaining fixed vertices number to rely on subgraph as splitting the behavior segment obtained;

2.2) adopt heuristic rule housing choice behavior fragment, obtain the behavior segment set comprising many group fixed qty behavior segments;

2.3) through type (1) calculates the local characteristics weights of each behavior segment;

In formula (1), F _ifor behavior segment t _ithe frequency occurred in normal procedure behavior, H _ifor comprising the ratio of behavior fragment in Malware sample; W (t _i) be behavior segment t _ilocal characteristics weights; t _ibe i-th behavior segment;

2.4) the local characteristics weights of Behavior-based control fragment build the local characteristics vector of Malware sample, the dimension of local characteristics vector is the arrangement of behavior segment according to a definite sequence, the value of certain dimension of local characteristics vector of a certain Malware sample is determined by following two kinds of situations: if having the behavior of the behavior segment representated by this dimension in the behavior of Malware sample, then described value equals the local characteristics weights of behavior fragment; Otherwise then described value equals zero;

2.5) through type (2) calculates the similarity of each group behavior segment and original Malware sample in behavior segment set, the one group of behavior segment representatively behavior segment selecting similarity maximum from described behavior segment set; Meanwhile, if select number of times to reach preset times time and the similarity that behavior segment set exists one group of behavior segment and original Malware sample is more than or equal to 0.8, then stop selection and by described behavior segment representatively behavior segment;

In formula (2), gram_set represents all characteristic sets that original sample is possible, rep is one group of behavior segment in behavior segment set, orig is original Malware sample, similar (rep, orig) the Similarity measures result between one group of behavior segment rep in the set of expression behavior segment and original Malware sample orig, gram_i is that in rep, sequence number is the behavior segment of i, gram_j is that in gram_set, sequence number is the behavior segment of j, the local characteristics weight computing function that w (gram_i) is behavior segment gram_i, the local characteristics weight computing function that w (gram_j) is behavior segment gram_j,

2.6) export and describedly represent behavior segment.

3. Behavior-based control fragment according to claim 1 share characteristic of malware clustering method, it is characterized in that, described step 3) detailed step as follows:

3.1) described collection analysis node calls the key value that distributed hashtable module obtains described behavior segment;

3.2) in described this collection analysis of collection analysis node statistics node all Malwares sample, behavior comprises the local statistical property of the sample number of described behavior segment;

3.2) described behavior segment and local statistical property thereof are encapsulated as distributed hashtable message by described collection analysis node, then described key value and distributed hashtable message are sent to distributed hashtable module, and the collection analysis node address of behavior segment issued in record; Described distributed hashtable module searches the collection analysis node of this key value responsible according to key value, and is stored to the collection analysis node being responsible for this key value by distributed hashtable Message routing;

3.3) different collection analysis nodes is responsible for the global property statistics of different behavior segment, described collection analysis node returns the distributed hashtable message of be responsible for behavior segment respectively according to collection analysis node address and distributed hashtable module, statistics is included in distributed hashtable the Malware number of samples of the same behavior fragment comprised from different acquisition analysis node, issue the collection analysis number of nodes of behavior fragment at interior global property, behavior segment and global property thereof are assembled into return messages and return to the collection analysis node address issuing behavior segment, the collection analysis node that described collection analysis node address is corresponding receives the behavior segment set with global property that each behavior segment and global property thereof obtain.

4. Behavior-based control fragment according to claim 1 share characteristic of malware clustering method, it is characterized in that, described step 4) detailed step as follows:

4.1) after receiving the behavior segment set with global property, described collection analysis node calculates the weights of each behavior segment in described behavior segment set according to formula (3);

In formula (3), N _i, S _i, F _ithe global property of constituting action fragment, N _ifor sharing the collection analysis nodes of behavior fragment in whole network to distributed hashtable module, S _ifor there is the Malware sample number of behavior fragment behavior, F _ifor the frequency that behavior segment occurs in normal procedure, V _ifor the weights of behavior segment;

4.2) described collection analysis node builds the eigen vector of described Malware sample according to the weights of each behavior segment in Malware sample; The dimension of described eigen vector is the arrangement of behavior segment set according to a definite sequence of collection analysis node this locality, the value of the eigen vector dimension of any Malware sample is determined by following two kinds of situations: if having the behavior of the behavior segment representated by this dimension in the sample behavior of Malware sample, then value equals the weights of behavior fragment; Otherwise value equals zero.

5. according to the Behavior-based control fragment in Claims 1 to 4 described in any one share characteristic of malware clustering method, it is characterized in that, described step 5) detailed step as follows:

5.1) distance between the eigen vector of any two Malware samples of described collection analysis node calculate, the less similarity then between two Malware samples of distance is larger;

5.2) described collection analysis node is according to the distance between the eigen vector of Malware sample, adopts and does not need in advance to know that the clustering algorithm of clusters number carries out cluster to each Malware sample and obtains cluster set;

5.3) each cluster in cluster set described in described collection analysis node traverses, extract the feature of joint act as this cluster of all Malware samples in this cluster, add up the network attribute of all Malware samples in this cluster and the local behavior property attribute as cluster simultaneously, the feature of described cluster and the attribute analysis result as described Malware is exported.

6. the characteristic of malware cluster analysis system that a Behavior-based control fragment is shared, it is characterized in that, comprise multiple collection analysis node for gathering Malware and analyzing, the geographic position dispersion in a network of described collection analysis node, each collection analysis node is responsible for collection and the analysis of Malware sample in a slice network area, and described collection analysis node comprises:

Behavior segment segmentation module, the behavior for the Malware sample by collection is divided into multiple behavior segment;

Distributed hashtable module, for building distributed hashtable;

Behavior segment coordination sharing module, for obtaining the local statistical property of behavior segment, behavior segment and local statistical property thereof are shared to distributed hashtable module by behavior segment coordination sharing module, by distributed hashtable module, behavior segment and local statistical property thereof is stored to distributed hashtable; The global property of behavior segment is obtained by all local statistical property statistics of the described behavior segment stored in distributed hashtable, and return the behavior segment set with global property to each collection analysis node issuing described behavior segment, the global property statistics of the different behavior segment of behavior segment coordination sharing module in charge of different collection analysis nodes;

Malware sample characterization module, global property for Behavior-based control fragment after receiving the behavior segment set with global property calculates the weights of each behavior segment in behavior segment set, and builds the eigen vector of described Malware sample according to the weights of each behavior segment in Malware sample;

The local cluster module of Malware sample, obtains cluster set for carrying out cluster according to the distance between the eigen vector of Malware sample to each Malware sample;

Malware sample local analytics module, for analyzing each cluster in cluster set, extracting the feature of this cluster and attribute and exporting as the Comprehensive analysis results of described Malware.

7. the characteristic of malware cluster analysis system that Behavior-based control fragment according to claim 6 is shared, is characterized in that, described behavior segment segmentation module is that Continuous behavior subsequence segmentation module or behavior rely on dividing sub-picture module; The behavior of Malware sample is considered as the behavior sequence that order performs by described Continuous behavior subsequence segmentation module, selects the Continuous behavior subsequence of regular length as splitting the behavior segment obtained from the behavior sequence that described order performs; Rely on dividing sub-picture module and set up behavior dependency graph according to the dependence between the behavior service data of Malware sample, from described behavior dependency graph, select the behavior obtaining fixed vertices number to rely on subgraph as splitting the behavior segment obtained; Described behavior segment segmentation module also selects module that behavior segment is distributed to behavior segment coordination sharing module by behavior segment, and described behavior segment selects module to comprise:

Behavior segment chooser module, comprises the behavior segment set of many group fixed qty behavior segments for adopting heuristic rule housing choice behavior fragment to obtain;

Behavior segment representativeness checking submodule, the local characteristics weights of each behavior segment are calculated for through type (1), the local characteristics weights of Behavior-based control fragment build the local characteristics vector of Malware sample, the dimension of local characteristics vector is the arrangement of behavior segment according to a definite sequence, the value of certain dimension of local characteristics vector of a certain Malware sample is determined by following two kinds of situations: if having the behavior of the behavior segment representated by this dimension in the behavior of Malware sample, then described value equals the local characteristics weights of behavior fragment; Otherwise then described value equals zero; Through type (2) calculates the similarity of each group behavior segment and original Malware sample in behavior segment set, the one group of behavior segment representatively behavior segment selecting similarity maximum from described behavior segment set; Meanwhile, if select number of times to reach preset times time and the similarity that behavior segment set exists one group of behavior segment and original Malware sample is more than or equal to 0.8, then stop selection and by described behavior segment representatively behavior segment; And the described behavior segment that represents is distributed to behavior segment coordination sharing module;

In formula (1), F _ifor behavior segment t _ithe frequency occurred in normal procedure behavior, H _ifor comprising the ratio of behavior fragment in Malware sample; W (t _i) be behavior segment t _ilocal characteristics weights; t _ibe i-th behavior segment;

In formula (2), gram_set represents all characteristic sets that original sample is possible, rep is one group of behavior segment in behavior segment set, orig is original Malware sample, similar (rep, orig) the Similarity measures result between one group of behavior segment rep in the set of expression behavior segment and original Malware sample orig, gram_i is that in rep, sequence number is the behavior segment of i, gram_j is that in gram_set, sequence number is the behavior segment of j, the local characteristics weight computing function that w (gram_i) is behavior segment gram_i, the local characteristics weight computing function that w (gram_j) is behavior segment gram_j.

8. the characteristic of malware cluster analysis system that Behavior-based control fragment according to claim 6 is shared, it is characterized in that, described behavior segment coordination sharing module comprises:

Behavior segment issues submodule, the key value of described behavior segment is obtained for calling distributed hashtable module, behavior in this collection analysis node all Malwares sample of adding up comprises the local statistical property of the sample number of described behavior segment, described behavior segment and local statistical property thereof are encapsulated as distributed hashtable message, then by described key value and distributed hashtable news release to distributed hashtable module;

Behavior segment receives submodule, for receiving the distributed hashtable message that distributed hashtable module returns and the collection analysis node address recording issue behavior segment;

Behavior segment statistics submodule, for after receiving the behavior segment set with global property according to the distributed hashtable message that collection analysis node address and distributed hashtable module return, statistics is included in distributed hashtable the Malware number of samples of the same behavior fragment comprised from different acquisition analysis node, issues the global property of the collection analysis number of nodes of behavior fragment, and the behavior segment statistics submodule of different collection analysis nodes is responsible for the global property statistics of different behavior segment;

Behavior segment global property returns submodule, returns to for behavior segment and global property thereof being assembled into return messages the collection analysis node address issuing behavior segment;

Behavior segment global property receives submodule, behavior segment global property for receiving other collection analysis nodes returns each behavior segment and the global property thereof of submodule transmission, obtains the behavior segment set with global property comprising the behavior segment that this collection analysis node is issued;

Described distributed hashtable module comprises:

Behavior segment key word mapping submodule, carries out for utilizing the behavior segment of hash function to input the key value of cryptographic hash as behavior segment that Hash calculation is fixed length;

Key word route submodule, for being stored into according to the key value of input the Hash routing table Message routing comprising behavior segment and local statistical property thereof the collection analysis node of this key value responsible.

9. the characteristic of malware cluster analysis system that Behavior-based control fragment according to claim 6 is shared, it is characterized in that, Malware sample characterization module comprises:

Behavior segment weight computing submodule, for calculating the weights of each behavior segment according to formula (3);

Malware sample properties vector characterizes submodule, for building the eigen vector of described Malware sample according to the weights of each behavior segment in Malware sample; The dimension of described eigen vector is the arrangement of behavior segment set according to a definite sequence of collection analysis node this locality, the value of the eigen vector dimension of any Malware sample is determined by following two kinds of situations: if having the behavior of the behavior segment representated by this dimension in the sample behavior of Malware sample, then value equals the weights of behavior fragment; Otherwise value equals zero;

In formula (3), N _i, S _i, F _ithe global property of constituting action fragment, N _ifor sharing the collection analysis nodes of behavior fragment in whole network to distributed hashtable module, S _ifor there is the Malware sample number of behavior fragment behavior, F _ifor the frequency that behavior segment occurs in normal procedure, V _ifor the weights of behavior segment.

10. according to the characteristic of malware cluster analysis system that the Behavior-based control fragment in claim 6 ~ 9 described in any one is shared, it is characterized in that, the local cluster module of described Malware sample comprises:

Eigen vector distance calculating sub module, for calculate any two Malware samples eigen vector between distance, the less similarity then between two Malware samples of distance is larger;

The local cluster submodule of Malware sample, for according to the distance between the eigen vector of Malware sample, adopts and does not need in advance to know that the clustering algorithm of clusters number carries out cluster to each Malware sample and obtains cluster set;

Each cluster in the cluster set that the local cluster submodule of Malware sample described in described Malware sample local analytics module walks exports, extract the feature of joint act as this cluster of all Malware samples in this cluster, add up the network attribute of all Malware samples in this cluster and the local behavior property attribute as cluster simultaneously, the feature of described cluster and the attribute analysis result as described Malware is exported.