US7464089B2 - System and method for processing a data stream to determine presence of search terms - Google Patents

System and method for processing a data stream to determine presence of search terms Download PDF

Info

Publication number
US7464089B2
US7464089B2 US11/208,222 US20822205A US7464089B2 US 7464089 B2 US7464089 B2 US 7464089B2 US 20822205 A US20822205 A US 20822205A US 7464089 B2 US7464089 B2 US 7464089B2
Authority
US
United States
Prior art keywords
search term
data
data stream
matrix
character
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related, expires
Application number
US11/208,222
Other versions
US20060050968A1 (en
Inventor
Jintae Oh
Ilsup Kim
Hojae Lee
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Connect Technologies Corp
Original Assignee
Connect Technologies Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Connect Technologies Corp filed Critical Connect Technologies Corp
Priority to US11/208,222 priority Critical patent/US7464089B2/en
Publication of US20060050968A1 publication Critical patent/US20060050968A1/en
Assigned to WINNOW TECHNOLOGIES INC. reassignment WINNOW TECHNOLOGIES INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: WINNOW TECHNOLOGY, LLC
Assigned to CONNECT TECHNOLOGIES CORPORATION reassignment CONNECT TECHNOLOGIES CORPORATION NUNC PRO TUNC ASSIGNMENT (SEE DOCUMENT FOR DETAILS). Assignors: WINNOW TECHNOLOGIES, INC.
Application granted granted Critical
Publication of US7464089B2 publication Critical patent/US7464089B2/en
Adjusted expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/903Querying
    • G06F16/90335Query processing
    • G06F16/90344Query processing by using string matching techniques
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00Data processing: database and file management or data structures
    • Y10S707/99931Database or file accessing
    • Y10S707/99933Query processing, i.e. searching
    • Y10S707/99936Pattern matching access
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00Data processing: database and file management or data structures
    • Y10S707/99941Database schema or data structure
    • Y10S707/99942Manipulating data structure, e.g. compression, compaction, compilation

Definitions

  • This invention relates generally to content searching engines and, more particularly, to a content searching engine using trap matrix elements.
  • the invention is a method and system for using a trap matrix to search the entire contents of a data stream for a pattern that matches the pattern for a search term.
  • the method and system can proceed to an exact match operation.
  • the present invention generates a pointer matrix and a corresponding active control matrix according to a set of terms in a rule table.
  • Data is sequenced through the trap matrix according to the hierarchy of its trap elements.
  • the trap elements perform a pattern match check, and also preferably perform a character match check, between the sequenced data stream and any search term in the set of terms in the rule table. Results from a positive pattern match, and any corresponding positive character match, are preferably communicated from the matching trap element to an exact match lookup.
  • FIG. 1 illustrates a schematic diagram view of a content searching engine according to the present invention
  • FIG. 2 illustrates a schematic diagram view of a preferred embodiment of the trap matrix according to the preferred embodiment of the present invention
  • FIG. 4 illustrates a block flow diagram of the operation of the content searching engine according to the present invention
  • FIG. 5 illustrates a schematic diagram view of an alternative embodiment of the present invention
  • FIGS. 6A-6C illustrate a detailed block flow diagram of the process for generating the pointer matrix and active control matrix according to the rule table and in accordance with the preferred embodiment of the present invention
  • FIGS. 7A-7C illustrate a detailed block flow diagram of the processes for pattern matching, character matching, and the exact match lookup in accordance with the preferred embodiment of the present invention
  • FIG. 8 illustrates a schematic diagram view of a scaled embodiment of the present invention.
  • FIGS. 9 and 10 illustrate schematic diagram views of the content searching engine integrated into an intrusion detection system according to the present invention.
  • FIG. 1 illustrates a diagrammatic view of a content searching engine 10 according to the present invention.
  • the content searching engine 10 has a trap matrix 12 in communication with an input device 14 , an exact match lookup 16 and a rule table 18 .
  • the content searching engine 10 also includes a buffer memory 20 between the input device 14 and the exact match lookup 16 and may also include an optional case control module 22 between the input device 14 and the trap matrix 12 .
  • the system of the content searching engine 10 operates on a data stream 24 that is communicated through the input device 14 .
  • the rule table 18 includes a set of terms 26 and a set of table addresses 28 .
  • Each of the terms in the set of terms 26 is stored in the rule table 18 at a location 30 defined by the set of table addresses 28 .
  • the term “get passwd file” and any corresponding prefix, such as “http” that may be used for a protocol is stored at a location defined by table addresses 0 - 18 .
  • Each successive term, along with any corresponding prefix can immediately follow the preceding term at the next location, with the next location being defined by the next table address.
  • the term following “get passwd file” could begin immediately after the last character in the preceding term, at address 19 .
  • the set of terms 26 may include as few as a single term.
  • the terms 26 have a number of characters 32 in a number of lengths 34 and combinations 36 .
  • a term following “get passwd file” could be “get pwl file type” and may also have a prefix.
  • the first several characters (“get p”) of both terms are identical, but the subsequent characters are different and represent a different character combination.
  • the lengths of the terms are different; the length of the term “get passwd file” is fifteen (15) characters, whereas the length of the term “get pwl file type” is seventeen (17) characters. In the case where the rule table 18 has a single term, such term would have an individual character length and an individual character combination.
  • the content searching engine 10 also includes a pointer matrix 38 that is defined by the terms 26 and addresses 28 in the rule table 18 .
  • the pointer matrix 38 includes a set of one-to-one pointers 40 that are particularly defined by table addresses 28 for each one of the terms 26 in the rule table 18 .
  • the pointer matrix 38 has row and column coordinates 42 that are defined according to character lengths 44 and character combinations 46 .
  • the character lengths 44 and character combinations 46 correspond with the set of terms 26 according to the lengths of the characters 34 and the combinations of the characters 36 , respectively. Therefore, for each one of the terms 26 in the rule table 18 , the pointer matrix 38 stores the corresponding table addresses 28 at the row and column coordinates 42 according to the lengths 32 and combinations 34 of the characters in each one of the particular terms 26 .
  • the address identifying the term “get pwl file type” (“ 70 ” in Table 1 above) would be stored as the term's corresponding one-to-one pointer 40 in the pointer matrix 38 .
  • the particular address (“ 70 ”) is stored in and can be read out from the pointer matrix 38 according to the rows and columns 42 that also correspond with the length of the characters in each of the terms 26 and the combination of the characters in each of the terms 26 .
  • the set of one-to-one pointers 40 correlate the table addresses 28 for each one of the terms 26 in the rule table 18 .
  • the character combinations 46 used for storing the pointers 40 in the pointer matrix 38 are generally defined by a set of patterns that uniquely correspond with the set of terms 26 according to the characters within each one of the terms 26 .
  • the set of patterns is a set of compressed character values that is produced by compressing the characters in each one of the terms 26 according to a numerical operation and a truncation operation. It will be appreciated that terms 26 with different character lengths may have the same compressed character value because at least one of the row and column coordinates 42 for two terms is different due to the different character lengths.
  • the compressed value for one of the terms 26 may be calculated based on a fewer number of characters and the pointer address would be stored according to the shortened character length to ensure that the set of pointers 40 has a one-to-one correspondence with each one of the terms 26 .
  • the set of patterns uniquely corresponds with the character combinations for those each one of the terms 26 having the particular character length. Given that the set of terms 26 may have a single term, the row and column coordinates 42 for such a term would be the length of the term's characters and the pattern resulting from the term's characters.
  • the trap matrix 12 includes a set of trap elements 48 and a corresponding set of delay elements 50 .
  • Each of the trap elements 48 has a memory 52 in communication with a corresponding comparator 54 and are generally referred to as memory-comparator pairs 56 for each of the trap elements 48 .
  • the trap elements 48 are in multiplexed communication with the input device 14 and have a hierarchy ( 1 to N) according to the set of delay elements 50 .
  • the hierarchy of the trap elements 48 particularly including the memory-comparator pairs 56 , is preferably in a one-to-one correspondence with the character lengths 34 of the set of terms 26 .
  • the trap matrix 12 has an increasing series of delay elements 50 between each of the trap elements 48 and the input device 14 .
  • the increasing series of delay elements 50 define an increasing order in the hierarchy of the trap elements 48 , and the increasing order of hierarchy corresponds with an increasing character length of the data stream 24 . Therefore, with each increasing order, the trap elements are able to examine the character patterns of increasing lengths of the data stream 24 .
  • the input device 14 receives the data stream 24 during a time period 58 as illustrated in Table 2 below.
  • the data stream 24 includes a set of data characters 60 having a length 62 and a combination 64 , and the time period 58 is made up of clock cycles 66 .
  • the input device 14 also communicates the data stream 24 to the buffer memory 20 through a bypass 68 .
  • the trap matrix 12 receives the data stream 24 from the input device 14 and sequences the set of characters 60 in the data stream 24 through each of the trap elements 48 .
  • the sequenced set of data characters enter the memory-comparator pairs 56 as a set of sequenced data 70 according to the hierarchy of the trap elements 48 .
  • the memory-comparator pairs 56 perform a pattern match check between patterns (P) of the sequenced data 70 and the set of patterns defined by the combination of characters 36 for each one of the terms 26 in the rule table 18 .
  • Each of the memory-comparator pairs 56 simultaneously performs the pattern match check according to the one-to-one correspondence between the hierarchy of the trap elements 48 and the character lengths 34 of the set of terms 26 .
  • a positive pattern match by one of the matching memory-comparator pairs particularly defines a particular one-to-one pointer 72 that contains one of the addresses 28 of one of the terms 26 in the rule table 18 .
  • the particular one-to-one pointer 72 has a row and column coordinate 74 according to the hierarchy of the matching trap element (k) and the pattern (P) of the sequenced data 70 in the matching trap element (P[k]).
  • the exact match lookup 16 communicates with the buffer memory 20 , trap matrix 12 and the rule table 18 .
  • the exact match lookup 16 receives the particular one-to-one pointer 72 from the pointer matrix 38 .
  • the exact match lookup 16 retrieves one of the terms 26 from the rule table 18 according to the table address corresponding with the one-to-one pointer 72 from the pointer matrix 38 .
  • the exact match lookup 16 examines the retrieved term with the portion of the data stream 24 from the buffer memory 20 that corresponds with the time period according to the matching memory-comparator pair. With the exact term retrieved from the rule table 18 and the corresponding portion of the data stream 24 from the buffer memory 20 , the exact match lookup checks for an exact match therebetween.
  • each of the terms 26 in the rule table 18 can be defined according to a set of compressed character values.
  • each one of the terms 26 can be compressed by expressing each character in a term according to a defined value and performing a set of operations on the set of values.
  • the operation may be a numerical and/or logical operation, such as summation, subtraction, multiplication, division, XOR, XNOR, and concatenation.
  • the set of operations may be a single numerical operation, such as summation, performed on each character in each of the terms 26 . It is also possible that the set of operations may be multiple operations.
  • the summation operation exemplifies the compression of the terms 26 .
  • the hexadecimal (0x) representation of the characters in the term “get passwd” can be summed into a compressed character value of 0x3E2.
  • this preferred method of compression is a one-way compression according to Equation 1 below because there is not a one-to-one correspondence between the compressed character value and the term being compressed.
  • Equation 1 the number of characters in the term increase, the number of other combinations of characters that would also have the same compressed character value increases exponentially. Therefore, once the term is compressed according to the sum of the character values (or any other operation on the character values), the compressed character value cannot be expanded back into the term as a one-to-one representation. Instead of being a one-to-one representation of the term, the compressed character value is a probability of the term.
  • the compressed character value for the 10-character term “get passwd” is 0x3E2 (i.e., ⁇ “get passwd” ⁇ 0x3E2)
  • the compressed character value for the 10-character term “got pissed” is also 0x3E2 (i.e., ⁇ “got pissed” ⁇ 0x3E2). Therefore, it is evident that, by itself, 0x3E2 is not a one-to-one representation of “get passwd”.
  • the compressed character value may be further compressed by removing the most significant bits (MSB) and thereby truncating the summation total to its least significant bits (LSB) according to Equation 2 below.
  • MSB most significant bits
  • LSB least significant bits
  • the most significant hexadecimal bit of 0x3 in the summation total of 0x3E2 can be removed, thereby truncating the summation total and resulting in a truncated compressed character value of 0xE2. Therefore, the term “get passwd” can be compressed and checked with the sequenced data stream 70 based on its character length of 10 and its pattern value of 0xE2.
  • the trap matrix 12 sequences the data stream 24 through the trap elements 48 , and the memory-comparator pairs 56 perform the pattern match check between patterns (P) of the sequenced data 70 and the set of patterns defined by each one of the terms 26 in the rule table 18 . Accordingly, the trap matrix 12 performs the same set of operations on the data stream 24 that is performed on the terms 26 in the rule table 18 . This allows for each one of the trap elements 48 to simultaneously perform the pattern match check between the sequenced data 70 and each one of the terms 26 according to each of the hierarchies in the trap matrix 12 .
  • the trap matrix 12 includes a set of compression operators 76 located between the input and each one of the memory-comparator pairs 56 .
  • the summation operation 78 is performed in the set of operations, adding each current character 80 with a set of prior summed characters 82 according to the hierarchy of the trap elements 48 , based on the series of delays 50 through which the data 24 is sequenced.
  • sequenced data 70 from each summation 78 is preferably truncated in an LSB operation 84 .
  • the one-to-one pointers 40 in the pointer matrix 38 can then be examined based on the pattern of the summed and truncated sequenced data 70 as it is communicated from the input device 14 to each of the trap elements 48 as discussed above.
  • each one of the comparators 54 can query the pointer matrix 38 at the row and column coordinate corresponding with the hierarchy of the trap elements 48 and with the pattern of the summed and truncated sequenced data 70 in each one of the corresponding memories 52 .
  • the corresponding comparator 54 identifies the positive pattern match and the content searching engine 10 can proceed to the exact match lookup 16 .
  • the first table address in the rule table 18 may be a zero value and that the first table address does not need to be used for the first character of any of the terms 26 .
  • the one-to-one pointers 40 can be set up to identify the last character in the term (which may be non-zero, even when the first term begins as the first table address in the rule table 18 ).
  • the memories 52 in each one of the trap elements 48 can be the size for the sequenced data 70 , regardless of hierarchy, due to the compression and truncation of the characters in the data stream 24 .
  • the compressed and truncated sequenced data 70 can be stored in a single byte ([7:0]), which is between 0 and FF in hexadecimal format and between 0 and 255 in decimal format (between 00000000 and 11111111 in binary format).
  • the memories 52 may be larger, such as being sized to accommodate at least one additional bit for case control and additional size for a prefix, such as a header in a data packet.
  • an active control matrix 86 may be used to perform the pattern match check between the sequenced data 70 and the terms 26 .
  • the active control matrix 86 is very similar to the pointer matrix 38 .
  • the active control matrix 86 uses the same row and column coordinates 42 as the pointer matrix 38 , including the definitions for the row and column coordinates 42 (i.e., identical row and column coordinate system defined according to the character lengths 44 and the character combinations 46 ). The difference between the active control matrix 86 and the pointer matrix 38 is in their contents.
  • the pointer matrix 38 contains the set of addresses 28 for the terms 26 in the rule table 18 .
  • the active control matrix 86 contains a set of flags 88 , such as a set of binary numbers having one and zero bits. Therefore, the set of flags 88 in the active control matrix 86 correspond with the addresses in the pointer matrix 38 according to Equation 3 below. In general, the set of flags 88 identify whether there is a pointer 72 in the pointer matrix 38 at each corresponding row and column coordinate 42 .
  • the pattern match is first identified by one of the flags 88 in the active control matrix 86 , and the pointer 72 for the term with the corresponding pattern and length may then be read from the same row and column coordinate 42 in the pointer matrix 38 as the matching flag in the active control matrix 86 .
  • any one of the trap elements 48 that identifies a positive pattern match then performs a character match check prior to proceeding to the exact match lookup 16 .
  • a character match check a pair of data characters 90 from the data stream 24 are compared with two uncompressed term characters 92 from a comparison character matrix 94 .
  • the uncompressed term characters 92 in the comparison character matrix 94 are character segments from each one of the terms 26 in the rule table 18 and are stored in the comparison character matrix 94 according to the length of the terms 34 .
  • the uncompressed term characters may be taken from any corresponding location in each of the terms 26 , and the trap matrix 12 can be designed accordingly.
  • the uncompressed term characters are the last two characters in each one of the terms 26
  • the trap matrix is designed such that the pair of data characters 90 are the last two characters from the data stream 24 that entered the trap elements 48 .
  • the pair of data characters 90 are in a multiplexed communication through a pair of communication paths 96 .
  • the pair of data characters 90 are not routed through a compression operator and are instead communicated through a delay element 98 before the communication paths 96 and through a delay element 100 between the communication paths 96 .
  • the last character and third to last character can be the characters to be compared in the character match check by using a delay of two clock cycles 66 , i.e., a pair of delay elements, for the delay element 100 between the communication paths 96 . It will be further appreciated that a single character can also be used for the character match check and more than two characters may also be used for the character match check.
  • the terms 26 in the rule table 18 can be sorted and stored based on their lengths 32 .
  • the rule table 18 can be virtually segmented into a set of N rule tables 102 , with all rules of a single length being stored in each one of the rule tables 104 in the set of rule tables 102 .
  • the lengths of the rules in the set of rule tables 102 may preferably range from a two-character length in the 2 nd rule table 106 to an N-character length in the N th rule table 108 and have a one-to-one correspondence with the number of trap elements 48 .
  • the N th rule table preferably contains terms having a 24-character length 110 . It will be appreciated that there may be terms in the rule table 18 with lengths significantly greater than twenty-four (24) characters and that it is possible to use pattern matching with these longer terms based on patterns generated using the 24-character length. For example, when the content searching engine 10 is incorporated into an intrusion detection system, the terms 26 would be based on intrusion detection rules, some of which are known to be greater than 150-characters in length.
  • the rule table 18 and corresponding pointer matrix 38 are defined at preparation step 210 , and the data stream 24 is communicated through the input 14 at input step 220 .
  • the buffer memory 20 is preferably a first-in-first-out (FIFO) memory for bypassing the trap matrix 12 with the multiplex bypassed data stream 68 .
  • the content searching engine 10 sequences the data stream 24 through the trap matrix 12 at process step 230 .
  • the data stream 24 is communicated in the multiplexed manner through the series of delay elements 50 and each one of the memories 52 according to the hierarchy of trap elements 48 .
  • Process step 240 defines the row and column coordinates 42 of the pointer matrix 38 that are queried 232 by each one of the trap elements 48 based on the sequenced data stream 70 .
  • Each one of the trap elements 48 performs the pattern match check 252 according to decision step 250 .
  • each one of the comparators 54 can query 232 the pointer matrix 38 by defining the row and column coordinate according to the hierarchy of the trap elements 48 and according to the pattern of the sequenced data 70 in each one of the corresponding memories 52 .
  • the content searching engine 10 proceeds to the exact match lookup 16 following the positive pattern match 254 .
  • the matching trap element defines the row and column coordinate for the one-to-one pointer, and it is this trapped one-to-one pointer that defines the table address for the potentially matched term in the rule table 18 .
  • the content searching engine continues to sequence data 234 according to process step 230 .
  • the comparators 54 can query 232 the active control matrix 86 at the row and column coordinate corresponding with the hierarchy of the trap elements 48 and with the pattern of the summed and truncated sequenced data 70 . Additionally, as discussed above with reference to the preferred embodiment, the comparators 54 may also perform the character match check 256 when there is a positive pattern match 254 . The combination of the pattern match check 252 and the character match check 256 significantly reduces any chance for a false positive based on the patterns in the compressed and truncated data stream 70 . Therefore, to proceed to the exact match lookup 16 , the preferred embodiment requires both the positive pattern match 254 and the positive character match 258 . According to the preferred embodiment, when either the pattern match check 252 or the character match check 256 is negative, the content searching engine continues to sequence data 234 according to process step 230 .
  • the potentially matched term 262 is read from the rule table 18 at the address corresponding with the trapped one-to-one pointer.
  • the exact match lookup 16 compares the potentially matched term 262 with the bypassed data flow 68 .
  • the exact match lookup 16 identifies an exact match 272 when the bypassed data flow 68 has the exact same characters in the same combination, and the same case for case sensitive terms, as the potentially matched term 262 .
  • a system executing the content searching engine 10 will generally proceed with some policy based on the exact match 272 .
  • the tenth (10 th ) trap element would identify a positive pattern match and a positive character match for the data stream between times 13 and 22 , inclusive (i.e., trap g-e-t- -p-a-s-s-w-d).
  • the 10 th trap element may also identify a positive pattern match for the data stream between times 1 and 10 , inclusive (G-o-t- -P-i-s-s-e-d), a false positive is averted by the character match.
  • the character match between the final two characters in the term, “w-d” is not the same as the final two characters in the data stream, “e-d”.
  • the data stream 24 is sequenced through each one of the trap elements 48 according to the hierarchy of the trap elements 48 .
  • the hierarchy of the 10 th trap element corresponds with a combined character length of 10.
  • the values for the “G-o-t- -P-i-s-s-e-d” stream of data characters would be sequenced through the trap elements 48 according to the values for “g-o-t- -p-i-s-s-e-d” data characters.
  • the character values for “g-o-t- -p-i-s-s-e-d” in the sequenced data stream are summed and truncated according to Equations 1 and 2, respectively (the summation and truncation is a particular example of generally combining the ten characters).
  • the summation and truncation is a particular example of generally combining the ten characters.
  • summing and truncating the character values for “g-o-t- -p-i-s-s-e-d” results in the hexadecimal value of 0xE2.
  • the 10 th trap element queries the active control matrix 86 at the row and column coordinate corresponding with its hierarchy (10) and with the pattern of the summed and truncated sequenced data (E2).
  • the table address of the term is stored at the corresponding row and column coordinate, i.e., [10, E2].
  • “got pissed” also has a length of 10 characters and its truncated compressed character value is also E2.
  • the rule table 18 does not contain the term “got pissed”. Therefore, the memory in the 10 th trap element traps the truncated compressed character value for “g-o-t- -p-i-s-s-e-d” and the comparator in the 10 th trap element identified a positive pattern match.
  • the set of flags 88 in the active control matrix 86 correspond with the set of table addresses 28 in the pointer matrix 38 . Therefore, since the active control matrix 86 and pointer matrix 38 use the same row and column coordinate system, the flag is set at the row and column coordinate [10, E2]. Accordingly, the character length and truncated compressed character value for “g-o-t- -p-i-s-s-e-d” is [10, E2], resulting in the positive pattern match. However, the character match between the final two characters in the term, “wd”, is not the same as the final two characters in the data stream, “e-d”.
  • the comparator in the 10 th trap element would not identify any character match, averting the false positive that could result from using the pattern match alone.
  • a false positive would not necessarily result in any error because, as a result of the positive match, the content searching engine 10 merely proceeds to the exact match lookup 16 , at which point any false positive would be identified as such because the uncompressed data characters communicated through the buffer memory 20 that correspond with the false positive match would not be identical to the term in the rule table (i.e., “g-o-t- -p-i-s-s-e-d” we “get passwd”).
  • FIG. 5 An alternative embodiment of the trap matrix 12 in the content searching engine 10 is illustrated in FIG. 5 .
  • the entire data stream 24 may be sequenced through the trap elements 48 without any compression.
  • the characters in the sequenced data stream 70 that are combined in each of the memories 52 are based on the hierarchy of the trap elements 48 , and the hierarchy of the trap elements is defined by the series of delay elements 50 between the input device 14 and each one of the memories 52 .
  • the character-length of the sequenced data stream 70 that is trapped in each one of the memories 52 corresponds with the size of the memories 52 .
  • the size of the memories 52 increases exponentially with each additional character to be stored in the memories 52 .
  • a 2-byte memory engine must be able to trap two (2) single-byte characters, and a 3-byte memory engine must be able to trap three (3) single-byte characters.
  • the 3-byte memory engine has a memory size of 2,097,152 bytes (about 2 mega-bytes), which is (2 8 ) 3 .
  • RAM Random Access Memory
  • CAM Content Addressable Memory
  • larger single-byte characters may be effectively trapped by the CAM because the sequenced data 70 can be used as the operand to the CAM memory registers and the CAM only returns an address from the CAM if the operand corresponds with one of the one-to-one pointers.
  • the sequenced data 70 must be stored at specific address locations, resulting in the exponential growth of the memory.
  • the sequenced data 70 is supplied to the memory as the operand and, in a single clock cycle, the CAM returns the address in the CAM if a corresponding match is found. It will be appreciated that the use of the CAM can be used either with compression or without compression of the characters in the sequenced data stream 70 . Without compression of the sequenced data 70 , it will be appreciated that the CAM can be implemented in the trap matrix 12 as the memory-comparator pairs 56 , and in such a trap matrix 12 , the contents of the CAM would contain the set of one-to-one pointers.
  • the trap matrix 12 can simultaneously search the sequenced data stream 70 for character patterns, whether the characters are uncompressed or compressed and/or truncated, that match the corresponding patterns of the characters for each one of the terms 26 in the rule table 18 .
  • the simultaneous search is conducted in each of the trap elements 48 .
  • the memory-comparator pairs 56 in each one of the trap elements simultaneously trap the combination of characters (again, whether the characters are uncompressed or compressed and/or truncated) and compare the patterns of the combination of characters in the sequenced data stream 70 with the corresponding patterns of the terms 26 .
  • the length of the characters being combined and trapped in the memory-comparator pairs is defined according to the hierarchy of the trap elements 48 and corresponds with the length of the terms 26 being compared to the trapped combined characters.
  • FIGS. 6 and 7 the operation of the content searching engine is now described in detail for the preferred embodiment of the invention.
  • the generation of the pointer matrix 38 and active control matrix 86 according to the rule table 18 is illustrated in FIGS. 6A-6C .
  • the operation of the trap matrix 12 , including the pattern match check and the character match check, and the exact match lookup 16 are illustrated in FIGS. 7A-7C .
  • the generation of the rule table 18 preferably begins by sorting the set of terms 26 in the rule table 18 by length and case 300 , resulting in the set of N rule tables 102 discussed above.
  • Terms that are not case sensitive are preferably set to lower case characters, and case sensitivity is preferably applied only for the ASCII values of the terms.
  • the set of terms 26 includes at least one term and there may be many terms 26 in the rule table 18 .
  • the i th term, i th term length, and i th term case sensitivity are saved in term[i], term_length[i], and case[i], respectively.
  • a lower limit for term length (low_limit) is set to an integer value, preferably a low number such as three (3).
  • the generation of the pointer matrix 38 and the active control matrix 86 begins with the initialization of the variables and arrays are initialized 302 .
  • the set of pointers 40 in the pointer matrix 38 are preferably set to zero and the set of flags 88 in the active control matrix 86 are also preferably set to zero.
  • the variable “N” is used as a reference to the number of trap elements, and as discussed above, the particular embodiment uses both a compression operation and truncation for the sequenced data 70 resulting in a single byte pattern. Accordingly, the size of the matrices, the number of rows by the number of columns, can be limited to N ⁇ 256.
  • the number of trap elements (N) can be equal to the maximum character length of the longest term.
  • the pointer matrix 28 and the active control matrix 86 can be set up such that the trap matrix 12 can trap sequenced data 70 for terms 26 that have character lengths that are longer than the number of trap elements (term_length>N) 304 . This is done by limiting the number of the characters used for the pattern match and character match to, at most, the first N-characters in the terms (cost_end) 306 . Of course, for terms that have a number of characters less than or equal to N-characters, the actual length of the term can alternatively be used to define the number of the characters (cost_end) that are used for the pattern match and character match 308 .
  • defining the pattern for each term begins by performing a numerical operation on each character in the term and truncating the result of the numerical operation 310 .
  • the trap matrix 12 is not case sensitive, and any upper case character in the term is operated on as a lower case character (+0x20) 312 .
  • the character values of each term are summed and truncated to the least significant bits of a single byte (sum %256) 312 , 314 .
  • sub-process A ensures that each of the equal-length terms has a different truncated sum 316 .
  • Sub-process A uses a set of uniqueness flags (end_sum[cost_end][sum]) to determine whether a particular truncated sum has already been used for terms having the equal character length 318 .
  • the matrix containing the set of uniqueness flags (end_sum[cost_end][sum]) is based on the same row and column coordinates as the pointer matrix 38 , and the uniqueness flags are initialized to zero (0). Whenever a truncated sum is calculated for a term, the uniqueness flag bit is set high to one (1) for the particular character length (cost-end) and truncated sum (sum %256) of the term 320 .
  • the pointer value (real_ptr) is added to the set of pointers in the pointer matrix (pointer[cost_end][sum]) according to the particular character length and truncated sum 322 .
  • the corresponding case control flag (case[i]) and active control flag (high bit) are respectively stored in the case control matrix (case_control[cost_end][sum]) and active control matrix (active[cost_end][sum]) of the trap matrix 12 .
  • the number of characters in the term that are used to calculate the truncated sum is reduced and a new truncated sum is computed accordingly 322 .
  • the number of characters is greater than the lower limit for term length. If this character reduction process does not work, it is likely to be an indication that the trap matrix 12 should be increased 326 . For example, if there are N terms that all begin with the same N characters, the trap matrix 12 may be increased by the number of trap elements.
  • the truncated sums of a number of terms may have identical least significant bits even though the non-truncated sums may be quite different according to their most significant bits, and the trap matrix 12 in this case could be increased by the size of the memory 52 in each one of the trap elements 48 .
  • the truncation of the compressed character value is optional and the size of the memories in each one of the trap elements can be sized according to the hierarchy of the trap elements and the corresponding maximum possible compressed character value that could be stored in the memory.
  • sub-process B selects a couple of characters from the end of the truncated and summed term and stores the uncompressed characters in a pair of character matrices (compare_char[0,1][cost_end][sum]) 328 .
  • the entire length of the term can be stored in the rule table 18 , and the pointer value (real_ptr) is set accordingly 330 .
  • the pointer value is incremented to the starting pointer value for the next term (term[i+1]) by adding the entire character length of the current term (term_length[i]) and any additional length (k bytes), such as for a header information that may be used in a protocol comparison.
  • the term counter is incremented by one and the process is repeated for the next term until the end of the rule table is reached 332 .
  • the pattern match check, character match check, and the exact match lookup begin by reading the data from the input and sequencing the data stream through the set of trap elements 350 .
  • the truncated sum in the memory 52 of each one of the trap elements 48 is initialized to zero (0) 352 .
  • the truncated sum can be initialized when the header is identified and the summation and truncation of the payload can begin 354 .
  • numerical operations other than summation may be used according to the operation performed on the terms in the rule table.
  • the trap matrix 12 of the preferred embodiment is not case sensitive, and any upper case character in the data stream is operated on as a lower case character (+0x20) 356 .
  • the character values are simultaneously summed and truncated in each one of the trap elements 358 .
  • the number of characters used for the summation in each of the trap elements depends on the hierarchy of the trap elements.
  • uncompressed characters are also communicated to the trap elements for the character match.
  • the original character and the current character are both delayed by two clock cycles so that the second to last and third to last characters (byte_delay[1,2], current_delay[1,2]) are ready to be compared with the corresponding second to last character and third to last character of the term, as stored in the pair of character matrices (compare_char[0,1][cost_end][sum]), should a pattern match occur in any one of the trap elements 48 .
  • the original character is delayed for the case in which the pattern match occurs for a term that is case sensitive 360 .
  • the current character is delayed for the case in which the pattern match occurs for a term that is not case sensitive 362 .
  • the process for the pattern match and character match is simultaneously performed in each one of the trap elements 48 .
  • the process is generally described with reference to the k th trap element in the trap matrix 364 .
  • the comparator for the k th trap element performs the pattern match check by determining whether the flag in the active control matrix is set high at the row and column corresponding to the hierarchy of the k th trap element and the truncated sum value in the memory of the k th trap element (k, sum[k]) 368 .
  • a match indicates a positive pattern match between the sequenced data and the term and the comparator proceeds to the character match check based on the case sensitivity of the term.
  • the trap element provides the pointer to the k th exact match lookup 372 .
  • the pointer for the k th exact match lookup is taken from the row and column of the pointer matrix that corresponds with the matching trap element (k, sum[k]).
  • the particular term for the k th exact match lookup is read from the rule table 374 .
  • the term is read from the address equal to the pointer for the k th exact match lookup (pointer[k][sum[k]]).
  • the actual content that had been bypassed through the buffer memory is then compared with the term from the rule table 376 . If the content searching engine 10 identifies an exact match between particular content in the data stream and any term in the rule table, the system executing the process will generally proceed with some policy based on the exact match 378 . For example, the system may capture the packet as well as perform other steps based on the exact match. The process for sequencing data and performing the pattern match check, the character match check and the exact match lookup can then be repeated 380 .
  • the trap matrix 12 can be scaled with multiple sets of trap elements 400 .
  • the data characters 404 can be alternatively sequenced through the sets of trap elements 400 .
  • the most significant bit data characters 406 and least significant bit data characters 408 can be respectively sequenced through the first pair of trap elements 410 .
  • the characters can be summed or otherwise sequenced through the pairs of hierarchical trap elements 412 and a pattern match may be determined in any one of the hierarchical trap elements 412 .
  • the data stream may be trapped in either one of the 10th hierarchical trap elements 414 .
  • the stream being trapped may begin with “g” in the most significant bit and would be trapped in the corresponding one of the 10th hierarchical trap elements 414 .
  • the data stream could begin with “g” in the least significant bit and would be trapped in the other one of the 10th hierarchical trap elements 414 .
  • the scalability of the trap matrix 12 allows for greater bit-widths with even more sets of trap elements.
  • the content searching engine 10 can be incorporated into any content searching system.
  • the content searching engine 10 may be incorporated into an intrusion detection system 422 between an external computer network 424 and an internal computer network 426 , as shown in FIGS. 9 and 10 .
  • the terms 26 in the rule table 18 are a set of security rules 428 that contain terms that are known to be used in computer hacking, such as attempting to gain unauthorized access to computer networks 426 , 428 .
  • the set of security rules 428 may be used to check entering data packets 430 and/or emanating data packets 432 .
  • the intrusion detection system 422 can be used even more generally as a computer security system.
  • the security rules 428 can also be defined to search for other types of security breaches that may be attempted through the computer system. For example, there could be certain terms that would indicate an attempt to communicate certain trade secrets through computer networks 426 , 428 .
  • the content searching engine 10 could even reside within the internal computer network 426 to check for hacking amongst computers within the network and to check for evidence of corporate espionage, including unauthorized communications containing trade secrets and/or other valuable corporate intelligence, such as customer lists, vendor lists, employee salaries, and competitor information.
  • the data stream 24 is communicated between the computer networks 426 , 428 through at least one transceiver 434 in the content searching engine 10 .
  • the data stream 24 is sequenced through the content searching engine 10 as discussed in detail above. If an exact match is found between characters in the data stream 24 and the security rules 428 , the matching portion of the data stream is trapped and sent to a detection control computer 436 .
  • the detection control computer 436 may log the trapped data stream and/or may implement a pre-defined policy rule according to the trapped data stream.
  • another transceiver 438 may be used for communications between the detection control computer 436 and the content searching engine 10 ( FIG.
  • the detection control computer 436 can also be in communication with the content searching engine 10 through another type of interface, such as a fast ethernet connection.
  • the transceivers can be an optical ethernet interface.
  • the set of patterns may be an exact character pattern set, a near-exact character pattern set, a character segment pattern set, a character operation pattern set, a truncated character operation pattern set, or even a combination of these pattern sets.
  • An exact character pattern set is exemplified by an exact combination of characters, having the same identical characters in the same order and with same case
  • a near-exact character pattern is exemplified by a combination of characters having the same characters in the same order but without any case sensitivity, such as may be implemented in the alternative embodiment and CAM described above.
  • a character segment pattern set is exemplified by a combination of characters having the same characters in the same order, but only for a segment of each one of the terms, such as the character match operation and CAM described above.
  • a character operation pattern set is exemplified by a combination of characters that are operated on by a numerical operator and/or logic operator, and a truncated character operation pattern set is exemplified by truncating the character operation pattern set, such as implemented in the preferred embodiment described above.

Abstract

A trap matrix searches the entire contents of a data stream for a pattern that matches the pattern for a search term. In those circumstances where there is a match between patterns of the data stream and the search term, the method and system can proceed to an exact match operation. In particular, a pointer matrix and a corresponding active control matrix are generated according to a set of terms in a rule table. Data is sequenced the trap matrix according to the hierarchy of its trap elements. The trap elements perform a pattern match check between the sequenced data stream and any search term in the set of terms in the rule table. Results from a positive pattern match are preferably communicated from the matching trap element to an exact match lookup.

Description

CROSS-REFERENCE TO RELATED APPLICATION
This application is a continuation application of U.S. application Ser. No. 10/132,336, filed on Apr. 25, 2002, and entitled SYSTEM AND PROCESS FOR SEARCHING WITHIN A DATA STREAM, which issued on Oct. 25, 2005 as U.S. Pat. No. 6,959,297, the disclosures of which are incorporated herein by reference.
STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT
Not Applicable.
BACKGROUND OF THE INVENTION
1. Field of the Invention
This invention relates generally to content searching engines and, more particularly, to a content searching engine using trap matrix elements.
2. Description of Related Art
It is currently known for content searching engines to use binary search methods to find a particular term in a stream of data. In using such search methods, it is further known for the stream of data to be read out from memory, such information stored in a database or other memory device, or for the stream of data to be communicated through computer networks, such as from the internet (the internet itself may even be considered to be a form of a decentralized database system). Regardless of the stream of data being searched, systems that use such binary search engines must search all potential combinations in the data stream for the search term. For example, to search a data stream for a ten-character term (such as “get passwd”), a binary search engine would have to search over 1.2*1024 combinations (25610 combinations). The entire data stream cannot be efficiently searched using such a method. Either many processors would need to work in parallel to compute all of the operations necessary for the binary searching engine, or the searches are performed statically, capturing and searching only a segment of the data stream and letting other segments pass without searching the exact contents.
BRIEF SUMMARY OF THE INVENTION
It is in view of the above problems that the present invention was developed. The invention is a method and system for using a trap matrix to search the entire contents of a data stream for a pattern that matches the pattern for a search term. In those circumstances where there is a match between patterns of the data stream and the search term, the method and system can proceed to an exact match operation. In particular, the present invention generates a pointer matrix and a corresponding active control matrix according to a set of terms in a rule table. Data is sequenced through the trap matrix according to the hierarchy of its trap elements. The trap elements perform a pattern match check, and also preferably perform a character match check, between the sequenced data stream and any search term in the set of terms in the rule table. Results from a positive pattern match, and any corresponding positive character match, are preferably communicated from the matching trap element to an exact match lookup.
Further features and advantages of the present invention, as well as the structure and operation of various embodiments of the present invention, are described in detail below with reference to the accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
The accompanying drawings, which are incorporated in and form a part of the specification, illustrate the embodiments of the present invention and together with the description, serve to explain the principles of the invention. In the drawings:
FIG. 1 illustrates a schematic diagram view of a content searching engine according to the present invention;
FIG. 2 illustrates a schematic diagram view of a preferred embodiment of the trap matrix according to the preferred embodiment of the present invention;
FIG. 3 illustrates a detailed view of a rule table used in the present invention;
FIG. 4 illustrates a block flow diagram of the operation of the content searching engine according to the present invention;
FIG. 5 illustrates a schematic diagram view of an alternative embodiment of the present invention;
FIGS. 6A-6C illustrate a detailed block flow diagram of the process for generating the pointer matrix and active control matrix according to the rule table and in accordance with the preferred embodiment of the present invention;
FIGS. 7A-7C illustrate a detailed block flow diagram of the processes for pattern matching, character matching, and the exact match lookup in accordance with the preferred embodiment of the present invention;
FIG. 8 illustrates a schematic diagram view of a scaled embodiment of the present invention; and
FIGS. 9 and 10 illustrate schematic diagram views of the content searching engine integrated into an intrusion detection system according to the present invention.
DETAILED DESCRIPTION OF THE INVENTION
Referring to the accompanying drawings in which like reference numbers indicate like elements, FIG. 1 illustrates a diagrammatic view of a content searching engine 10 according to the present invention. Generally, the content searching engine 10 has a trap matrix 12 in communication with an input device 14, an exact match lookup 16 and a rule table 18. The content searching engine 10 also includes a buffer memory 20 between the input device 14 and the exact match lookup 16 and may also include an optional case control module 22 between the input device 14 and the trap matrix 12. As discussed below, the system of the content searching engine 10 operates on a data stream 24 that is communicated through the input device 14.
The rule table 18 includes a set of terms 26 and a set of table addresses 28. Each of the terms in the set of terms 26 is stored in the rule table 18 at a location 30 defined by the set of table addresses 28. For example, the term “get passwd file” and any corresponding prefix, such as “http” that may be used for a protocol, is stored at a location defined by table addresses 0-18. Each successive term, along with any corresponding prefix, can immediately follow the preceding term at the next location, with the next location being defined by the next table address. According to the particular example and with reference to the table below, the term following “get passwd file” could begin immediately after the last character in the preceding term, at address 19. Although the preferred embodiment of the rule table 18 is illustrated with multiple terms, it should be recognized that the set of terms 26 may include as few as a single term.
It will also be recognized that the terms 26 have a number of characters 32 in a number of lengths 34 and combinations 36. For example, as illustrated in Table 1 below, a term following “get passwd file” could be “get pwl file type” and may also have a prefix. According to this particular example, the first several characters (“get p”) of both terms are identical, but the subsequent characters are different and represent a different character combination. Additionally, the lengths of the terms are different; the length of the term “get passwd file” is fifteen (15) characters, whereas the length of the term “get pwl file type” is seventeen (17) characters. In the case where the rule table 18 has a single term, such term would have an individual character length and an individual character combination.
TABLE 1
EXAMPLE OF RULE TABLE
ADDRESS
. . . 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 . . .
TERM . . . g e t p w l f i l e t y p e . . .
The content searching engine 10 also includes a pointer matrix 38 that is defined by the terms 26 and addresses 28 in the rule table 18. The pointer matrix 38 includes a set of one-to-one pointers 40 that are particularly defined by table addresses 28 for each one of the terms 26 in the rule table 18. The pointer matrix 38 has row and column coordinates 42 that are defined according to character lengths 44 and character combinations 46. The character lengths 44 and character combinations 46 correspond with the set of terms 26 according to the lengths of the characters 34 and the combinations of the characters 36, respectively. Therefore, for each one of the terms 26 in the rule table 18, the pointer matrix 38 stores the corresponding table addresses 28 at the row and column coordinates 42 according to the lengths 32 and combinations 34 of the characters in each one of the particular terms 26. For example, the address identifying the term “get pwl file type” (“70” in Table 1 above) would be stored as the term's corresponding one-to-one pointer 40 in the pointer matrix 38. The particular address (“70”) is stored in and can be read out from the pointer matrix 38 according to the rows and columns 42 that also correspond with the length of the characters in each of the terms 26 and the combination of the characters in each of the terms 26. In this manner, the set of one-to-one pointers 40 correlate the table addresses 28 for each one of the terms 26 in the rule table 18.
According to the present invention, the character combinations 46 used for storing the pointers 40 in the pointer matrix 38 are generally defined by a set of patterns that uniquely correspond with the set of terms 26 according to the characters within each one of the terms 26. As discussed in detail below, there are many types of patterns that can be defined using the character combinations 46. In the preferred embodiment of the present invention, the set of patterns is a set of compressed character values that is produced by compressing the characters in each one of the terms 26 according to a numerical operation and a truncation operation. It will be appreciated that terms 26 with different character lengths may have the same compressed character value because at least one of the row and column coordinates 42 for two terms is different due to the different character lengths. As a corollary, it will be appreciated that when a pair of terms 26 with the same character length also have the same compressed value, the compressed value for one of the terms 26 may be calculated based on a fewer number of characters and the pointer address would be stored according to the shortened character length to ensure that the set of pointers 40 has a one-to-one correspondence with each one of the terms 26. Accordingly, for those terms 26 that have equal character lengths, the set of patterns uniquely corresponds with the character combinations for those each one of the terms 26 having the particular character length. Given that the set of terms 26 may have a single term, the row and column coordinates 42 for such a term would be the length of the term's characters and the pattern resulting from the term's characters.
The trap matrix 12 includes a set of trap elements 48 and a corresponding set of delay elements 50. Each of the trap elements 48 has a memory 52 in communication with a corresponding comparator 54 and are generally referred to as memory-comparator pairs 56 for each of the trap elements 48. The trap elements 48 are in multiplexed communication with the input device 14 and have a hierarchy (1 to N) according to the set of delay elements 50. The hierarchy of the trap elements 48, particularly including the memory-comparator pairs 56, is preferably in a one-to-one correspondence with the character lengths 34 of the set of terms 26. In particular, the trap matrix 12 has an increasing series of delay elements 50 between each of the trap elements 48 and the input device 14. The increasing series of delay elements 50 define an increasing order in the hierarchy of the trap elements 48, and the increasing order of hierarchy corresponds with an increasing character length of the data stream 24. Therefore, with each increasing order, the trap elements are able to examine the character patterns of increasing lengths of the data stream 24.
In operation, the input device 14 receives the data stream 24 during a time period 58 as illustrated in Table 2 below. The data stream 24 includes a set of data characters 60 having a length 62 and a combination 64, and the time period 58 is made up of clock cycles 66. The input device 14 also communicates the data stream 24 to the buffer memory 20 through a bypass 68. The trap matrix 12 receives the data stream 24 from the input device 14 and sequences the set of characters 60 in the data stream 24 through each of the trap elements 48. The sequenced set of data characters enter the memory-comparator pairs 56 as a set of sequenced data 70 according to the hierarchy of the trap elements 48.
As the trap matrix 12 sequences the data stream 24 through the trap elements 48, the memory-comparator pairs 56 perform a pattern match check between patterns (P) of the sequenced data 70 and the set of patterns defined by the combination of characters 36 for each one of the terms 26 in the rule table 18. Each of the memory-comparator pairs 56 simultaneously performs the pattern match check according to the one-to-one correspondence between the hierarchy of the trap elements 48 and the character lengths 34 of the set of terms 26. A positive pattern match by one of the matching memory-comparator pairs particularly defines a particular one-to-one pointer 72 that contains one of the addresses 28 of one of the terms 26 in the rule table 18. The particular one-to-one pointer 72 has a row and column coordinate 74 according to the hierarchy of the matching trap element (k) and the pattern (P) of the sequenced data 70 in the matching trap element (P[k]).
The exact match lookup 16 communicates with the buffer memory 20, trap matrix 12 and the rule table 18. The exact match lookup 16 receives the particular one-to-one pointer 72 from the pointer matrix 38. The exact match lookup 16 retrieves one of the terms 26 from the rule table 18 according to the table address corresponding with the one-to-one pointer 72 from the pointer matrix 38. The exact match lookup 16 examines the retrieved term with the portion of the data stream 24 from the buffer memory 20 that corresponds with the time period according to the matching memory-comparator pair. With the exact term retrieved from the rule table 18 and the corresponding portion of the data stream 24 from the buffer memory 20, the exact match lookup checks for an exact match therebetween.
The preferred embodiment of the trap matrix 12 and rule table 18 is now particularly described with reference to FIG. 2. As discussed above, the pattern for each of the terms 26 in the rule table 18 can be defined according to a set of compressed character values. Generally, each one of the terms 26 can be compressed by expressing each character in a term according to a defined value and performing a set of operations on the set of values. For example, the operation may be a numerical and/or logical operation, such as summation, subtraction, multiplication, division, XOR, XNOR, and concatenation. The set of operations may be a single numerical operation, such as summation, performed on each character in each of the terms 26. It is also possible that the set of operations may be multiple operations. According to the preferred embodiment of the present invention, the summation operation exemplifies the compression of the terms 26. For example, as summarized in Table 2 below, the hexadecimal (0x) representation of the characters in the term “get passwd” can be summed into a compressed character value of 0x3E2.
TABLE 2
EXAMPLE OF COMPRESSED CHARACTER VALUE
HEX Value(0x) 67 65 74 20 70 61 73 73 77 64 Compressed Character Value
Term Character g e t p a s s w d = 0x3E2 (Summation Total)
Table Address 11 12 13 14 15 16 17 18 19 20 Term Length = 10
It should be recognized that this preferred method of compression is a one-way compression according to Equation 1 below because there is not a one-to-one correspondence between the compressed character value and the term being compressed. As the number of characters in the term increase, the number of other combinations of characters that would also have the same compressed character value increases exponentially. Therefore, once the term is compressed according to the sum of the character values (or any other operation on the character values), the compressed character value cannot be expanded back into the term as a one-to-one representation. Instead of being a one-to-one representation of the term, the compressed character value is a probability of the term. For example, the compressed character value for the 10-character term “get passwd” is 0x3E2 (i.e., Σ“get passwd”→0x3E2), and the compressed character value for the 10-character term “got pissed” is also 0x3E2 (i.e., Σ“got pissed”→0x3E2). Therefore, it is evident that, by itself, 0x3E2 is not a one-to-one representation of “get passwd”.
Σ(0x Term), @0x Character in Term→Compressed Character Value for Term   (1)
The compressed character value may be further compressed by removing the most significant bits (MSB) and thereby truncating the summation total to its least significant bits (LSB) according to Equation 2 below. For example, the most significant hexadecimal bit of 0x3 in the summation total of 0x3E2 can be removed, thereby truncating the summation total and resulting in a truncated compressed character value of 0xE2. Therefore, the term “get passwd” can be compressed and checked with the sequenced data stream 70 based on its character length of 10 and its pattern value of 0xE2. As discussed above, a table address for the term “get passwd” would be stored as the one-to-one pointer in the pointer matrix 38 and would be stored at the row and column coordinate corresponding to the character length of 10 and pattern value of 0xE2. It will be further appreciated that the set of one-to-one pointers 40 may correspond with the first or last character in each of the terms 26 in the rule table 18 and all of the characters in the term can be read out from the rule table 18 based on the address of the first character and the character length of the term.
Truncated Compressed Character Value=LSB(Compressed Character Value)   (2)
As discussed above, the trap matrix 12 sequences the data stream 24 through the trap elements 48, and the memory-comparator pairs 56 perform the pattern match check between patterns (P) of the sequenced data 70 and the set of patterns defined by each one of the terms 26 in the rule table 18. Accordingly, the trap matrix 12 performs the same set of operations on the data stream 24 that is performed on the terms 26 in the rule table 18. This allows for each one of the trap elements 48 to simultaneously perform the pattern match check between the sequenced data 70 and each one of the terms 26 according to each of the hierarchies in the trap matrix 12.
Accordingly, for the preferred embodiment, the trap matrix 12 includes a set of compression operators 76 located between the input and each one of the memory-comparator pairs 56. In the preferred embodiment the summation operation 78 is performed in the set of operations, adding each current character 80 with a set of prior summed characters 82 according to the hierarchy of the trap elements 48, based on the series of delays 50 through which the data 24 is sequenced.
Additionally, the sequenced data 70 from each summation 78 is preferably truncated in an LSB operation 84. The one-to-one pointers 40 in the pointer matrix 38 can then be examined based on the pattern of the summed and truncated sequenced data 70 as it is communicated from the input device 14 to each of the trap elements 48 as discussed above. In particular, each one of the comparators 54 can query the pointer matrix 38 at the row and column coordinate corresponding with the hierarchy of the trap elements 48 and with the pattern of the summed and truncated sequenced data 70 in each one of the corresponding memories 52. When the pointer matrix 38 contains a pointer 72 at the row and column coordinate, the corresponding comparator 54 identifies the positive pattern match and the content searching engine 10 can proceed to the exact match lookup 16. It will be recognized that the first table address in the rule table 18 may be a zero value and that the first table address does not need to be used for the first character of any of the terms 26. Additionally, it will be recognized that the one-to-one pointers 40 can be set up to identify the last character in the term (which may be non-zero, even when the first term begins as the first table address in the rule table 18).
It will be appreciated that the memories 52 in each one of the trap elements 48 can be the size for the sequenced data 70, regardless of hierarchy, due to the compression and truncation of the characters in the data stream 24. In particular, for the preferred embodiment, the compressed and truncated sequenced data 70 can be stored in a single byte ([7:0]), which is between 0 and FF in hexadecimal format and between 0 and 255 in decimal format (between 00000000 and 11111111 in binary format). It will also be appreciated that the memories 52 may be larger, such as being sized to accommodate at least one additional bit for case control and additional size for a prefix, such as a header in a data packet.
As discussed above, a direct examination of the pointer matrix 38 can be used to perform the pattern match check between the sequenced data 70 in any one of the trap elements 48 and the terms 26 in the rule table 18. Additionally, according to the preferred embodiment of the present invention, an active control matrix 86 may be used to perform the pattern match check between the sequenced data 70 and the terms 26. The active control matrix 86 is very similar to the pointer matrix 38. The active control matrix 86 uses the same row and column coordinates 42 as the pointer matrix 38, including the definitions for the row and column coordinates 42 (i.e., identical row and column coordinate system defined according to the character lengths 44 and the character combinations 46). The difference between the active control matrix 86 and the pointer matrix 38 is in their contents. As discussed above, the pointer matrix 38 contains the set of addresses 28 for the terms 26 in the rule table 18. In comparison, the active control matrix 86 contains a set of flags 88, such as a set of binary numbers having one and zero bits. Therefore, the set of flags 88 in the active control matrix 86 correspond with the addresses in the pointer matrix 38 according to Equation 3 below. In general, the set of flags 88 identify whether there is a pointer 72 in the pointer matrix 38 at each corresponding row and column coordinate 42. Therefore, according to the preferred embodiment, the pattern match is first identified by one of the flags 88 in the active control matrix 86, and the pointer 72 for the term with the corresponding pattern and length may then be read from the same row and column coordinate 42 in the pointer matrix 38 as the matching flag in the active control matrix 86.
[Table Address in Pointer Matrix]→[Flag in Active Control Matrix]  (3)
In the preferred embodiment, any one of the trap elements 48 that identifies a positive pattern match then performs a character match check prior to proceeding to the exact match lookup 16. In the character match check, a pair of data characters 90 from the data stream 24 are compared with two uncompressed term characters 92 from a comparison character matrix 94. The uncompressed term characters 92 in the comparison character matrix 94 are character segments from each one of the terms 26 in the rule table 18 and are stored in the comparison character matrix 94 according to the length of the terms 34.
It will be appreciated that the uncompressed term characters may be taken from any corresponding location in each of the terms 26, and the trap matrix 12 can be designed accordingly. For example, in the preferred embodiment, the uncompressed term characters are the last two characters in each one of the terms 26, and the trap matrix is designed such that the pair of data characters 90 are the last two characters from the data stream 24 that entered the trap elements 48. In particular, the pair of data characters 90 are in a multiplexed communication through a pair of communication paths 96. The pair of data characters 90 are not routed through a compression operator and are instead communicated through a delay element 98 before the communication paths 96 and through a delay element 100 between the communication paths 96.
It will also be appreciated that other combinations of characters are possible by altering the series of delays 98, 100. For example, the last character and third to last character can be the characters to be compared in the character match check by using a delay of two clock cycles 66, i.e., a pair of delay elements, for the delay element 100 between the communication paths 96. It will be further appreciated that a single character can also be used for the character match check and more than two characters may also be used for the character match check.
Based on the general description of the content searching engine 10 and the particular description of the rule table 18 above, it will be appreciated that the terms 26 in the rule table 18 can be sorted and stored based on their lengths 32. The rule table 18 can be virtually segmented into a set of N rule tables 102, with all rules of a single length being stored in each one of the rule tables 104 in the set of rule tables 102.
As particularly illustrated in FIG. 3, the lengths of the rules in the set of rule tables 102 may preferably range from a two-character length in the 2nd rule table 106 to an N-character length in the Nth rule table 108 and have a one-to-one correspondence with the number of trap elements 48. The Nth rule table preferably contains terms having a 24-character length 110. It will be appreciated that there may be terms in the rule table 18 with lengths significantly greater than twenty-four (24) characters and that it is possible to use pattern matching with these longer terms based on patterns generated using the 24-character length. For example, when the content searching engine 10 is incorporated into an intrusion detection system, the terms 26 would be based on intrusion detection rules, some of which are known to be greater than 150-characters in length.
The operation of the system is now generally described with reference to FIG. 4 and is described again in more detail below with reference to the preferred embodiment of the invention. Generally, the rule table 18 and corresponding pointer matrix 38 are defined at preparation step 210, and the data stream 24 is communicated through the input 14 at input step 220. According to step 222, the buffer memory 20 is preferably a first-in-first-out (FIFO) memory for bypassing the trap matrix 12 with the multiplex bypassed data stream 68.
The content searching engine 10 sequences the data stream 24 through the trap matrix 12 at process step 230. In particular, the data stream 24 is communicated in the multiplexed manner through the series of delay elements 50 and each one of the memories 52 according to the hierarchy of trap elements 48. Process step 240 defines the row and column coordinates 42 of the pointer matrix 38 that are queried 232 by each one of the trap elements 48 based on the sequenced data stream 70. Each one of the trap elements 48 performs the pattern match check 252 according to decision step 250. In particular, each one of the comparators 54 can query 232 the pointer matrix 38 by defining the row and column coordinate according to the hierarchy of the trap elements 48 and according to the pattern of the sequenced data 70 in each one of the corresponding memories 52. Generally, the content searching engine 10 proceeds to the exact match lookup 16 following the positive pattern match 254. When the content searching engine 10 proceeds to the exact match lookup 16, it should be appreciated that the matching trap element defines the row and column coordinate for the one-to-one pointer, and it is this trapped one-to-one pointer that defines the table address for the potentially matched term in the rule table 18. When there is not a positive pattern match 254, the content searching engine continues to sequence data 234 according to process step 230.
As particularly discussed above with reference to the preferred embodiment of the trap matrix 12 for the content searching engine 10, the comparators 54 can query 232 the active control matrix 86 at the row and column coordinate corresponding with the hierarchy of the trap elements 48 and with the pattern of the summed and truncated sequenced data 70. Additionally, as discussed above with reference to the preferred embodiment, the comparators 54 may also perform the character match check 256 when there is a positive pattern match 254. The combination of the pattern match check 252 and the character match check 256 significantly reduces any chance for a false positive based on the patterns in the compressed and truncated data stream 70. Therefore, to proceed to the exact match lookup 16, the preferred embodiment requires both the positive pattern match 254 and the positive character match 258. According to the preferred embodiment, when either the pattern match check 252 or the character match check 256 is negative, the content searching engine continues to sequence data 234 according to process step 230.
As discussed above, one of the terms 26 from the rule table 18 must be read to perform the exact match lookup 16. Therefore, as discussed above and according to process step 260, the potentially matched term 262 is read from the rule table 18 at the address corresponding with the trapped one-to-one pointer. In process step 270, the exact match lookup 16 compares the potentially matched term 262 with the bypassed data flow 68. The exact match lookup 16 identifies an exact match 272 when the bypassed data flow 68 has the exact same characters in the same combination, and the same case for case sensitive terms, as the potentially matched term 262. According to process step 280, a system executing the content searching engine 10 will generally proceed with some policy based on the exact match 272.
Based on the example of the character compression discussed with reference to Table 2 above, an example of using the character match check 256 in combination with the pattern match check 252 to reduce the chance for a false positive based on the patterns in the compressed data stream 70. As discussed above, when the rule table 18 includes the term “get passwd”, a pattern for the term can be equivalently expressed according to the sum of its hexadecimal character values, i.e., “get passwd”→0x3E2. The term “get passwd” has ten (10) characters. Therefore, with reference to Table 3 below and according to the above description of the preferred embodiment of the invention, the tenth (10th) trap element would identify a positive pattern match and a positive character match for the data stream between times 13 and 22, inclusive (i.e., trap g-e-t- -p-a-s-s-w-d). As discussed in detail below, although the 10th trap element may also identify a positive pattern match for the data stream between times 1 and 10, inclusive (G-o-t- -P-i-s-s-e-d), a false positive is averted by the character match. In particular, the character match between the final two characters in the term, “w-d”, is not the same as the final two characters in the data stream, “e-d”.
It will be appreciated that none of the trap elements 48 other than the 10th trap element would trap “g-e-t- -p-a-s-s-w-d” because the hierarchy of the 10th trap element corresponds with terms having character lengths equal to ten (Hierarchy=10≈Character Length=10) whereas the hierarchy of the other trap elements corresponds with terms having character lengths greater than or less than ten (Hierarchy>10≈Character Length>10 Hierarchy<10≈Character Length<10).
TABLE 3
EXAMPLE OF DATA STREAM
TIME
22 21 20 19 18 17 16 15 14 13 12 11 10 09 08 07 06 05 04 03 02 01
DATA d w s s a p t e g Z A d e s s i P t o G
Figure US07464089-20081209-P00001
According to the description of the preferred embodiment above, the data stream 24 is sequenced through each one of the trap elements 48 according to the hierarchy of the trap elements 48. In particular, the hierarchy of the 10th trap element corresponds with a combined character length of 10. Assuming that the values of the characters are set using the case control module 22, the values for the “G-o-t- -P-i-s-s-e-d” stream of data characters would be sequenced through the trap elements 48 according to the values for “g-o-t- -p-i-s-s-e-d” data characters. Therefore, in times 1 through 10, inclusive, the character values for “g-o-t- -p-i-s-s-e-d” in the sequenced data stream are summed and truncated according to Equations 1 and 2, respectively (the summation and truncation is a particular example of generally combining the ten characters). As discussed above, summing and truncating the character values for “g-o-t- -p-i-s-s-e-d” results in the hexadecimal value of 0xE2.
The 10th trap element queries the active control matrix 86 at the row and column coordinate corresponding with its hierarchy (10) and with the pattern of the summed and truncated sequenced data (E2). Of course, since the term “get passwd” has a length of 10 characters and its truncated compressed character value is E2, the table address of the term is stored at the corresponding row and column coordinate, i.e., [10, E2]. Similarly, “got pissed” also has a length of 10 characters and its truncated compressed character value is also E2. However, the rule table 18 does not contain the term “got pissed”. Therefore, the memory in the 10th trap element traps the truncated compressed character value for “g-o-t- -p-i-s-s-e-d” and the comparator in the 10th trap element identified a positive pattern match.
It will be appreciated that the set of flags 88 in the active control matrix 86 correspond with the set of table addresses 28 in the pointer matrix 38. Therefore, since the active control matrix 86 and pointer matrix 38 use the same row and column coordinate system, the flag is set at the row and column coordinate [10, E2]. Accordingly, the character length and truncated compressed character value for “g-o-t- -p-i-s-s-e-d” is [10, E2], resulting in the positive pattern match. However, the character match between the final two characters in the term, “wd”, is not the same as the final two characters in the data stream, “e-d”. Therefore, the comparator in the 10th trap element would not identify any character match, averting the false positive that could result from using the pattern match alone. Of course, a false positive would not necessarily result in any error because, as a result of the positive match, the content searching engine 10 merely proceeds to the exact match lookup 16, at which point any false positive would be identified as such because the uncompressed data characters communicated through the buffer memory 20 that correspond with the false positive match would not be identical to the term in the rule table (i.e., “g-o-t- -p-i-s-s-e-d” we “get passwd”).
An alternative embodiment of the trap matrix 12 in the content searching engine 10 is illustrated in FIG. 5. According to this embodiment, the entire data stream 24 may be sequenced through the trap elements 48 without any compression. As discussed above, the characters in the sequenced data stream 70 that are combined in each of the memories 52 are based on the hierarchy of the trap elements 48, and the hierarchy of the trap elements is defined by the series of delay elements 50 between the input device 14 and each one of the memories 52. It should be appreciated that the character-length of the sequenced data stream 70 that is trapped in each one of the memories 52 corresponds with the size of the memories 52. However, without any compression of the sequenced data stream 70, the size of the memories 52 increases exponentially with each additional character to be stored in the memories 52.
For example, a 2-byte memory engine must be able to trap two (2) single-byte characters, and a 3-byte memory engine must be able to trap three (3) single-byte characters. Each single, eight-bit byte can be used to define up to 256 potential characters (28=256). Therefore, for the 2-byte memory engine to trap two (2) successive single-byte characters, the 2-byte memory engine has a memory size of 8,192 bytes, which is (28)2. Similarly, for the 3-byte memory engine to trap three (3) successive single-byte characters, the 3-byte memory engine has a memory size of 2,097,152 bytes (about 2 mega-bytes), which is (28)3. The reason that the size of the memories increases exponentially without compression is because the memory engines are accommodating for any potential combination of characters in the data stream 24 so that the memory engine can trap the uncompressed characters according to each of the various combinations for the set of terms 26 in the rule table 18. Generally, the largest N-byte memory engine can trap the longest uncompressed sequenced data stream 70 which has N-successive single-byte characters, and the N-byte memory engine has a memory size generally defined by Equation 4.
Memory Size=(28)N, where N=Number of Trapped Single-Byte Characters   (4)
It will be appreciated that current technology for standard Random Access Memory (RAM) only allows for memory engines with the capacity to trap several single-byte characters due to the number of permutations between the sequenced characters and the potential combination of characters for the set of terms 26 in the rule table 18. Based on advances in Content Addressable Memory (CAM), larger single-byte characters may be effectively trapped by the CAM because the sequenced data 70 can be used as the operand to the CAM memory registers and the CAM only returns an address from the CAM if the operand corresponds with one of the one-to-one pointers. In RAM, the sequenced data 70 must be stored at specific address locations, resulting in the exponential growth of the memory. In the CAM, the sequenced data 70 is supplied to the memory as the operand and, in a single clock cycle, the CAM returns the address in the CAM if a corresponding match is found. It will be appreciated that the use of the CAM can be used either with compression or without compression of the characters in the sequenced data stream 70. Without compression of the sequenced data 70, it will be appreciated that the CAM can be implemented in the trap matrix 12 as the memory-comparator pairs 56, and in such a trap matrix 12, the contents of the CAM would contain the set of one-to-one pointers.
Generally, the trap matrix 12 can simultaneously search the sequenced data stream 70 for character patterns, whether the characters are uncompressed or compressed and/or truncated, that match the corresponding patterns of the characters for each one of the terms 26 in the rule table 18. As described above, the simultaneous search is conducted in each of the trap elements 48. In particular, the memory-comparator pairs 56 in each one of the trap elements simultaneously trap the combination of characters (again, whether the characters are uncompressed or compressed and/or truncated) and compare the patterns of the combination of characters in the sequenced data stream 70 with the corresponding patterns of the terms 26. The length of the characters being combined and trapped in the memory-comparator pairs is defined according to the hierarchy of the trap elements 48 and corresponds with the length of the terms 26 being compared to the trapped combined characters.
With reference to FIGS. 6 and 7, the operation of the content searching engine is now described in detail for the preferred embodiment of the invention. In particular, the generation of the pointer matrix 38 and active control matrix 86 according to the rule table 18 is illustrated in FIGS. 6A-6C. The operation of the trap matrix 12, including the pattern match check and the character match check, and the exact match lookup 16 are illustrated in FIGS. 7A-7C.
Referring to FIG. 6A, the generation of the rule table 18 preferably begins by sorting the set of terms 26 in the rule table 18 by length and case 300, resulting in the set of N rule tables 102 discussed above. Terms that are not case sensitive are preferably set to lower case characters, and case sensitivity is preferably applied only for the ASCII values of the terms. As discussed above, the set of terms 26 includes at least one term and there may be many terms 26 in the rule table 18. For the set of terms 26, the ith term, ith term length, and ith term case sensitivity are saved in term[i], term_length[i], and case[i], respectively. A lower limit for term length (low_limit) is set to an integer value, preferably a low number such as three (3).
The generation of the pointer matrix 38 and the active control matrix 86 begins with the initialization of the variables and arrays are initialized 302. In particular, the set of pointers 40 in the pointer matrix 38 are preferably set to zero and the set of flags 88 in the active control matrix 86 are also preferably set to zero. The variable “N” is used as a reference to the number of trap elements, and as discussed above, the particular embodiment uses both a compression operation and truncation for the sequenced data 70 resulting in a single byte pattern. Accordingly, the size of the matrices, the number of rows by the number of columns, can be limited to N×256.
As discussed above, it is preferable for the number of trap elements (N) to be equal to the maximum character length of the longest term. However, the pointer matrix 28 and the active control matrix 86 can be set up such that the trap matrix 12 can trap sequenced data 70 for terms 26 that have character lengths that are longer than the number of trap elements (term_length>N) 304. This is done by limiting the number of the characters used for the pattern match and character match to, at most, the first N-characters in the terms (cost_end) 306. Of course, for terms that have a number of characters less than or equal to N-characters, the actual length of the term can alternatively be used to define the number of the characters (cost_end) that are used for the pattern match and character match 308.
As discussed above with reference to the preferred embodiment, defining the pattern for each term begins by performing a numerical operation on each character in the term and truncating the result of the numerical operation 310. According to the preferred embodiment, the trap matrix 12 is not case sensitive, and any upper case character in the term is operated on as a lower case character (+0x20) 312. According to the particular example, the character values of each term are summed and truncated to the least significant bits of a single byte (sum %256) 312, 314.
As discussed above, for each one of the terms having an equal character length (cost_end=i), the set of pointers for these equal-length terms are stored according to the character length and the truncated sum value. Therefore, for these equal-length terms, the present invention ensures that each one of these equal-length terms has a different truncated sum for its particular pattern. Referring to FIG. 6B, sub-process A ensures that each of the equal-length terms has a different truncated sum 316. Sub-process A uses a set of uniqueness flags (end_sum[cost_end][sum]) to determine whether a particular truncated sum has already been used for terms having the equal character length 318. The matrix containing the set of uniqueness flags (end_sum[cost_end][sum]) is based on the same row and column coordinates as the pointer matrix 38, and the uniqueness flags are initialized to zero (0). Whenever a truncated sum is calculated for a term, the uniqueness flag bit is set high to one (1) for the particular character length (cost-end) and truncated sum (sum %256) of the term 320.
If no other term with the same character length produces the same truncated sum, the pointer value (real_ptr) is added to the set of pointers in the pointer matrix (pointer[cost_end][sum]) according to the particular character length and truncated sum 322. Additionally, the corresponding case control flag (case[i]) and active control flag (high bit) are respectively stored in the case control matrix (case_control[cost_end][sum]) and active control matrix (active[cost_end][sum]) of the trap matrix 12. In comparison, if another term with the same character length produces the same truncated sum, the number of characters in the term that are used to calculate the truncated sum is reduced and a new truncated sum is computed accordingly 322. When reducing the number of characters used for calculating the truncated sum, it is preferable to ensure that the number of characters is greater than the lower limit for term length. If this character reduction process does not work, it is likely to be an indication that the trap matrix 12 should be increased 326. For example, if there are N terms that all begin with the same N characters, the trap matrix 12 may be increased by the number of trap elements. As another example, it is possible that the truncated sums of a number of terms may have identical least significant bits even though the non-truncated sums may be quite different according to their most significant bits, and the trap matrix 12 in this case could be increased by the size of the memory 52 in each one of the trap elements 48. It will be appreciated that the truncation of the compressed character value is optional and the size of the memories in each one of the trap elements can be sized according to the hierarchy of the trap elements and the corresponding maximum possible compressed character value that could be stored in the memory. In particular, without any truncation of the compressed character value, at each hierarchy of the trap elements, the memories could be sized according to Equation 5.
Memory Size=(28)*(2x), x≧log(k)/log(2), k=k th trap element hierarchy   (5)
As discussed above, when the truncated sum is used in the pattern match, it is preferable to use a second matching technique to avoid false positives. For example, two uncompressed characters from each term can be used in the character match. Therefore, after sub-process A has ensured that a particular term has a different truncated sum from other terms with the same length, these uncompressed characters can be selected from the term. Referring to FIG. 6C, sub-process B selects a couple of characters from the end of the truncated and summed term and stores the uncompressed characters in a pair of character matrices (compare_char[0,1][cost_end][sum]) 328. Additionally, even though the length of the term used for the pattern match and character match does not necessarily use every character in the term, the entire length of the term can be stored in the rule table 18, and the pointer value (real_ptr) is set accordingly 330. In particular, the pointer value is incremented to the starting pointer value for the next term (term[i+1]) by adding the entire character length of the current term (term_length[i]) and any additional length (k bytes), such as for a header information that may be used in a protocol comparison. The term counter is incremented by one and the process is repeated for the next term until the end of the rule table is reached 332.
Referring to FIG. 7A, the pattern match check, character match check, and the exact match lookup begin by reading the data from the input and sequencing the data stream through the set of trap elements 350. At the beginning of the data stream, the truncated sum in the memory 52 of each one of the trap elements 48 is initialized to zero (0) 352. For example, if the data stream is a data packet having a header and a payload, the truncated sum can be initialized when the header is identified and the summation and truncation of the payload can begin 354. As discussed above, numerical operations other than summation may be used according to the operation performed on the terms in the rule table. As discussed above, the trap matrix 12 of the preferred embodiment is not case sensitive, and any upper case character in the data stream is operated on as a lower case character (+0x20) 356. According to the particular example, the character values are simultaneously summed and truncated in each one of the trap elements 358. As discussed in detail above, the number of characters used for the summation in each of the trap elements depends on the hierarchy of the trap elements.
As discussed above, uncompressed characters are also communicated to the trap elements for the character match. According to the particular example, the original character and the current character are both delayed by two clock cycles so that the second to last and third to last characters (byte_delay[1,2], current_delay[1,2]) are ready to be compared with the corresponding second to last character and third to last character of the term, as stored in the pair of character matrices (compare_char[0,1][cost_end][sum]), should a pattern match occur in any one of the trap elements 48. The original character is delayed for the case in which the pattern match occurs for a term that is case sensitive 360. The current character is delayed for the case in which the pattern match occurs for a term that is not case sensitive 362.
Referring now to FIG. 7B, the process for the pattern match and character match is simultaneously performed in each one of the trap elements 48. The process is generally described with reference to the kth trap element in the trap matrix 364. The comparator for the kth trap element performs the pattern match check by determining whether the flag in the active control matrix is set high at the row and column corresponding to the hierarchy of the kth trap element and the truncated sum value in the memory of the kth trap element (k, sum[k]) 368. A match indicates a positive pattern match between the sequenced data and the term and the comparator proceeds to the character match check based on the case sensitivity of the term. If there is no positive match in either the pattern match check or the character match check, the next character is sequenced through the trap elements and the process begins again. When there is a positive match in both the pattern match check and the character match check, the trap element provides the pointer to the kth exact match lookup 372. In particular, the pointer for the kth exact match lookup is taken from the row and column of the pointer matrix that corresponds with the matching trap element (k, sum[k]).
Referring now to FIG. 7C, the particular term for the kth exact match lookup is read from the rule table 374. In particular, the term is read from the address equal to the pointer for the kth exact match lookup (pointer[k][sum[k]]). Accounting for any case sensitivity, the actual content that had been bypassed through the buffer memory is then compared with the term from the rule table 376. If the content searching engine 10 identifies an exact match between particular content in the data stream and any term in the rule table, the system executing the process will generally proceed with some policy based on the exact match 378. For example, the system may capture the packet as well as perform other steps based on the exact match. The process for sequencing data and performing the pattern match check, the character match check and the exact match lookup can then be repeated 380.
Referring to FIG. 8, the trap matrix 12 can be scaled with multiple sets of trap elements 400. For example, in the case in which the data stream 24 has a bit-width of sixteen (16) bits 402, the data characters 404 can be alternatively sequenced through the sets of trap elements 400. In particular, the most significant bit data characters 406 and least significant bit data characters 408 can be respectively sequenced through the first pair of trap elements 410. The characters can be summed or otherwise sequenced through the pairs of hierarchical trap elements 412 and a pattern match may be determined in any one of the hierarchical trap elements 412. For example, when “g-e-t- -p-a-s-s-w-d” is in the data stream 24 and the term “get passwd” is in the rule table 18, the data stream may be trapped in either one of the 10th hierarchical trap elements 414. As particularly illustrated, the stream being trapped may begin with “g” in the most significant bit and would be trapped in the corresponding one of the 10th hierarchical trap elements 414. It will be appreciated that the data stream could begin with “g” in the least significant bit and would be trapped in the other one of the 10th hierarchical trap elements 414. It will be further appreciated that the scalability of the trap matrix 12 allows for greater bit-widths with even more sets of trap elements.
The content searching engine 10 can be incorporated into any content searching system. For example, the content searching engine 10 may be incorporated into an intrusion detection system 422 between an external computer network 424 and an internal computer network 426, as shown in FIGS. 9 and 10. In each application of the intrusion detection system 422, the terms 26 in the rule table 18 are a set of security rules 428 that contain terms that are known to be used in computer hacking, such as attempting to gain unauthorized access to computer networks 426, 428. For the particular application of the intrusion detection system 422 shown in FIG. 10, it is evident that the set of security rules 428 may be used to check entering data packets 430 and/or emanating data packets 432. It will be appreciated that, by checking the entering and emanating data packets 428, 430, the intrusion detection system 422 can be used even more generally as a computer security system. For example, in addition to checking for hacking terms, the security rules 428 can also be defined to search for other types of security breaches that may be attempted through the computer system. For example, there could be certain terms that would indicate an attempt to communicate certain trade secrets through computer networks 426, 428. It will be further appreciated that the content searching engine 10 could even reside within the internal computer network 426 to check for hacking amongst computers within the network and to check for evidence of corporate espionage, including unauthorized communications containing trade secrets and/or other valuable corporate intelligence, such as customer lists, vendor lists, employee salaries, and competitor information.
In the particular embodiments shown in FIGS. 9 and 10, the data stream 24 is communicated between the computer networks 426, 428 through at least one transceiver 434 in the content searching engine 10. The data stream 24 is sequenced through the content searching engine 10 as discussed in detail above. If an exact match is found between characters in the data stream 24 and the security rules 428, the matching portion of the data stream is trapped and sent to a detection control computer 436. The detection control computer 436 may log the trapped data stream and/or may implement a pre-defined policy rule according to the trapped data stream. According to the particular embodiments, another transceiver 438 may be used for communications between the detection control computer 436 and the content searching engine 10 (FIG. 9) or between the computer networks 426, 428 when checking both entering and emanating data packets 428, 430 (FIG. 10). As illustrated in FIG. 10, the detection control computer 436 can also be in communication with the content searching engine 10 through another type of interface, such as a fast ethernet connection. The transceivers can be an optical ethernet interface.
It will be appreciated that the set of patterns may be an exact character pattern set, a near-exact character pattern set, a character segment pattern set, a character operation pattern set, a truncated character operation pattern set, or even a combination of these pattern sets. An exact character pattern set is exemplified by an exact combination of characters, having the same identical characters in the same order and with same case, and a near-exact character pattern is exemplified by a combination of characters having the same characters in the same order but without any case sensitivity, such as may be implemented in the alternative embodiment and CAM described above. A character segment pattern set is exemplified by a combination of characters having the same characters in the same order, but only for a segment of each one of the terms, such as the character match operation and CAM described above. A character operation pattern set is exemplified by a combination of characters that are operated on by a numerical operator and/or logic operator, and a truncated character operation pattern set is exemplified by truncating the character operation pattern set, such as implemented in the preferred embodiment described above.
In view of the foregoing, it will be seen that the several advantages of the invention are achieved and attained. The embodiments were chosen and described in order to best explain the principles of the invention and its practical application to thereby enable others skilled in the art to best utilize the invention in various embodiments and with various modifications as are suited to the particular use contemplated.
As various modifications could be made in the constructions and methods herein described and illustrated without departing from the scope of the invention, it is intended that all matter contained in the foregoing description or shown in the accompanying drawings shall be interpreted as illustrative rather than limiting. For example, it will be appreciated that the numerical operation may include an arithmetic operator, a comparison operator, and/or an aggregation operator (such as the summation operator). Thus, the breadth and scope of the present invention should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims appended hereto and their equivalents.

Claims (19)

1. A method of processing a data stream to determine whether any of a plurality of search terms are present therein, each of the data stream and the search terms comprising a plurality of data characters, and wherein each search term has a character length and a corresponding search term data pattern, the method comprising:
simultaneously processing the data stream through a plurality of paths, each path being configured to search the data stream for a data pattern corresponding to a search term character length that is unique to that path and that matches a search term data pattern; and
if a matching data pattern is found within the data stream by any of the plurality of paths, performing an exact match operation to determine if the search term corresponding to the matching search term data pattern found is present within the data stream.
2. The method of claim 1 further comprising:
storing a rule table, the rule table comprising (1) a plurality of search terms and (2) for each search term, an address in a memory where one or more data characters comprising that search term is located;
generating a search term data pattern for each search term in the rule table, the search term data pattern being expressed as a byte value;
generating a matrix comprising a plurality of data values indexed by a plurality of coordinate pairs; and
populating the matrix with the addresses for each search term in the rule table such that each search term's rule table address is located in the matrix at a coordinate pair defined by (1) that search term's character length and (2) that search term's corresponding search term data pattern.
3. The method of claim 2 wherein the simultaneously processing step comprises:
within each path, (1) generating a data stream data pattern using the same methodology as was used to generate the search term data patterns, (2) accessing the matrix to identify the data value stored therein at the coordinate pair defined by (a) the character length unique to that path and (b) the data stream data pattern that was generated within that path, and (3) determining that a search term data pattern match exists if a rule table address is found in the matrix at the unique character length-data stream data pattern coordinate pair.
4. The method of claim 3 further comprising:
if a search term data pattern match exists, retrieving from memory the search term located at the rule table address found in the matrix at the unique character length-data stream data pattern; and
wherein the exact match operation performing step comprises comparing the data stream with the retrieved search term to determine whether the retrieved search term is present in the data stream.
5. The method of claim 4 wherein the data pattern generation methodology comprises a compression operation.
6. The method of claim 5 wherein each data character has a corresponding byte value, and wherein the compression operation is further configured to (1) generate the search term data patterns by, for each search term, summing each of the corresponding byte values for the characters comprising that search term, and (2) generate the data stream data patterns by, for a portion of the data stream within each path that is equal in length to that path's corresponding unique character length, summing each of the corresponding byte values for the characters comprising that portion of the data stream.
7. The method of claim 5 wherein the compression operation is configured to generate at least one selected from the group consisting of a plurality of exact character patterns, a plurality of near-exact character patterns, a plurality of character segment patterns, and a plurality of truncated character operation patterns.
8. The method of claim 4 further comprising:
performing at least one additional processing operation on the data stream prior to the exact match operation performing step, the at least one additional processing operation being configured to reduce a likelihood of the exact match operation being performed on a portion of the data stream that does not include at least one of the search terms.
9. The method of claim 4 further comprising:
receiving the data stream from a public computer network, wherein the data stream comprises at least one data packet.
10. The method of claim 4 further comprising:
receiving the data stream from a local area computer network, wherein the data stream comprises at least one data packet.
11. A system for processing a data stream to determine whether any of a plurality of search terms are present therein, the data stream and the search terms both comprising a plurality of data characters, the system comprising:
a plurality of trap elements configured to simultaneously process the data stream to thereby determine whether any of a plurality of search term data patterns are present therein, each search term data pattern corresponding to at least one search term, each search term having a character length, each trap element being further configured to search the data stream to determine whether at least one search term data pattern corresponding to a search term character length that is unique to that trap element is present therein; and
an exact match lookup unit in communication with the plurality of trap elements, the exact match lookup unit being configured to, dependent upon a search term data pattern being found within the data stream by any of the plurality of trap elements, process at least a portion of the data stream to determine if the search term corresponding to the search term data pattern found within the data stream is exactly present therein.
12. The system of claim 11 further comprising:
a rule table comprising (1) a plurality of search terms and (2) for each search term, an address in a memory where one or more data characters comprising that search term is located;
a search term compression operator configured to generate a search term data pattern for each search term in the rule table, the search term data pattern being expressed as a byte value;
a matrix accessible by the plurality of trap elements, the matrix comprising a plurality of data values indexed by a plurality of coordinate pairs; and
a code segment executable by a processor, the code segment configured to populate the matrix with the addresses for each search term in the rule table such that each search term's rule table address is located in the matrix at a coordinate pair defined by (1) that search term's character length and (2) that search term's corresponding search term data pattern.
13. The system of claim 12 further comprising a plurality of data stream compression operators, each data stream compression operator in communication with a different trap element, each data stream compression operator being configured to generate a data stream data pattern using the same methodology as used by the search term compression operator, and wherein each trap element is further configured to (1) access the matrix to identify the data value stored therein at the coordinate pair defined by (a) the character length unique to that trap element and (b) the data stream data pattern that was generated by the data stream compression operator m communication with that trap element, and (2) determine that a search term data pattern match exists if a rule table address is found in the matrix at the unique character length-data stream data pattern coordinate pair.
14. The system of claim 13 further comprising a code segment executable by a processor, the code segment configured to, if a search term data pattern match exists, retrieve from memory the search term located at the rule table address found in the matrix at the unique character length-data stream data pattern, and wherein the exact match lookup unit is further configured to compare at least a portion of the data stream with the retrieved search term to determine whether the retrieved search term is exactly present therein.
15. The system of claim 14 wherein each data character has a corresponding byte value, and wherein the search term compression operation is further configured to generate the search term data patterns by, for each search term, summing each of the corresponding byte values for the characters comprising that search term, and wherein each data stream compression operator is further configured to generate the data stream data patterns by, for a portion of the data stream within each trap element that is equal in length to that trap element's corresponding unique character length, summing each of the corresponding byte values for the characters comprising that portion of the data stream.
16. A method of generating a matrix for use in searching a data stream to determine whether any of a plurality of search terms are present therein, the method comprising:
storing a rule table, the rule table comprising (1) a plurality of search terms and (2) for each search term, an address in a memory where at least a portion of that search term is located, each search term comprising a plurality of data characters, each search term having a character length;
generating a search term data pattern for each search term in the rule table, the search term data pattern being expressed as a byte value;
generating a matrix comprising a plurality of data values indexed by a plurality of coordinate pairs;
populating the matrix with the addresses for each search term in the rule table such that each search term's rule table address is located in the matrix at a coordinate pair defined by (1) that search term's character length and (2) that search term's corresponding search term data pattern.
17. The method of claim 16 wherein the search term data pattern generating step comprises performing a compression operation on each of the search terms in the rule table.
18. The method of claim 17 wherein each data character has a corresponding byte value, and wherein the search term data pattern generating step further comprises, for each search term, summing each of the corresponding byte values for the characters comprising that search term.
19. The method of claim 16 further comprising:
accessing the matrix when performing a search operation on a data stream to determine whether a search term whose rule table address is located in the matrix is present in that data stream.
US11/208,222 2002-04-25 2005-08-19 System and method for processing a data stream to determine presence of search terms Expired - Fee Related US7464089B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US11/208,222 US7464089B2 (en) 2002-04-25 2005-08-19 System and method for processing a data stream to determine presence of search terms

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US10/132,336 US6959297B2 (en) 2002-04-25 2002-04-25 System and process for searching within a data stream using a pointer matrix and a trap matrix
US11/208,222 US7464089B2 (en) 2002-04-25 2005-08-19 System and method for processing a data stream to determine presence of search terms

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US10/132,336 Continuation US6959297B2 (en) 2002-04-25 2002-04-25 System and process for searching within a data stream using a pointer matrix and a trap matrix

Publications (2)

Publication Number Publication Date
US20060050968A1 US20060050968A1 (en) 2006-03-09
US7464089B2 true US7464089B2 (en) 2008-12-09

Family

ID=29268757

Family Applications (2)

Application Number Title Priority Date Filing Date
US10/132,336 Expired - Fee Related US6959297B2 (en) 2002-04-25 2002-04-25 System and process for searching within a data stream using a pointer matrix and a trap matrix
US11/208,222 Expired - Fee Related US7464089B2 (en) 2002-04-25 2005-08-19 System and method for processing a data stream to determine presence of search terms

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US10/132,336 Expired - Fee Related US6959297B2 (en) 2002-04-25 2002-04-25 System and process for searching within a data stream using a pointer matrix and a trap matrix

Country Status (4)

Country Link
US (2) US6959297B2 (en)
JP (1) JP2005524149A (en)
AU (1) AU2003225183A1 (en)
WO (1) WO2003091910A2 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7685195B2 (en) * 2005-03-24 2010-03-23 Sas Institute Inc. Systems and methods for analyzing web site search terms
US20100287034A1 (en) * 2009-05-08 2010-11-11 David Carter Pope Computer-Implemented Systems and Methods for Determining Future Profitability
US20100318591A1 (en) * 2009-06-12 2010-12-16 Cray Inc. Inclusive or bit matrix to compare multiple corresponding subfields
US20230037564A1 (en) * 2021-08-06 2023-02-09 Bank Of America Corporation System and method for generating optimized data queries to improve hardware efficiency and utilization
US11934402B2 (en) * 2021-08-06 2024-03-19 Bank Of America Corporation System and method for generating optimized data queries to improve hardware efficiency and utilization

Families Citing this family (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6959297B2 (en) 2002-04-25 2005-10-25 Winnow Technology, Llc System and process for searching within a data stream using a pointer matrix and a trap matrix
US7870161B2 (en) 2003-11-07 2011-01-11 Qiang Wang Fast signature scan
US7308561B2 (en) * 2003-12-12 2007-12-11 Alcatel Lucent Fast, scalable pattern-matching engine
US7526804B2 (en) * 2004-02-02 2009-04-28 Microsoft Corporation Hardware assist for pattern matches
US7486673B2 (en) 2005-08-29 2009-02-03 Connect Technologies Corporation Method and system for reassembling packets prior to searching
US7984180B2 (en) 2005-10-20 2011-07-19 Solarflare Communications, Inc. Hashing algorithm for network receive filtering
KR100772523B1 (en) * 2006-08-01 2007-11-01 한국전자통신연구원 Apparatus for detecting intrusion using pattern and method thereof
US8392174B2 (en) 2006-08-07 2013-03-05 International Characters, Inc. Method and apparatus for lexical analysis using parallel bit streams
US20080033974A1 (en) * 2006-08-07 2008-02-07 International Characters, Inc. Method and Apparatus for XML Parsing Using Parallel Bit streams
US20080040345A1 (en) * 2006-08-07 2008-02-14 International Characters, Inc. Method and Apparatus for String Search Using Parallel Bit Streams
US7764205B2 (en) * 2007-08-27 2010-07-27 Comtech Aha Corporation Decompressing dynamic huffman coded bit streams
US8843523B2 (en) * 2009-01-12 2014-09-23 Micron Technology, Inc. Devices, systems, and methods for communicating pattern matching results of a parallel pattern search engine
US9098274B2 (en) 2009-12-03 2015-08-04 Intel Corporation Methods and apparatuses to improve turbo performance for events handling
US20110145205A1 (en) * 2009-12-14 2011-06-16 Sanjeev Jain Packet Boundary Spanning Pattern Matching Based At Least In Part Upon History Information
US8504510B2 (en) * 2010-01-07 2013-08-06 Interdisciplinary Center Herzliya State machine compression for scalable pattern matching
US8458354B2 (en) * 2010-01-27 2013-06-04 Interdisciplinary Center Herzliya Multi-pattern matching in compressed communication traffic
US8627462B2 (en) * 2010-05-10 2014-01-07 Mcafee, Inc. Token processing
US8954661B2 (en) * 2010-11-02 2015-02-10 Intel Corporation Binary search pipeline
US20120150887A1 (en) * 2010-12-08 2012-06-14 Clark Christopher F Pattern matching
US8909813B2 (en) * 2011-03-22 2014-12-09 Ramot At Tel-Aviv University Ltd. Efficient processing of compressed communication traffic
US9223618B2 (en) 2011-09-20 2015-12-29 Intel Corporation Multi-threaded queuing system for pattern matching
US8903715B2 (en) * 2012-05-04 2014-12-02 International Business Machines Corporation High bandwidth parsing of data encoding languages
US20140164434A1 (en) * 2012-12-10 2014-06-12 International Business Machines Corporation Streaming data pattern recognition and processing
US9002846B2 (en) * 2013-03-15 2015-04-07 International Business Machines Corporation Ending tuple processing in a stream-based computing application
US20140324530A1 (en) * 2013-04-30 2014-10-30 Liveops, Inc. Method and system for detecting patters in data streams
US11436593B2 (en) 2016-03-31 2022-09-06 Bitflyer Blockchain, Inc. Transaction processing device, transaction processing method, and program for same

Citations (96)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4593353A (en) 1981-10-26 1986-06-03 Telecommunications Associates, Inc. Software protection method and apparatus
US4868376A (en) 1987-05-15 1989-09-19 Smartcard International Inc. Intelligent portable interactive personal data system
US5051947A (en) * 1985-12-10 1991-09-24 Trw Inc. High-speed single-pass textual search processor for locating exact and inexact matches of a search pattern in a textual stream
US5369605A (en) 1993-07-07 1994-11-29 Dell Usa, L.P. Incremental search content addressable memory for increased data compression efficiency
US5469161A (en) 1992-08-13 1995-11-21 International Business Machines Corporation Algorithm for the implementation of Ziv-Lempel data compression using content addressable memory
US5497488A (en) 1990-06-12 1996-03-05 Hitachi, Ltd. System for parallel string search with a function-directed parallel collation of a first partition of each string followed by matching of second partitions
US5511213A (en) 1992-05-08 1996-04-23 Correa; Nelson Associative memory processor architecture for the efficient execution of parsing algorithms for natural language processing and pattern recognition
US5525982A (en) 1994-04-15 1996-06-11 International Business Machines Corporation Method and means for character string pattern matching for compression and the like using minimal cycles per character
US5532693A (en) 1994-06-13 1996-07-02 Advanced Hardware Architectures Adaptive data compression system with systolic string matching logic
US5542045A (en) 1993-10-15 1996-07-30 Software Security, Inc. Method for interposing a security function in a computer program
US5546463A (en) 1994-07-12 1996-08-13 Information Resource Engineering, Inc. Pocket encrypting and authenticating communications device
US5576985A (en) 1996-03-25 1996-11-19 Holtz; Klaus Integrated content addressable read only memory
US5602764A (en) 1993-12-22 1997-02-11 Storage Technology Corporation Comparing prioritizing memory for string searching in a data compression system
US5631971A (en) 1994-05-24 1997-05-20 Sparrow; Malcolm K. Vector based topological fingerprint matching
US5680612A (en) 1994-03-02 1997-10-21 Ricoh Company, Ltd. Document retrieval apparatus retrieving document data using calculated record identifier
US5737424A (en) 1996-06-04 1998-04-07 Software Security, Inc. Method and system for secure distribution of protected data using elliptic curve systems
US5777608A (en) 1989-03-10 1998-07-07 Board Of Regents, The University Of Texas System Apparatus and method for in-parallel scan-line graphics rendering using content-searchable memories
US5778071A (en) 1994-07-12 1998-07-07 Information Resource Engineering, Inc. Pocket encrypting and authenticating communications device
US5805801A (en) 1997-01-09 1998-09-08 International Business Machines Corporation System and method for detecting and preventing security
US5809145A (en) 1996-06-28 1998-09-15 Paradata Systems Inc. System for distributing digital information
US5826011A (en) 1995-12-26 1998-10-20 Rainbow Technologies, Inc. Method of metering and protecting computer software
US5829051A (en) 1994-04-04 1998-10-27 Digital Equipment Corporation Apparatus and method for intelligent multiple-probe cache allocation
US5845298A (en) 1997-04-23 1998-12-01 Sun Microsystems, Inc. Write barrier system and method for trapping garbage collection page boundary crossing pointer stores
US5856977A (en) 1997-05-15 1999-01-05 Yang; Muh-Rong Distribution network switch for very large gigabit switching architecture
US5867609A (en) 1995-12-07 1999-02-02 Nec Research Institute, Inc. Method for computing correlation operations on partially occluded data
US5907838A (en) 1996-12-10 1999-05-25 Seiko Epson Corporation Information search and collection method and system
US5913216A (en) 1996-03-19 1999-06-15 Lucent Technologies, Inc. Sequential pattern memory searching and storage management technique
US5940389A (en) 1997-05-12 1999-08-17 Computer And Communication Research Laboratories Enhanced partially self-routing algorithm for controller Benes networks
US5987028A (en) 1997-05-12 1999-11-16 Industrial Technology Research Insitute Multiple channel ATM switch
US6005940A (en) 1997-05-16 1999-12-21 Software Security, Inc. System for securely storing and reading encrypted data on a data medium using a transponder
US6079621A (en) 1998-11-13 2000-06-27 Chrysalis-Its Inc. Secure card for E-commerce and identification
US6098089A (en) 1997-04-23 2000-08-01 Sun Microsystems, Inc. Generation isolation system and method for garbage collection
US6119120A (en) 1996-06-28 2000-09-12 Microsoft Corporation Computer implemented methods for constructing a compressed data structure from a data string and for using the data structure to find data patterns in the data string
US6128741A (en) 1998-03-05 2000-10-03 Rainbow Technologies, Inc. Compact transparent dongle device
US6147890A (en) 1997-12-30 2000-11-14 Kawasaki Steel Corporation FPGA with embedded content-addressable memory
US6151598A (en) 1995-08-14 2000-11-21 Shaw; Venson M. Digital dictionary with a communication system for the creating, updating, editing, storing, maintaining, referencing, and managing the digital dictionary
US6167136A (en) 1997-05-16 2000-12-26 Software Security, Inc. Method for preventing copying of digital video disks
US6167393A (en) 1996-09-20 2000-12-26 Novell, Inc. Heterogeneous record search apparatus and method
US6223172B1 (en) 1997-10-31 2001-04-24 Nortel Networks Limited Address routing using address-sensitive mask decimation scheme
US6240436B1 (en) 1998-03-30 2001-05-29 Rainbow Technologies, Inc. High speed montgomery value calculation
US6240407B1 (en) 1998-04-29 2001-05-29 International Business Machines Corp. Method and apparatus for creating an index in a database system
US6253243B1 (en) 1998-12-04 2001-06-26 Sun Microsystems, Inc. Automated trap control for a distributed network management system
US6263313B1 (en) 1998-08-13 2001-07-17 International Business Machines Corporation Method and apparatus to create encoded digital content
US20010013802A1 (en) 1999-07-07 2001-08-16 Ghene Faulcon System and process for high speed interface clock skew correction
US6278782B1 (en) 1997-09-16 2001-08-21 Safenet, Inc. Method of implementing a key recovery system
US6282290B1 (en) 1997-03-28 2001-08-28 Mykotronx, Inc. High speed modular exponentiator
US6282657B1 (en) 1997-09-16 2001-08-28 Safenet, Inc. Kernel mode protection
US6307936B1 (en) 1997-09-16 2001-10-23 Safenet, Inc. Cryptographic key management scheme
US6314506B1 (en) 1998-12-28 2001-11-06 Intel Corporation Method and apparatus for determining a next address within a binary search algorithm
US20010043702A1 (en) 1999-01-15 2001-11-22 Laszlo Elteto USB hub keypad
US20010056540A1 (en) 1997-09-16 2001-12-27 Timothy Ober Secure memory area
US6338056B1 (en) 1998-12-14 2002-01-08 International Business Machines Corporation Relational database extender that supports user-defined index types and user-defined search
US20020034329A1 (en) 1997-02-12 2002-03-21 Stmicroelectronics S.R.I Word recognition device and method
US20020046342A1 (en) 1999-01-15 2002-04-18 Laszlo Elteto Secure IR communication between a keypad and a token
US6397331B1 (en) 1997-09-16 2002-05-28 Safenet, Inc. Method for expanding secure kernel program memory
US6412069B1 (en) 1997-09-16 2002-06-25 Safenet, Inc. Extending crytographic services to the kernel space of a computer operating system
US20020104004A1 (en) 2001-02-01 2002-08-01 Bruno Couillard Method and apparatus for synchronizing real-time clocks of time stamping cryptographic modules
US6438612B1 (en) 1998-09-11 2002-08-20 Ssh Communications Security, Ltd. Method and arrangement for secure tunneling of data between virtual routers
US20020120810A1 (en) 2001-02-28 2002-08-29 Brouwer Roger J. Method and system for patching ROM code
US20020129290A1 (en) 2001-03-06 2002-09-12 Bruno Couillard Method and system for time synchronization
US6453415B1 (en) 1997-09-16 2002-09-17 Safenet, Inc. Method of communicating securely between an application program and a secure kernel
US6463538B1 (en) 1998-12-30 2002-10-08 Rainbow Technologies, Inc. Method of software protection using a random code generator
US20020157003A1 (en) 2001-04-18 2002-10-24 Rouslan Beletski Apparatus for secure digital signing of documents
US20020188858A1 (en) 2001-06-08 2002-12-12 Pijnenburg Beheer N.V. Method and device for making information contents of a volatile semiconductor memory irretrievable
US20020191548A1 (en) 2001-03-22 2002-12-19 Tatu Ylonen Security system for a data communications network
US20030026427A1 (en) 2001-08-02 2003-02-06 Bruno Couillard Method and system providing improved security for the transfer of root keys
US20030028778A1 (en) 2001-08-02 2003-02-06 Bruno Couillard Method and system for securely timestamping digital data
US20030028664A1 (en) 2001-08-02 2003-02-06 Kaijun Tan Method and system for secure distribution and utilization of data over a network
US6523119B2 (en) 1996-12-04 2003-02-18 Rainbow Technologies, Inc. Software protection device and method
US20030065800A1 (en) 2001-09-12 2003-04-03 Raqia Networks Inc. Method of generating of DFA state machine that groups transitions into classes in order to conserve memory
US20030097577A1 (en) 2001-11-20 2003-05-22 Rainbow Technologies, Inc. Software protection method utilizing hidden application code in a protection dynamic link library object
US20030110389A1 (en) 2001-11-06 2003-06-12 Rainbow Technologies, Inc. Token for storing installation software and drivers
US20030108193A1 (en) 2001-10-29 2003-06-12 Rainbow Technologies, Inc. Key generation performance improvement
US20030110379A1 (en) 2001-12-07 2003-06-12 Tatu Ylonen Application gateway system, and method for maintaining security in a packet-switched information network
US20030110208A1 (en) 2001-09-12 2003-06-12 Raqia Networks, Inc. Processing data across packet boundaries
US20030123491A1 (en) 2000-12-13 2003-07-03 Bruno Couillard Method and system for time synchronization
US20030163803A1 (en) 2002-02-25 2003-08-28 Daniel Wyschogrod Fast hardware processing of regular expressions containing sub-expressions
US20030163738A1 (en) 2002-02-25 2003-08-28 Bruno Couillard Universal password generator
US6671808B1 (en) 1999-01-15 2003-12-30 Rainbow Technologies, Inc. USB-compliant personal key
US6678734B1 (en) 1999-11-13 2004-01-13 Ssh Communications Security Ltd. Method for intercepting network packets in a computing device
US20040015905A1 (en) 2000-10-27 2004-01-22 Antti Huima Method for managing compiled filter code
US20040034674A1 (en) 2002-08-16 2004-02-19 Safenet B.V. Generator circuit for generating large numbers
US6704871B1 (en) 1997-09-16 2004-03-09 Safenet, Inc. Cryptographic co-processor
US6708273B1 (en) 1997-09-16 2004-03-16 Safenet, Inc. Apparatus and method for implementing IPSEC transforms within an integrated circuit
US20040059907A1 (en) 2002-09-20 2004-03-25 Rainbow Technologies, Inc. Boot-up and hard drive protection using a USB-compliant token
US20040057430A1 (en) 2002-06-28 2004-03-25 Ssh Communications Security Corp. Transmission of broadcast packets in secure communication connections between computers
US20040066274A1 (en) 2002-10-07 2004-04-08 Doug Bailey Tamper detection and secure power failure recovery circuit
US20040098585A1 (en) 2002-11-05 2004-05-20 Rainbow Technologies, Inc. Secure authentication using hardware token and computer fingerprint
US20040098596A1 (en) 2002-11-15 2004-05-20 Rainbow Technologies, Inc. Driverless USB security token
US6807553B2 (en) 2001-04-23 2004-10-19 Safenet B.V. Digital true random number generator circuit
US6842896B1 (en) 1999-09-03 2005-01-11 Rainbow Technologies, Inc. System and method for selecting a server in a multiple server license management system
US6848045B2 (en) 1999-01-15 2005-01-25 Rainbow Technologies, Inc. Integrated USB connector for personal token
US6856981B2 (en) 2001-09-12 2005-02-15 Safenet, Inc. High speed data stream pattern recognition
US6941404B2 (en) 2000-12-19 2005-09-06 Safenet B.V. Data transfer device, transaction system and method for exchanging control and I/O data with a data processing system
US6959297B2 (en) 2002-04-25 2005-10-25 Winnow Technology, Llc System and process for searching within a data stream using a pointer matrix and a trap matrix
US6973565B2 (en) 2001-05-09 2005-12-06 Safenet Canada, Inc. Biometrically secured memory IC

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0859333A1 (en) 1997-02-12 1998-08-19 STMicroelectronics S.r.l. Method of coding characters for word recognition and word recognition device using that coding
US20020019629A1 (en) 1998-07-10 2002-02-14 Medtronic, Inc. Devices, systems and methods for transluminally and controllably forming intramyocardial channels in cardiac tissue
US6311183B1 (en) * 1998-08-07 2001-10-30 The United States Of America As Represented By The Director Of National Security Agency Method for finding large numbers of keywords in continuous text streams

Patent Citations (100)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4593353A (en) 1981-10-26 1986-06-03 Telecommunications Associates, Inc. Software protection method and apparatus
US5051947A (en) * 1985-12-10 1991-09-24 Trw Inc. High-speed single-pass textual search processor for locating exact and inexact matches of a search pattern in a textual stream
US4868376A (en) 1987-05-15 1989-09-19 Smartcard International Inc. Intelligent portable interactive personal data system
US5777608A (en) 1989-03-10 1998-07-07 Board Of Regents, The University Of Texas System Apparatus and method for in-parallel scan-line graphics rendering using content-searchable memories
US5497488A (en) 1990-06-12 1996-03-05 Hitachi, Ltd. System for parallel string search with a function-directed parallel collation of a first partition of each string followed by matching of second partitions
US5511213A (en) 1992-05-08 1996-04-23 Correa; Nelson Associative memory processor architecture for the efficient execution of parsing algorithms for natural language processing and pattern recognition
US5469161A (en) 1992-08-13 1995-11-21 International Business Machines Corporation Algorithm for the implementation of Ziv-Lempel data compression using content addressable memory
US5369605A (en) 1993-07-07 1994-11-29 Dell Usa, L.P. Incremental search content addressable memory for increased data compression efficiency
US5542045A (en) 1993-10-15 1996-07-30 Software Security, Inc. Method for interposing a security function in a computer program
US5602764A (en) 1993-12-22 1997-02-11 Storage Technology Corporation Comparing prioritizing memory for string searching in a data compression system
US5680612A (en) 1994-03-02 1997-10-21 Ricoh Company, Ltd. Document retrieval apparatus retrieving document data using calculated record identifier
US5829051A (en) 1994-04-04 1998-10-27 Digital Equipment Corporation Apparatus and method for intelligent multiple-probe cache allocation
US5525982A (en) 1994-04-15 1996-06-11 International Business Machines Corporation Method and means for character string pattern matching for compression and the like using minimal cycles per character
US5631971A (en) 1994-05-24 1997-05-20 Sparrow; Malcolm K. Vector based topological fingerprint matching
US5532693A (en) 1994-06-13 1996-07-02 Advanced Hardware Architectures Adaptive data compression system with systolic string matching logic
US5546463A (en) 1994-07-12 1996-08-13 Information Resource Engineering, Inc. Pocket encrypting and authenticating communications device
US5778071A (en) 1994-07-12 1998-07-07 Information Resource Engineering, Inc. Pocket encrypting and authenticating communications device
US5878142A (en) 1994-07-12 1999-03-02 Information Resource Engineering, Inc. Pocket encrypting and authenticating communications device
US6151598A (en) 1995-08-14 2000-11-21 Shaw; Venson M. Digital dictionary with a communication system for the creating, updating, editing, storing, maintaining, referencing, and managing the digital dictionary
US5867609A (en) 1995-12-07 1999-02-02 Nec Research Institute, Inc. Method for computing correlation operations on partially occluded data
US5826011A (en) 1995-12-26 1998-10-20 Rainbow Technologies, Inc. Method of metering and protecting computer software
US5913216A (en) 1996-03-19 1999-06-15 Lucent Technologies, Inc. Sequential pattern memory searching and storage management technique
US5576985A (en) 1996-03-25 1996-11-19 Holtz; Klaus Integrated content addressable read only memory
US5737424A (en) 1996-06-04 1998-04-07 Software Security, Inc. Method and system for secure distribution of protected data using elliptic curve systems
US5809145A (en) 1996-06-28 1998-09-15 Paradata Systems Inc. System for distributing digital information
US6119120A (en) 1996-06-28 2000-09-12 Microsoft Corporation Computer implemented methods for constructing a compressed data structure from a data string and for using the data structure to find data patterns in the data string
US6167393A (en) 1996-09-20 2000-12-26 Novell, Inc. Heterogeneous record search apparatus and method
US6523119B2 (en) 1996-12-04 2003-02-18 Rainbow Technologies, Inc. Software protection device and method
US5907838A (en) 1996-12-10 1999-05-25 Seiko Epson Corporation Information search and collection method and system
US6070162A (en) 1996-12-10 2000-05-30 Seiko Epson Corporation Information search and collection system
US5805801A (en) 1997-01-09 1998-09-08 International Business Machines Corporation System and method for detecting and preventing security
US6442295B2 (en) 1997-02-12 2002-08-27 Stmicroelectronics S.R.L. Word recognition device and method
US20020034329A1 (en) 1997-02-12 2002-03-21 Stmicroelectronics S.R.I Word recognition device and method
US6282290B1 (en) 1997-03-28 2001-08-28 Mykotronx, Inc. High speed modular exponentiator
US6098089A (en) 1997-04-23 2000-08-01 Sun Microsystems, Inc. Generation isolation system and method for garbage collection
US5845298A (en) 1997-04-23 1998-12-01 Sun Microsystems, Inc. Write barrier system and method for trapping garbage collection page boundary crossing pointer stores
US5987028A (en) 1997-05-12 1999-11-16 Industrial Technology Research Insitute Multiple channel ATM switch
US5940389A (en) 1997-05-12 1999-08-17 Computer And Communication Research Laboratories Enhanced partially self-routing algorithm for controller Benes networks
US5856977A (en) 1997-05-15 1999-01-05 Yang; Muh-Rong Distribution network switch for very large gigabit switching architecture
US6005940A (en) 1997-05-16 1999-12-21 Software Security, Inc. System for securely storing and reading encrypted data on a data medium using a transponder
US6167136A (en) 1997-05-16 2000-12-26 Software Security, Inc. Method for preventing copying of digital video disks
US6307936B1 (en) 1997-09-16 2001-10-23 Safenet, Inc. Cryptographic key management scheme
US6453415B1 (en) 1997-09-16 2002-09-17 Safenet, Inc. Method of communicating securely between an application program and a secure kernel
US6412069B1 (en) 1997-09-16 2002-06-25 Safenet, Inc. Extending crytographic services to the kernel space of a computer operating system
US6708273B1 (en) 1997-09-16 2004-03-16 Safenet, Inc. Apparatus and method for implementing IPSEC transforms within an integrated circuit
US6278782B1 (en) 1997-09-16 2001-08-21 Safenet, Inc. Method of implementing a key recovery system
US6397331B1 (en) 1997-09-16 2002-05-28 Safenet, Inc. Method for expanding secure kernel program memory
US6282657B1 (en) 1997-09-16 2001-08-28 Safenet, Inc. Kernel mode protection
US6704871B1 (en) 1997-09-16 2004-03-09 Safenet, Inc. Cryptographic co-processor
US20010056540A1 (en) 1997-09-16 2001-12-27 Timothy Ober Secure memory area
US6223172B1 (en) 1997-10-31 2001-04-24 Nortel Networks Limited Address routing using address-sensitive mask decimation scheme
US6147890A (en) 1997-12-30 2000-11-14 Kawasaki Steel Corporation FPGA with embedded content-addressable memory
US6128741A (en) 1998-03-05 2000-10-03 Rainbow Technologies, Inc. Compact transparent dongle device
US6240436B1 (en) 1998-03-30 2001-05-29 Rainbow Technologies, Inc. High speed montgomery value calculation
US6240407B1 (en) 1998-04-29 2001-05-29 International Business Machines Corp. Method and apparatus for creating an index in a database system
US6345256B1 (en) 1998-08-13 2002-02-05 International Business Machines Corporation Automated method and apparatus to package digital content for electronic distribution using the identity of the source content
US6263313B1 (en) 1998-08-13 2001-07-17 International Business Machines Corporation Method and apparatus to create encoded digital content
US6438612B1 (en) 1998-09-11 2002-08-20 Ssh Communications Security, Ltd. Method and arrangement for secure tunneling of data between virtual routers
US6079621A (en) 1998-11-13 2000-06-27 Chrysalis-Its Inc. Secure card for E-commerce and identification
US6253243B1 (en) 1998-12-04 2001-06-26 Sun Microsystems, Inc. Automated trap control for a distributed network management system
US6338056B1 (en) 1998-12-14 2002-01-08 International Business Machines Corporation Relational database extender that supports user-defined index types and user-defined search
US6314506B1 (en) 1998-12-28 2001-11-06 Intel Corporation Method and apparatus for determining a next address within a binary search algorithm
US6463538B1 (en) 1998-12-30 2002-10-08 Rainbow Technologies, Inc. Method of software protection using a random code generator
US6848045B2 (en) 1999-01-15 2005-01-25 Rainbow Technologies, Inc. Integrated USB connector for personal token
US6671808B1 (en) 1999-01-15 2003-12-30 Rainbow Technologies, Inc. USB-compliant personal key
US20010043702A1 (en) 1999-01-15 2001-11-22 Laszlo Elteto USB hub keypad
US20020046342A1 (en) 1999-01-15 2002-04-18 Laszlo Elteto Secure IR communication between a keypad and a token
US20010013802A1 (en) 1999-07-07 2001-08-16 Ghene Faulcon System and process for high speed interface clock skew correction
US6842896B1 (en) 1999-09-03 2005-01-11 Rainbow Technologies, Inc. System and method for selecting a server in a multiple server license management system
US6678734B1 (en) 1999-11-13 2004-01-13 Ssh Communications Security Ltd. Method for intercepting network packets in a computing device
US20040015905A1 (en) 2000-10-27 2004-01-22 Antti Huima Method for managing compiled filter code
US20030123491A1 (en) 2000-12-13 2003-07-03 Bruno Couillard Method and system for time synchronization
US6941404B2 (en) 2000-12-19 2005-09-06 Safenet B.V. Data transfer device, transaction system and method for exchanging control and I/O data with a data processing system
US20020104004A1 (en) 2001-02-01 2002-08-01 Bruno Couillard Method and apparatus for synchronizing real-time clocks of time stamping cryptographic modules
US20020120810A1 (en) 2001-02-28 2002-08-29 Brouwer Roger J. Method and system for patching ROM code
US20020129290A1 (en) 2001-03-06 2002-09-12 Bruno Couillard Method and system for time synchronization
US20020191548A1 (en) 2001-03-22 2002-12-19 Tatu Ylonen Security system for a data communications network
US20020157003A1 (en) 2001-04-18 2002-10-24 Rouslan Beletski Apparatus for secure digital signing of documents
US6807553B2 (en) 2001-04-23 2004-10-19 Safenet B.V. Digital true random number generator circuit
US6973565B2 (en) 2001-05-09 2005-12-06 Safenet Canada, Inc. Biometrically secured memory IC
US20020188858A1 (en) 2001-06-08 2002-12-12 Pijnenburg Beheer N.V. Method and device for making information contents of a volatile semiconductor memory irretrievable
US20030026427A1 (en) 2001-08-02 2003-02-06 Bruno Couillard Method and system providing improved security for the transfer of root keys
US20030028664A1 (en) 2001-08-02 2003-02-06 Kaijun Tan Method and system for secure distribution and utilization of data over a network
US20030028778A1 (en) 2001-08-02 2003-02-06 Bruno Couillard Method and system for securely timestamping digital data
US20030110208A1 (en) 2001-09-12 2003-06-12 Raqia Networks, Inc. Processing data across packet boundaries
US20030065800A1 (en) 2001-09-12 2003-04-03 Raqia Networks Inc. Method of generating of DFA state machine that groups transitions into classes in order to conserve memory
US6856981B2 (en) 2001-09-12 2005-02-15 Safenet, Inc. High speed data stream pattern recognition
US20030108193A1 (en) 2001-10-29 2003-06-12 Rainbow Technologies, Inc. Key generation performance improvement
US20030110389A1 (en) 2001-11-06 2003-06-12 Rainbow Technologies, Inc. Token for storing installation software and drivers
US20030097577A1 (en) 2001-11-20 2003-05-22 Rainbow Technologies, Inc. Software protection method utilizing hidden application code in a protection dynamic link library object
US20030110379A1 (en) 2001-12-07 2003-06-12 Tatu Ylonen Application gateway system, and method for maintaining security in a packet-switched information network
US20030163803A1 (en) 2002-02-25 2003-08-28 Daniel Wyschogrod Fast hardware processing of regular expressions containing sub-expressions
US20030163738A1 (en) 2002-02-25 2003-08-28 Bruno Couillard Universal password generator
US6959297B2 (en) 2002-04-25 2005-10-25 Winnow Technology, Llc System and process for searching within a data stream using a pointer matrix and a trap matrix
US20040057430A1 (en) 2002-06-28 2004-03-25 Ssh Communications Security Corp. Transmission of broadcast packets in secure communication connections between computers
US20040034674A1 (en) 2002-08-16 2004-02-19 Safenet B.V. Generator circuit for generating large numbers
US20040059907A1 (en) 2002-09-20 2004-03-25 Rainbow Technologies, Inc. Boot-up and hard drive protection using a USB-compliant token
US20040066274A1 (en) 2002-10-07 2004-04-08 Doug Bailey Tamper detection and secure power failure recovery circuit
US20040098585A1 (en) 2002-11-05 2004-05-20 Rainbow Technologies, Inc. Secure authentication using hardware token and computer fingerprint
US20040098596A1 (en) 2002-11-15 2004-05-20 Rainbow Technologies, Inc. Driverless USB security token

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
Brenner, "Extending APL for pattern matching", ACM 1983, pp. 275-280.
Cohen, "An n-gram hash and skip algorithm for finding large numbers of keywords in continuous text streams," Software-Practice and Experience, 1998, pp. 1605-1635, vol. 28.
Cohen, "Hardware-assisted algorithm for full-text large-dictionary string matching using n-gram hashing," Information Processing & Management, 1998, pp. 443-464, vol. 34.
Cohen, "Recursive hashing functions for n-grams", ACM 1997, pp. 291-320.

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7685195B2 (en) * 2005-03-24 2010-03-23 Sas Institute Inc. Systems and methods for analyzing web site search terms
US20100287034A1 (en) * 2009-05-08 2010-11-11 David Carter Pope Computer-Implemented Systems and Methods for Determining Future Profitability
US8185432B2 (en) 2009-05-08 2012-05-22 Sas Institute Inc. Computer-implemented systems and methods for determining future profitability
US8473331B2 (en) 2009-05-08 2013-06-25 Sas Institute Inc. Computer-implemented systems and methods for determining future profitability
US20100318591A1 (en) * 2009-06-12 2010-12-16 Cray Inc. Inclusive or bit matrix to compare multiple corresponding subfields
US8954484B2 (en) * 2009-06-12 2015-02-10 Cray Inc. Inclusive or bit matrix to compare multiple corresponding subfields
US9547474B2 (en) 2009-06-12 2017-01-17 Cray Inc. Inclusive or bit matrix to compare multiple corresponding subfields
US20230037564A1 (en) * 2021-08-06 2023-02-09 Bank Of America Corporation System and method for generating optimized data queries to improve hardware efficiency and utilization
US11934402B2 (en) * 2021-08-06 2024-03-19 Bank Of America Corporation System and method for generating optimized data queries to improve hardware efficiency and utilization

Also Published As

Publication number Publication date
WO2003091910A2 (en) 2003-11-06
US6959297B2 (en) 2005-10-25
WO2003091910A3 (en) 2004-05-27
AU2003225183A1 (en) 2003-11-10
JP2005524149A (en) 2005-08-11
US20030208487A1 (en) 2003-11-06
AU2003225183A8 (en) 2003-11-10
US20060050968A1 (en) 2006-03-09

Similar Documents

Publication Publication Date Title
US7464089B2 (en) System and method for processing a data stream to determine presence of search terms
US10949641B2 (en) Fast signature scan
US7411418B2 (en) Efficient representation of state transition tables
Lakshminarayanan et al. Algorithms for advanced packet classification with ternary CAMs
US7110540B2 (en) Multi-pass hierarchical pattern matching
US7454418B1 (en) Fast signature scan
US7917509B2 (en) String search scheme in a distributed architecture
US7134143B2 (en) Method and apparatus for data packet pattern matching
EP2421223A2 (en) Method and Apparatus for Approximate Pattern Matching
US20080189784A1 (en) Method and Apparatus for Deep Packet Inspection
WO2004013777A1 (en) System and method of parallel pattern matching
JP2014238892A (en) Method, computer system and computer program for retrieving secured data
CA2281103C (en) N-way processing of bit strings in a dataflow architecture
Pao et al. String searching engine for virus scanning
US6687715B2 (en) Parallel lookups that keep order
US7567568B2 (en) Method and apparatus for user identification in computer traffic
US7536717B2 (en) Fast searching of list for IP filtering
JP3443356B2 (en) Packet classifier
Nguyen et al. BBFex: A Flexible and Efficient FPGA-based Pattern Matching Engine for Large Database
Wang High speed string matching for virus scanning-Quick Sampling and Verification
Kim High performance DFA-based string matching

Legal Events

Date Code Title Description
AS Assignment

Owner name: WINNOW TECHNOLOGIES INC., JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:WINNOW TECHNOLOGY, LLC;REEL/FRAME:017882/0152

Effective date: 20041112

AS Assignment

Owner name: CONNECT TECHNOLOGIES CORPORATION, JAPAN

Free format text: NUNC PRO TUNC ASSIGNMENT;ASSIGNOR:WINNOW TECHNOLOGIES, INC.;REEL/FRAME:021104/0493

Effective date: 20080617

CC Certificate of correction
REMI Maintenance fee reminder mailed
LAPS Lapse for failure to pay maintenance fees
STCH Information on status: patent discontinuation

Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362

FP Lapsed due to failure to pay maintenance fee

Effective date: 20121209