WO2005017702A2 - Integrated circuit apparatus and method for high throughput signature based network applications - Google Patents
Integrated circuit apparatus and method for high throughput signature based network applications Download PDFInfo
- Publication number
- WO2005017702A2 WO2005017702A2 PCT/US2004/026335 US2004026335W WO2005017702A2 WO 2005017702 A2 WO2005017702 A2 WO 2005017702A2 US 2004026335 W US2004026335 W US 2004026335W WO 2005017702 A2 WO2005017702 A2 WO 2005017702A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- network
- module
- coupled
- support member
- memory
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 103
- 230000009471 action Effects 0.000 claims abstract description 19
- 238000001914 filtration Methods 0.000 claims abstract description 13
- 230000015654 memory Effects 0.000 claims description 121
- 230000007704 transition Effects 0.000 claims description 47
- 230000008569 process Effects 0.000 claims description 36
- 238000004422 calculation algorithm Methods 0.000 claims description 29
- 208000037656 Respiratory Sounds Diseases 0.000 claims description 26
- 206010037833 rales Diseases 0.000 claims description 26
- 238000000605 extraction Methods 0.000 claims description 24
- 238000012545 processing Methods 0.000 claims description 23
- 230000006870 function Effects 0.000 claims description 22
- 230000014509 gene expression Effects 0.000 claims description 22
- 238000007726 management method Methods 0.000 claims description 16
- 241000288140 Gruiformes Species 0.000 claims description 15
- 238000004891 communication Methods 0.000 claims description 12
- 230000002123 temporal effect Effects 0.000 claims description 10
- 238000012546 transfer Methods 0.000 claims description 10
- 238000001514 detection method Methods 0.000 claims description 9
- 238000012544 monitoring process Methods 0.000 claims description 6
- 230000002265 prevention Effects 0.000 claims description 6
- 238000003909 pattern recognition Methods 0.000 claims description 5
- 239000000758 substrate Substances 0.000 claims description 5
- 230000002155 anti-virotic effect Effects 0.000 claims description 4
- 230000006837 decompression Effects 0.000 claims description 4
- 230000011664 signaling Effects 0.000 claims description 4
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 claims description 3
- 238000003780 insertion Methods 0.000 claims description 3
- 230000037431 insertion Effects 0.000 claims description 3
- 229910052710 silicon Inorganic materials 0.000 claims description 3
- 239000010703 silicon Substances 0.000 claims description 3
- 238000003860 storage Methods 0.000 claims description 3
- 238000012550 audit Methods 0.000 claims description 2
- 230000008859 change Effects 0.000 claims description 2
- 238000012217 deletion Methods 0.000 claims description 2
- 230000037430 deletion Effects 0.000 claims description 2
- 230000000977 initiatory effect Effects 0.000 claims description 2
- 230000008447 perception Effects 0.000 claims description 2
- 238000006467 substitution reaction Methods 0.000 claims description 2
- 230000002093 peripheral effect Effects 0.000 claims 4
- 125000004122 cyclic group Chemical group 0.000 claims 2
- 238000012805 post-processing Methods 0.000 claims 2
- 230000004044 response Effects 0.000 claims 2
- 230000002457 bidirectional effect Effects 0.000 claims 1
- 238000007781 pre-processing Methods 0.000 abstract description 5
- 230000006855 networking Effects 0.000 description 12
- 230000004048 modification Effects 0.000 description 10
- 238000012986 modification Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 9
- 230000008901 benefit Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 4
- 230000001960 triggered effect Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 241000700605 Viruses Species 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000007796 conventional method Methods 0.000 description 2
- 235000014510 cooky Nutrition 0.000 description 2
- 230000007812 deficiency Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 101000701853 Rattus norvegicus Serine protease inhibitor A3N Proteins 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000001627 detrimental effect Effects 0.000 description 1
- 230000003090 exacerbative effect Effects 0.000 description 1
- RGNPBRKPHBKNKX-UHFFFAOYSA-N hexaflumuron Chemical compound C1=C(Cl)C(OC(F)(F)C(F)F)=C(Cl)C=C1NC(=O)NC(=O)C1=C(F)C=CC=C1F RGNPBRKPHBKNKX-UHFFFAOYSA-N 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 238000000059 patterning Methods 0.000 description 1
- 230000001012 protector Effects 0.000 description 1
- 229910000679 solder Inorganic materials 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 238000012384 transportation and delivery Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
- H04L63/0227—Filtering policies
- H04L63/0236—Filtering by address, protocol, port number or service, e.g. IP-address or URL
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
- H04L63/0227—Filtering policies
- H04L63/0245—Filtering by information in the payload
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
- H04L65/752—Media network packet handling adapting media to network capabilities
Definitions
- the invention relates to computer networking security applications. More particularly, the invention includes an integrated circuit implementation of an apparatus for signature based network applications acting upon network packets and stream data at wire- speed. According to a specific embodiment, the invention includes an apparatus and method for high throughput flow classification of packets into network streams, packet reassembly of such streams (where desired), filtering and pre-processing of such streams (including protocol decoding where desired), pattern matching on header and payload content of such streams, and action execution based upon rule-based policy for multiple network applications, simultaneously at wire speed.
- the invention has been applied to networking devices, which are been distributed throughout local, wide area, and world wide area networks.
- Packets are routed between computers using specially developed algorithms that allow computers and network equipment to decide along which path the packet should be sent to arrive at its final destination. These algorithms examine the packet header (typically a fixed sized portion of the packet containing information such as the source and destination address of the packet added to the payload to be transported) to make routing decisions. The algorithms need to examine the packet and make the decision very quickly to allow large numbers of packets to be sent with very small delay.
- the contents of the packet may be examined for information to aid in making decisions about the path and priority given to a packet; this examination of the data however adds an overhead that can limit the throughput and delay imposed by the device examining the data - typically the more data to be searched the longer the delay incurred by searching it.
- a piece of email which is sent across a network as a series of packets may be examined to see if it is an unwanted email message (commonly referred to as 'spam'); this examination often desires looking at the contents of the message, which is the payload portion of the packets involved in carrying the email. Similarly the email may be scanned to see if it contains a computer virus. Packets may also be examined to look for copyright infringements, illegal activity such as computer 'hacking' or corporate espionage, or simply to analyze usage to offer a better quality of service. By examining packets in a network new applications are now being offered, and it can reasonably be expected that new network applications based on the examining of packets will continue to be developed.
- Network equipment also works under several constraints; the total time that a packet takes to get from an ingress interface to an egress interface needs to be kept to a minimum. The time it takes for a packet to travel through a communication device or channel is called latency. The latency introduced by a device must not only be kept to a minimum, but must also be kept relatively constant; change in latency, is known as jitter. Jitter, in particular, adversely affects multimedia streams.
- Routing and other decisions are typically done wholly on the information provided within the single packet, but if a particular pattern is being searched for in a stream, it is desirable to find it even if it spans across the boundaries between two or more packets. Thus, to do proper searching of streams it is essential to provide some mechanism for dealing with fragmented and out of order packets.
- This finite automaton can be "executed" to search for patterns; this execution involves the calculation of a transition function, which defines transitions from one state of the finite automaton to another state of the finite automaton, each transition being triggered by a single piece of input, called a symbol, from the data being searched.
- the invention includes an integrated circuit implementation of an apparatus for signature based network applications acting upon network packets and stream data at wire-speed.
- the invention includes an apparatus and method for high throughput (e.g., 10,000,000 bits per second and greater) flow classification of packets into network streams, packet reassembly of such streams (where desired), filtering and pre-processing of such streams (including protocol decoding where desired), pattern matching on header and payload content of such streams, and action execution based upon rule-based policy for multiple network applications, simultaneously at wire speed.
- high throughput e.g. 10,000,000 bits per second and greater
- the invention has been applied to networking devices, which are been distributed throughout local, wide area, and world wide area networks.
- the apparatus comprises a rigid support member (e.g., printed circuit board, substrate, silicon substrate, integrated circuit module) comprising a connector region, which has a network connection region and a host connection region.
- the rigid support member has a selected width and a selected length. The selected width and selected length are adapted to couple via the connector region into a network system.
- the connector region is directly connected into a common interface bus.
- One or more hardware modules e.g., integrated circuits, integrated circuit modules
- is disposed e.g., solder bumps
- the one or more hardware modules includes a network interface module coupled to the rigid support member.
- the network interface module includes one or more network interface ports.
- the one or more network interface ports is coupled via the connector region to a packet based network.
- the one or more network interface ports contains one or more ingress network ports.
- a network interface bus is coupled to the rigid support member.
- the network interface bus is adapted to interface the network interface module to the network module.
- a network module is coupled to the rigid support member.
- the network module is coupled to the network interface bus.
- a network event module is coupled to the rigid support member.
- the network event module is coupled to the network module.
- a memory module is coupled to the rigid support member and the memory module is coupled to the network event module and the network module.
- the memory module includes a pattern memory. The pattern memory is associated with a plurality of pre-stored patterns.
- a host interface module is coupled to the rigid support member and is coupled to the network event module, or the network module, or both.
- a host interface bus is coupled to the rigid support member.
- the host interface bus is coupled to the host interface module and is capable of connecting to the host system via the connector region.
- the invention can use one or more pre-stored patterns.
- the pre-stored patterns can include regular expressions, n-gram expressions (e.g., tuple of symbols), among others.
- the memory module additionally comprises a feature memory; which is associated with a plurality of pre-stored features.
- a rule memory is also associated with a plurality of pre-stored rules.
- the network module includes a feature extraction device, which is coupled to the network module and the memory module.
- the feature extraction device is also capable of identifying a feature association according to a feature extraction algorithm.
- the feature extraction algorithm identifies a feature association based upon examination of one or more packets according to some predetermined functionality.
- the feature association identifies one or more of a plurality of pre- stored features.
- the pre-stored features are stored in a feature memory.
- a policy device is coupled to the feature extraction device and the memory module.
- the policy device identifies a rale association based upon the feature association identified by the feature extraction device according to a policy algorithm.
- the policy algorithm identifies the rule association by examining the feature association according to some pre-determined functionality.
- the rule association identifies one or more of a plurality of pre-stored rules, which are stored in a rule memory.
- the feature extraction algorithm can be an approximate pattern matching process for at least one or more of the predetermined patterns.
- the approximate pattern matching process is performed on streams of data from text files of data, text streams of data, binary files of data, binary streams of data, audio streams of data, audio files of data, video streams of data, video files of data, multimedia streams of data, and multimedia files of data, any combination of these, and the like.
- the measure of approximation in the approximate pattern matching process is an edit distance, which can be the number of insertions, deletions or substitutions desired to exactly match the pattern.
- the measure of approximation in the approximate pattern matching process can also be related to human perception, among other factors.
- the invention provides a method for performing high throughput pattern matching.
- the high throughput pattern matching operation is performed using one or more of a plurality of patterns; which are defined by a Regular Language as understood in the art.
- the patterns are defined by a Regular Language.
- the Regular Language is implemented as a Finite Automaton.
- the Finite Automaton includes a transition table representation of the Regular Language.
- the transition table describes a transition function for the Finite Automaton.
- the transition table is adapted to be stored in a compressed form.
- the compressed form is adapted such that the transition function of the Finite Automaton is able to be computed from the compressed form in a maximum time that is constant with respect to the size of the compressed form.
- the pattern matching is provided at wire speed in an efficient and cost effective manner.
- the invention provides an apparatus for performing high throughput pattern matching.
- the high throughput pattern matching operation is performed using one or more of a plurality of patterns.
- the patterns are represented as a single pattern database.
- the single pattern database comprises the patterns from one or more of a plurality of applications.
- the pattern matching operation is able to uniquely identify the application from the matching pattern.
- the Finite Automaton includes a transition table representation of the Regular Language.
- the transition table describes a transition function for the Finite Automaton.
- the invention provides a method for converting a network system into an accelerated signature based network system.
- the method includes providing a network system.
- the network system comprises a host memory coupled to the host processor, a host interface bus coupled to the host processor, and a host connector coupled to the host interface bus.
- the method also includes providing an Integrated Circuit Apparatus for high throughput pattern matching for network applications.
- the apparatus a rigid support member comprises a connector region, which includes a network connection region and a host connection region.
- the rigid support member has a selected width and a selected length. The selected width and selected length are adapted to couple via the connector region into a network system.
- one or more hardware modules is disposed onto and coupled to the rigid support member.
- the one or more hardware modules includes a Network Interface Module coupled to the rigid support member.
- the Network Interface Module includes one or more network interface ports.
- the one or more network interface ports is coupled via the connector region to a Packet Based Network.
- the one or more network interface ports contains one or more ingress network ports.
- a Network Interface Bus is coupled to the rigid support member.
- the Network Interface Bus is adapted to interface the Network Interface Module to the Network Module.
- a Network Module is coupled to the rigid support member.
- the Network Module is coupled to the Network Interface Bus.
- a Network Event Module is coupled to the rigid support member.
- the Network Event Module is coupled to the Network Module.
- a Memory Module is coupled to the rigid support member.
- the Memory Module is coupled to the Network Event Module and the Network Module.
- the Memory Module includes a Pattern Memory.
- the Pattern Memory is associated with a plurality of pre-stored patterns.
- a Host Interface Module is coupled to the rigid support member.
- the Host Interface Module is coupled to the Network Event Module and/or the Network Module.
- a Host Interface Bus is coupled to the rigid support member.
- the Host Interface Bus is coupled to the Host Interface Module.
- the Host Interface Bus is capable of connecting to the host system via the connector region.
- the method includes connecting the host interface connector region of the Integrated Circuit Apparatus with the host connector on the network system to mechanically and electrically couple the host interface bus of the network system to the host interface bus of the Integrated Circuit Apparatus.
- the method includes transferring selected driver software to the network system.
- the driver software is configured to facilitate communication between the Integrated Circuit Apparatus and the network system via the host interface bus.
- the method includes initializing the Integrated Circuit Apparatus via the driver software.
- the invention provides a method for signature based pattern recognition using an Integrated Circuit Apparatus.
- the method includes providing an Integrated Circuit Apparatus for high throughput pattern matching for network applications.
- the apparatus includes a rigid support member comprising a connector region.
- the connector region includes a network connection region and a host connection region.
- the rigid support member has a selected width and a selected length.
- the selected width and selected length are adapted to couple via the connector region into a network system.
- one or more hardware modules is disposed onto and coupled to the rigid support member.
- the one or more hardware modules including A Network Interface Module coupled to the rigid support member.
- the Network Interface Module includes one or more network interface ports.
- the one or more network interface ports is coupled via the connector region to a Packet Based Network.
- the one or more network interface ports contains one or more ingress network ports.
- a Network Interface Bus is coupled to the rigid support member.
- the Network Interface Bus is adapted to interface the Network Interface Module to the Network Module.
- a Network Module is coupled to the rigid support member.
- the Network Module is coupled to the Network Interface Bus.
- a Network Event Module is coupled to the rigid support member.
- the Network Event Module is coupled to the Network Module.
- a Memory Module is coupled to the rigid support member.
- the Memory Module is coupled to the Network Event Module and the Network Module.
- the Memory Module includes a Pattern Memory.
- the Pattern Memory is associated with a plurality of pre-stored patterns.
- a Host Interface Module is coupled to the rigid support member.
- the Host Interface Module is coupled to the Network Event Module and/or the Network Module.
- a Host Interface Bus is coupled to the rigid support member.
- the Host Interface Bus is coupled to the Host Interface Module.
- the Host Interface Bus is capable of connecting to the host system via the connector region.
- the method includes transferring information from a Packet Based Network to a network interface port and transferring the information from the network interface port through a network interface bus.
- the method includes receiving the information from the network interface bus at a processing unit and identifying an association between one or more packets and a flow from the information using the processing unit.
- the one or more packets are reordered into one or more respective flows.
- the method also includes determining if the one or more packets for the one or more respective flows is associated with a signature based pattern stored in memory through a memory bus coupled to the processing unit, where upon the determining occurs using the memory having a random access time of less than 8 nanoseconds.
- a signal is initiated to a policy engine based upon the determining step.
- the invention can also perform pattern matching with high throughput.
- the transition function used by the Finite Automaton should have a constant time complexity that guarantees transitions can be achieved within a fixed bound, the fixed bound being defined by the throughput to be achieved. This is achieved, in part, by using memories with low random access times, such as modern static RAMs.
- the invention also conserves memory usage by the pattern database, without unduly restricting the number of patterns in the pattern database.
- This can be achieved using compression technologies such as those described in United States provisional patent 60/473,373 filed May 23, 2003, commonly assigned, and titled “Apparatus and Method for Large Hardware Finite State Machine with Embedded Equivalence,” and United States provisional patent No. 60/454,398 filed on March 12, 2003, commonly assigned, and titled “Apparatus and Method for Memory Efficient Programmable Pattern Matching Finite State Machine Hardware.”
- compression technologies such as those described in United States provisional patent 60/473,373 filed May 23, 2003, commonly assigned, and titled “Apparatus and Method for Large Hardware Finite State Machine with Embedded Equivalence," and United States provisional patent No. 60/454,398 filed on March 12, 2003, commonly assigned, and titled “Apparatus and Method for Memory Efficient Programmable Pattern Matching Finite State Machine Hardware.”
- other similar technologies obvious to those trained in the art, to reduce the
- a key to these technologies is their low and constant latency overhead, which not only results in compact memory usage, but also high throughput. This lower memory usage results in either a lower cost for production of a given system, or a larger capacity of signatures for a given cost of system.
- the present invention including the apparatus can be adapted to fit within a wide range of existing and new network systems by being of a generic form factor and connecting through a standard hardware interface requiring no hardware re-engineering of the network system in order for it to be adapted to use the apparatus.
- Multiple applications can run simultaneously.
- the multiple applications are able to have separate databases and separate rule databases yet have the hardware apparatus run all applications simultaneously at wire speed; wire speed being the maximum throughput possible for the given physical medium in use according to other embodiments.
- the invention provides pattern databases, rule sets, and hence applications that can be updated through the host or the network without manual intervention as either new signatures are provided or new applications.
- the architecture being designed in such a way as to provide a common format for signature based services.
- the invention provides for minimizing upper bound worst case jitter and latency. This is accomplished through implementing core network functions in hardware, rather than in software such as in the kernel of a computer operating system or in a software TCP/IP stack. Furthermore combining these network functions with pattern matching functions in hardware, so that they are tightly coupled, results in a system with lower latency and jitter.
- this invention allows for protocol decoding to be tightly coupled to these network and pattern matching functions so that, in hardware, packets can: be received, classified and reordered; be decoded according to protocol definitions, and have multiple application pattern matching applied.
- protocol decoding allows for protocol decoding to be tightly coupled to these network and pattern matching functions so that, in hardware, packets can: be received, classified and reordered; be decoded according to protocol definitions, and have multiple application pattern matching applied.
- Temporal regular expressions being any expanded set of regular expressions that contain a temporal component. This temporal component allows searching across the data content, but with the additional benefit of being able to utilize information about relative and absolute timing information.
- Figure 1 depicts a typical network environment including of a Packet Based Network [100], a number of network systems [101], [102], [103] and a number of hosts connected to a Local Area Network (LAN) [104] according to an embodiment of the present invention.
- LAN Local Area Network
- Figure 2 depicts an embodiment of the Integrated Circuit Apparatus of this invention on a rigid support member (such as a card) [201] according to an embodiment of the present invention.
- Figure 3 depicts a block diagram of an embodiment of the Integrated Circuit Apparatus [300] according to an embodiment of the present invention.
- Figure 4 depicts a functional block diagram of an embodiment of the Integrated Circuit Apparatus running in a look-aside (passive) mode of operation according to an embodiment of the present invention.
- Figure 5 depicts a functional diagram of the Integrated Circuit Apparatus running in a look-aside (passive) mode of operation with the inclusion of a Protocol Decoder [513] according to an embodiment of the present invention.
- Figure 6 depicts a functional diagram of an embodiment of the Integrated Circuit Apparatus running in a look-aside (passive) mode of operation with the inclusion of an Update module [614] according to an embodiment of the present invention.
- Figure 7 illustrates that in one embodiment of the present invention multiple sets of patterns [701, 702, 703, 704], one for each application that is executing on the Apparatus, will be present in the Memory [700] of the Apparatus according to an embodiment of the present invention.
- Figure 8 is a flowchart of several of the processes running according to an embodiment of the present invention.
- Figure 9 depicts a flow classification process according to an embodiment of the present invention.
- Figure 10 depicts a functional block diagram of the present invention including the configurable insertion of flexible Stream Processor Blocks [1005] between each of the functional units [1000, 1001, 1002, 1003, 1004] according to an embodiment of the present invention.
- Figure 11 depicts an example taxonomy of Stream Processors according to an embodiment of the present invention.
- Figure 12 depicts an example representation of a plurality of patterns by a Regular Language and method for matching against compressed representation of the Regular Language according to an embodiment of the present invention.
- Figure 13 is a flowchart for converting an existing network system into an accelerated signature based network system according to an embodiment of the present invention.
- the invention includes an integrated circuit implementation of an apparatus for signature based network applications acting upon network packets and stream data at wire-speed.
- the invention includes an apparatus and method for high throughput flow classification of packets into network streams, packet reassembly of such streams (where desired), filtering and preprocessing of such streams (including protocol decoding where desired), pattern matching on header and payload content of such streams, and action execution based upon rule-based policy for multiple network applications, simultaneously at wire speed.
- the invention has been applied to networking devices, which are been distributed throughout local, wide area, and world wide area networks.
- the invention comprises an apparatus and method for performing pattern matching for network applications using specialized hardware.
- This present architecture allows the implementation of high throughput signature based network applications on packet based networks up to wire speed.
- the novel architecture specifically includes hardware support for pattern matching networking and security operations.
- This architecture is suited to high performance security systems based upon signature matching. These systems include Intrusion Detection Systems, Intrusion Prevention Systems, Antivirus Gateways, Email Scanning Gateways, Content Filtering Systems, Anti-spam Systems, Content Protection Systems, Bandwidth/Quality of Service Management, Content Monitoring Systems, Network Monitoring Systems, and many others.
- Another novel aspect of the invention is that the apparatus is adapted to couple to a variety of network systems including Firewalls, Network Appliances, Security Appliances, Servers and other Network Equipment, which have been described in more detail below.
- Figure 1 depicts several examples of network systems which could be coupled to different embodiments of the apparatus. These examples are merely illustrative and should not limit the scope of the claims herein. One of ordinary skill in the art would recognize many variations, alternatives, and modifications. These examples include a look-aside network system at [101], an inline network system at [102] and a network server at [103], and possibly other elements.
- the network systems has a Look-aside Gateway Monitoring Device (e.g. network monitor or Intrusion Detection System) [101], a Gateway System (e.g. Router, Firewall or Switch) [102] connecting the LAN to the Packet Based Network [100] and a Host System (e.g.
- a Host System e.g.
- the apparatus [201] is shown in Figure 2. This figure is merely an example, which should not unduly limit the scope of the claims herein. One of ordinary skill in the art would recognizes many variations, modifications, and alternatives.
- This apparatus may be coupled to a network system [200] through a connector region.
- Embodiments of the connector region which connect to the Host System include PCI and Compact-PCI standards which define the electrical and mechanical interfaces.
- the rigid support member has a selected width and selected length, being adapted to couple into a network system [200] such as network appliance, server or network node.
- the rigid support member is suitable to server as a substrate (e.g., printed circuit board, silicon substrate, integrated circuit package) for a number of integrated circuit devices and other hardware, which will be used to implement an embodiment of the present invention.
- the rigid support member also includes a common bus, which can be coupled to any conventional network appliance, server, or network node.
- the apparatus includes a number of modules for performing high throughput analysis (e.g., wire speed) on network traffic as shown in Figure 3.
- This figure is merely an example, which should not unduly limit the scope of the claims herein.
- Signals are received from the ingress network port within the Network Interface Module [301] according to the physical transmission medium (e.g. optical, electrical). Data is extracted from these signals in the form of bits. This data is passed to the Network Module [302] over the Network Interface Bus [301] (These bits then undergo a number of network preprocessing functions in order to extract the relevant data content). The data is packed into packets before being classified into a flow by the Flow Classification Device.
- the packet is then placed in Flow Memory (within the Memory Module [308]) until the Flow Assembler Device uses the packet to reconstruct a flow.
- the flow is then decoded according to pre-defined protocols (e.g. by the Protocol Decoder), filters and preprocessors to produce data content streams.
- the feature extraction can be thought of, in one embodiment, as a pattern matching process with a database of signatures provided by Pattern Memory within the Memory Module).
- the extracted features then trigger a message to the Policy Device, which interprets these features according to policies and rules (as provided by the Rule Memory), generating events and actions which are communicated to the Host System [304] via the Host Interface Module [309] and Host Interface Bus [310].
- the Host Interface Bus being a standard hardware bus (e.g. PCI) so that the Integrated Circuit Apparatus can easily be integrated with a wide range of existing network equipment.
- an Update Module [311] which is controlled either by the Host System or a remote device across the Packet Based Network (coupled to the Network Interface Port via the Connector Region [301]).
- the Update Module adapting to update any of the memories within the Memory Module, so as to provide updates to patterns, protocol definitions, rales and other device properties.
- the apparatus connects to a Packet Based Network through the connector region [303].
- this connector region is the RJ-45 connector for IEEE 802 Ethernet.
- the network can include, among others, SONET, ATM, and others. Packets are received from the Packet Based Network through this region by the Network Interface Module [301], which may include a number of ingress network ports.
- One embodiment of the Packet Based Network is an Internet Protocol (IP) network.
- IP Internet Protocol
- the Network Interface Module handles the translation of incoming electrical or optical signals into digital bits, and assembles those bits into packets according to a predefined specification (e.g. in one embodiment the IEEE802 Ethernet specification).
- the Network Interface Module couples to a Network Module [302] via a Network Interface Bus [305].
- the Network Interface Bus in several embodiments includes the UTOPIA, SPI-3 and CSIX bus standards.
- the Network Module includes a number of devices which take these digital bits and perform network processing functions.
- the Network Module receives packets of data from the Network Interface Module and provides the Network Event Module [307] with decoded, contiguous streams of data.
- the Network Module may be provided by a single Network Processing Unit (NPU), and in others by a combination of integrated circuits, such as an NPU and Classification Processor.
- the Network Module is coupled to a Memory Module [308], which provides memory for a variety of devices and databases as explained herein.
- the Network Module provides a Flow Classification Device, which is responsible for identifying an association between each incoming packet and a flow, where a flow is a predetermined sequence of packets from a source address to a destination network address.
- the Flow Classification device then identifies the flow queue within Flow Memory (provided by the Memory Module) on which to place the packet, according to this association.
- the Flow Classification Device is coupled to a Flow Assembler Device, which manages the flow queues on a per-flow basis for these incoming packets, and effectively reorders the packets, according to a predetermined specification. In one embodiment, this specification would be TCP/IP.
- the Flow Assembler may, in one embodiment, couple to a Protocol Decoder which in turn is coupled to Protocol Memory, provided by the Memory Module.
- the Protocol may, in one embodiment, couple to a Protocol Decoder which in turn is coupled to Protocol Memory, provided by the Memory Module.
- Memory contains a plurality of network protocol definitions, which are used by the Protocol Decoder to identify salient protocol features from the network flow.
- examples of such features maybe source and destination email addresses as part of an SMTP e-mail message.
- Figure 9 depicts the flow classification process for one embodiment of the present invention, h [900] packets from multiple flows arrive serially and possibly out of order.
- the first step in flow classification is to determine on which flow queue to place each packet.
- each packet is placed in such a queue and the queue is sorted into correct sequence as determined by some pre-determined algorithm (e.g. sequence numbers in TCP/IP).
- the Network Event Module [307] includes a number of devices, and analyses whole network streams to extract relevant features and then apply rules (or policy) to these features in order to signal, via events, the Host Interface Module [309].
- the Network Event Module is searching streams of data using pattern matching algorithms, and then analyzing these matches according to a rule set, in order to then notify the Host System of relevant network events.
- the Network Event Module is provided by a Field Programmable Gate Array (FPGA). Incoming data streams from the Network Module are passed to the Feature Extraction Device, which identifies features of importance; a matchable representation of these features being stored within a Pattern Memory provided by the Memory Module.
- FPGA Field Programmable Gate Array
- these patterns may be compiled representations of Regular Expressions, Deterministic Finite Automata, Berkeley Packet Filter expressions, or Approximate Signatures.
- these databases of signatures may relate to a plurality of distinct applications executing simultaneously. Matched features are passed to a Policy Device, which analyses the features in relation to a database of rules, provided by the Rule Memory within the Memory Module. These rales are used to make higher level decisions based upon a predetermined schema, as provided by the applications related to these rules. In some embodiments, this allows aggregation of matched features (important in denial-of-service attack detection for Intrusion Detection Systems), or selective rule set enabling (e.g.
- these databases of rales may relate to a plurality of separate applications executing simultaneously.
- the Policy Device may, as a result of a rale, identify an action that needs to be performed.
- such actions may include signaling the Host System via the Host Interface Module, signaling the Network Module to drop or modify (in the case that the apparatus is inline) a packet or plurality of packets, or triggering a counter or timer.
- the Host Interface Module may be coupled to the Network Event Module and/or the Network Module.
- the Host Interface Module is responsible for the interfacing of the apparatus modules with the Host System.
- the Host Interface Module is coupled to the Host System via the Host Interface Bus [310], via the host component of the connector region [304]. In one embodiment, this may be communications across a PCI bus, where the PCI standard defines the characteristics of the Host Interface Bus and Connector Region.
- the Host Interface Module is provided by a separate ASIC or FPGA.
- a suitable FPGA or ASIC has interfaces to low latency RAM, at least 5,000 logic cells, multiple clocking domains, internal block RAM and a high speed data bus.
- the FPGA can be one such as the Virtex 2 Pro manufactured by Xilinx, hie, but can be others.
- it may include an NPU, where the NPU has multiple processing units (e.g. micro-engines), an interface to multiple banks of low latency RAM and a high speed data bus.
- the NPU can be an IXP 2400 manufactured by Intel Co ⁇ oration.
- the Host Interface Module will facilitate the signaling of the Host System by the Network Event Module according to triggered rales and/or actions.
- the Host Interface Module is coupled to an Update Module [311], and facilitates communications between the Update Module and the Host System, so that the Update Module may update one or more of the databases provided within the Memory Module.
- the Update Module is responsible for the management of the databases provided within the Memory Module.
- the Update Module is responsible for the updating of the patterns in the Pattern Memory, the protocol definitions in the Protocol Memory and the Rule databases in the Rule Memory.
- the Update Module may authenticate this process via the Authentication Device according to a pre-determined specification.
- the Authentication Device in some embodiments, will do so in a cryptographically strong manner to maintain authenticity, integrity and confidentiality of the updates.
- the Authentication Device may provide hardware support for the acceleration of cryptographic primitives.
- the updates are provided by the Host System via the Host Interface Module, and in other embodiments, by a remote system on the Packet Based Network via the Network Module (possibly connected to the Apparatus on a separate Management Interface).
- the Integrated Circuit Apparatus may be operating in-line such that triggered rales make decisions to drop or modify packets, before passing such packets out on an egress network interface, being provided by the Network Interface Module.
- the Network Event Module will identify such a decision, and signal the Network Module to perform the operation in its Flow Post Processor.
- FIGs [4, 5, 6] different embodiments of the Integrated Circuit Apparatus are represented, showing the data flow from the Packet Based Network, through to the Host System.
- the Integrated Circuit Apparatus executes network applications on packets arriving from the Packet Based Network [400].
- the packets are first received via the Network Interface Port [401], where they are translated from physical signals (e.g. electrical, optical) into bits and arranged into packets of data.
- These packets of data are then passed to a Flow Classification Device [402] that associates each packet with a network flow.
- These packets are then assembled into flows by the Flow Assembler Device [403].
- the Flow Assembler Device then passes data, in the form of reassembled flows, through to a Feature Extraction Device [404].
- the Feature Extraction Device identifies patterns or signatures within these flows from a database of patterns [410] stored within a Pattern Memory [409], and signals successful matches to the Policy Device [305].
- the Policy Device associates one or more matches with events according to a database of rales [412] stored in a Rule Memory [411], translating the matches into network events and associated actions.
- the Policy Device communicates to the Host System [406] messages about these events, actions and other state information via the Host Interface Device [407].
- the messages can include an access control list update message, an audit message, an event message, an alarm message, a status message, a query message, an update message, a management message, an error message, a warning message, any combination of these and the like.
- the Host Interface Device couples to the Host System through the Host Interface Port [407], which translates the message bits into physical signals suitable for transmission.
- the packets are received via the Network Interface Port [501], where they are translated from physical signals (e.g. electrical, optical) into bits and arranged into packets of data. These packets of data are then passed to a Flow Classification Device [502] that associates each packet with a network flow. These packets are then assembled into flows by the Flow Assembler Device [503]. The Flow Assembler Device then passes data in the form of reassembled flows through to a Protocol Decoder, which parses the flows according to network protocol descriptions into protocol content flows. These protocol content flows are then passed to the Feature Extraction Device [504].
- a Protocol Decoder parses the flows according to network protocol descriptions into protocol content flows.
- the Feature Extraction Device identifies patterns or signatures within these protocol content flows from a database of patterns [510] stored within a Pattern Memory [509], and signals successful matches to the Policy Device [505].
- the Policy Device associates one or more matches with events according to a database of rales [512] stored in a Rule Memory [511], translating the matches into network events and associated network actions.
- the Policy Device communicates to the Host System [508] messages about these events, actions and other state information via the Host Interface Device [506].
- the Host Interface Device couples to the Host System through the Host Interface Port [507], which translates the message bits into physical signals suitable for transmission.
- Figure 8 shows logical operations within the apparatus in embodiments of the invention.
- a high level description of these operations is as follows: in one process [800], packets are received from an ingress network interface, classified as belonging to a flow and queued in Flow Memory. In a second process [801], packets are read from the Flow Memory, reassembled into a contiguous flow. In a third process [802], these reassembled flows are then analyzed for relevant features, the identification of which, desires a decision to be made, based upon a rale database, as to whether to trigger an action, notify the host and the like.
- the Integrated Circuit Apparatus is operating in a flow through mode of operation, hi this mode, a fourth process [803], takes packets that have been processed, and may drop them completely or modify them before they are transmitted on an egress network interface.
- Diagram [800] shows the packet receipt process, which includes: waiting for a packet to become available on an ingress network interface port, receiving such packet, classifying the packet according to a flow, then placing the packet in Flow Memory.
- Diagram [801] shows another process that waits for such packets to be queued in Flow
- Diagram [802] depicts a further process which checks the pattern queues for ready data; then removes such data off the queue, updating the context of the device to that of the flow of the current data, extracting the features that are found from such a flow. If no features are found, then the process waits for the next available packet, otherwise it triggers any rales that may be associated with the triggered feature. If the rule is associated with an action, the process then triggers the associated action (e.g. flagging, the notification of the Host System, to drop or modify the packet). Should the host warrant notification by the rule, a message is then passed to the Host System with any relevant information (e.g.
- [803] is a process which rans for some embodiments of the invention (when the apparatus is running in the "flow-through", otherwise known as “active” or “inline", mode of operation). In this case, the process waits for packets in the Flow Memory to be flagged as processed, it then removes the packet from the queue and either drops or retransmits the packet on the egress interface depending on the action being executed.
- FIG. 7 illustrates that the Integrated Circuit Apparatus may have multiple procedures running simultaneously on network traffic.
- each application may have its own rule definitions within rule memory.
- the operation of the modules within this device [600, 601, 602, 603, 604, 605, 606, 607, 608, 609, 610, 611, 612, 613] are the same as for Figure 5, with the exception that the Host System [608] may, via the Host Interface Device [606], communicate to the Update Device updates of either of the pattern database, or the rale database.
- the Update Device controls the management of these updates within the memories [609, 611].
- the databases may be updated through a management protocol over the Packet Based Network [600] via the Network Interface Module.
- each procedure may have its own pattern database in Pattern Memory, and rale database in Rule Memory.
- Such databases may not necessarily be stored within separate memory blocks in hardware form, and may instead be compact hardware representations within a single database.
- Some embodiments of the invention include Stream Processor Blocks [1005], which can contain several Stream Processors [1006], as shown in Figure 10. Each Stream Processor Block may include one or more Stream Processors [1006].
- the Stream Processors can be one or more in a series of algorithmic units that act upon a packet or stream of packets; several examples of the blocks that can be placed in [1006] are shown in Figure 11.
- Figure 11 depicts an example taxonomy of Stream Processors including a Null Processor [1100] which copies data input directly to output with no modification, a MIME Decoder [1101] which decodes MIME encoded data, a Digest Generator [1102] which takes a data stream and outputs some subset or digest of such data (e.g. packet headers), a Unicode Decoder [1103] which decodes Unicode encoded data, an XML Parser [1104] which parses and decodes XML encoded data according to some predetermined specification, a Checksum Verifier [1105] which performs a checksum operation of input data according to some predetermined specification (e.g.
- a Decompression Processor [1106] which decompresses input data streams according to some predetermined algorithm (e.g. zip), a URL Decoder [1107] which decodes an HTTP encoded URL, a Packet Filter [1108] which filters input data according to some predetermined specification (e.g. BPF), an HTTP Cookie Handler [1109] which parses input data according to the HTML or related specification and decodes a Cookie within the stream and then performs some predetermined function, a Decryption Processor [1110] which decrypts input data according to some predetermined specification (e.g.
- a method for performing high throughput pattern matching according to the present invention is outlined as follows.
- transition table in compressed form such the transition function of the finite automata is able to be computed from the compressed form in a predetermined (e.g., maximum) time that is constant with respect to the size of the compressed form;
- the above sequence of steps provides a method for high throughput pattern matching using a Regular language.
- the method performs high throughput pattern matching using, for example, the hardware and software described herein. That is, the pattern matching process and storage of patterns can be implemented in the hardware and software features described in one or more of the figures and descriptions.
- the high throughput pattern matching operation is performed using one or more of a plurality of patterns.
- the patterns are preferably defined by a regular language; which has been implemented as a finite automaton.
- the finite automaton includes a transition table representation of the regular language.
- the transition table describes a transition function for the finite automaton.
- the transition table is adapted to be stored in a compressed fonn, which is adapted such that the transition function of the finite automaton is able to be computed from the compressed form in a predetermined time (e.g., maximum time) that is constant with respect to the size of the compressed form.
- a predetermined time e.g., maximum time
- the computation of the next state of the finite automata from the current state and incoming data is independent of the size of the compressed transition table, and is constant. In order that high throughput be achieved, this computation should take less than 40 nanoseconds. In another embodiment of the invention, the compressed transition table should occupy less than one-fifth the space of the original transition table.
- [1200] shows the Regular Language for expressing two example patterns.
- the first pattern represents the character "a” followed zero or more "b” characters, followed by the character "c”.
- the second pattern represents the literal string "de”.
- the patterns are combined by the "
- the ".*" at the front of the Regular Language expression indicates that it can match the patterns anywhere within given data.
- the finite automata for implementing the Regular Language defined by [1200] is depicted in [1210]. Only the main transitions are shown for clarity. Those trained in the art will recognize the finite automata [1210] as being an implementation of the patterns defined by the Regular Language [1200].
- the transition table [1220] expression of the finite automata fully defines all transitions within the automata. This transition table should be compressed in order to conserve memory, and used for matching the patterns against incoming data.
- the method for performing high throughput pattern matching according to the present invention is outlined in flowchart [1230]. As shown, the flow chart includes processes of start (e.g., initiation), express patterns by regular expression, implement regular language as finite automata, compress transition table from finite automata, store (e.g., memory) transition table in compressed form, and perform patterning matching process. Depending upon the embodiment, certain steps may be combined or even separated further. Additionally, one or more steps may be inserted or even exchanged for others. Depending upon the embodiment, the functionality can be performed in software, hardware, or a combination of hardware and software without departing from the scope of the claims herein.
- a method for converting a network system into an accelerated signature based network system is outlined as follows.
- a network system e.g., conventional network, IP based, network
- the integrated circuit apparatus Connect the integrated circuit apparatus to the network system, e.g., a firewall, a network management system, an intrusion prevention system, a router, a network switch, a logging system, a network appliance, a security system; an anti- viras system, an anti-spam system, an intrusion detection system, a content filtering system, a network monitoring system, a file server, a mail server, a web server, a proxy server, and
- the network system e.g., a firewall, a network management system, an intrusion prevention system, a router, a network switch, a logging system, a network appliance, a security system; an anti- viras system, an anti-spam system, an intrusion detection system, a content filtering system, a network monitoring system, a file server, a mail server, a web server, a proxy server, and
- the method involves replacing one or more existing network interface cards in the network system with the apparatus.
- the present invention provides a method for converting a network system into an accelerated signature based network system. Further details of the present method are provided according to Figure 13. This diagram is merely an example, which should not unduly limit the scope of the claims herein.
- the method includes providing a network system.
- the network system has one or more input ports.
- a host processor is coupled to the one or more input ports.
- a host memory is coupled to the host processor.
- a host interface bus is coupled to the host processor and a host connector is coupled to the host interface bus.
- the method also includes providing an integrated circuit apparatus for high throughput pattern matching for network applications. As merely an example, the present apparatus described herein can be used, as well as others.
- the method also includes connecting the host interface connector region of the integrated circuit apparatus with the host connector on the network system to mechanically and electrically couple the host interface bus of the network system to the host interface bus of the integrated circuit apparatus.
- the method also transfers selected driver software to the network system.
- the driver software is configured to facilitate communication between the integrated circuit apparatus and the network system via the host interface bus.
- the method also initializes the integrated circuit apparatus via the driver software. Once the apparatus has been integrated into the networking system, various methods can be performed. An example of such a method is provided in more detail below and well as other portions of the present specification.
- a method for signature based pattern recognition using an integrated circuit apparatus according to the present invention is outlined as follows.
- the present invention includes a method for signature based pattern recognition using an integrated circuit apparatus.
- the method includes providing an integrated circuit apparatus for high throughput pattern matching for network applications.
- the apparatus can be the one described herein, but can also be others depending upon the embodiment.
- the apparatus is integrated into a pre-existing network via common interface bus without substantial hardware modifications.
- the apparatus is merely inserted into the connector for the common interface bus for preferred embodiments.
- the method then transfers information from a packet based network to a network interface port through the connector and transfers the information from the network interface port through a network interface bus also through the connector.
- the method receives information from the network interface bus at a processing unit and identifies an association between one or more packets and a flow from the information using the processing unit.
- the method reorders the one or more packets into one or more respective flows and determines if the one or more packets for the one or more respective flows is associated with a signature based pattern stored in memory through a memory bus coupled to the processing unit. The determining occurs using the memory having a random access time of less than 8 nanoseconds in preferred embodiments.
- the method initiates a signal to a policy engine on the apparatus if an association occurs.
- various methods can be performed. An example of such a method is provided in more detail below as well as other portions of the present specification.
- the method for signature based pattern recognition further requires the decoding of reordered packets according to specific protocols. The decoding is performed by the processing unit. Some protocols, such as [1104] XML Parsing are shown in Figure 11.
- the present system can also be applied to a variety of applications including intrusion detection, intrusion prevention, firewalling, content filtering, access control, antivirus, network monitoring, traffic filtering, spam filtering, content classification, application-level switching, bandwidth/quality of service management, surveillance, and XML web services, among others. Therefore, the described embodiments should not be limited to the details given herein, but should be defined by the following claims and their full scope of equivalents.
Abstract
Description
Claims
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP04781080A EP1656631A2 (en) | 2003-08-13 | 2004-08-12 | Integrated circuit apparatus and method for high throughput signature based network applications |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/640,870 | 2003-08-13 | ||
US10/640,870 US20050114700A1 (en) | 2003-08-13 | 2003-08-13 | Integrated circuit apparatus and method for high throughput signature based network applications |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2005017702A2 true WO2005017702A2 (en) | 2005-02-24 |
WO2005017702A3 WO2005017702A3 (en) | 2005-07-21 |
Family
ID=34193602
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2004/026335 WO2005017702A2 (en) | 2003-08-13 | 2004-08-12 | Integrated circuit apparatus and method for high throughput signature based network applications |
Country Status (5)
Country | Link |
---|---|
US (3) | US20050114700A1 (en) |
EP (1) | EP1656631A2 (en) |
KR (1) | KR20060080176A (en) |
CN (1) | CN1836245A (en) |
WO (1) | WO2005017702A2 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2008097710A2 (en) * | 2007-02-02 | 2008-08-14 | Tarari, Inc. | Systems and methods for processing access control lists (acls) in network switches using regular expression matching logic |
Families Citing this family (118)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020133842A1 (en) * | 2000-12-08 | 2002-09-19 | Leviten Michael W. | Transgenic mice containing deubiquitinated enzyme gene disruptions |
US7082044B2 (en) * | 2003-03-12 | 2006-07-25 | Sensory Networks, Inc. | Apparatus and method for memory efficient, programmable, pattern matching finite state machine hardware |
US7496662B1 (en) | 2003-05-12 | 2009-02-24 | Sourcefire, Inc. | Systems and methods for determining characteristics of a network and assessing confidence |
CN100499451C (en) * | 2003-08-26 | 2009-06-10 | 中兴通讯股份有限公司 | Network communication safe processor and its data processing method |
US9614772B1 (en) * | 2003-10-20 | 2017-04-04 | F5 Networks, Inc. | System and method for directing network traffic in tunneling applications |
US7002943B2 (en) * | 2003-12-08 | 2006-02-21 | Airtight Networks, Inc. | Method and system for monitoring a selected region of an airspace associated with local area networks of computing devices |
US7440434B2 (en) * | 2004-02-11 | 2008-10-21 | Airtight Networks, Inc. | Method and system for detecting wireless access devices operably coupled to computer local area networks and related methods |
US7536723B1 (en) * | 2004-02-11 | 2009-05-19 | Airtight Networks, Inc. | Automated method and system for monitoring local area computer networks for unauthorized wireless access |
US7216365B2 (en) * | 2004-02-11 | 2007-05-08 | Airtight Networks, Inc. | Automated sniffer apparatus and method for wireless local area network security |
US7219319B2 (en) * | 2004-03-12 | 2007-05-15 | Sensory Networks, Inc. | Apparatus and method for generating state transition rules for memory efficient programmable pattern matching finite state machine hardware |
US7861304B1 (en) * | 2004-05-07 | 2010-12-28 | Symantec Corporation | Pattern matching using embedded functions |
US7539681B2 (en) * | 2004-07-26 | 2009-05-26 | Sourcefire, Inc. | Methods and systems for multi-pattern searching |
US7496962B2 (en) * | 2004-07-29 | 2009-02-24 | Sourcefire, Inc. | Intrusion detection strategies for hypertext transport protocol |
US20060075093A1 (en) * | 2004-10-05 | 2006-04-06 | Enterasys Networks, Inc. | Using flow metric events to control network operation |
US8010685B2 (en) * | 2004-11-09 | 2011-08-30 | Cisco Technology, Inc. | Method and apparatus for content classification |
US7535909B2 (en) | 2004-11-09 | 2009-05-19 | Cisco Technology, Inc. | Method and apparatus to process packets in a network |
US7936682B2 (en) * | 2004-11-09 | 2011-05-03 | Cisco Technology, Inc. | Detecting malicious attacks using network behavior and header analysis |
US8367105B2 (en) * | 2004-11-10 | 2013-02-05 | Teva Pharmaceutical Industries, Ltd. | Compressed solid dosage form manufacturing process well-suited for use with drugs of low aqueous solubility and compressed solid dosage forms made thereby |
US20060198375A1 (en) * | 2004-12-07 | 2006-09-07 | Baik Kwang H | Method and apparatus for pattern matching based on packet reassembly |
US7634584B2 (en) | 2005-04-27 | 2009-12-15 | Solarflare Communications, Inc. | Packet validation in virtual network interface architecture |
US20060277267A1 (en) * | 2005-05-16 | 2006-12-07 | Simon Lok | Unified memory IP packet processing platform |
US8533308B1 (en) | 2005-08-12 | 2013-09-10 | F5 Networks, Inc. | Network traffic management through protocol-configurable transaction processing |
GB0517304D0 (en) * | 2005-08-23 | 2005-10-05 | Netronome Systems Inc | A system and method for processing and forwarding transmitted information |
US7733803B2 (en) * | 2005-11-14 | 2010-06-08 | Sourcefire, Inc. | Systems and methods for modifying network map attributes |
US8046833B2 (en) * | 2005-11-14 | 2011-10-25 | Sourcefire, Inc. | Intrusion event correlation with network discovery information |
US7716577B2 (en) * | 2005-11-14 | 2010-05-11 | Oracle America, Inc. | Method and apparatus for hardware XML acceleration |
US7710933B1 (en) | 2005-12-08 | 2010-05-04 | Airtight Networks, Inc. | Method and system for classification of wireless devices in local area computer networks |
US8565088B1 (en) | 2006-02-01 | 2013-10-22 | F5 Networks, Inc. | Selectively enabling packet concatenation based on a transaction boundary |
US7840726B2 (en) * | 2006-04-12 | 2010-11-23 | Dell Products L.P. | System and method for identifying and transferring serial data to a programmable logic device |
US20080022401A1 (en) * | 2006-07-21 | 2008-01-24 | Sensory Networks Inc. | Apparatus and Method for Multicore Network Security Processing |
US7948988B2 (en) * | 2006-07-27 | 2011-05-24 | Sourcefire, Inc. | Device, system and method for analysis of fragments in a fragment train |
US7725510B2 (en) * | 2006-08-01 | 2010-05-25 | Alcatel-Lucent Usa Inc. | Method and system for multi-character multi-pattern pattern matching |
US7701945B2 (en) | 2006-08-10 | 2010-04-20 | Sourcefire, Inc. | Device, system and method for analysis of segments in a transmission control protocol (TCP) session |
US8136162B2 (en) | 2006-08-31 | 2012-03-13 | Broadcom Corporation | Intelligent network interface controller |
CA2672908A1 (en) * | 2006-10-06 | 2008-04-17 | Sourcefire, Inc. | Device, system and method for use of micro-policies in intrusion detection/prevention |
US8042184B1 (en) * | 2006-10-18 | 2011-10-18 | Kaspersky Lab, Zao | Rapid analysis of data stream for malware presence |
US9015301B2 (en) * | 2007-01-05 | 2015-04-21 | Digital Doors, Inc. | Information infrastructure management tools with extractor, secure storage, content analysis and classification and method therefor |
US20080186971A1 (en) * | 2007-02-02 | 2008-08-07 | Tarari, Inc. | Systems and methods for processing access control lists (acls) in network switches using regular expression matching logic |
US9106606B1 (en) | 2007-02-05 | 2015-08-11 | F5 Networks, Inc. | Method, intermediate device and computer program code for maintaining persistency |
US20080196104A1 (en) * | 2007-02-09 | 2008-08-14 | George Tuvell | Off-line mms malware scanning system and method |
US8069352B2 (en) | 2007-02-28 | 2011-11-29 | Sourcefire, Inc. | Device, system and method for timestamp analysis of segments in a transmission control protocol (TCP) session |
US8127353B2 (en) * | 2007-04-30 | 2012-02-28 | Sourcefire, Inc. | Real-time user awareness for a computer network |
US8416773B2 (en) * | 2007-07-11 | 2013-04-09 | Hewlett-Packard Development Company, L.P. | Packet monitoring |
US8645960B2 (en) * | 2007-07-23 | 2014-02-04 | Redknee Inc. | Method and apparatus for data processing using queuing |
US7733805B2 (en) * | 2007-07-25 | 2010-06-08 | Brocade Communications Systems, Inc. | Method and apparatus for determining bandwidth-consuming frame flows in a network |
CN101360088B (en) * | 2007-07-30 | 2011-09-14 | 华为技术有限公司 | Regular expression compiling, matching system and compiling, matching method |
US8291495B1 (en) * | 2007-08-08 | 2012-10-16 | Juniper Networks, Inc. | Identifying applications for intrusion detection systems |
US9083609B2 (en) * | 2007-09-26 | 2015-07-14 | Nicira, Inc. | Network operating system for managing and securing networks |
US8042185B1 (en) * | 2007-09-27 | 2011-10-18 | Netapp, Inc. | Anti-virus blade |
US8305896B2 (en) * | 2007-10-31 | 2012-11-06 | Cisco Technology, Inc. | Selective performance enhancement of traffic flows |
US8112800B1 (en) | 2007-11-08 | 2012-02-07 | Juniper Networks, Inc. | Multi-layered application classification and decoding |
US7970894B1 (en) | 2007-11-15 | 2011-06-28 | Airtight Networks, Inc. | Method and system for monitoring of wireless devices in local area computer networks |
US20090165139A1 (en) * | 2007-12-21 | 2009-06-25 | Yerazunis William S | Secure Computer System and Method |
US8930926B2 (en) * | 2008-02-08 | 2015-01-06 | Reservoir Labs, Inc. | System, methods and apparatus for program optimization for multi-threaded processor architectures |
US9858053B2 (en) | 2008-02-08 | 2018-01-02 | Reservoir Labs, Inc. | Methods and apparatus for data transfer optimization |
US8661422B2 (en) * | 2008-02-08 | 2014-02-25 | Reservoir Labs, Inc. | Methods and apparatus for local memory compaction |
US8474043B2 (en) * | 2008-04-17 | 2013-06-25 | Sourcefire, Inc. | Speed and memory optimization of intrusion detection system (IDS) and intrusion prevention system (IPS) rule processing |
US20090292775A1 (en) * | 2008-05-20 | 2009-11-26 | Scott Wayne Flenniken | Method and process for the Forensic Inspection of real time streams FIRST Engine |
US8339959B1 (en) | 2008-05-20 | 2012-12-25 | Juniper Networks, Inc. | Streamlined packet forwarding using dynamic filters for routing and security in a shared forwarding plane |
US8955107B2 (en) * | 2008-09-12 | 2015-02-10 | Juniper Networks, Inc. | Hierarchical application of security services within a computer network |
WO2010033622A2 (en) * | 2008-09-17 | 2010-03-25 | Reservoir Labs, Inc. | Methods and apparatus for joint parallelism and locality optimization in source code compilation |
US8272055B2 (en) | 2008-10-08 | 2012-09-18 | Sourcefire, Inc. | Target-based SMB and DCE/RPC processing for an intrusion detection system or intrusion prevention system |
US8572717B2 (en) | 2008-10-09 | 2013-10-29 | Juniper Networks, Inc. | Dynamic access control policy with port restrictions for a network security appliance |
US8040808B1 (en) | 2008-10-20 | 2011-10-18 | Juniper Networks, Inc. | Service aware path selection with a network acceleration device |
KR101276796B1 (en) * | 2008-12-03 | 2013-07-30 | 한국전자통신연구원 | Apparatus and method for matching pattern |
US8688619B1 (en) | 2009-03-09 | 2014-04-01 | Reservoir Labs | Systems, methods and apparatus for distributed decision processing |
US9398043B1 (en) | 2009-03-24 | 2016-07-19 | Juniper Networks, Inc. | Applying fine-grain policy action to encapsulated network attacks |
WO2010127173A2 (en) * | 2009-04-30 | 2010-11-04 | Reservoir Labs, Inc. | System, apparatus and methods to implement high-speed network analyzers |
US8572014B2 (en) * | 2009-10-16 | 2013-10-29 | Mcafee, Inc. | Pattern recognition using transition table templates |
CA2789824C (en) | 2010-04-16 | 2018-11-06 | Sourcefire, Inc. | System and method for near-real time network attack detection, and system and method for unified detection via detection routing |
US8892483B1 (en) | 2010-06-01 | 2014-11-18 | Reservoir Labs, Inc. | Systems and methods for planning a solution to a dynamically changing problem |
US8433790B2 (en) | 2010-06-11 | 2013-04-30 | Sourcefire, Inc. | System and method for assigning network blocks to sensors |
US8671182B2 (en) | 2010-06-22 | 2014-03-11 | Sourcefire, Inc. | System and method for resolving operating system or service identity conflicts |
US8509071B1 (en) | 2010-10-06 | 2013-08-13 | Juniper Networks, Inc. | Multi-dimensional traffic management |
US8914601B1 (en) | 2010-10-18 | 2014-12-16 | Reservoir Labs, Inc. | Systems and methods for a fast interconnect table |
US9134976B1 (en) | 2010-12-13 | 2015-09-15 | Reservoir Labs, Inc. | Cross-format analysis of software systems |
US8848554B2 (en) * | 2011-03-07 | 2014-09-30 | Oracle International Corporation | Packet sniffing with packet filtering hooks |
US8601034B2 (en) | 2011-03-11 | 2013-12-03 | Sourcefire, Inc. | System and method for real time data awareness |
US9489180B1 (en) | 2011-11-18 | 2016-11-08 | Reservoir Labs, Inc. | Methods and apparatus for joint scheduling and layout optimization to enable multi-level vectorization |
US8681794B2 (en) | 2011-11-30 | 2014-03-25 | Broadcom Corporation | System and method for efficient matching of regular expression patterns across multiple packets |
US8724496B2 (en) * | 2011-11-30 | 2014-05-13 | Broadcom Corporation | System and method for integrating line-rate application recognition in a switch ASIC |
US9830133B1 (en) | 2011-12-12 | 2017-11-28 | Significs And Elements, Llc | Methods and apparatus for automatic communication optimizations in a compiler based on a polyhedral representation |
US9251535B1 (en) | 2012-01-05 | 2016-02-02 | Juniper Networks, Inc. | Offload of data transfer statistics from a mobile access gateway |
US9648133B2 (en) * | 2012-03-12 | 2017-05-09 | Telefonaktiebolaget L M Ericsson | Optimizing traffic load in a communications network |
KR101222486B1 (en) * | 2012-04-13 | 2013-01-16 | 주식회사 페타바이 | Method, server, terminal, and computer-readable recording medium for selectively eliminating nondeterministic element of nondeterministic finite automata |
US9798588B1 (en) | 2012-04-25 | 2017-10-24 | Significs And Elements, Llc | Efficient packet forwarding using cyber-security aware policies |
US10936569B1 (en) | 2012-05-18 | 2021-03-02 | Reservoir Labs, Inc. | Efficient and scalable computations with sparse tensors |
US9684865B1 (en) | 2012-06-05 | 2017-06-20 | Significs And Elements, Llc | System and method for configuration of an ensemble solver |
US10771448B2 (en) * | 2012-08-10 | 2020-09-08 | Cryptography Research, Inc. | Secure feature and key management in integrated circuits |
US9250954B2 (en) * | 2013-01-17 | 2016-02-02 | Xockets, Inc. | Offload processor modules for connection to system memory, and corresponding methods and systems |
US20140236908A1 (en) * | 2013-02-20 | 2014-08-21 | Verizon Patent And Licensing Inc. | Method and apparatus for providing enhanced data retrieval with improved response time |
US10599697B2 (en) | 2013-03-15 | 2020-03-24 | Uda, Llc | Automatic topic discovery in streams of unstructured data |
US10204026B2 (en) | 2013-03-15 | 2019-02-12 | Uda, Llc | Realtime data stream cluster summarization and labeling system |
US10698935B2 (en) | 2013-03-15 | 2020-06-30 | Uda, Llc | Optimization for real-time, parallel execution of models for extracting high-value information from data streams |
US10430111B2 (en) | 2013-03-15 | 2019-10-01 | Uda, Llc | Optimization for real-time, parallel execution of models for extracting high-value information from data streams |
WO2014145092A2 (en) | 2013-03-15 | 2014-09-18 | Akuda Labs Llc | Hierarchical, parallel models for extracting in real time high-value information from data streams and system and method for creation of same |
US9426124B2 (en) | 2013-04-08 | 2016-08-23 | Solarflare Communications, Inc. | Locked down network interface |
US10742604B2 (en) | 2013-04-08 | 2020-08-11 | Xilinx, Inc. | Locked down network interface |
US9563399B2 (en) | 2013-08-30 | 2017-02-07 | Cavium, Inc. | Generating a non-deterministic finite automata (NFA) graph for regular expression patterns with advanced features |
US10002326B2 (en) | 2014-04-14 | 2018-06-19 | Cavium, Inc. | Compilation of finite automata based on memory hierarchy |
US10110558B2 (en) | 2014-04-14 | 2018-10-23 | Cavium, Inc. | Processing of finite automata based on memory hierarchy |
US10097582B2 (en) * | 2014-11-25 | 2018-10-09 | International Business Machines Corporation | Secure data redaction and masking in intercepted data interactions |
US9787638B1 (en) | 2014-12-30 | 2017-10-10 | Juniper Networks, Inc. | Filtering data using malicious reference information |
US9807117B2 (en) | 2015-03-17 | 2017-10-31 | Solarflare Communications, Inc. | System and apparatus for providing network security |
US20160308669A1 (en) * | 2015-04-20 | 2016-10-20 | Jian Ho | Method and System for Real Time Data Protection with Private Key and Algorithm for Transmission and Storage |
US9729329B2 (en) * | 2015-05-19 | 2017-08-08 | Nxp B.V. | Communications security |
US10200391B2 (en) * | 2015-09-23 | 2019-02-05 | AVAST Software s.r.o. | Detection of malware in derived pattern space |
US20170093770A1 (en) * | 2015-09-25 | 2017-03-30 | Intel Corporation | Technologies for receive side message inspection and filtering |
CN105450543B (en) * | 2015-12-01 | 2018-07-20 | 四川神琥科技有限公司 | Voice data transmission method |
US10075416B2 (en) | 2015-12-30 | 2018-09-11 | Juniper Networks, Inc. | Network session data sharing |
US10608992B2 (en) * | 2016-02-26 | 2020-03-31 | Microsoft Technology Licensing, Llc | Hybrid hardware-software distributed threat analysis |
CN107301222B (en) * | 2016-03-07 | 2020-11-10 | 杭州海存信息技术有限公司 | Big data memory with data analysis function |
CN106708532B (en) * | 2016-12-30 | 2020-12-04 | 中国人民解放军国防科学技术大学 | Multilevel regular expression matching method based on TCAM |
CN106776456B (en) * | 2017-01-18 | 2019-06-18 | 中国人民解放军国防科学技术大学 | High speed regular expression matching hybrid system and method based on FPGA+NPU |
WO2019089131A1 (en) | 2017-11-06 | 2019-05-09 | Intel Corporation | Technologies for programming flexible accelerated network pipeline using ebpf |
EP3788512A4 (en) | 2017-12-30 | 2022-03-09 | Target Brands, Inc. | Hierarchical, parallel models for extracting in real time high-value information from data streams and system and method for creation of same |
US10944770B2 (en) * | 2018-10-25 | 2021-03-09 | EMC IP Holding Company LLC | Protecting against and learning attack vectors on web artifacts |
CN113098844B (en) * | 2021-03-08 | 2023-03-21 | 黑龙江大学 | Intelligent network intrusion detection system of hardware protocol |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6304973B1 (en) * | 1998-08-06 | 2001-10-16 | Cryptek Secure Communications, Llc | Multi-level security network system |
US20020077995A1 (en) * | 1998-04-28 | 2002-06-20 | Samuel Steven Allison | Pattern matching in communications network |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5608662A (en) * | 1995-01-12 | 1997-03-04 | Television Computer, Inc. | Packet filter engine |
US6167047A (en) * | 1998-05-18 | 2000-12-26 | Solidum Systems Corp. | Packet classification state machine |
US6424934B2 (en) * | 1998-05-18 | 2002-07-23 | Solidum Systems Corp. | Packet classification state machine having reduced memory storage requirements |
US6349405B1 (en) * | 1999-05-18 | 2002-02-19 | Solidum Systems Corp. | Packet classification state machine |
CA2321466C (en) * | 2000-09-29 | 2006-06-06 | Mosaid Technologies Incorporated | Priority encoder circuit and method |
-
2003
- 2003-08-13 US US10/640,870 patent/US20050114700A1/en not_active Abandoned
-
2004
- 2004-08-12 EP EP04781080A patent/EP1656631A2/en not_active Withdrawn
- 2004-08-12 CN CNA2004800230864A patent/CN1836245A/en active Pending
- 2004-08-12 KR KR1020067002962A patent/KR20060080176A/en not_active Application Discontinuation
- 2004-08-12 WO PCT/US2004/026335 patent/WO2005017702A2/en active Application Filing
-
2006
- 2006-10-06 US US11/539,607 patent/US20070230445A1/en not_active Abandoned
- 2006-10-06 US US11/539,603 patent/US20070195814A1/en not_active Abandoned
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020077995A1 (en) * | 1998-04-28 | 2002-06-20 | Samuel Steven Allison | Pattern matching in communications network |
US6304973B1 (en) * | 1998-08-06 | 2001-10-16 | Cryptek Secure Communications, Llc | Multi-level security network system |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2008097710A2 (en) * | 2007-02-02 | 2008-08-14 | Tarari, Inc. | Systems and methods for processing access control lists (acls) in network switches using regular expression matching logic |
WO2008097710A3 (en) * | 2007-02-02 | 2009-03-26 | Tarari Inc | Systems and methods for processing access control lists (acls) in network switches using regular expression matching logic |
Also Published As
Publication number | Publication date |
---|---|
US20070230445A1 (en) | 2007-10-04 |
KR20060080176A (en) | 2006-07-07 |
EP1656631A2 (en) | 2006-05-17 |
CN1836245A (en) | 2006-09-20 |
WO2005017702A3 (en) | 2005-07-21 |
US20070195814A1 (en) | 2007-08-23 |
US20050114700A1 (en) | 2005-05-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20050114700A1 (en) | Integrated circuit apparatus and method for high throughput signature based network applications | |
Kumar et al. | Curing regular expressions matching algorithms from insomnia, amnesia, and acalculia | |
US9092471B2 (en) | Rule parser | |
US7957378B2 (en) | Stateful flow of network packets within a packet parsing processor | |
US7706378B2 (en) | Method and apparatus for processing network packets | |
US7225188B1 (en) | System and method for performing regular expression matching with high parallelism | |
US7031316B2 (en) | Content processor | |
US8977744B2 (en) | Real-time network monitoring and security | |
US20050216770A1 (en) | Intrusion detection system | |
US8060633B2 (en) | Method and apparatus for identifying data content | |
US9491143B2 (en) | Context-aware pattern matching accelerator | |
US20040049596A1 (en) | Reliable packet monitoring methods and apparatus for high speed networks | |
Schuehler et al. | A modular system for FPGA-based TCP flow processing in high-speed networks | |
JP2009510815A (en) | Method and system for reassembling packets before search | |
WO2006069041A2 (en) | Network interface and firewall device | |
US7451216B2 (en) | Content intelligent network recognition system and method | |
Yang et al. | Intrusion detection system for high-speed network | |
WO2008125112A1 (en) | Method and apparatus for inspection of compressed data packages | |
EP1757039A2 (en) | Programmable packet parsing processor | |
US20030229708A1 (en) | Complex pattern matching engine for matching patterns in IP data streams | |
Lockwood | Network Packet Processing in Reconfigurable Hardware | |
Kapoor | Data Mining and Deep Learning Systems for Network Traffic Classification and Characterization at Scale | |
Daskalakis | Snort DPI on FPGA with GigE | |
Ibrahim | SPIMN Stateful Packet Inspection for Multi Gigabits Networks | |
Attig | Architectures for rule processing intrusion detection and prevention systems |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 200480023086.4 Country of ref document: CN |
|
AK | Designated states |
Kind code of ref document: A2 Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW |
|
AL | Designated countries for regional patents |
Kind code of ref document: A2 Designated state(s): BW GH GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
WWE | Wipo information: entry into national phase |
Ref document number: 1020067002962 Country of ref document: KR |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2004781080 Country of ref document: EP |
|
WWP | Wipo information: published in national office |
Ref document number: 2004781080 Country of ref document: EP |
|
WWP | Wipo information: published in national office |
Ref document number: 1020067002962 Country of ref document: KR |