US7085846B2 - Buffer to buffer credit flow control for computer network - Google Patents

Buffer to buffer credit flow control for computer network Download PDF

Info

Publication number
US7085846B2
US7085846B2 US10/039,184 US3918401A US7085846B2 US 7085846 B2 US7085846 B2 US 7085846B2 US 3918401 A US3918401 A US 3918401A US 7085846 B2 US7085846 B2 US 7085846B2
Authority
US
United States
Prior art keywords
buffer
credit
port
egress
ingress
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related, expires
Application number
US10/039,184
Other versions
US20030126223A1 (en
Inventor
John E. Jenne
Mark Lyndon Oelke
Sompong Paul Olarig
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
SAICO INFORMATION TECHNOLOGY (WUHAN) Co Ltd
Original Assignee
MaXXan Systems Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by MaXXan Systems Inc filed Critical MaXXan Systems Inc
Priority to US10/039,184 priority Critical patent/US7085846B2/en
Assigned to MAXXAN SYSTEMS, INCORPORATED reassignment MAXXAN SYSTEMS, INCORPORATED ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: OLARIG, SOMPONG PAUL, JENNE, JOHN E., OELKE, MARK LYNDON
Publication of US20030126223A1 publication Critical patent/US20030126223A1/en
Application granted granted Critical
Publication of US7085846B2 publication Critical patent/US7085846B2/en
Assigned to CIPHERMAX, INC. reassignment CIPHERMAX, INC. CHANGE OF NAME (SEE DOCUMENT FOR DETAILS). Assignors: MAXXAN SYSTEMS, INC.
Assigned to SAICO INFORMATION TECHNOLOGY (WUHAN) CO., LTD. reassignment SAICO INFORMATION TECHNOLOGY (WUHAN) CO., LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: CIPHERMAX, INC.
Adjusted expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00Packet switching elements
    • H04L49/30Peripheral units, e.g. input or output ports
    • H04L49/3045Virtual queuing
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/10Flow control; Congestion control
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/10Flow control; Congestion control
    • H04L47/39Credit based
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/50Queue scheduling
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/50Queue scheduling
    • H04L47/52Queue scheduling by attributing bandwidth to queues
    • H04L47/527Quantum based scheduling, e.g. credit or deficit based scheduling or token bank
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/50Queue scheduling
    • H04L47/62Queue scheduling characterised by scheduling criteria
    • H04L47/6205Arrangements for avoiding head of line blocking
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00Packet switching elements
    • H04L49/35Switches specially adapted for specific applications
    • H04L49/356Switches specially adapted for specific applications for storage area networks
    • H04L49/357Fibre channel switches

Definitions

  • the present invention is related to computer networks. More specifically, the present invention is related to providing flow control of information for a computer network.
  • Fibre Channel standards define protocols for link-level and end-to-end congestion control. However, these standard protocols do not eliminate head of line (HOL) blocking within a switch. HOL blocking is a problem for internal switching that occurs when several packets at the head of an input queue block packets from being forwarded to output ports.
  • HOL blocking is a problem for internal switching that occurs when several packets at the head of an input queue block packets from being forwarded to output ports.
  • Storage Area Network (SAN) switches that share egress buffer resources are particularly susceptible to HOL blocking when they become congested because, unlike typical TCP/IP switches and routers, a SAN switch does not discard traffic when it becomes congested.
  • the Fibre Channel link-level flow control mechanism (buffer to buffer credits or BB Credits) is typically provided on a per-link basis to devices attached to the SAN switch. Occasionally, several ingress ports may share a pool of BB credits to receive traffic. In most SAN switches, egress congestion is not communicated to the ingress ports to limit the amount of ingress traffic. As a result, HOL blocking may occur within the switch as pools of shared memory become congested. Another major problem with buffer-to-buffer flow control model is that it is difficult to determine the number of BB Credits that are needed to efficiently move the frames. This is critical because the system needs enough credits to be able to provide a continuous flow between ports.
  • the invention overcomes the above-identified problems as well as other shortcomings and deficiencies of existing technologies by providing an end-to-end, e.g., ingress port to egress port, traffic flow control through a computer network at the system level.
  • the present invention is directed to a method for providing buffer-to-buffer credit port-level flow control for a computer network in operative communication with a plurality of ingress and egress network processors, each having an egress port and an ingress port that is associated with a buffer-to-buffer credit value corresponding to the current number of frames the ingress port may send, a buffer value corresponding to the current total frame size the ingress port may send, and a pending buffer-to-buffer value corresponding to the pending buffer-to-buffer credits an egress port may issue the ingress port.
  • buffer-to-buffer flow control is implemented to manage frame traffic from a selected ingress port based on the number and size of the frames the port is permitted to send.
  • the port is issued credits that correspond to the number and size of the frame that the port may send. These credits are decremented when a frame is sent and may be incremented when the frame reaches its destination.
  • the present invention is directed to a method comprising the steps of: sending a frame from the ingress port to a destination egress port, if the ingress port has a sufficient buffer-to-buffer credit value and buffer value; decrementing the buffer-to-buffer credit value associated with the ingress port; decrementing the buffer value associated with the ingress port; determining whether to increment the buffer-to-buffer credit value associated with the ingress port; incrementing the pending credit value associated with the ingress port; and determining whether to send a credit message to the ingress port.
  • a set of network processors is associated with a bridge.
  • the computer system may further comprise a switch fabric; and the network processors may be in operative communication with the switch fabric via the associated bridge.
  • the step of determining whether to increment the buffer-to-buffer credit value may further comprise: incrementing the buffer-to-buffer credit value associated with the ingress port if the product of one plus the buffer-to-buffer credit value times the maximum frame size in bytes is less than or equal to a minimum egress buffering value.
  • the minimum egress buffering value may correspond to the minimum amount of egress buffering that is available for any one egress port.
  • the step of determining whether to send a credit message to the ingress port may further comprise: sending the credit message if the pending credit value is greater than, or equal to, a credit watermark value.
  • the method may further comprise the step of: increasing the buffer value if the credit message is sent.
  • An advantage of the present flow control schemes is that HOL blocking is substantially eliminated.
  • the present flow control schemes alleviate the problems of increased system latency, unintentionally dropped packets, and time-out situations.
  • Another advantage of the present flow control schemes is that more efficient data streaming is provided for the computer network.
  • FIG. 1 is a block diagram of one exemplary embodiment of a computer network
  • FIG. 2 is a block diagram of one exemplary embodiment of a computer network
  • FIG. 3 is an exemplary embodiment of a BB Credit flow control scheme of the present invention.
  • FIG. 4 is an exemplary embodiment of a BB Credit flow control scheme of the present invention.
  • FIG. 5 is an exemplary embodiment of a BB Credit flow control scheme of the present invention.
  • the present invention relates to a system and method for providing traffic flow control through a computer network, e.g., a SAN, at the system level.
  • a computer network e.g., a SAN
  • the presently disclosed system and method for flow control involves dynamic buffer-to-buffer flow control scheme that uses a credit/debit based scheme to manage traffic to a particular port.
  • the flow control scheme limits the traffic associated with a selected port in the computer network based on the size and number of frames or packets that are to be passed through that port.
  • FIG. 1 shows a functional block diagram of a computer network 10 .
  • the exemplary embodiment of computer network 10 shown in FIG. 1 is a Fibre Channel SAN.
  • Computer network 10 comprises a Fibre Channel protocol switching fabric 15 .
  • Switching fabric 15 comprises network switch 20 .
  • Network switch 20 may be any switch suitable for handling network traffic, such as a fibre channel switch or cross bar switch, for example.
  • Computer network 10 also contains one or more bridge devices 25 .
  • Bridge 25 provides a connection between one or more network processors 30 and switch 20 .
  • Bridge 25 contains a switch fabric interface 55 to connect bridge 25 and switch 20 .
  • Bridge 25 may also comprise a VOQ 35 to provide a bandwidth matching function.
  • a 512-port SAN switch may be implemented by connecting 256 network processors 30 to a 64-port switch fabric 15 .
  • Each network processor 30 has an ingress switch interface 45 and an egress switch interface 50 .
  • the network processor 30 may merely provide limited computational capabilities and may be satisfied by even rudimentary digital processors. Any of these digital processors need not exist within the present invention. Any necessary processing can be satisfied from remote processors. In a preferred embodiment of the present invention, latency may be reduced by having the network processor 30 within the system.
  • each network processor 30 supports two 1 Gbps Fibre Channel ports. Each Fibre Channel port is made up of an ingress port 60 and an egress port 65 . Each network processor 30 may include an interface 40 . For example, network processor 30 may have an OC-48 (2.5 Gbps) CSIX interface to switch fabric 15 . Switch 20 has one or more switch fabric ports 55 to provide a connection with bridges 25 . Switch fabric port 55 may support any protocol operable to conduct traffic across network 10 . For example, each of the switch fabric ports is an OC-192 (10 Gbps) CSIX interface. In the exemplary embodiment shown in FIG. 1 , each bridge 25 connects four network processors 30 to each switch fabric port 55 .
  • Each network processor 30 has ingress buffering that is used to implement a VOQ for each egress Fibre Channel port 65 in the system. In the example discussed above, each network processor 30 implement 512 VOQs, one for each egress Fibre Channel port 65 in the system. Each network processor 30 also has egress buffering that is used to implement at least two outbound queues, one for each egress Fibre Channel port 65 connected to the network processor 30 . The network processors 30 monitor the depth of the egress buffers for each of its two Fibre Channel ports 60 and 65 .
  • the flow-control scheme of the present disclosure utilizes a dynamic buffer-to-buffer flow control mechanism to control traffic between ports.
  • Standard fibre channel buffer-to-buffer flow control mechanisms use a credit/debit based algorithm to control traffic between the N-Port and F-Port of a fibre channel link.
  • Table I below shows an example of how fibre channel buffer-to-buffer flow control can be extended across the fabric switch to handle frame transfers between two network processors, “NP 1” and “NP 250.”
  • Port 1 sends Frame A to Network Processor 1 (NP 1) 2 Port 1 decrements BB Credit 3 NP 1 receives Frame A and determines that Frame A's destination is Port 500 (attached to NP 250) 4 NP 1 sends Frame A to NP 250 across switch fabric 5 NP 250 receives Frame A 6 NP 250 queues Frame A for transmission to Port 500 7 NP 250 sends Frame A to Port 500 8 NP 250 creates a credit message for Frame A to send to NP 1 9 NP 250 sends credit message to NP 1 10 NP 1 receives credit message for Frame A 11 NP gives BB Credit to Port 1 12 Port 1 increments BB Credit
  • This example shown in Table I illustrates the fibre channel buffer-to-buffer flow control scheme may be extended all the way to the egress port within the network switch.
  • the egress NP determines when to issue a credit message that translates into a BB Credit to the ingress port. Therefore, the egress network processor throttles the ingress port transmission rate by controlling when the BB Credit is sent.
  • a major problem with this flow control model is that it is difficult to determine the number of BB Credits that are needed to efficiently moves the frames. It is important that the system has enough credits to be able to provide a continuous flow between a single ingress and egress port.
  • the presently disclosed dynamic BB Credit flow control scheme involves byte-based connectivity between the ingress and egress network processors.
  • the ingress network processor is permitted to send a predefined amount of traffic to an egress network processor.
  • This traffic can be made up of a large number of small frames or a small number of large frames. Regardless of size characteristics of the traffic, the ingress network processor preferably never sends more than the predefined amount of traffic to the egress network processor.
  • the egress network processor After the egress network processor has transmitted the frame out the egress port, it sends a credit to the ingress network processor with the frame size in bytes. The ingress network processor then uses this credit to increase its pool of permissible traffic.
  • This BB Credit flow control scheme dynamically allocates BB Credits based on the amount of egress buffering available at the egress network processor.
  • This fibre channel buffer-to-buffer flow control scheme operates on a per-frame basis, so if there are a lot of small fibre channel frames, the BB Credits are given quickly which permits the small frames to stream. If there are a lot of large fibre channel frames, the BB Credits are given out at a slower rate that the egress port can handle. If an egress port is congested, the egress network processor gives credits back to ingress network processor at the rate it is transmitting traffic out of the network switch.
  • the byte-based credits sent from the egress network processor to the ingress network processor of a network switch may generate overhead that uses valuable switch fabric bandwidth. Accordingly, another embodiment of the dynamic BB Credit flow control scheme combines credit messages. Instead of generating a byte-based credit for every frame transmitted out the egress port, the BB Credit flow control scheme may combine multiple credits.
  • the egress network processor tracks the amount of byte-based credits for each ingress port, and once a credit level was reached, the egress network processor generates a credit for the accumulated byte total.
  • the credit level may be programmable so that the level may be defined by a user for a particular system or network.
  • the ingress network processor tracks the amount of available egress buffer available for each egress port.
  • BB Credits are based on the minimum amount of egress buffering available at any one egress port.
  • Tables 2 and 3 are tracked at each ingress and egress port, respectively.
  • Table II shows the variables to be tracked at the ingress port, where N corresponds to the number of ports in the system.
  • the Max_BB_Credit variable corresponds to the maximum number of BB Credit that may be negotiated for that port.
  • the Cur_BB_Credit variable is the current BB Credit value for the port.
  • Max_Buff represents the maximum amount, e.g., in bytes, of egress buffering for a single egress port.
  • Cur_Buff is the currently available amount, e.g., in bytes, of egress buffering for a single ingress port per egress port.
  • Table III shows the variables to be tracked at the egress port, where N corresponds to the number of ports in the system.
  • Credit_Watermark is a variable that is used by the system to determine when to issue a credit back to an ingress port.
  • Pending_Credit represents the amount of pending egress buffering credit for each ingress port.
  • FIG. 2 shows a functional block diagram of computer network 400 that includes a network switch or cross bar 410 .
  • computer network 400 is preferably a fibre channel SAN and switch 410 is preferably a fibre-channel switch.
  • Computer network 400 includes an ingress network processor 415 and an egress network processor 420 .
  • Ingress network processor 415 comprises an ingress fabric interface 425 and a egress fabric interface 430 that allows ingress network processor 415 to be connected to switch 410 such that traffic may be conducted between ingress network processor 415 and switch 410 .
  • Ingress network processor 415 also has one or more ingress ports 435 and egress ports 440 for accommodating traffic between ingress network processor 415 and ingress devices 445 .
  • Egress network processor 420 comprises an ingress fabric interface 450 and an egress fabric interface 455 that allows egress network processor 420 to be connected to switch 410 such that traffic may be conducted between egress network processor 420 and switch 410 .
  • Egress network processor 420 also has one or more ingress ports 460 and egress ports 465 for accommodating traffic between egress network processor 420 and egress devices 470 .
  • FIG. 3 illustrates an exemplary embodiment of dynamic BB Credit flow control at the ingress network processor using the exemplary network shown in FIG. 2 .
  • the ingress port negotiates with the ingress device for a BB Credit count and stores this value in Max_BB_Credit.
  • the negotiation for the BB Credit value generally does not have to take into account the latency of the switch, but typically does need to recognize the latencies of the fibre channel link due to distance.
  • a small BB Credit is generally preferred.
  • the Max_Buf value is based on configuration information that indicates the maximum buffering capability at a single egress for a single ingress.
  • Cur_BB_Credit may be set equal to Max_BB_Credit and Cur_Buf[N] may be set equal to Max_Buf.
  • the Credit_Watermark may be configured by software at system initialization.
  • the egress port variable, Pending_Credit[N] is initially set to zero.
  • the ingress port is on standby to receive a frame from the ingress device.
  • an ingress device sends a frame to the ingress port, it must be determined whether the ingress port has sufficient BB Credit to send a frame.
  • ingress device 445 only tracks the number of BB Credits that were negotiated. If the ingress port has insufficient BB Credit, then the frame cannot be sent at this time. Accordingly, at step 515 , the ingress port must wait for a BB Credit before it may send a frame.
  • the ingress port sends the frame to the ingress network processor at step 520 .
  • the ingress device decrements its BB Credit count at step 525 .
  • the ingress NP receives the frame and decrements the ingress port's Cur_BB_Credit at step 530 .
  • the ingress network processor determines whether there is sufficient Cur_Buf[n] at the egress network processor to send the frame. If Cur_Buf[n] is insufficient at the egress network processor, the ingress processor must wait. If Cur_Buf[n] is sufficient, the ingress network processor sends the frame to the egress network processor at step 540 . The ingress network processor then decrements Cur_Buf[n] at step 545 by the frame size plus the associated overhead.
  • the ingress network processor determines whether or not to give a BB Credit to the ingress device. If the system determines that there is sufficient egress buffering for any one egress port, then the network processor gives a BB Credit to the ingress device at step 555 and then increments the Cur_BB_Credit for the ingress port at step 560 . In one embodiment of the present BB Credit flow control scheme, if ((Cur_BB_Credit+1)*MAX — FC_FRAME_SIZE) ⁇ Minimum (Cur_Buf[n]), then the ingress network processor will immediately gives a BB Credit to the ingress device or port and increments Cur_BB_Credit.
  • the variable Minimum (Cur_Buf[n]) is the minimum amount of egress buffering available for any one egress port. Otherwise, the ingress network processor must wait until it receives a credit message from the egress network processor, before it may increment the ingress port's BB Credit. In this particular embodiment, the system checks to see if there is enough egress buffering for all the current BB Credits plus one (assuming each BB Credit is associated with a maximum sized fibre channel frame). Typically, for Fibre Channel networks, the smallest frame is 36 bytes and the largest is 2148 bytes.
  • FIG. 4 illustrates an exemplary embodiment of dynamic BB Credit flow control at the egress network processor using the exemplary network show in FIG. 2 .
  • the egress network processor negotiates with the egress device for an acceptable BB Credit count.
  • the egress network processor waits for a frame to send to the egress device. Once a frame is received, the egress processor checks the BB Credit level for the egress device at step 610 . If the BB Credit is insufficient, the egress network processor waits to receive a BB Credit at step 615 . If the BB Credit is sufficient, the egress network processor transmits the frame to the egress device at step 620 .
  • the egress processor then decrements the BB Credit value at step 625 .
  • the egress network processor de-allocates the buffering needed for the frame and increments Pending_Credit[N] by the frame size plus the associated overhead for the source ingress port at step 630 .
  • the system must determine whether a credit message should be sent to the ingress network port. Generally, the egress network processor will send a credit message if a sufficient amount of buffering has been made available after the frame has been routed to an egress port. Accordingly, at step 635 , it is determined whether there is sufficient Pending_Credit.
  • the egress network processor creates a credit message to send back to the ingress network processor. If the system determines that a sufficient amount of buffering is available, then a credit message is generated and sent at step 640 .
  • the credit message includes the amount of buffering freed up, which is equal to Pending_Credit[n].
  • the variable Pending_Credit[N] is then preferably set to zero at step 645 before preceding back to step 605 . Otherwise, the flow control process proceeds to step 505 and the ingress port waits for the next frame to send.
  • FIG. 5 illustrates an exemplary embodiment of dynamic BB Credit flow control where the ingress network processor receives a credit message.
  • the credit message may be a special inband control cell that is set from the egress network processor to the source ingress network processor.
  • the credit message cell may have fields indicating the source ingress port, the destination egress port, and the amount of buffer credit in bytes. The source and destination ports may simply be indicated in the cell header addresses.
  • the ingress network processor receives the credit message from the egress network processor at step 700 , it increments Cur_Buf[n] by the amount of buffering freed up specified in the credit message at step 705 .
  • the ingress processor must determine whether to give a BB Credit to the ingress port at step 710 .
  • this determination depends on whether there is sufficient egress buffering. If so, then the ingress network processor sends a BB Credit to the ingress device at step 715 and then increments the ingress port's Cur_BB_Credit at step 720 .
  • the ingress network processor gives a BB Credit to the ingress port and increments Cur_BB_Credit. Otherwise, the ingress port must wait until Cur_Buf[n] is incremented by additional credit messages before sending a BB Credit to the ingress port. The flow control scheme then goes back to step 700 and the ingress port stands by for the next frame.
  • Table IV shows an example of how the BB Credit flow control scheme controls frame traffic across the network switch 410 .
  • the example shown in Table IV is based on the following assumptions: all fibre channel frames are equal to the maximum fibre channel frame size; there is no overhead; and the same egress is used throughout Table IV to simplify the example.
  • one frame occupies one buffer/credit.
  • one buffer contains usually 2 KB of memory.
  • a small frame pocket of 36 bytes for example, still consumes an entire 2 KB of memory.
  • several smaller frame packets may be logically grouped and associated with a single buffer. For example, several small frames of about 36 bytes may be collapsed into one 2 KB buffer.
  • the system may keep track of the ordering via hardware logic. For example, the system may maintain a scoreboard to track the order in which the frame packets are to be sent. In this exemplary embodiment, the system maximizes the number of available credit for larger frames and thus potentially increases the throughput between the sender and receiver.
  • the presently disclosed flow control schemes provide a number of advantages.
  • One advantage of the present invention is that the flow control scheme substantially eliminates head of line (HOL) blocking.
  • HOL blocking is a problem for internal switching that occurs when several packets at the head of an input queue block packets from being forwarded to output ports.
  • the dynamic BB Credit flow control scheme prevents problems caused by HOL blocking such as increased system latency, unintentionally dropped packets, and time-out problems.
  • the dynamic BB Credit flow control scheme provides byte-based connectivity between the ingress and egress network processors.
  • the ingress network processor is permitted to send a predefined amount of traffic to the egress network processor.
  • This flow control scheme allows for the system to dynamically give BB Credits based on the amount of buffering available.

Abstract

A system and method for providing flow control for a computer network is disclosed. In one embodiment of the present invention, buffer-to-buffer credit flow control is implemented to limit the amount of data a port may send based on the number and size of the frames sent from that port.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS
This application is related to U.S. patent application publication number 20020078299, titled “Caching System and Method for a Network Storage System” by Lin-Sheng Chiou, Mike Witkowski, Hawkins Yao, Cheh-Suei Yang, and Sompong Paul Olarig, which was filed on Dec. 14, 2000, now U.S. Pat. No. 6,792,507 and which is incorporated herein by reference in its entirety for all purposes; U.S. patent application Ser. No. 10/015,047, titled “System, Apparatus and Method for Address Forwarding for a Computer Network” by Hawkins Yao, Cheh-Suei Yang, Richard Gunlock, Michael L. Witkowski, and Sompong Paul Olarig, which was filed on Oct. 26, 2001 and which is incorporated herein by reference in its entirety for all purposes; U.S. patent application Ser. No. 20030200330, titled “System And Method For Load-Sharing Computer Network Switch” by Sompong Paul Olarig, Mark Lyndon Oelke, and John E. Jenne, which was filed on Apr. 22, 2002, and which is incorporated herein by reference in its entirety for all purposes; U.S. patent application Ser. No. 10/039,190, titled “Network Processor Interface System” by Sompong Paul Olarig, Mark Lyndon Oelke and John E. Jenne, which is being filed concurrently on Dec. 31, 2001, and which is incorporated herein by reference in its entirety for all purposes; and U.S. patent application Ser. No. 10/039,189, titled “Xon/Xoff Flow Control for Computer Network” by Hawkins Yao, John E. Jenne and Mark Lyndon Oelke, which is being filed concurrently on Dec. 31, 2001, and which is incorporated herein by reference in its entirety for all purposes.
FIELD OF THE INVENTION
The present invention is related to computer networks. More specifically, the present invention is related to providing flow control of information for a computer network.
BACKGROUND OF THE INVENTION TECHNOLOGY
Fibre Channel standards define protocols for link-level and end-to-end congestion control. However, these standard protocols do not eliminate head of line (HOL) blocking within a switch. HOL blocking is a problem for internal switching that occurs when several packets at the head of an input queue block packets from being forwarded to output ports. Storage Area Network (SAN) switches that share egress buffer resources are particularly susceptible to HOL blocking when they become congested because, unlike typical TCP/IP switches and routers, a SAN switch does not discard traffic when it becomes congested.
The Fibre Channel link-level flow control mechanism (buffer to buffer credits or BB Credits) is typically provided on a per-link basis to devices attached to the SAN switch. Occasionally, several ingress ports may share a pool of BB credits to receive traffic. In most SAN switches, egress congestion is not communicated to the ingress ports to limit the amount of ingress traffic. As a result, HOL blocking may occur within the switch as pools of shared memory become congested. Another major problem with buffer-to-buffer flow control model is that it is difficult to determine the number of BB Credits that are needed to efficiently move the frames. This is critical because the system needs enough credits to be able to provide a continuous flow between ports.
SUMMARY OF THE INVENTION
The invention overcomes the above-identified problems as well as other shortcomings and deficiencies of existing technologies by providing an end-to-end, e.g., ingress port to egress port, traffic flow control through a computer network at the system level.
The present invention is directed to a method for providing buffer-to-buffer credit port-level flow control for a computer network in operative communication with a plurality of ingress and egress network processors, each having an egress port and an ingress port that is associated with a buffer-to-buffer credit value corresponding to the current number of frames the ingress port may send, a buffer value corresponding to the current total frame size the ingress port may send, and a pending buffer-to-buffer value corresponding to the pending buffer-to-buffer credits an egress port may issue the ingress port.
In an exemplary embodiment of the present system and method for flow control, buffer-to-buffer flow control is implemented to manage frame traffic from a selected ingress port based on the number and size of the frames the port is permitted to send. The port is issued credits that correspond to the number and size of the frame that the port may send. These credits are decremented when a frame is sent and may be incremented when the frame reaches its destination.
The present invention is directed to a method comprising the steps of: sending a frame from the ingress port to a destination egress port, if the ingress port has a sufficient buffer-to-buffer credit value and buffer value; decrementing the buffer-to-buffer credit value associated with the ingress port; decrementing the buffer value associated with the ingress port; determining whether to increment the buffer-to-buffer credit value associated with the ingress port; incrementing the pending credit value associated with the ingress port; and determining whether to send a credit message to the ingress port. A set of network processors is associated with a bridge. The computer system may further comprise a switch fabric; and the network processors may be in operative communication with the switch fabric via the associated bridge. The step of determining whether to increment the buffer-to-buffer credit value may further comprise: incrementing the buffer-to-buffer credit value associated with the ingress port if the product of one plus the buffer-to-buffer credit value times the maximum frame size in bytes is less than or equal to a minimum egress buffering value. The minimum egress buffering value may correspond to the minimum amount of egress buffering that is available for any one egress port. The step of determining whether to send a credit message to the ingress port may further comprise: sending the credit message if the pending credit value is greater than, or equal to, a credit watermark value. The method may further comprise the step of: increasing the buffer value if the credit message is sent.
An advantage of the present flow control schemes is that HOL blocking is substantially eliminated. The present flow control schemes alleviate the problems of increased system latency, unintentionally dropped packets, and time-out situations. Another advantage of the present flow control schemes is that more efficient data streaming is provided for the computer network. Other advantages will be apparent in view of the figures and detailed description below.
BRIEF DESCRIPTION OF THE DRAWINGS
A more complete understanding of the present disclosure and advantages thereof may be acquired by referring to the following description taken in conjunction with the accompanying drawings, wherein:
FIG. 1 is a block diagram of one exemplary embodiment of a computer network;
FIG. 2 is a block diagram of one exemplary embodiment of a computer network;
FIG. 3 is an exemplary embodiment of a BB Credit flow control scheme of the present invention;
FIG. 4 is an exemplary embodiment of a BB Credit flow control scheme of the present invention; and
FIG. 5 is an exemplary embodiment of a BB Credit flow control scheme of the present invention.
While the present invention is susceptible to various modifications and alternative forms, specific exemplary embodiments thereof have been shown by way of example in the drawings and are herein described in detail. It should be understood, however, that the description herein of specific embodiments is not intended to limit the invention to the particular forms disclosed, but on the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention as defined by the appended claims.
DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS
The present invention relates to a system and method for providing traffic flow control through a computer network, e.g., a SAN, at the system level. The presently disclosed system and method for flow control involves dynamic buffer-to-buffer flow control scheme that uses a credit/debit based scheme to manage traffic to a particular port. Generally, the flow control scheme limits the traffic associated with a selected port in the computer network based on the size and number of frames or packets that are to be passed through that port.
FIG. 1 shows a functional block diagram of a computer network 10. The exemplary embodiment of computer network 10 shown in FIG. 1 is a Fibre Channel SAN. Computer network 10 comprises a Fibre Channel protocol switching fabric 15. Switching fabric 15 comprises network switch 20. Network switch 20 may be any switch suitable for handling network traffic, such as a fibre channel switch or cross bar switch, for example. Computer network 10 also contains one or more bridge devices 25. Bridge 25 provides a connection between one or more network processors 30 and switch 20. Bridge 25 contains a switch fabric interface 55 to connect bridge 25 and switch 20. Bridge 25 may also comprise a VOQ 35 to provide a bandwidth matching function. In the particular embodiment illustrated in FIG. 1, a 512-port SAN switch may be implemented by connecting 256 network processors 30 to a 64-port switch fabric 15. Each network processor 30 has an ingress switch interface 45 and an egress switch interface 50.
For the present disclosure, the network processor 30 may merely provide limited computational capabilities and may be satisfied by even rudimentary digital processors. Any of these digital processors need not exist within the present invention. Any necessary processing can be satisfied from remote processors. In a preferred embodiment of the present invention, latency may be reduced by having the network processor 30 within the system.
In the exemplary embodiment shown in FIG. 1, each network processor 30 supports two 1 Gbps Fibre Channel ports. Each Fibre Channel port is made up of an ingress port 60 and an egress port 65. Each network processor 30 may include an interface 40. For example, network processor 30 may have an OC-48 (2.5 Gbps) CSIX interface to switch fabric 15. Switch 20 has one or more switch fabric ports 55 to provide a connection with bridges 25. Switch fabric port 55 may support any protocol operable to conduct traffic across network 10. For example, each of the switch fabric ports is an OC-192 (10 Gbps) CSIX interface. In the exemplary embodiment shown in FIG. 1, each bridge 25 connects four network processors 30 to each switch fabric port 55.
Each network processor 30 has ingress buffering that is used to implement a VOQ for each egress Fibre Channel port 65 in the system. In the example discussed above, each network processor 30 implement 512 VOQs, one for each egress Fibre Channel port 65 in the system. Each network processor 30 also has egress buffering that is used to implement at least two outbound queues, one for each egress Fibre Channel port 65 connected to the network processor 30. The network processors 30 monitor the depth of the egress buffers for each of its two Fibre Channel ports 60 and 65.
The flow-control scheme of the present disclosure utilizes a dynamic buffer-to-buffer flow control mechanism to control traffic between ports. Standard fibre channel buffer-to-buffer flow control mechanisms use a credit/debit based algorithm to control traffic between the N-Port and F-Port of a fibre channel link. Table I below shows an example of how fibre channel buffer-to-buffer flow control can be extended across the fabric switch to handle frame transfers between two network processors, “NP 1” and “NP 250.”
TABLE I
Example of Two-Level Flow Control
Step # Action
1 Port 1 sends Frame A to Network Processor 1 (NP 1)
2 Port 1 decrements BB Credit
3 NP 1 receives Frame A and determines that Frame A's
destination is Port 500 (attached to NP 250)
4 NP 1 sends Frame A to NP 250 across switch fabric
5 NP 250 receives Frame A
6 NP 250 queues Frame A for transmission to Port 500
7 NP 250 sends Frame A to Port 500
8 NP 250 creates a credit message for Frame A to send to NP 1
9 NP 250 sends credit message to NP 1
10 NP 1 receives credit message for Frame A
11 NP gives BB Credit to Port 1
12 Port 1 increments BB Credit
This example shown in Table I illustrates the fibre channel buffer-to-buffer flow control scheme may be extended all the way to the egress port within the network switch. The egress NP determines when to issue a credit message that translates into a BB Credit to the ingress port. Therefore, the egress network processor throttles the ingress port transmission rate by controlling when the BB Credit is sent. As discussed above, a major problem with this flow control model is that it is difficult to determine the number of BB Credits that are needed to efficiently moves the frames. It is important that the system has enough credits to be able to provide a continuous flow between a single ingress and egress port.
One challenge is that there is a wide range of fibre channel frame sizes. If the system uses the minimum number of BB Credits needed for a continuous flow of the largest fibre channel frames, then a stream of small fibre channel frames would be unnecessarily throttled due to a lack of BB credits. On the other hand, if the number of BB Credits is set to the number of credits needed to stream the smallest fibre channel frames, then large fibre channel frames place extreme buffering requirements on the system. To minimize the buffering requirements for large fibre channels and to enable small fibre channel frames to stream, a dynamic BB credit level flow control is needed.
The presently disclosed dynamic BB Credit flow control scheme involves byte-based connectivity between the ingress and egress network processors. The ingress network processor is permitted to send a predefined amount of traffic to an egress network processor. This traffic can be made up of a large number of small frames or a small number of large frames. Regardless of size characteristics of the traffic, the ingress network processor preferably never sends more than the predefined amount of traffic to the egress network processor. After the egress network processor has transmitted the frame out the egress port, it sends a credit to the ingress network processor with the frame size in bytes. The ingress network processor then uses this credit to increase its pool of permissible traffic.
This BB Credit flow control scheme dynamically allocates BB Credits based on the amount of egress buffering available at the egress network processor. This fibre channel buffer-to-buffer flow control scheme operates on a per-frame basis, so if there are a lot of small fibre channel frames, the BB Credits are given quickly which permits the small frames to stream. If there are a lot of large fibre channel frames, the BB Credits are given out at a slower rate that the egress port can handle. If an egress port is congested, the egress network processor gives credits back to ingress network processor at the rate it is transmitting traffic out of the network switch.
The byte-based credits sent from the egress network processor to the ingress network processor of a network switch may generate overhead that uses valuable switch fabric bandwidth. Accordingly, another embodiment of the dynamic BB Credit flow control scheme combines credit messages. Instead of generating a byte-based credit for every frame transmitted out the egress port, the BB Credit flow control scheme may combine multiple credits. The egress network processor tracks the amount of byte-based credits for each ingress port, and once a credit level was reached, the egress network processor generates a credit for the accumulated byte total. The credit level may be programmable so that the level may be defined by a user for a particular system or network. The ingress network processor tracks the amount of available egress buffer available for each egress port. BB Credits are based on the minimum amount of egress buffering available at any one egress port.
In another embodiment of the BB Credit flow control scheme, the information shown in Tables 2 and 3 is tracked at each ingress and egress port, respectively. Table II below shows the variables to be tracked at the ingress port, where N corresponds to the number of ports in the system. The Max_BB_Credit variable corresponds to the maximum number of BB Credit that may be negotiated for that port. The Cur_BB_Credit variable is the current BB Credit value for the port. Max_Buff represents the maximum amount, e.g., in bytes, of egress buffering for a single egress port. Cur_Buff is the currently available amount, e.g., in bytes, of egress buffering for a single ingress port per egress port.
TABLE II
Ingress Port Variables
Variable Description
Max_BB_Credit maximum negotiated BB Credit count
Cur_BB_Credit current BB Credit count
Max_Buf maximum amount of egress buffering for a single
ingress port
Cur_Buf[N] current available egress buffering for a single ingress
port on a per egress port basis
Table III below shows the variables to be tracked at the egress port, where N corresponds to the number of ports in the system. Credit_Watermark is a variable that is used by the system to determine when to issue a credit back to an ingress port. Pending_Credit represents the amount of pending egress buffering credit for each ingress port.
TABLE III
Egress Port Variables
Variable Description
Credit_Watermark watermark for determining when to issue a credit
back to the ingress network processor
Pending_Credit[N] amount of pending egress buffering credit for each
ingress port
FIG. 2 shows a functional block diagram of computer network 400 that includes a network switch or cross bar 410. As discussed above, computer network 400 is preferably a fibre channel SAN and switch 410 is preferably a fibre-channel switch. Computer network 400 includes an ingress network processor 415 and an egress network processor 420. Ingress network processor 415 comprises an ingress fabric interface 425 and a egress fabric interface 430 that allows ingress network processor 415 to be connected to switch 410 such that traffic may be conducted between ingress network processor 415 and switch 410. Ingress network processor 415 also has one or more ingress ports 435 and egress ports 440 for accommodating traffic between ingress network processor 415 and ingress devices 445. Egress network processor 420 comprises an ingress fabric interface 450 and an egress fabric interface 455 that allows egress network processor 420 to be connected to switch 410 such that traffic may be conducted between egress network processor 420 and switch 410. Egress network processor 420 also has one or more ingress ports 460 and egress ports 465 for accommodating traffic between egress network processor 420 and egress devices 470.
FIG. 3 illustrates an exemplary embodiment of dynamic BB Credit flow control at the ingress network processor using the exemplary network shown in FIG. 2. Initially, at step 500, the ingress port negotiates with the ingress device for a BB Credit count and stores this value in Max_BB_Credit. The negotiation for the BB Credit value generally does not have to take into account the latency of the switch, but typically does need to recognize the latencies of the fibre channel link due to distance. For the dynamic BB Credit flow control scheme, a small BB Credit is generally preferred. Preferably, the Max_Buf value is based on configuration information that indicates the maximum buffering capability at a single egress for a single ingress. Initially, Cur_BB_Credit may be set equal to Max_BB_Credit and Cur_Buf[N] may be set equal to Max_Buf. The Credit_Watermark may be configured by software at system initialization. Preferably, the egress port variable, Pending_Credit[N] is initially set to zero.
At step 505, the ingress port is on standby to receive a frame from the ingress device. When an ingress device sends a frame to the ingress port, it must be determined whether the ingress port has sufficient BB Credit to send a frame. Generally, ingress device 445 only tracks the number of BB Credits that were negotiated. If the ingress port has insufficient BB Credit, then the frame cannot be sent at this time. Accordingly, at step 515, the ingress port must wait for a BB Credit before it may send a frame.
If it is determined at step 510 that the ingress port has a sufficient BB Credit value, then the ingress port sends the frame to the ingress network processor at step 520. The ingress device decrements its BB Credit count at step 525. The ingress NP receives the frame and decrements the ingress port's Cur_BB_Credit at step 530.
At step 530, the ingress network processor determines whether there is sufficient Cur_Buf[n] at the egress network processor to send the frame. If Cur_Buf[n] is insufficient at the egress network processor, the ingress processor must wait. If Cur_Buf[n] is sufficient, the ingress network processor sends the frame to the egress network processor at step 540. The ingress network processor then decrements Cur_Buf[n] at step 545 by the frame size plus the associated overhead.
At step 550, the ingress network processor then determines whether or not to give a BB Credit to the ingress device. If the system determines that there is sufficient egress buffering for any one egress port, then the network processor gives a BB Credit to the ingress device at step 555 and then increments the Cur_BB_Credit for the ingress port at step 560. In one embodiment of the present BB Credit flow control scheme, if ((Cur_BB_Credit+1)*MAXFC_FRAME_SIZE)≦Minimum (Cur_Buf[n]), then the ingress network processor will immediately gives a BB Credit to the ingress device or port and increments Cur_BB_Credit. The variable Minimum (Cur_Buf[n]) is the minimum amount of egress buffering available for any one egress port. Otherwise, the ingress network processor must wait until it receives a credit message from the egress network processor, before it may increment the ingress port's BB Credit. In this particular embodiment, the system checks to see if there is enough egress buffering for all the current BB Credits plus one (assuming each BB Credit is associated with a maximum sized fibre channel frame). Typically, for Fibre Channel networks, the smallest frame is 36 bytes and the largest is 2148 bytes.
FIG. 4 illustrates an exemplary embodiment of dynamic BB Credit flow control at the egress network processor using the exemplary network show in FIG. 2. At step 600, the egress network processor negotiates with the egress device for an acceptable BB Credit count. At step 605, the egress network processor waits for a frame to send to the egress device. Once a frame is received, the egress processor checks the BB Credit level for the egress device at step 610. If the BB Credit is insufficient, the egress network processor waits to receive a BB Credit at step 615. If the BB Credit is sufficient, the egress network processor transmits the frame to the egress device at step 620. The egress processor then decrements the BB Credit value at step 625. After transmitting the frame to an egress port, the egress network processor de-allocates the buffering needed for the frame and increments Pending_Credit[N] by the frame size plus the associated overhead for the source ingress port at step 630. Next, the system must determine whether a credit message should be sent to the ingress network port. Generally, the egress network processor will send a credit message if a sufficient amount of buffering has been made available after the frame has been routed to an egress port. Accordingly, at step 635, it is determined whether there is sufficient Pending_Credit.
In one embodiment of the present BB Credit flow control scheme, if Pending_Credit[N]≧Credit_Watermark, the egress network processor creates a credit message to send back to the ingress network processor. If the system determines that a sufficient amount of buffering is available, then a credit message is generated and sent at step 640. The credit message includes the amount of buffering freed up, which is equal to Pending_Credit[n]. The variable Pending_Credit[N] is then preferably set to zero at step 645 before preceding back to step 605. Otherwise, the flow control process proceeds to step 505 and the ingress port waits for the next frame to send.
FIG. 5 illustrates an exemplary embodiment of dynamic BB Credit flow control where the ingress network processor receives a credit message. The credit message may be a special inband control cell that is set from the egress network processor to the source ingress network processor. The credit message cell may have fields indicating the source ingress port, the destination egress port, and the amount of buffer credit in bytes. The source and destination ports may simply be indicated in the cell header addresses. When the ingress network processor receives the credit message from the egress network processor at step 700, it increments Cur_Buf[n] by the amount of buffering freed up specified in the credit message at step 705. Next, the ingress processor must determine whether to give a BB Credit to the ingress port at step 710. As discussed above, this determination depends on whether there is sufficient egress buffering. If so, then the ingress network processor sends a BB Credit to the ingress device at step 715 and then increments the ingress port's Cur_BB_Credit at step 720. In one embodiment of the present BB Credit flow control scheme, if ((Cur_BB_Credit+1)*MAXFC_FRAME_SIZE)≦Minimum(Cur_Buf[n]), then the ingress network processor gives a BB Credit to the ingress port and increments Cur_BB_Credit. Otherwise, the ingress port must wait until Cur_Buf[n] is incremented by additional credit messages before sending a BB Credit to the ingress port. The flow control scheme then goes back to step 700 and the ingress port stands by for the next frame.
Table IV below shows an example of how the BB Credit flow control scheme controls frame traffic across the network switch 410. For the purposes of illustration, the process shown in Table IV is based on the following values: Max_BB Credit=3; Max_Buf=8 KB; Credit_Watermark=4 KB; and the maximum fibre channel frame size equals 2K. Furthermore, for the purposes of illustration, the example shown in Table IV is based on the following assumptions: all fibre channel frames are equal to the maximum fibre channel frame size; there is no overhead; and the same egress is used throughout Table IV to simplify the example.
TABLE IV
Example of Dynamic BB Credit Flow Control
Step Cur_BB Cur Pending
# Action Credit Buf[1] Credit[1]
1 Initialize 3 8 KB 0 KB
2 Port 1 sends Frame 1 to NP 1; 2 8 KB 0 KB
decrement BB Credit
3 NP 1 decrements Cur_Buf[1] 2 6 KB 0 KB
for Frame 1
4 (2 + 1) * 2 KB <= 6 KB, so NP 3 6 KB 0 KB
1 immediately gives BB Credit
5 NP 1 sends Frame 1 to NP 250 3 6 KB 0 KB
via switch fabric
6 Port 1 sends Frame 2 to NP 1; 2 6 KB 0 KB
decrement BB Credit
7 NP 1 decrements Cur_Buf[1] 2 4 KB 0 KB
for Frame 2
8 (2 + 1) * 2 KB > 4 KB, so NP 1 2 4 KB 0 KB
can NOT give BB Credit until a
credit message is received from
Egress NP
9 NP 1 sends Frame 2 to NP 250 2 4 KB 0 KB
via switch fabric
10 Port 1 sends Frame 3 to NP 1; 1 4 KB 0 KB
decrement BB Credit
11 NP 1 decrements Cur_Buf[1] 1 2 KB 0 KB
for Frame 3
12 (1 + 1) * 2 KB > 2 KB, so NP 1 1 2 KB 0 KB
can NOT give BB Credit until a
credit message is received from
Egress NP
13 NP 1 sends Frame 3 to NP 250 1 2 KB 0 KB
via switch fabric
14 Port 1 sends Frame 4 to NP 1; 0 2 KB 0 KB
decrement BB Credit
15 NP 1 decrements Cur_Buf[1] 0 0 KB 0 KB
for Frame 4
16 (0 + 1) * 2 KB > 0 KB, so NP 1 0 0 KB 0 KB
can NOT give BB Credit until a
credit message is received from
Egress NP
17 NP 1 sends Frame 4 to NP 250 0 0 KB 0 KB
via switch fabric
18 NP 250 receives Frame 1 and 0 0 KB 0 KB
sends it to Port 500
19 NP 250 increments Pending 0 0 KB 2 KB
Credit[n]
20 2 KB < 4 KB, so NP 250 can 0 0 KB 2 KB
NOT give a credit
21 NP 250 receives Frame 2 and 0 0 KB 2 KB
sends it to Port 500
22 NP 250 increments Pending 0 0 KB 4 KB
Credit[n]
23 4 KB >= 4 KB, so NP 250 can 0 0 KB 0 KB
give a credit
24 NP 250 sends credit message = 0 0 KB 0 KB
4 KB back to NP 1
25 NP 1 receives credit message = 0 4 KB 0 KB
4 KB and increments Cur
Buf[1]
26 (0 + 2) * 2 KB <= 4 KB, so NP 2 4 KB 0 KB
1 gives 2 BB Credits
27 NP 250 receives Frame 3 and 2 4 KB 0 KB
sends it to Port 500
28 NP 250 increments Pending 2 4 KB 2 KB
Credit[n]
29 2 KB < 4 KB, so NP 250 can 2 4 KB 2 KB
NOT give a credit
30 NP 250 receives Frame 4 and 2 4 KB 2 KB
sends it to Port 500
31 NP 250 increments Pending 2 4 KB 4 KB
Credit[n]
32 4 KB >= 4 KB, so NP 250 can 2 4 KB 0 KB
give a credit
33 NP 250 sends credit message = 2 4 KB 0 KB
4 KB back to NP 1
34 NP 1 receives credit message = 2 8 KB 0 KB
4 KB and increments Cur
Buf[1]
35 (2 + 1) * 2 KB <= 8 KB, so NP 3 4 KB 0 KB
1 gives 1 BB Credit; only 1
because 3 = Max_BB Credit
In one exemplary embodiment of the BB Credit buffer scheme of the present invention, one frame occupies one buffer/credit. Typically, one buffer contains usually 2 KB of memory. Thus, a small frame pocket of 36 bytes, for example, still consumes an entire 2 KB of memory. Accordingly, for another exemplary embodiment of the BB Credit scheme, several smaller frame packets may be logically grouped and associated with a single buffer. For example, several small frames of about 36 bytes may be collapsed into one 2 KB buffer. The system may keep track of the ordering via hardware logic. For example, the system may maintain a scoreboard to track the order in which the frame packets are to be sent. In this exemplary embodiment, the system maximizes the number of available credit for larger frames and thus potentially increases the throughput between the sender and receiver.
The presently disclosed flow control schemes provide a number of advantages. One advantage of the present invention is that the flow control scheme substantially eliminates head of line (HOL) blocking. As discussed above Fibre Channel standards define link-level and end-to-end congestion control protocols but these standard protocols do not eliminate head of line (HOL) blocking. HOL blocking is a problem for internal switching that occurs when several packets at the head of an input queue block packets from being forwarded to output ports. The dynamic BB Credit flow control scheme prevents problems caused by HOL blocking such as increased system latency, unintentionally dropped packets, and time-out problems.
Another advantage of the presently disclosed flow control schemes is that they allow for more efficient data streaming. Instead of frame based flow control that does not account for frame size, the dynamic BB Credit flow control scheme provides byte-based connectivity between the ingress and egress network processors. The ingress network processor is permitted to send a predefined amount of traffic to the egress network processor. This flow control scheme allows for the system to dynamically give BB Credits based on the amount of buffering available.
The invention, therefore, is well adapted to carry out the objects and attain the ends and advantages mentioned, as well as others inherent therein. While the invention has been depicted, described, and is defined by reference to exemplary embodiments of the invention, such references do not imply a limitation on the invention, and no such limitation is to be inferred. The invention is capable of considerable modification, alternation, and equivalents in form and function, as will occur to those ordinarily skilled in the pertinent arts and having the benefit of this disclosure. The depicted and described embodiments of the invention are exemplary only, and are not exhaustive of the scope of the invention. Consequently, the invention is intended to be limited only by the spirit and scope of the appended claims, giving full cognizance to equivalents in all respects.

Claims (6)

1. A method for providing buffer-to-buffer credit port-level flow control for a computer network in operative communication with a plurality of ingress and egress network processors, each having an egress port and an ingress port that is associated with a buffer-to-buffer credit value corresponding to the current number of frames the ingress port sends, a buffer value corresponding to the current total frame size the ingress port sends, and a pending buffer-to-buffer value corresponding to the pending buffer-to-buffer credits an egress port issues the ingress port, and a set of network processors is associated with a bridge, the method comprising the steps of:
sending a frame from the ingress port to a destination egress port, if the ingress port has a sufficient buffer-to-buffer credit value and buffer value;
decrementing the buffer-to-buffer credit value associated with the ingress port;
decrementing the buffer value associated with the ingress port;
determining whether to increment the buffer-to-buffer credit value associated with the ingress port by incrementing the buffer-to-buffer credit value associated with the ingress port if the product of one plus the buffer-to-buffer credit value times the maximum frame size in bytes is less than or equal to a minimum egress buffering value; the minimum egress buffering value corresponds to the minimum amount of egress buffering that is available for any one egress port;
incrementing the pending credit value associated with the ingress port;
determining whether to send a credit message to the ingress port;
the computer network further comprises a switch fabric: and
wherein the network processors are in operative communication with the switch fabric via the associated bridge.
2. The method of claim 1, wherein the step of determining whether to send a credit message to the ingress port further comprises:
sending the credit message if the pending credit value is greater than, or equal to, a credit watermark value.
3. The method of claim 2, further comprising the step of:
increasing the buffer value if the credit message is sent.
4. A system for providing buffer-to-buffer credit port-level flow control for a computer network in operative communication with a plurality of ingress and egress network processors, said system comprising:
a plurality of ingress and egress network processors, each of said plurality of network processors having an egress port and an ingress port that is associated with a buffer-to-buffer credit value corresponding to the current number of frames the ingress port sends, a buffer value corresponding to the current total frame size the ingress port sends, and a pending buffer-to-buffer value corresponding to the pending buffer-to-buffer credits an egress port issues the ingress port, and a set of said plurality of network processors is associated with a bridge, wherein:
a frame is sent from the ingress port to a destination egress port, if the ingress port has a sufficient buffer-to-buffer credit value and buffer value;
the buffer-to-buffer credit value associated with the ingress port is decremented;
the buffer value associated with the ingress port is decremented;
the buffer-to-buffer credit value associated with the ingress port is determined whether to be incremented, the buffer-to-buffer credit value associated with the ingress port is incremented if the product of one plus the buffer-to-buffer credit value times the maximum frame size in bytes is less than or equal to a minimum egress buffering value, the minimum egress buffering value corresponds to the minimum amount of egress buffering that is available for any one egress port;
the pending credit value associated with the ingress port is incremented;
a credit message to the ingress port is determined whether to be sent;
the computer network further comprises a switch fabric; and
wherein the network processors are in operative communication with the switch fabric via the associated bridge.
5. The system of claim 4, wherein when the pending credit value is greater than, or equal to, a credit watermark value the credit message is sent to the ingress port.
6. The system of claim 5, further comprising the buffer value is increased if the credit message is sent.
US10/039,184 2001-12-31 2001-12-31 Buffer to buffer credit flow control for computer network Expired - Fee Related US7085846B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US10/039,184 US7085846B2 (en) 2001-12-31 2001-12-31 Buffer to buffer credit flow control for computer network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US10/039,184 US7085846B2 (en) 2001-12-31 2001-12-31 Buffer to buffer credit flow control for computer network

Publications (2)

Publication Number Publication Date
US20030126223A1 US20030126223A1 (en) 2003-07-03
US7085846B2 true US7085846B2 (en) 2006-08-01

Family

ID=21904108

Family Applications (1)

Application Number Title Priority Date Filing Date
US10/039,184 Expired - Fee Related US7085846B2 (en) 2001-12-31 2001-12-31 Buffer to buffer credit flow control for computer network

Country Status (1)

Country Link
US (1) US7085846B2 (en)

Cited By (46)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030118053A1 (en) * 2001-12-26 2003-06-26 Andiamo Systems, Inc. Methods and apparatus for encapsulating a frame for transmission in a storage area network
US20040008673A1 (en) * 2002-07-11 2004-01-15 Ygal Arbel Overhead processing in telecommunications nodes
US20040008708A1 (en) * 2002-07-11 2004-01-15 Giacomini Peter J. Overhead engine for telecommunications nodes
US20040008701A1 (en) * 2002-07-11 2004-01-15 Giacomini Peter J. Hierarchical finite-state machines
US20040100910A1 (en) * 2002-11-27 2004-05-27 Andiamo Systems, Inc. Methods and devices for exchanging peer parameters between network devices
US20040120335A1 (en) * 2002-12-23 2004-06-24 Alcatel Canada Inc. Efficient per-queue backpressure signaling
US20040153566A1 (en) * 2003-01-31 2004-08-05 Brocade Communications Systems, Inc. Dynamic link distance configuration for extended fabric
US20050036499A1 (en) * 2001-12-26 2005-02-17 Andiamo Systems, Inc., A Delaware Corporation Fibre Channel Switch that enables end devices in different fabrics to communicate with one another while retaining their unique Fibre Channel Domain_IDs
US20070094530A1 (en) * 2005-10-21 2007-04-26 Yoshio Kakizaki Data transfer method and remote copy system
US20070133415A1 (en) * 2005-12-13 2007-06-14 Intel Corporation Method and apparatus for flow control initialization
US20070153816A1 (en) * 2002-06-12 2007-07-05 Cisco Technology, Inc. Methods and apparatus for characterizing a route in a fibre channel fabric
US20070162559A1 (en) * 2006-01-12 2007-07-12 Amitabha Biswas Protocol flow control
US7263066B1 (en) * 2001-12-14 2007-08-28 Applied Micro Circuits Corporation Switch fabric backplane flow management using credit-based flow control
US7304987B1 (en) * 2001-12-14 2007-12-04 Applied Micro Circuits Corporation System and method for synchronizing switch fabric backplane link management credit counters
US7324441B1 (en) * 2003-07-14 2008-01-29 Cisco Technology, Inc. Methods and apparatus for alleviating deadlock in a fibre channel network
US20080028090A1 (en) * 2006-07-26 2008-01-31 Sophana Kok System for managing messages transmitted in an on-chip interconnect network
US7327680B1 (en) * 2002-11-05 2008-02-05 Cisco Technology, Inc. Methods and apparatus for network congestion control
US7349435B2 (en) 2002-07-11 2008-03-25 Bay Microsystems, Inc. Multiport overhead cell processor for telecommunications nodes
US7406034B1 (en) 2002-04-01 2008-07-29 Cisco Technology, Inc. Methods and apparatus for fibre channel frame delivery
US20080186961A1 (en) * 2001-12-20 2008-08-07 Kenneth Yi Yun System and Method for Reevaluating Granted Arbitrated Bids
US20090041057A1 (en) * 2007-08-06 2009-02-12 International Business Machines Corporation Performing a recovery action in response to a credit depletion notification
US20090043880A1 (en) * 2007-08-06 2009-02-12 International Business Machines Corporation Credit depletion notification for transmitting frames between a port pair
US7593324B2 (en) 2004-10-25 2009-09-22 Cisco Technology, Inc. Graceful port shutdown protocol for fibre channel interfaces
US7616637B1 (en) 2002-04-01 2009-11-10 Cisco Technology, Inc. Label switching in fibre channel networks
US7649844B2 (en) * 2004-12-29 2010-01-19 Cisco Technology, Inc. In-order fibre channel packet delivery
US7782770B1 (en) * 2006-06-30 2010-08-24 Marvell International, Ltd. System and method of cross-chip flow control
US20110029709A1 (en) * 2009-08-03 2011-02-03 Feiereisel Neil S Data Movement System and Method
US7916628B2 (en) 2004-11-01 2011-03-29 Cisco Technology, Inc. Trunking for fabric ports in fibre channel switches and attached devices
US20110154132A1 (en) * 2009-12-23 2011-06-23 Gunes Aybay Methods and apparatus for tracking data flow based on flow state values
US20110267942A1 (en) * 2010-04-30 2011-11-03 Gunes Aybay Methods and apparatus for flow control associated with a switch fabric
US20120008636A1 (en) * 2007-12-27 2012-01-12 Cellco Partnership D/B/A Verizon Wireless Dynamically adjusted credit based round robin scheduler
US8418129B1 (en) 2001-12-14 2013-04-09 Qualcomm Incorporated Method for automatically generating code to define a system of hardware elements
US8593970B2 (en) 2008-09-11 2013-11-26 Juniper Networks, Inc. Methods and apparatus for defining a flow control signal related to a transmit queue
US8683000B1 (en) * 2006-10-27 2014-03-25 Hewlett-Packard Development Company, L.P. Virtual network interface system with memory management
US20140086054A1 (en) * 2010-07-19 2014-03-27 Cisco Technology, Inc. Mitigating the effects of congested interfaces on a fabric
US8717889B2 (en) 2008-12-29 2014-05-06 Juniper Networks, Inc. Flow-control in a switch fabric
US8811183B1 (en) 2011-10-04 2014-08-19 Juniper Networks, Inc. Methods and apparatus for multi-path flow control within a multi-stage switch fabric
US20140269294A1 (en) * 2013-03-13 2014-09-18 Cisco Technology, Inc. Intra Switch Transport Protocol
US8964556B2 (en) 2008-09-11 2015-02-24 Juniper Networks, Inc. Methods and apparatus for flow-controllable multi-staged queues
US9032089B2 (en) 2011-03-09 2015-05-12 Juniper Networks, Inc. Methods and apparatus for path selection within a network based on flow duration
US9065773B2 (en) 2010-06-22 2015-06-23 Juniper Networks, Inc. Methods and apparatus for virtual channel flow control associated with a switch fabric
US9178832B2 (en) 2013-07-11 2015-11-03 International Business Machines Corporation Queue credit management
US9660940B2 (en) 2010-12-01 2017-05-23 Juniper Networks, Inc. Methods and apparatus for flow control associated with a switch fabric
US20170289066A1 (en) * 2016-03-31 2017-10-05 Mellanox Technologies Tlv Ltd. Facilitating communication of data packets using credit-based flow control
US9860185B2 (en) 2013-03-14 2018-01-02 Cisco Technology, Inc. Intra switch transport protocol
US10122645B2 (en) 2012-12-07 2018-11-06 Cisco Technology, Inc. Output queue latency behavior for input queue based device

Families Citing this family (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030084219A1 (en) * 2001-10-26 2003-05-01 Maxxan Systems, Inc. System, apparatus and method for address forwarding for a computer network
US7145914B2 (en) * 2001-12-31 2006-12-05 Maxxan Systems, Incorporated System and method for controlling data paths of a network processor subsystem
US20030195956A1 (en) * 2002-04-15 2003-10-16 Maxxan Systems, Inc. System and method for allocating unique zone membership
US20030200330A1 (en) * 2002-04-22 2003-10-23 Maxxan Systems, Inc. System and method for load-sharing computer network switch
US20030202510A1 (en) * 2002-04-26 2003-10-30 Maxxan Systems, Inc. System and method for scalable switch fabric for computer network
US20040030766A1 (en) * 2002-08-12 2004-02-12 Michael Witkowski Method and apparatus for switch fabric configuration
US7397768B1 (en) 2002-09-11 2008-07-08 Qlogic, Corporation Zone management in a multi-module fibre channel switch
US7388843B2 (en) * 2003-07-16 2008-06-17 Qlogic, Corporation Method and apparatus for testing loop pathway integrity in a fibre channel arbitrated loop
US7355966B2 (en) * 2003-07-16 2008-04-08 Qlogic, Corporation Method and system for minimizing disruption in common-access networks
US7630384B2 (en) * 2003-07-21 2009-12-08 Qlogic, Corporation Method and system for distributing credit in fibre channel systems
US7646767B2 (en) 2003-07-21 2010-01-12 Qlogic, Corporation Method and system for programmable data dependant network routing
US7684401B2 (en) 2003-07-21 2010-03-23 Qlogic, Corporation Method and system for using extended fabric features with fibre channel switch elements
US7894348B2 (en) 2003-07-21 2011-02-22 Qlogic, Corporation Method and system for congestion control in a fibre channel switch
US7792115B2 (en) 2003-07-21 2010-09-07 Qlogic, Corporation Method and system for routing and filtering network data packets in fibre channel systems
US7539143B2 (en) * 2003-08-11 2009-05-26 Netapp, Inc. Network switching device ingress memory system
US20050089054A1 (en) * 2003-08-11 2005-04-28 Gene Ciancaglini Methods and apparatus for provisioning connection oriented, quality of service capabilities and services
US8199764B2 (en) * 2003-08-25 2012-06-12 Cisco Technology, Inc. Scalable approach to large scale queuing through dynamic resource allocation
US20050114540A1 (en) * 2003-11-12 2005-05-26 International Business Machines Corporation Method and system of generically specifying congestion control and a voidance behavior
US7930377B2 (en) 2004-04-23 2011-04-19 Qlogic, Corporation Method and system for using boot servers in networks
US7404020B2 (en) * 2004-07-20 2008-07-22 Qlogic, Corporation Integrated fibre channel fabric controller
US8295299B2 (en) 2004-10-01 2012-10-23 Qlogic, Corporation High speed fibre channel switch element
WO2006109207A1 (en) * 2005-04-13 2006-10-19 Koninklijke Philips Electronics N.V. Electronic device and method for flow control
JP2006309579A (en) * 2005-04-28 2006-11-09 Hitachi Ltd Storage control apparatus and storage system
US8194547B1 (en) * 2006-08-07 2012-06-05 Emc Corporation Configuring flow control settings
US7724756B2 (en) * 2007-12-10 2010-05-25 Alcatel-Lucent Input/output buffer controller for optimized memory utilization and prevention of packet under-run errors
JP4557028B2 (en) * 2008-03-19 2010-10-06 ソニー株式会社 Information processing apparatus, information processing method, client device, information processing system
CN102577278B (en) * 2009-10-19 2016-01-20 国际商业机器公司 For the Dynamic Resource Allocation for Multimedia of distributed cluster storage network
US9237103B2 (en) * 2012-03-30 2016-01-12 Cisco Technology, Inc. Congestion alleviation in network systems
US20150254088A1 (en) * 2014-03-08 2015-09-10 Datawise Systems, Inc. Methods and systems for converged networking and storage
CN107979544A (en) * 2016-10-25 2018-05-01 华为技术有限公司 A kind of retransmission method of IP packet, equipment and system
US11190588B2 (en) 2020-04-28 2021-11-30 International Business Machines Corporation Host initiated link reset in a storage area network
US11277356B2 (en) 2020-08-12 2022-03-15 International Business Machines Corporation Network buffer credit allocation

Citations (81)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4755930A (en) 1985-06-27 1988-07-05 Encore Computer Corporation Hierarchical cache memory system and method
US5140682A (en) 1988-07-08 1992-08-18 Hitachi, Ltd Storage control apparatus
US5247649A (en) 1988-05-06 1993-09-21 Hitachi, Ltd. Multi-processor system having a multi-port cache memory
US5515376A (en) 1993-07-19 1996-05-07 Alantec, Inc. Communication apparatus and methods
US5530832A (en) 1993-10-14 1996-06-25 International Business Machines Corporation System and method for practicing essential inclusion in a multiprocessor and cache hierarchy
US5602841A (en) 1994-04-07 1997-02-11 International Business Machines Corporation Efficient point-to-point and multi-point routing mechanism for programmable packet switching nodes in high speed data transmission networks
US5611049A (en) 1992-06-03 1997-03-11 Pitts; William M. System for accessing distributed data cache channel at each network node to pass requests and data
US5699548A (en) 1995-06-01 1997-12-16 Intel Corporation Method and apparatus for selecting a mode for updating external memory
US5778429A (en) 1994-07-04 1998-07-07 Hitachi, Ltd. Parallel processor system including a cache memory subsystem that has independently addressable local and remote data areas
US5835943A (en) 1995-11-30 1998-11-10 Stampede Technologies, Inc. Apparatus and method for increased data access in a network file oriented caching system
US5835756A (en) 1993-06-04 1998-11-10 Digital Equipment Corporation Real-time open file cache hashing table restructuring in a server computer
US5845280A (en) 1995-09-25 1998-12-01 Microsoft Corporation Method and apparatus for transmitting a file in a network using a single transmit request from a user-mode process to a kernel-mode process
US5845324A (en) 1995-04-28 1998-12-01 Unisys Corporation Dual bus network cache controller system having rapid invalidation cycles and reduced latency for cache access
US5852717A (en) 1996-11-20 1998-12-22 Shiva Corporation Performance optimizations for computer networks utilizing HTTP
US5864854A (en) 1996-01-05 1999-01-26 Lsi Logic Corporation System and method for maintaining a shared cache look-up table
US5873100A (en) 1996-12-20 1999-02-16 Intel Corporation Internet browser that includes an enhanced cache for user-controlled document retention
US5878218A (en) 1997-03-17 1999-03-02 International Business Machines Corporation Method and system for creating and utilizing common caches for internetworks
US5881229A (en) 1995-04-26 1999-03-09 Shiva Corporation Method and product for enchancing performance of computer networks including shared storage objects
US5918244A (en) 1994-05-06 1999-06-29 Eec Systems, Inc. Method and system for coherently caching I/O devices across a network
US5930253A (en) 1995-02-09 1999-07-27 Northern Telecom Limited Narrow band ATM switch arrangement for a communications network
US5933849A (en) 1997-04-10 1999-08-03 At&T Corp Scalable distributed caching system and method
US5944780A (en) 1997-05-05 1999-08-31 At&T Corp Network with shared caching
US5944789A (en) 1996-08-14 1999-08-31 Emc Corporation Network file server maintaining local caches of file directory information in data mover computers
US5978951A (en) 1997-09-11 1999-11-02 3Com Corporation High speed cache management unit for use in a bridge/router
US5978841A (en) 1996-03-08 1999-11-02 Berger; Louis Look ahead caching process for improved information retrieval response time by caching bodies of information before they are requested by the user
US5987223A (en) 1997-10-17 1999-11-16 Sharp Kabushiki Kaisha Image processing device
US5991810A (en) 1997-08-01 1999-11-23 Novell, Inc. User name authentication for gateway clients accessing a proxy cache server
US6041058A (en) 1997-09-11 2000-03-21 3Com Corporation Hardware filtering method and apparatus
US6044406A (en) * 1997-04-08 2000-03-28 International Business Machines Corporation Credit-based flow control checking and correction method
US6081883A (en) 1997-12-05 2000-06-27 Auspex Systems, Incorporated Processing system with dynamically allocatable buffer memory
US6085234A (en) 1994-11-28 2000-07-04 Inca Technology, Inc. Remote file services network-infrastructure cache
US6098096A (en) 1996-12-09 2000-08-01 Sun Microsystems, Inc. Method and apparatus for dynamic cache preloading across a network
US6128306A (en) 1996-08-30 2000-10-03 Sgs-Thomson Microelectronics Limited Cell queue formation in an ATM switch
US6138209A (en) 1997-09-05 2000-10-24 International Business Machines Corporation Data processing system and multi-way set associative cache utilizing class predict data structure and method thereof
US6147976A (en) 1996-06-24 2000-11-14 Cabletron Systems, Inc. Fast network layer packet filter
US6243358B1 (en) * 1997-02-07 2001-06-05 France Telecom Process and device for allocating resources in a packet transmission digital network
US6289386B1 (en) * 1998-05-11 2001-09-11 Lsi Logic Corporation Implementation of a divide algorithm for buffer credit calculation in a high speed serial channel
US20010037435A1 (en) 2000-05-31 2001-11-01 Van Doren Stephen R. Distributed address mapping and routing table mechanism that supports flexible configuration and partitioning in a modular switch-based, shared-memory multiprocessor computer system
US20010043564A1 (en) * 2000-01-10 2001-11-22 Mellanox Technologies Ltd. Packet communication buffering with dynamic flow control
US20020004842A1 (en) * 2000-06-30 2002-01-10 Kanad Ghose System and method for fast, reliable byte stream transport
US20020010790A1 (en) 2000-07-17 2002-01-24 Ellis Donald R. Architecture and addressing scheme for storage interconnect and emerging storage service providers
US20020012344A1 (en) * 2000-06-06 2002-01-31 Johnson Ian David Switching system
US20020024953A1 (en) 2000-07-05 2002-02-28 Davis Simon Paul Switching devices
US6400730B1 (en) 1999-03-10 2002-06-04 Nishan Systems, Inc. Method and apparatus for transferring data between IP network devices and SCSI and fibre channel devices over an IP network
US20020071439A1 (en) 2000-12-08 2002-06-13 Mike Reeves System and method of operating a communication network associated with an MPLS implementation of an ATM platform
US6484209B1 (en) 1997-10-31 2002-11-19 Nortel Networks Limited Efficient path based forwarding and multicast forwarding
US20020186703A1 (en) * 2001-05-31 2002-12-12 Steve West Distributed control of data flow in a network switch
US20020188786A1 (en) 2001-06-07 2002-12-12 Barrow Jonathan J. Data storage system with integrated switching
US20030002506A1 (en) 2001-07-02 2003-01-02 Hitachi, Ltd. Packet switching apparatus, method of transmitting multicast packet at packet switching apparatus, and setup method of packet switching apparatus
US20030012204A1 (en) 2001-07-11 2003-01-16 Sancastle Technologies, Ltd Extension of fibre channel addressing
US20030014540A1 (en) 2001-07-06 2003-01-16 Nortel Networks Limited Policy-based forwarding in open shortest path first (OSPF) networks
US6532501B1 (en) * 1999-09-30 2003-03-11 Silicon Graphics, Inc. System and method for distributing output queue space
US20030048792A1 (en) 2001-09-04 2003-03-13 Qq Technology, Inc. Forwarding device for communication networks
US20030063348A1 (en) 2000-10-27 2003-04-03 Posey Nolan J. System and method for packet classification
US20030074449A1 (en) * 2001-10-12 2003-04-17 Rory Smith Bandwidth allocation in a synchronous transmission network for packet oriented signals
US20030084219A1 (en) 2001-10-26 2003-05-01 Maxxan Systems, Inc. System, apparatus and method for address forwarding for a computer network
US20030093541A1 (en) 2001-09-28 2003-05-15 Lolayekar Santosh C. Protocol translation in a storage system
US20030093567A1 (en) 2001-09-28 2003-05-15 Lolayekar Santosh C. Serverless storage services
US6584101B2 (en) 1998-12-04 2003-06-24 Pmc-Sierra Ltd. Communication method for packet switching systems
US20030126297A1 (en) 2001-12-31 2003-07-03 Maxxan Systems, Inc. Network processor interface system
US20030128703A1 (en) 2002-01-03 2003-07-10 Yongdong Zhao Switch queue predictive protocol (SQPP) based packet switching technique
US6594701B1 (en) * 1998-08-04 2003-07-15 Microsoft Corporation Credit-based methods and systems for controlling data flow between a sender and a receiver with reduced copying of data
US6597699B1 (en) 1999-09-28 2003-07-22 Telefonaktiebolaget Lm Ericsson (Publ) Quality of service management in a packet data router system having multiple virtual router instances
US20030154301A1 (en) 2002-01-24 2003-08-14 Mceachern William Ross System and method of downloading data for a communication switch
US20030163555A1 (en) 2001-02-28 2003-08-28 Abdella Battou Multi-tiered control architecture for adaptive optical networks, and methods and apparatus therefor
US6615271B1 (en) * 1994-07-18 2003-09-02 Enterasys Networks, Inc. Traffic control system having distributed rate calculation and link flow control
US20030195956A1 (en) 2002-04-15 2003-10-16 Maxxan Systems, Inc. System and method for allocating unique zone membership
US20030198231A1 (en) 1999-06-30 2003-10-23 Broadcom Corporation Frame forwarding in a switch fabric
US20030202520A1 (en) 2002-04-26 2003-10-30 Maxxan Systems, Inc. Scalable switch fabric system and apparatus for computer networks
US6657962B1 (en) * 2000-04-10 2003-12-02 International Business Machines Corporation Method and system for managing congestion in a network
US6687247B1 (en) 1999-10-27 2004-02-03 Cisco Technology, Inc. Architecture for high speed class of service enabled linecard
US6721818B1 (en) 1998-08-24 2004-04-13 Canon Kabushiki Kaisha Electronic device that stores information on its location based on information obtained from a node
US6731644B1 (en) 2000-02-14 2004-05-04 Cisco Technology, Inc. Flexible DMA engine for packet header modification
US6735174B1 (en) * 2000-03-29 2004-05-11 Intel Corporation Method and systems for flow control of transmissions over channel-based switched fabric connections
US6747949B1 (en) * 1999-05-21 2004-06-08 Intel Corporation Register based remote data flow control
US6757791B1 (en) 1999-03-30 2004-06-29 Cisco Technology, Inc. Method and apparatus for reordering packet data units in storage queues for reading and writing memory
US6765919B1 (en) 1998-10-23 2004-07-20 Brocade Communications Systems, Inc. Method and system for creating and implementing zones within a fibre channel system
US6765871B1 (en) * 2000-11-29 2004-07-20 Akara Corporation Fiber channel flow control method and apparatus for interface to metro area transport link
US6785241B1 (en) * 1999-12-02 2004-08-31 International Business Machines Corporation Method for pacing buffered data transfers over a network such as fibre channel
US6792507B2 (en) 2000-12-14 2004-09-14 Maxxan Systems, Inc. Caching system and method for a network storage system
US20050243734A1 (en) 2001-09-28 2005-11-03 Nemirovsky Mario D Multi-threaded packet processing engine for stateful packet processing

Family Cites Families (35)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4442504A (en) * 1981-03-09 1984-04-10 Allen-Bradley Company Modular programmable controller
US4598404A (en) * 1983-12-22 1986-07-01 Gte Automatic Electric Inc. Data format arrangement for communication between the peripheral processors of a telecommunications switching network
CA1299274C (en) * 1987-07-24 1992-04-21 Shin-Ichiro Hayano Time-division multiplex switching network
US5289460A (en) * 1992-07-31 1994-02-22 International Business Machines Corp. Maintenance of message distribution trees in a communications network
US5933607A (en) * 1993-06-07 1999-08-03 Telstra Corporation Limited Digital communication system for simultaneous transmission of data from constant and variable rate sources
US5918224A (en) * 1995-07-26 1999-06-29 Borland International, Inc. Client/server database system with methods for providing clients with server-based bi-directional scrolling at the server
GB2304255B (en) * 1995-08-07 2000-04-12 Be Aerospace Inc Multi-stage switch
US5805785A (en) * 1996-02-27 1998-09-08 International Business Machines Corporation Method for monitoring and recovery of subsystems in a distributed/clustered system
US6470013B1 (en) * 1998-10-13 2002-10-22 Cisco Technology, Inc. Use of enhanced ethernet link—loop packets to automate configuration of intelligent linecards attached to a router
US6574632B2 (en) * 1998-11-18 2003-06-03 Harris Corporation Multiple engine information retrieval and visualization system
US6597689B1 (en) * 1998-12-30 2003-07-22 Nortel Networks Limited SVC signaling system and method
US6438705B1 (en) * 1999-01-29 2002-08-20 International Business Machines Corporation Method and apparatus for building and managing multi-clustered computer systems
US6674756B1 (en) * 1999-02-23 2004-01-06 Alcatel Multi-service network switch with multiple virtual routers
US6850531B1 (en) * 1999-02-23 2005-02-01 Alcatel Multi-service network switch
US6401120B1 (en) * 1999-03-26 2002-06-04 Microsoft Corporation Method and system for consistent cluster operational data in a server cluster using a quorum of replicas
GB2360168B (en) * 2000-03-11 2003-07-16 3Com Corp Network switch including hysteresis in signalling fullness of transmit queues
US7320034B2 (en) * 2000-03-20 2008-01-15 International Business Machines Corporation System and method for reserving a virtual connection in an IP network
EP1297662A2 (en) * 2000-06-02 2003-04-02 Inrange Technologies Corporation Address conversion method and device in a fibre channel switch
US6865602B1 (en) * 2000-07-24 2005-03-08 Alcatel Canada Inc. Network management support for OAM functionality and method therefore
US20030152182A1 (en) * 2000-08-22 2003-08-14 Pai B. Anand Optical exchange method, apparatus and system for facilitating data transport between WAN, SAN and LAN and for enabling enterprise computing into networks
US6823453B1 (en) * 2000-10-06 2004-11-23 Hewlett-Packard Development Company, L.P. Apparatus and method for implementing spoofing-and replay-attack-resistant virtual zones on storage area networks
US20030097439A1 (en) * 2000-10-23 2003-05-22 Strayer William Timothy Systems and methods for identifying anomalies in network data streams
US7031320B2 (en) * 2000-12-22 2006-04-18 Samsung Electronics Co., Ltd. Apparatus and method for performing high-speed IP route lookup and managing routing/forwarding tables
US20020103921A1 (en) * 2001-01-31 2002-08-01 Shekar Nair Method and system for routing broadband internet traffic
US20030091267A1 (en) * 2001-02-28 2003-05-15 Alvarez Mario F. Node management architecture with customized line card handlers for a modular optical network, and methods and apparatus therefor
US7023869B2 (en) * 2001-05-10 2006-04-04 Emc Corporation Data storage system with one or more integrated server-like behaviors
US7310644B2 (en) * 2001-06-06 2007-12-18 Microsoft Corporation Locating potentially identical objects across multiple computers
US20030015430A1 (en) * 2001-07-16 2003-01-23 Ronald Frowein Electrodeposition coating process in continuous electrodeposition coating plants
US20030026267A1 (en) * 2001-07-31 2003-02-06 Oberman Stuart F. Virtual channels in a network switch
US20030033346A1 (en) * 2001-08-10 2003-02-13 Sun Microsystems, Inc. Method, system, and program for managing multiple resources in a system
US6944829B2 (en) * 2001-09-25 2005-09-13 Wind River Systems, Inc. Configurable user-interface component management system
US20030097445A1 (en) * 2001-11-20 2003-05-22 Stephen Todd Pluggable devices services and events for a scalable storage service architecture
US6845431B2 (en) * 2001-12-28 2005-01-18 Hewlett-Packard Development Company, L.P. System and method for intermediating communication with a moveable media library utilizing a plurality of partitions
US6983303B2 (en) * 2002-01-31 2006-01-03 Hewlett-Packard Development Company, Lp. Storage aggregator for enhancing virtualization in data storage networks
US6988149B2 (en) * 2002-02-26 2006-01-17 Lsi Logic Corporation Integrated target masking

Patent Citations (82)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4755930A (en) 1985-06-27 1988-07-05 Encore Computer Corporation Hierarchical cache memory system and method
US5247649A (en) 1988-05-06 1993-09-21 Hitachi, Ltd. Multi-processor system having a multi-port cache memory
US5140682A (en) 1988-07-08 1992-08-18 Hitachi, Ltd Storage control apparatus
US5611049A (en) 1992-06-03 1997-03-11 Pitts; William M. System for accessing distributed data cache channel at each network node to pass requests and data
US5835756A (en) 1993-06-04 1998-11-10 Digital Equipment Corporation Real-time open file cache hashing table restructuring in a server computer
US5515376A (en) 1993-07-19 1996-05-07 Alantec, Inc. Communication apparatus and methods
US5530832A (en) 1993-10-14 1996-06-25 International Business Machines Corporation System and method for practicing essential inclusion in a multiprocessor and cache hierarchy
US5602841A (en) 1994-04-07 1997-02-11 International Business Machines Corporation Efficient point-to-point and multi-point routing mechanism for programmable packet switching nodes in high speed data transmission networks
US5918244A (en) 1994-05-06 1999-06-29 Eec Systems, Inc. Method and system for coherently caching I/O devices across a network
US5778429A (en) 1994-07-04 1998-07-07 Hitachi, Ltd. Parallel processor system including a cache memory subsystem that has independently addressable local and remote data areas
US6615271B1 (en) * 1994-07-18 2003-09-02 Enterasys Networks, Inc. Traffic control system having distributed rate calculation and link flow control
US6085234A (en) 1994-11-28 2000-07-04 Inca Technology, Inc. Remote file services network-infrastructure cache
US5930253A (en) 1995-02-09 1999-07-27 Northern Telecom Limited Narrow band ATM switch arrangement for a communications network
US5881229A (en) 1995-04-26 1999-03-09 Shiva Corporation Method and product for enchancing performance of computer networks including shared storage objects
US5845324A (en) 1995-04-28 1998-12-01 Unisys Corporation Dual bus network cache controller system having rapid invalidation cycles and reduced latency for cache access
US5699548A (en) 1995-06-01 1997-12-16 Intel Corporation Method and apparatus for selecting a mode for updating external memory
US5845280A (en) 1995-09-25 1998-12-01 Microsoft Corporation Method and apparatus for transmitting a file in a network using a single transmit request from a user-mode process to a kernel-mode process
US5835943A (en) 1995-11-30 1998-11-10 Stampede Technologies, Inc. Apparatus and method for increased data access in a network file oriented caching system
US5864854A (en) 1996-01-05 1999-01-26 Lsi Logic Corporation System and method for maintaining a shared cache look-up table
US5978841A (en) 1996-03-08 1999-11-02 Berger; Louis Look ahead caching process for improved information retrieval response time by caching bodies of information before they are requested by the user
US6147976A (en) 1996-06-24 2000-11-14 Cabletron Systems, Inc. Fast network layer packet filter
US5944789A (en) 1996-08-14 1999-08-31 Emc Corporation Network file server maintaining local caches of file directory information in data mover computers
US6128306A (en) 1996-08-30 2000-10-03 Sgs-Thomson Microelectronics Limited Cell queue formation in an ATM switch
US5852717A (en) 1996-11-20 1998-12-22 Shiva Corporation Performance optimizations for computer networks utilizing HTTP
US6098096A (en) 1996-12-09 2000-08-01 Sun Microsystems, Inc. Method and apparatus for dynamic cache preloading across a network
US5873100A (en) 1996-12-20 1999-02-16 Intel Corporation Internet browser that includes an enhanced cache for user-controlled document retention
US6243358B1 (en) * 1997-02-07 2001-06-05 France Telecom Process and device for allocating resources in a packet transmission digital network
US5878218A (en) 1997-03-17 1999-03-02 International Business Machines Corporation Method and system for creating and utilizing common caches for internetworks
US6044406A (en) * 1997-04-08 2000-03-28 International Business Machines Corporation Credit-based flow control checking and correction method
US5933849A (en) 1997-04-10 1999-08-03 At&T Corp Scalable distributed caching system and method
US5944780A (en) 1997-05-05 1999-08-31 At&T Corp Network with shared caching
US5991810A (en) 1997-08-01 1999-11-23 Novell, Inc. User name authentication for gateway clients accessing a proxy cache server
US6138209A (en) 1997-09-05 2000-10-24 International Business Machines Corporation Data processing system and multi-way set associative cache utilizing class predict data structure and method thereof
US6041058A (en) 1997-09-11 2000-03-21 3Com Corporation Hardware filtering method and apparatus
US5978951A (en) 1997-09-11 1999-11-02 3Com Corporation High speed cache management unit for use in a bridge/router
US5987223A (en) 1997-10-17 1999-11-16 Sharp Kabushiki Kaisha Image processing device
US6484209B1 (en) 1997-10-31 2002-11-19 Nortel Networks Limited Efficient path based forwarding and multicast forwarding
US6081883A (en) 1997-12-05 2000-06-27 Auspex Systems, Incorporated Processing system with dynamically allocatable buffer memory
US6289386B1 (en) * 1998-05-11 2001-09-11 Lsi Logic Corporation Implementation of a divide algorithm for buffer credit calculation in a high speed serial channel
US6594701B1 (en) * 1998-08-04 2003-07-15 Microsoft Corporation Credit-based methods and systems for controlling data flow between a sender and a receiver with reduced copying of data
US6721818B1 (en) 1998-08-24 2004-04-13 Canon Kabushiki Kaisha Electronic device that stores information on its location based on information obtained from a node
US20050018619A1 (en) 1998-10-23 2005-01-27 David Banks Method and system for creating and implementing zones within a fibre channel system
US6765919B1 (en) 1998-10-23 2004-07-20 Brocade Communications Systems, Inc. Method and system for creating and implementing zones within a fibre channel system
US6584101B2 (en) 1998-12-04 2003-06-24 Pmc-Sierra Ltd. Communication method for packet switching systems
US6400730B1 (en) 1999-03-10 2002-06-04 Nishan Systems, Inc. Method and apparatus for transferring data between IP network devices and SCSI and fibre channel devices over an IP network
US6757791B1 (en) 1999-03-30 2004-06-29 Cisco Technology, Inc. Method and apparatus for reordering packet data units in storage queues for reading and writing memory
US6747949B1 (en) * 1999-05-21 2004-06-08 Intel Corporation Register based remote data flow control
US20030198231A1 (en) 1999-06-30 2003-10-23 Broadcom Corporation Frame forwarding in a switch fabric
US6597699B1 (en) 1999-09-28 2003-07-22 Telefonaktiebolaget Lm Ericsson (Publ) Quality of service management in a packet data router system having multiple virtual router instances
US6532501B1 (en) * 1999-09-30 2003-03-11 Silicon Graphics, Inc. System and method for distributing output queue space
US6687247B1 (en) 1999-10-27 2004-02-03 Cisco Technology, Inc. Architecture for high speed class of service enabled linecard
US6785241B1 (en) * 1999-12-02 2004-08-31 International Business Machines Corporation Method for pacing buffered data transfers over a network such as fibre channel
US20010043564A1 (en) * 2000-01-10 2001-11-22 Mellanox Technologies Ltd. Packet communication buffering with dynamic flow control
US6731644B1 (en) 2000-02-14 2004-05-04 Cisco Technology, Inc. Flexible DMA engine for packet header modification
US6735174B1 (en) * 2000-03-29 2004-05-11 Intel Corporation Method and systems for flow control of transmissions over channel-based switched fabric connections
US6657962B1 (en) * 2000-04-10 2003-12-02 International Business Machines Corporation Method and system for managing congestion in a network
US20010037435A1 (en) 2000-05-31 2001-11-01 Van Doren Stephen R. Distributed address mapping and routing table mechanism that supports flexible configuration and partitioning in a modular switch-based, shared-memory multiprocessor computer system
US20020012344A1 (en) * 2000-06-06 2002-01-31 Johnson Ian David Switching system
US20020004842A1 (en) * 2000-06-30 2002-01-10 Kanad Ghose System and method for fast, reliable byte stream transport
US20020024953A1 (en) 2000-07-05 2002-02-28 Davis Simon Paul Switching devices
US20020010790A1 (en) 2000-07-17 2002-01-24 Ellis Donald R. Architecture and addressing scheme for storage interconnect and emerging storage service providers
US20030063348A1 (en) 2000-10-27 2003-04-03 Posey Nolan J. System and method for packet classification
US6765871B1 (en) * 2000-11-29 2004-07-20 Akara Corporation Fiber channel flow control method and apparatus for interface to metro area transport link
US20020071439A1 (en) 2000-12-08 2002-06-13 Mike Reeves System and method of operating a communication network associated with an MPLS implementation of an ATM platform
US6792507B2 (en) 2000-12-14 2004-09-14 Maxxan Systems, Inc. Caching system and method for a network storage system
US20030163555A1 (en) 2001-02-28 2003-08-28 Abdella Battou Multi-tiered control architecture for adaptive optical networks, and methods and apparatus therefor
US20020186703A1 (en) * 2001-05-31 2002-12-12 Steve West Distributed control of data flow in a network switch
US20020188786A1 (en) 2001-06-07 2002-12-12 Barrow Jonathan J. Data storage system with integrated switching
US20030002506A1 (en) 2001-07-02 2003-01-02 Hitachi, Ltd. Packet switching apparatus, method of transmitting multicast packet at packet switching apparatus, and setup method of packet switching apparatus
US20030014540A1 (en) 2001-07-06 2003-01-16 Nortel Networks Limited Policy-based forwarding in open shortest path first (OSPF) networks
US20030012204A1 (en) 2001-07-11 2003-01-16 Sancastle Technologies, Ltd Extension of fibre channel addressing
US20030048792A1 (en) 2001-09-04 2003-03-13 Qq Technology, Inc. Forwarding device for communication networks
US20030093541A1 (en) 2001-09-28 2003-05-15 Lolayekar Santosh C. Protocol translation in a storage system
US20030093567A1 (en) 2001-09-28 2003-05-15 Lolayekar Santosh C. Serverless storage services
US20050243734A1 (en) 2001-09-28 2005-11-03 Nemirovsky Mario D Multi-threaded packet processing engine for stateful packet processing
US20030074449A1 (en) * 2001-10-12 2003-04-17 Rory Smith Bandwidth allocation in a synchronous transmission network for packet oriented signals
US20030084219A1 (en) 2001-10-26 2003-05-01 Maxxan Systems, Inc. System, apparatus and method for address forwarding for a computer network
US20030126297A1 (en) 2001-12-31 2003-07-03 Maxxan Systems, Inc. Network processor interface system
US20030128703A1 (en) 2002-01-03 2003-07-10 Yongdong Zhao Switch queue predictive protocol (SQPP) based packet switching technique
US20030154301A1 (en) 2002-01-24 2003-08-14 Mceachern William Ross System and method of downloading data for a communication switch
US20030195956A1 (en) 2002-04-15 2003-10-16 Maxxan Systems, Inc. System and method for allocating unique zone membership
US20030202520A1 (en) 2002-04-26 2003-10-30 Maxxan Systems, Inc. Scalable switch fabric system and apparatus for computer networks

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
PCT International Search Report PCT/US01/47769, 3 pages, Mailing Date Apr. 23, 2002.
U.S. Appl. No. 10/117,040 entitled "System and Method for Expansion of Computer Network Switching System Without Disruption Thereof," filed Apr. 5, 2002 by Mark Oelke et al.
U.S. Appl. No. 10/117,266 entitled "System and Method for Guarnateed Link Layer Flow Control," filed Apr. 5, 2002 by Hani Ajus et al.
U.S. Appl. No. 10/117,290 entitled "Method and System for Reduced Distributed Event Handling in a Network Environment," filed Apr. 5, 2005 by Huang Ruotao et al.
U.S. Appl. No. 10/117,418 entitled "System and Method for Linking a Plurality of Network Switches," filed Apr. 5, 2002 by Ram Iyer et al.
U.S. Appl. No. 10/117,638 entitled "Fibre Channel Implementation Using Network Processors," filed Apr. 5, 2002 by Yao Hawkins et al.

Cited By (86)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7263066B1 (en) * 2001-12-14 2007-08-28 Applied Micro Circuits Corporation Switch fabric backplane flow management using credit-based flow control
US7965624B2 (en) 2001-12-14 2011-06-21 Qualcomm Incorporated Data link fault tolerance
US20080253294A1 (en) * 2001-12-14 2008-10-16 Alberto Alessandro Della Ripa Data link fault tolerance
US8418129B1 (en) 2001-12-14 2013-04-09 Qualcomm Incorporated Method for automatically generating code to define a system of hardware elements
US7304987B1 (en) * 2001-12-14 2007-12-04 Applied Micro Circuits Corporation System and method for synchronizing switch fabric backplane link management credit counters
US7889729B2 (en) 2001-12-20 2011-02-15 Qualcomm Incorporated System and method for reevaluating granted arbitrated bids
US20080186961A1 (en) * 2001-12-20 2008-08-07 Kenneth Yi Yun System and Method for Reevaluating Granted Arbitrated Bids
US20050036499A1 (en) * 2001-12-26 2005-02-17 Andiamo Systems, Inc., A Delaware Corporation Fibre Channel Switch that enables end devices in different fabrics to communicate with one another while retaining their unique Fibre Channel Domain_IDs
US7599360B2 (en) 2001-12-26 2009-10-06 Cisco Technology, Inc. Methods and apparatus for encapsulating a frame for transmission in a storage area network
US20030118053A1 (en) * 2001-12-26 2003-06-26 Andiamo Systems, Inc. Methods and apparatus for encapsulating a frame for transmission in a storage area network
US7406034B1 (en) 2002-04-01 2008-07-29 Cisco Technology, Inc. Methods and apparatus for fibre channel frame delivery
US7616637B1 (en) 2002-04-01 2009-11-10 Cisco Technology, Inc. Label switching in fibre channel networks
US9350653B2 (en) 2002-04-01 2016-05-24 Cisco Technology, Inc. Label switching in fibre channel networks
US8462790B2 (en) 2002-04-01 2013-06-11 Cisco Technology, Inc. Label switching in fibre channel networks
US20070153816A1 (en) * 2002-06-12 2007-07-05 Cisco Technology, Inc. Methods and apparatus for characterizing a route in a fibre channel fabric
US7830809B2 (en) 2002-06-12 2010-11-09 Cisco Technology, Inc. Methods and apparatus for characterizing a route in a fibre channel fabric
US20040008708A1 (en) * 2002-07-11 2004-01-15 Giacomini Peter J. Overhead engine for telecommunications nodes
US20040008701A1 (en) * 2002-07-11 2004-01-15 Giacomini Peter J. Hierarchical finite-state machines
US7349435B2 (en) 2002-07-11 2008-03-25 Bay Microsystems, Inc. Multiport overhead cell processor for telecommunications nodes
US20040008673A1 (en) * 2002-07-11 2004-01-15 Ygal Arbel Overhead processing in telecommunications nodes
US7327680B1 (en) * 2002-11-05 2008-02-05 Cisco Technology, Inc. Methods and apparatus for network congestion control
US8605624B2 (en) 2002-11-27 2013-12-10 Cisco Technology, Inc. Methods and devices for exchanging peer parameters between network devices
US20040100910A1 (en) * 2002-11-27 2004-05-27 Andiamo Systems, Inc. Methods and devices for exchanging peer parameters between network devices
US7433326B2 (en) 2002-11-27 2008-10-07 Cisco Technology, Inc. Methods and devices for exchanging peer parameters between network devices
US20040120335A1 (en) * 2002-12-23 2004-06-24 Alcatel Canada Inc. Efficient per-queue backpressure signaling
US20040153566A1 (en) * 2003-01-31 2004-08-05 Brocade Communications Systems, Inc. Dynamic link distance configuration for extended fabric
US7600035B2 (en) * 2003-01-31 2009-10-06 Brocade Communications Systems, Inc. Dynamic link distance configuration for extended fabric
US20090274169A1 (en) * 2003-01-31 2009-11-05 Brocade Communications Systems, Inc. Dynamic link distance configuration for extended fabric
US7876711B2 (en) 2003-06-26 2011-01-25 Cisco Technology, Inc. Fibre channel switch that enables end devices in different fabrics to communicate with one another while retaining their unique fibre channel domain—IDs
US8625460B2 (en) 2003-06-26 2014-01-07 Cisco Technology, Inc. Fibre channel switch that enables end devices in different fabrics to communicate with one another while retaining their unique fibre channel domain—IDs
US7324441B1 (en) * 2003-07-14 2008-01-29 Cisco Technology, Inc. Methods and apparatus for alleviating deadlock in a fibre channel network
US7593324B2 (en) 2004-10-25 2009-09-22 Cisco Technology, Inc. Graceful port shutdown protocol for fibre channel interfaces
US7916628B2 (en) 2004-11-01 2011-03-29 Cisco Technology, Inc. Trunking for fabric ports in fibre channel switches and attached devices
US8750094B2 (en) 2004-11-01 2014-06-10 Cisco Technology, Inc. Trunking for fabric ports in Fibre channel switches and attached devices
US7649844B2 (en) * 2004-12-29 2010-01-19 Cisco Technology, Inc. In-order fibre channel packet delivery
US20070094530A1 (en) * 2005-10-21 2007-04-26 Yoshio Kakizaki Data transfer method and remote copy system
US7441142B2 (en) * 2005-10-21 2008-10-21 Hitachi, Ltd. Data transfer method and remote copy system
US20070133415A1 (en) * 2005-12-13 2007-06-14 Intel Corporation Method and apparatus for flow control initialization
US7924708B2 (en) * 2005-12-13 2011-04-12 Intel Corporation Method and apparatus for flow control initialization
US7895329B2 (en) * 2006-01-12 2011-02-22 Hewlett-Packard Development Company, L.P. Protocol flow control
US20070162559A1 (en) * 2006-01-12 2007-07-12 Amitabha Biswas Protocol flow control
US7782770B1 (en) * 2006-06-30 2010-08-24 Marvell International, Ltd. System and method of cross-chip flow control
US8085658B1 (en) 2006-06-30 2011-12-27 Marvell International Ltd. System and method of cross-chip flow control
US20080028090A1 (en) * 2006-07-26 2008-01-31 Sophana Kok System for managing messages transmitted in an on-chip interconnect network
US8683000B1 (en) * 2006-10-27 2014-03-25 Hewlett-Packard Development Company, L.P. Virtual network interface system with memory management
US7975027B2 (en) * 2007-08-06 2011-07-05 International Business Machines Corporation Credit depletion notification for transmitting frames between a port pair
US20090043880A1 (en) * 2007-08-06 2009-02-12 International Business Machines Corporation Credit depletion notification for transmitting frames between a port pair
US7787375B2 (en) 2007-08-06 2010-08-31 International Business Machines Corporation Performing a recovery action in response to a credit depletion notification
US20090041057A1 (en) * 2007-08-06 2009-02-12 International Business Machines Corporation Performing a recovery action in response to a credit depletion notification
US20120008636A1 (en) * 2007-12-27 2012-01-12 Cellco Partnership D/B/A Verizon Wireless Dynamically adjusted credit based round robin scheduler
US9042398B2 (en) * 2007-12-27 2015-05-26 Cellco Partnership Dynamically adjusted credit based round robin scheduler
US10931589B2 (en) 2008-09-11 2021-02-23 Juniper Networks, Inc. Methods and apparatus for flow-controllable multi-staged queues
US8593970B2 (en) 2008-09-11 2013-11-26 Juniper Networks, Inc. Methods and apparatus for defining a flow control signal related to a transmit queue
US9876725B2 (en) 2008-09-11 2018-01-23 Juniper Networks, Inc. Methods and apparatus for flow-controllable multi-staged queues
US8964556B2 (en) 2008-09-11 2015-02-24 Juniper Networks, Inc. Methods and apparatus for flow-controllable multi-staged queues
US8717889B2 (en) 2008-12-29 2014-05-06 Juniper Networks, Inc. Flow-control in a switch fabric
US8307136B2 (en) * 2009-08-03 2012-11-06 National Instruments Corporation Data movement system and method
US20110029709A1 (en) * 2009-08-03 2011-02-03 Feiereisel Neil S Data Movement System and Method
US20110154132A1 (en) * 2009-12-23 2011-06-23 Gunes Aybay Methods and apparatus for tracking data flow based on flow state values
US9264321B2 (en) 2009-12-23 2016-02-16 Juniper Networks, Inc. Methods and apparatus for tracking data flow based on flow state values
US10554528B2 (en) 2009-12-23 2020-02-04 Juniper Networks, Inc. Methods and apparatus for tracking data flow based on flow state values
US9967167B2 (en) 2009-12-23 2018-05-08 Juniper Networks, Inc. Methods and apparatus for tracking data flow based on flow state values
US11323350B2 (en) 2009-12-23 2022-05-03 Juniper Networks, Inc. Methods and apparatus for tracking data flow based on flow state values
US20110267942A1 (en) * 2010-04-30 2011-11-03 Gunes Aybay Methods and apparatus for flow control associated with a switch fabric
US11398991B1 (en) 2010-04-30 2022-07-26 Juniper Networks, Inc. Methods and apparatus for flow control associated with a switch fabric
US9602439B2 (en) * 2010-04-30 2017-03-21 Juniper Networks, Inc. Methods and apparatus for flow control associated with a switch fabric
US10560381B1 (en) 2010-04-30 2020-02-11 Juniper Networks, Inc. Methods and apparatus for flow control associated with a switch fabric
US9065773B2 (en) 2010-06-22 2015-06-23 Juniper Networks, Inc. Methods and apparatus for virtual channel flow control associated with a switch fabric
US9705827B2 (en) 2010-06-22 2017-07-11 Juniper Networks, Inc. Methods and apparatus for virtual channel flow control associated with a switch fabric
US9444742B2 (en) * 2010-07-19 2016-09-13 Cisco Technology, Inc. Mitigating the effects of congested interfaces on a fabric
US20140086054A1 (en) * 2010-07-19 2014-03-27 Cisco Technology, Inc. Mitigating the effects of congested interfaces on a fabric
US10616143B2 (en) 2010-12-01 2020-04-07 Juniper Networks, Inc. Methods and apparatus for flow control associated with a switch fabric
US9660940B2 (en) 2010-12-01 2017-05-23 Juniper Networks, Inc. Methods and apparatus for flow control associated with a switch fabric
US11711319B2 (en) 2010-12-01 2023-07-25 Juniper Networks, Inc. Methods and apparatus for flow control associated with a switch fabric
US9032089B2 (en) 2011-03-09 2015-05-12 Juniper Networks, Inc. Methods and apparatus for path selection within a network based on flow duration
US9716661B2 (en) 2011-03-09 2017-07-25 Juniper Networks, Inc. Methods and apparatus for path selection within a network based on flow duration
US9426085B1 (en) 2011-10-04 2016-08-23 Juniper Networks, Inc. Methods and apparatus for multi-path flow control within a multi-stage switch fabric
US8811183B1 (en) 2011-10-04 2014-08-19 Juniper Networks, Inc. Methods and apparatus for multi-path flow control within a multi-stage switch fabric
US10122645B2 (en) 2012-12-07 2018-11-06 Cisco Technology, Inc. Output queue latency behavior for input queue based device
US9628406B2 (en) * 2013-03-13 2017-04-18 Cisco Technology, Inc. Intra switch transport protocol
US20140269294A1 (en) * 2013-03-13 2014-09-18 Cisco Technology, Inc. Intra Switch Transport Protocol
US9860185B2 (en) 2013-03-14 2018-01-02 Cisco Technology, Inc. Intra switch transport protocol
US9455926B2 (en) 2013-07-11 2016-09-27 Globalfoundries Inc. Queue credit management
US9178832B2 (en) 2013-07-11 2015-11-03 International Business Machines Corporation Queue credit management
US10079782B2 (en) * 2016-03-31 2018-09-18 Mellanox Technologies Tlv Ltd. Facilitating communication of data packets using credit-based flow control
US20170289066A1 (en) * 2016-03-31 2017-10-05 Mellanox Technologies Tlv Ltd. Facilitating communication of data packets using credit-based flow control

Also Published As

Publication number Publication date
US20030126223A1 (en) 2003-07-03

Similar Documents

Publication Publication Date Title
US7085846B2 (en) Buffer to buffer credit flow control for computer network
US7760752B2 (en) Programmable pseudo virtual lanes for fibre channel systems
US7296093B1 (en) Network processor interface system
US7512067B2 (en) Method and system for congestion control based on optimum bandwidth allocation in a fibre channel switch
JP2753468B2 (en) Digital communication controller
US6424626B1 (en) Method and system for discarding and regenerating acknowledgment packets in ADSL communications
US7596627B2 (en) Methods and apparatus for network congestion control
US7062568B1 (en) Point-to-point protocol flow control extension
US8081650B2 (en) Method and system for selecting virtual lanes in fibre channel switches
US7161907B2 (en) System and method for dynamic rate flow control
EP1280301B1 (en) Method and switch for congestion control
US7894348B2 (en) Method and system for congestion control in a fibre channel switch
US6600721B2 (en) End node pacing for QOS and bandwidth management
US20030126280A1 (en) XON/XOFF flow control for computer network
US7391728B2 (en) Apparatus and method for improved Fibre Channel oversubscription over transport
US8072988B2 (en) Method and system for buffer-to-buffer credit recovery in fibre channel systems using virtual and/or pseudo virtual lanes
US7020133B2 (en) Switch queue predictive protocol (SQPP) based packet switching method
US7724666B1 (en) Credit-based flow control over unreliable links
US20050030893A1 (en) Method and system for detecting congestion and over subscription in a fibre channel network
JPH07321842A (en) Apparatus for interfacing packet switching network to plurality of data terminals,module for interfacing plurality of end points to system that switches frame relay packet,and method for interfacing terminal to system that switches data packet
US7203171B1 (en) Ingress discard in output buffered switching devices
US20070147404A1 (en) Method and apparatus for policing connections using a leaky bucket algorithm with token bucket queuing
US7734808B1 (en) End-to-end congestion control in a Fibre Channel network
US8130652B2 (en) Control of data in communication networks

Legal Events

Date Code Title Description
AS Assignment

Owner name: MAXXAN SYSTEMS, INCORPORATED, CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:JENNE, JOHN E.;OELKE, MARK LYNDON;OLARIG, SOMPONG PAUL;REEL/FRAME:012695/0561;SIGNING DATES FROM 20020214 TO 20020301

AS Assignment

Owner name: CIPHERMAX, INC., TEXAS

Free format text: CHANGE OF NAME;ASSIGNOR:MAXXAN SYSTEMS, INC.;REEL/FRAME:019984/0549

Effective date: 20070117

FPAY Fee payment

Year of fee payment: 4

AS Assignment

Owner name: SAICO INFORMATION TECHNOLOGY (WUHAN) CO., LTD., CH

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:CIPHERMAX, INC.;REEL/FRAME:028592/0887

Effective date: 20120614

REMI Maintenance fee reminder mailed
LAPS Lapse for failure to pay maintenance fees
STCH Information on status: patent discontinuation

Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362

FP Lapsed due to failure to pay maintenance fee

Effective date: 20140801