CA1311819C

CA1311819C - Broadband packet switch with combined queuing

Info

Publication number: CA1311819C
Application number: CA000599575A
Authority: CA
Inventors: James Nicholas Giacopelli; Morgan Littlewood
Original assignee: Bell Communications Research Inc
Current assignee: Iconectiv LLC
Priority date: 1988-09-02
Filing date: 1989-05-12
Publication date: 1992-12-22
Anticipated expiration: 2009-12-22
Also published as: US4893304A

Abstract

Abstract of the Disclosure A novel packet switch architecture is disclosed. The packet switch utilizes internal queuing (i.e. recirculation loops) and output queuing (i.e. multiple paths to each destination) to provide a packet switch which offers superior performance in comparison to a packet switch which utilizes either of these queuing strategies alone. The combination of recirculation and output queues have complimentary effects. The output queuing reduces the number of recirculation loops needed and recirculation reduces the bandwidth requirements for an output buffered switch.

Description

Ficld of thc /nven~ion The present invention relates to an architecture for a high speed and efficient packet switch. More particularly, the invention relates to a packet switch architecture which minimizes packet loss and ma~imizes utilization for a wide variety of 5 traffic conditions by combining packet recirculation with the use of multiple paths to each destination port.

Bacl~ground of thc /n~cntion An important element for providing advanced telecommunications services requiring large bandwidth is a high capacity packet switch capable of 15 interconnecting a plurality of input ports and a plurality of output ports. A packet switch that can connect any specific input port to any specific output port is known as a full access packet switch.
Typically, such a packet switch is synchronous. The packets routed therein are of fi~ed length and are contained in time slots. During a packet switch cycle, 20 packets present at the input ports are routed through an interconnection network comprising part of the packet switch to specific output ports. If the interconnection network is self-routing, each packet arriving at an input port is provided with a header which contains the address of a specific output port to which the packet is to be routed. The interconnection network utilizes this address information to route the packets to the specific output ports.
A packet switch is internally non-blocking if it can deliver all packets from the input ports to the requested output ports when the packets are addressed to distinct output ports. However, there is still the possibility of external blocking, i.e. an internally non-blocking packet switch can still block if there are two simultaneous requests for the same output port. In this case, one or both packets to the same output port will be blocked.
30 Accordingly, it is desirable for a packet switching architecture to be both internally and externally non-blocking.
- One e~cample of a minimally sized interconnection network is a banyan routing network. Even though a banyan network is sufficient for routing packets, routing decisions may cause internal collisions, even for a distinct set of addresses, reducing the l3~Q319 throughput to an unacceptably low level. In short, the banyan network is an internally blocking network. The internal collisions in the banyan network can be eliminated by arranging packets in either ascending or descending order based on destination address before routing through the banyan network. The arrangement of packets in ascending or 5 descending order can be accomplished through use of a Batcher sorting network connected in front of the banyan network. However, the resulting Batcher-banyan network is still externally blocking when two or more packets are simultaneously addressed to the same output.
Various packet switch architectures using Batcher and/or banyan 10 networks have been proposed. These various architectures utilize a variety of techniques to resolve output port conflicts among packets and use a variety of techniques to buffer or queue packets that are not routed as a result of a conflict resolution process. The techniques used impact the size and complexity as well as the overall performance and efficiency of the packet switch architecture.
The performance of an architecture is generally characterized by its packet loss rate and the delay for a given link utilization. Both delay and loss are dependent on congestion due to traffic profiles, the ability of the interconnection network to route to the appropriate destination and the amount of and placement of packet buffers.
Conceptually, zero packet loss can be achieved with an ideal switch 20 design. The ideal switch design requires full interconnectivity from each input to every output and infinitely long queues at each output. Arriving packets can be moved immediately from the inputs to the outputs where they are queued for access to outgoing trunks. In reality, full interconnectivity is expensive and the number of buffers must be finite. All packet switch architectures make design trade offs between the complexity of the 25 interconnection networl~ and the number and location of packet buffers provided.
Alternative buffering locations are at the switch inputs [see e.g. "A
Broadband Packet Switch for Integrated Transport," IEEE J-SAC Vol. SAC-5 No. 8, October 1987, J. Y. Hui and E. Arthurs; and "Reservation-Based Contention Resolution Mechanism for Batcher-Banyan Packet Switches," Electronics Letters Vol. 24 No. 13, June 30 23, 1988, B. Bingham and H. Bussey], at the switch outputs [see e.g. "The Knockout Switch:
A Simple, Modular Architecture for High Performance Packet Switching, Proc. ISS '87, March 1987, Y.S. Yeh, M. G. Hluchyj and A.S. Acampora; and "A Broadband Packet Switch for Integrated Transport," IEEE J-SAC Vol. SAC-5 No. 8, October 1987, J. Y. Hui and E. Arthurs], and internally to the switch [see e.g. "Starlite: A Wideband Digital 35 Switch," Proc. Globecom '84, November 1984, A. Huang and S. Knauer; "Applications of Self-Routing Switches to LATA Fiber Optic Networks," Poc. ISS '87, March 1987, C. Day, J. Giacopelli, and J. Hickey; and "Design of an Integrated Services Packet Network," IEEE

~311~ 9 JSAC, Vol. SAC-4, No. 8, November 1986, J. Turner]. The gosl is to minimizc packet loss and mazimize utilization for a wide range of traffic conditions while minimizing the comple~ity of the switch architectures.
Input buffered switches senice packets on a first-come first-served S basis by storing new arrivals in input queues to await service. This arrangement suffers from head of the queue blocking. Head of the queue blocking occurs since a paclcet at the top of the queue which cannot be transmitted to a particular output, bloclcs other packets within the queue from e~iting even though they may be addressed to idle outputs. A variety of relatively comple~c techniques such as queue depth scarch have been used to solve this 10 problem.
Output queuing generally involves the use of multiple routing paths to each output. Thus, a number of packets addressed to an output may be simultaneously routed thereto depending on the number of routing paths to the output. The packets are then queued at the output ports to obtain access to outgoing trunks. Thus, output queuing is 15 associated with the use of relatively comple~ interconnection networl~s necessary to achieve multiple routing paths to each output.
Internal queuing may be accomplished as follows. A trap network may be located in between a Batcher sorting networlc and a bsnyan routing network at the outputs of the Batcher networlc. The trap networ1~ identiffes packets with repeated output 20 port addresses. The repeats can then be discarded or recirculated baclc to the switch input ports for later transmission. Each recirculation loop typically includes an internal queue for use by recirculating packets. The use of recirculation loops and queues for recirculating pacl~ets solves the head of the queue blocl~ing problem for paclcet switches. However, prior art paclcet switches utilizing internal queuing are provided with a dedicated input at the 2S Batcher sorting networl~ for each recirculation path. Thus, for an interconnection network of given size, this substantially reduces thc number of input ports which can be used for servicing newly arriving paclcets. Another way of stating this is that a fi~ed bandwidth through the networl~ is allocated to recirculating packets.
In view of the above, it is an object of the present invention to provide 30 a pacl~et switch architecture which overcomes the shortcomings of switches which utilize input queuing alone, internal queuing alone, or output queuing alone. More particularly, it is an object of thc invention to provide a packet switch architecture which combines a plurality of queuing approaches to form a more efficient packet switch.
Summary of thc Invcntion 3S In a preferred embodiment, the present invention is a packet switch architecture that is built around a Batcher sorting network and a plurality of banyan routing networks. This architecture combines both internal queuing ti.e. recirculation) and output 131~

queuing (i.e. multiple paths to each output) to form a paclcet switch which offers superior performance in comparison to a packet switch which utilizes either output queuing or internal queuing by itself. The combination of recirculation and output queuing have complimentary effects. The output queuing drastically reduces the number of recirculation S loops, increasing the number of inputs that can serve newly arriving packets. Recirculation, on the other hand, reduces the bandwidth required for an output buffered switch.I~rlcf Dcscrip~lon of thc Drawing ~ IGS. lA and lB (arranged as shown in FIG. 1 and jointly referred to herein as FIG. 1) schematically illustrate a paclcet switch architecture utilizing internal and 10 output queuing, in accordance with an illustrative embodiment of the present invention;
FIG. 2 shows the format of packets routed through the switch of FIG. 1;
FIG. 3 shows a comparator or trap cell forming part of the packet switch of FIG. 1;
FIG. 4 shows an output port controller forming part of the packet switch of FIG. 1;
PIG. S schematically illustrates the architecture of a paclcet switch with prioritized overflow in accordance with an alternative illustrative embodiment of the invention;
FIG. 6 schematically illustrates a packet switch architecture in which inputs are dynamically allocated between newly arriving and recirculating pacl~ets;
FIG. 7 shows the format of pacl~ets utilized in the packet switch of FIG. 6;
FIG. 8 schematically illustrates the architecture of a pacl~et switch 25 utilizing trunk grouping, in accordance with an illustrative embodiment of the present invention, and FIGS. 9, 10 and 11 illustrate the format of paclcets utilized in the switch of FIG. 8.

1 3 ~
s Dctaikd Dcscrip~ion of thc Invcntion A. Packet Switch Architecture FIG. 1 shows an inventive packet switch arehitecture 10 that is built around an NxN Batcher sorting network 12 and a plurality of banyan routing nehvorks 14a, 5 14b. The Batcher network 12 has N input lines designated a1, a2...aM, aM+ 1...aN and N
output lines designated b1, b2...bN. The packet switch 10 also comprises a plurality of input port eontrollers 16-M+ 1, 16-M+ 2..., 16-N-1, 16-N and a plurality of output port eontrollers 18-1...18-N. The input port eontrollers are eonnected to incoming trunks 17-M+ 1, 17-M+ 2,...17-N and the outgoing port coDtrollers are connected to outgoing trunks 10 19-1...19-N.
In FIG. 1, the Ba~cher network inputs are divided into two groups. A
first group of inputs a1...aM receives recirculating packets from the recirculation loops 30.
A second group of inputs aM+1, aM+2...aN reeeives newly arriving packets via the trunks 17-M+ 1...17-N and input port controllers 16-M+ 1...16-N. It should be noted that in the 15 network 10 of FIG. 1, the number of Batcher inputs allocated to receiving newly arriving packets is fixed and the number of Batcher inputs allocated to receiving recirculating packets is fised. Another way of stating this is that a fixed fraetion of the total bandwidth through the switch 10 is alloeated to reeireulating paekets.
In any packet switehing cyele, the paekets present at the Bateher inputs 20 are synehronously transmitted through the Bateher network and sorted, for example, in aseending order aeeording to a destination address found in the paeket header. Thus, during any packet switching cycle, the paekets are sorted aeeording to destination addresses at the Batcher outputs b1, b2...bN. Since the packet switch 10 of FIG. 1 is a synchronous switch, the packets may be viewed as occupying timeslots. The input signal at any Batcher input 25 may be viewed as comprising a sequence of packets that are synchronous with the packets at any other input.
Illustratively, data in Asychronous Time Division Multiplexed format (see e.g. "Asynchronous Time-Division Techniques: An Experimental Packet Networlc Integrating Video Communications," ISS '84, May 1984, A. Thomas, J. P. Coudreuse, and 30 M. Servel) or Dynamie Time Division Multiplexed format (see e.g. "Dynamie TDM - A
Packet Approach to Broadband Networking," Proc. ICC '87, June 1987, L. T. Wu, S. H.
Lee and T. T. Lee) arrives via the trunlcs 17. The input port controllers 16 insert headers obtained from translation tables in front of each transmission paclcet found in the Asynchronous Time Division Multiplexed data or Dynamic Time Division Multiplexed data.
35 It should be noted that recirculating packets have previously been provided with headers by the input port controllers when they were newly arriving packcts.
The format of each resulting paclcet is shown in FIG. 2. Each packet comprises a data field which contains user data and a header. Thc header comprises a routing field and a priority field. Both fields are ordered with their most significant bit S first. The first bit of the routing field senes as an activity bit (A) wherc a logical "1"
represents an active paclcet and a logical "0" represents an idle packet. Idle paclcets appear to have lower addresses than active paclcets. This causes idle paclcets to be grouped together at the lower numbered outputs of the Batcher sorting network 12. The remainder of the bits in the routing field comprise the destination address. For paclcets addressed to the same 10 destination, the Batcher networl~ 12 will continue to sort over the priority field. The priority field contains two sub-fields. The senice priority (SP) sub-field identifies a class of senice to which the packet belongs. The switch priority (PR) sub-field maintains a count of the number of slots a packet has been delayed (through the use of the recirculation loop 30) with an initial value of all logical "l"s. Illustratively, for every slot a packet is delayed, the 15 PR-subfield is decremented to increase the pacl~et's priority relative to other packets within the samc senice class. Thus, the Batcher sorting network 12 of FIG. 1, produces at its outputs b1...bN a sorted list of packets ordered by destination address. Within each common destination, paclcets are ordered by priority of senice.
In the paclcet switch 10 of FIG. 1, two banyan networlcs 14a, 14b are 20 utilized. This means there are two routing paths directcd to each output port controller 18 and two packets can simultaneously be routed to each output port controller 18. The arriving packets are qucued at the output port controllers 18 to await access to the outgoing trunl~s 19.
If there are more than two paclcets present at the Batcher outputs, 25 b1...bN contending for the same output port controller, a conflict resolution process is utilized to determine which paclcets will be permitted to e~it to the output port controllers.
Those packets not "winning" the conflict resolution process are recirculated via the recirculation loops 30 baclc to the Batcher inputs for re-transmission in the ne~ct packet switch cycle.
In the paclcet switch of FIG. 1, contention is resol~ed by identifying the two top packets within a common address bloclc, since these will be the highest priority packets of the group. All other packets in a common address block are recirculated. The trap networlc 32 operates on the sorted output produced by the Batcher networl~ 12 wherein packets with common destination addresses appear on adjacent outputs.
The trap networlc 32 comprises a column of trap cells, 34-3, 34-4...34-N, one for each output b3...bN of the Batcher networlc. The trap cells are address comparators. The trap networl~ outputs are designated cl, c2... CN. As shown in FIG. 3, 1 3 ~

the eomparator (i.e. traF cell) 34-i compares the destination address on Batcher output iine bi with the destination address on Batcher output line bi 2 If the addresses are equal, then the packet on line bi losses the conflict resolution process and is recirculated. If the addresses are different, then the packet on line bi wins the contention resolution since it is S either the highest or second highest priority pacl~et to a particular destination. The pacl~et present on Batcher output line bi emerges from the trap networlc comparator 34-i on the trap networl~ output line ci. A trap bit in the packet is set if the eompared destination addresses are equal and the pacl~et is to be recirculated. The trap bit is not set if the compared destination addresses are not equal and the packet is to e~cit to an output port eontroller via a 10 banyan networlc 14. Note that the top two Batcher output lines bl, b2 are not assoeiated with comparators as these two lines contain two paclcets addressed to two different output port controllers or the two highest priority paekets addressed to the same output port controller. In either event these packets can be routed by one or both of the banyan networks 14. However, delay elements (not shown) are utilized on the lines bl, b2 to 15 insure that packets on lines bl, b2 remain synehronized with packets on lines b3...bN.
For a networlc with 1~ banyan routing networks 14,1~ packets may be routed to a particular output port controller 16 in a packet switching cycle. In this case, the trap network eompares the packet on Batcher output line bi with the packet on Batcher output line bi ~. If the addresses are equal, the paclcet on line bi is recirculated. Otherwise 20 the packet on line i e~cits to the appropriate output port controller.
After the trapping networlc identifies packets to be routed to the output port controllers and paclcets to be recirculated, the two groups of paelcets should be separated so that the paclcets to be recirculated can be steered to the recirculation loops 30 and the packets e~iting to the output port controllers can be steered to the banyan networks 25 14. In addition, to meet the non-blocking criterion for banyan networlcs, the e~iting packets must be repacked into an ascending list by destination address without any gaps between them.
A concentrator network 36 performs this task by regrouping the packets into two separate lists. The packets to be recirculated are put in one list and the 30 e~citing packets are placed in the other list. A routing header constructed by the concentrator networlc 36 steers packets to be recirculated to one edge of the concentrator network (i.e. to the lower numbered outputs) and the exiting packets to the opposite edge (i.e. to the higher numbered outputs). Lists are built starting at opposite edges with each list growing towards the center of the concentrator outputs d1...dN. This proeedure results 35 in two separate sorted lists of packets.

The boundary between paclcets to be recirculated and e~iting paclcets at the outputs of the concentrator 36 is arbitrary. Therefore each concentrator output desirably has aceess to both the recirculation loops 30 and one of the banyan networlcs 14a, 14b. Such aCCe58 i5 provided by the selector networl~ 38 which comprises a eolumn of eells 40-1, 5 40-2...40-N, each having an input d1, d2...dN and two outputs e1a, e1b...eNa, eNb. The outputs e1a, e2a...eMa form part of the recirculation loops 30. The outputs e(M+ 1)a~
e(M+ 2)a eNa handle overflow packets which are lost via the lines 29. Thus any packets to be recirculated and appearing on this latter set of outputs are lost as overflow pacl~ets.
Another way of looking at this is to note that the recirculation bandwidth is finite. If, in 10 any switch cycle, the number of paclcets to be recirculated e~cceeds the recirculation bandwidth, packets are lost. Note that each of the recirculating loop 30 includes a shift register 52 for queuing recirculating packets. The shift registers together form the shared queue 54. The queue 54 is designated a shared queue because a packet from any Batcher input can be queued therein. RecirculatiDg paclcets, stored in the queue 54, enter the 15 Bateher networlc via the inputs al...aM.
The packets present at the selector outputs e1b, e2b...eNb are transmitted to the banyan networlcs 14a, 14b for routing to particular output port controllers. When there are two packets addressed to the same output port controller present at the outputs elb' e2b eNb~ the two pacl~ets are routed using separate banyan 20 networks. To aceomplish this, an inverse perfect shuffle wiring pattern 42 connects the outputs elb, e2b...eNb to the banyan networlcs. This wiring pattern divides the sorted list of packets present at the outputs e1b, e2b...eNb into two separate but sorted lists having a unique set of destination addresses. In accordanee with the inverse perfect shuffle wiring p rn~ outputs e1b~ e3b~ eSb e(N-l)b are connected to the inputs f1, f3, fs-- fN 1 of the 25 banyan 14a and the outputs e2b, e4b...eNb are connected to the inputs g1. g3---gN-1 of the banyan networl~ 14b. The inputs f2- f4...fN of the banyan 14a are set to zero and the inputs g2~ g4~---gN of the banyan 14b are set to zero.
The outputs hl, h2...hN of the banyan networl~ 14a are connected to the inputs P1, P2---PN of the output port controllers 16 and the outputs l1~ 12...1N of the 30 banyan networlc 14b are eonnected to the inputs q1. q2...qN of the output port controllers 18.
Illustratively, as shown in FIG. 4, each output port controller 18-i includes a shift register 20 associated with the input Pi and a shift register 22 associated with the input fi. The shift registers 20, 22 serve as queues for packets arriving via the inputs Pi, 35 qi. The control device 24, enables the queues 20, 22 to access the outgoing trunl~ line 19-i.
Note, in an alternative embodiment of the invention, there may be separate queues in the output port controllers for paekets from different classes of services. Instead of shift 131~ 9 9.

registcrs, RAM devices may be used to queue packets at the output port controllers.
In FIG. 1, the network 12 is shown as a full Batcher networl~.
However, if there are N/2 input port controllers and N/2 recirculation lines, an N~N Batcher networlc i8 not necessary. Since paclcets are already sorted at the trap networlc, a 5 concentrator can maintain the relative positions of the trapped packets producing a sorted list of trapped packets. For this case an (N/2)~N/2 Batcher network is needed to sort the new arrivals. At this point, both the trapped packets and new arrivals are separate but sorted lists that are then merged together. This is accomplished using a merge network which is actually the last stage of an N~cN Batcher network to produce a sorted list of length 10 N.

B. Prioritized Ovcrflow One problem with the pacl~et switch architecture of FIG. 1, is that packets will be lost, when in any switching cyde, there are more packets to be circulated than there are recirculation loops. The sorting and trapping networlcs 12, 32 sort packets 15 based on their destination addresses. This same order is maintained by the selector network 38. When overflow occurs, the highest addressed paclcets, regardless of the priority level, are lost first resulting in a network that favors low addresses.
To correct this problem, packets are preferably given access to the recirculation loops based on priority, not destination address, so that the lowest priority 20 packets are the first to be lost at overload. To accomplish this, the pacl~eh are reordered before the recirculation. A packet switch 110 for carrying out such reordering is shown in FIG. 5. In FIG. 5, the Batcher network 66 replaces the concentrator networlc 36. In addition, the funcdons performed by the trap and selector networlcs 32 and 38 of FIG. 1 are modified in the networks 32' and 38' of FIG. 5. A concentrator network is limited to 25 partitioning a list of packets into two parts. The concentrator network 36 of FIG. 1 partitions paclcets arriving at its inputs into lists of trapped (i.e. non-e~iting packets inciuding recirculating and overflow packets) and e~iting packets. To partition trapped packets based on priority, multiple concentrator networlcs would be required. Such an approach would be both comple~c and e~pensive. Instead, the needed partitioning is carried 30 out using the Batcher network 66.
A Batcher network can be used to partition a list into as many pieces as necessary by constructing a two part header whose first part groups common packets and a second part that orders packets within the groups.

13~$J~9 ~o Thc routing and priority fields of the packets (see FIG. 2) contain the necessary information to accomplish tbe desired partitioning. Thus, packets are compared in the trap network 32' of PIG. S in the same manner as they are compared in tbe trap network 32 of PIG. 1. However, in contrast to the networlc 32 of FIG. 1, trapped pacl~ets leaving the S networlc 32' of FIG. S will have their routing and priority fields interchanged tbus marking them for recirculation. The first bit of the priority field is now defined as a logical zero.
Since an active packet's activity bit (see FIG. 2) is a logical one, trapped pacltets will have lower addresses than esiting packets because of tbe leading zero. Idle packcts are constructed by the input port controllers 17-M+ 1...17-N to havc an address between the 10 valid ranges for trapped and esiting packets. In this case, when the paclcets leaving the trap network 32' are sorted using the Batcher sortiDg network 66, the packets at the outputs of the sorting networlc are arranged so that the trapped paclcets appear at the outputs witb the lowest addresses, followed by the idle packets, followed by the esiting packets. The trapped packets are ordered from highest priority to lowest priority and the esiting paclcets are 15 ordered according to destination address. Thus, the trapped paclcets with lowest priority are most lil~ely to be lost as overflow packets.
The selector cells comprising the selector networlc 38' now mal~e a decision based on the first bit of each arriving pacl~et. All paclcets with leading zeros are directed towards the recirculation loops and all packets with leading ones are directed 20 toward the banyan networks 14a, 14b. At the point where packets overflow (i.e. at the point where there are more paclcets with leading zeros than recirculating loops), the overflow packets will also be directed to the banyan networlcs (rather than to the distinct overflow lincs 29 of FIG. 1). These packets, with leading zeros, will not effect the routing capabilities of tbe banyan networks. However, tbese overflow packets can be monitored by 25 the output port controllers to obtain some idea of the fraction of paclcets being lost. In the event corresponding input port and output port controllers form duples controllers, these overflow packets can be queued and resubmitted to the switch in a later switch cycle.
Before trapped packets enter the recirculation loops 30, the priority and routing fields are swapped again, placing the routing field in front. This header swap is 30 carried out by the selector cells in the selector networlc 28' and ensures that packets re-entering the switch via the inpuh a1...an of the Batcher network 12 have the appropriate header format. In addition, the priority field of each recirculating packet will be decremented to increase its priority relative to paclcets which have not been recirculated or have been recirculated fewer times.

35 C. Dynamic Allocation of Batcher Inputs i31~

In the packet switch architecture of FIG. 1, a fi~ed fraction of the packet switch bandwidth is dedicated to recirculating pacl~ets. This fraction may be represented as M/N whieh M is the number of Bateher inputs dedieated to reeireulating paelcets and N is the total number of Bateher input lines. The remainder of the inputs are S dedieated to newly arriving pael~ets. Illustratively, MIN is about a third so that 33% of available bandwidth is dedicated for recirculating packets.
It should be noted, however, that the capaeity of each Batcher input al...aN is allocated 100% to recirculating or newly arriving paclcets. However, in reality the offered load on each input line dedicated to newly arriving packets is much less than 100%.
10 For e~cample, the offered load on an input line allocated to newly arriving packets may be 50% so that the input line is idle half the time. To take advantage of this e~cess capaeity, Bateher inputs a1...aM may be dynamieally allocated between newly arriving and reeireulating pael~ets. When this is done, it is not neeessary to dedieate a fi~ced fraetion of the switeh bandwidth to reeireulating paekets.
An e~cample of a paelcet switeh with dynamie input alloeation is shown in FIG. 6. The pacl~et switch 80 of FIG. 6 comprises a plurality of input port controllers 16-1...16-N which reeeive newly arriving pacl~ets via the incoming trunlcs 17-1...17-N. The paclcet switch 80 also includes a plurality of output port controllers 18-1...18-N which interface with outgoing trunks 19-1...19-N. A eonfliet resolution and routing networlc 82 is 20 provided. This networlc serves to resolve eonfliets between paelcets addressed to the same output port and to route e~iting pacltets to the output port controllers 18-1...18-N. Pacl~ets which eannot be routed to the output port eontrollers are sent via lines 85 to the reeireulation networl~ 90 for return to the input port eontrollers. Illustratively, the network 82 eomprises a Bateher sorting network, a trap networlc for identifying e~iting and 25 reeireulating paekets, a eoneentrator networl~ for separating the e~iting and reeireulating paelcets into separate lists, and a seleetor network for transmitting reeireulating paekets to the reeireulation network 90 and for transmitting e~iting paekets to one or more banyan routing networ~s for routing to the output port controllers. Such a conflict-resolution and routing arrangement is illustrated in FIG. 1 and eomprises elements 12, 32, 36, 38 14a, 14b 30 of FIG. 1. The reeireulation networl~ 90 eomprises eoDeentrator 92, multipleser 94, and banyan networl~ 96.
The packet switch 80 of FIG. 6 operates as follows. All newly arriving pael~ets at eaeh input port controller are delayed for a period in e~cess of one paelcet slot. This provides time for an input port eontroller to deeide in time slot T if it will 35 be idle during the time slot T+ 2. If an input port controller will be idle during time slot T+2, then it serves as an aceess point for a recirculating packet. In such a case, the input port eontroller submits an active steering packet to the COnCeDtratOr 92 that will be paired 1 3 ~

with a recirculating packet to provide a routing header that returns the recirculating pa~.ket to the input port controller that issued the steering paclcet. The input port controller that receives the recirculating packet, will then resubmit the recirculating paclcet to the network 82 in the T+ 2 time slot.
S The packet formats for both the data packet and the steering paclcets are shown in FIG. 7. The data paclcet 91 in paclcet time slot T contains a destination address field (DA) 97, a data field 98, and an additional field 99 following the data field. When a data paclcet is submitted by an input port controller into the switch, the additional field 99 is in the don't care state as designated by the symbol xn. As shown in FIG. 7, the steering packet 101 is the same size as the data packet 91. However in the steering packet 101, the fields 107 and 108 (corresponding to the fields 97 and 98) are in the don't care state and the field 109 (corresponding to the field 99) contains the address of the input port controller which generated it.
The network 80 dynamically allocates inputs between newly arriving 15 packets and recirculating packets as follows. As indicated above, an input port controller that will bc idle during packct slot T+ 2 generatcs an activc stccring packct during the packet time slot T that includes its address in the source address field 109. An input port controller that will not be idle during the paclcet time slot T+ 2 generates an idle steering packet during the packet slot T as indicated by an activity bit being sct to zcro. Thc acti~e and idle steering packcts arc conccntratcd by thc concentrator network 92. Since the input port controllers 16-1...16-N are connected in order of address to the concentrator 92, thc resulting concentrated list meeh the non-blocking criteria of ~he banyan network 96. The stecring packets gencratcd during packet slot T are timcd so that thcy meet up synchronously with the don't care fields 99 of redrculating packets of packet slot T. Thc multipleser 94 gates the source ficld 109 of each steering packct with the ficld 99 of a corresponding rocirculating data packet of slot T. The source address which is now located in the ffeld 99 of a recirculating packet of slot T serves as a header to route a recirculating data packet of slot T+ 1 (i.e. the immediately following recirculating packet) back to the input port controller indicated by the source address. The input port controller then 30 resubmits the data packets to the switch during the slot T+2 when it would otherwise be idle.
The packet switch architecture 80 of FIG. 2, dynamically allocates recirculating lines by sharing access with underutilized input lines. The amount of bandwidth a~railablc for recirculating packets will fluctuate on a slot-by-slot basis. It should 35 be noted that even if there are more recirculating packets than idle port controllers for a particular slot, these escess recirculating packets will pair up with idle steering packets and emerge from the banyan networl~ 96 at random locations. These packets may then be - l 3 queued at the input port controllers for possible later re-submission to the switch.

D. Trunk Grouping Trunk grouping allows increased bandwidth to be engineered on a per route basis. (see e.g. MultichaDnel Bandwidth Allocation, Canadian Patent Application S No. 584,749 filed for Achille Pattavina on 1 Dec., 1988 and assigned to the assignee hereof). Trunk grouping pools switch rcsources by assigning multiple outgoing trunks to a common group. This group is then treated as a single high bandwidth trunk. A common address (logical address) represents each trunlc group but each individual trunlt in a group requires a routing address (physical address) that uniquely identifies it.
A packet switch 200 utilizing recirculation, multiple routing paths to each output and trunk groups is illustrated in FIG. 8. In the packet switch 200 of FIG. 8, contention is resolved using logical addresses while routing is performed using physical addresses. Translation from logical addresses to physical addresses is carried out using control packets. The control packets are generated using the control packet generator 202 15 which stores the necessary translation information.
The initial paclcet headers are shown in FIG. 9. Each newly arriving data packet is formed by the input port controllers and contains an acti~dty bit (A), a logical address field (LA), a logical address inde~ field (LA INDEX), a fised logic 1, a packet indes field (PKT INDEX), a trap nag (T), a priority field (PR) and a data field. The 20 symbol X indicates a don't care state. Initially for each data paclcet, the input port controllers set the LA INDEX, PKT INDEX and T nag to logic 0~. The activity bit is set to logic 1 for active pacl~ets and zero for idle packets. The LA field coDtains the logical address for the packet. Packets entering the Batcher network 12 from the recirculation loops 30 have the same format as the data packet of FIG. 9.
The format of the control paclcets generated by the control packet generator 202 is also shown in FIG. 9. Each control packet contains an activity bit (A), logical address field (LA), logical address indes field (LA INDEX), a fi~ed logic 0, a trap nag (T), and physical address field (PA).
During a switch cycle, the NsN Batcher network 12 produces a sorted 30 list based on the logical addresses and, within each common logical address, paclcets are ordered by priority. The outputs of the NsN Batcher network 12 are connected to two running sum adder networks 204a, 204b. Each of the networks 204a, 204b is an (N/2) s (N/2) network. The outputs of the Batcher networl~ 12 are connected to the running sum adder networks using the inverse perfect shuffle wiring pattern discussed in connection with 35 FIG. 1. The adder networlcs 204a, 204b performs two operations. First they compute a 131~.Q~ 9 running count of the number of packets having a particular logical address and store this value within each LA INDEX field. Secondly, they compute a running count over all data packets. This value is stored in the PKT inde~ field of each paclcet. The combination of fields A, LA, LA INDEX and fised logic "1" uniquely selects a control packet to be S assigned to a particular data packet. The fi~ced logic "1~ field serves as the least significant bit of the header of a data packet, thus making all data packets appear to have aD odd address. The fi~ed logic "0" field of the control packet makes all control packets appear to have an even address.
The control packets are merged with the data packets using the merge 10 networks 206a, 206b. At the outputs of the merge networks is a combined sorted list comprising data and control packets. The sorting is based on logical addresses. The trap networks 208a, 208b pair control packets to data packeh if there is a control packet on line i that matches the A, LA, LA INDEX fields of the data packet on liDe i+ 1. For paired packets, the physical address (PA) field is copied from the control packet into the data 15 packet with the priority field (PR) being shifted back one position. Data packets which do not pair up with control packets are not altered. After completing the process, the trap networks 208a, 208b rotate the A, LA, and LA INDEX fields to the back of the header.
The headers at the outputs of the trap network 208a, 208b are shown in FIG. 10. More particularly, FIG. 10 shows the header of a non-paired or trapped data paclcet and the 20 header of a paired packet as well as the header of a control packet. For paired packets, the trap flag is set at logic "1N. For unpaired packets, the trap flag is set at logic "0".
The non-paired packets are eventually recirculated back to the inputs of the Batcher network 12 using the recirculation loops 30. Illustratively, a paclcet is not paired in the following circumstances. In a packet switch cycle, the number of control 25 paclcets produced for each logical address is equal to the number of physical addresses corresponding to the particular logical address multiplied by the number of routing paths to each physical address. If, in a par~icular switch cycle, the number of data packets having a particular logical address e~ceeds the number of control packets for a particular logical address (and therefore e~cceeds the routing capacity of the switch to the logical address), the 30 packets will not be paired and will be recirculated.
The reverse banyan networks 210a, 210b separate control packets from data packets. A reverse banyan network is a mirror image of a banyan network e~cept it routes based on least significant bit first. It is non-blocking for a continuous set of ascending or descending addresses though idle packets may be interleaved with active 35 packets. The reversc banyan networks 210a, 210b direct all data packets to the second Batcher networl~ 212 using the fi~ed logical 1 and PKT inde~c fields as a routing header.
During the routing process, the fixed logical 1 and PKT INDEX fields are rotated to the 131~

bael~ of the header producing the format for tr&pped paekets (i.e. paclcets to be recirculated) and paired paelcets shown in FIG. 11. The Bateher 212 COnCeDtrates the list of data paelcets based on priority or physieal address. The seleetor networlc 214 separates the paired and unpaired paelcets. The paired paelcets are routed through the banyan networlcs 14a, 14b to S the output port eontrollers 18 based on physieal address. The unpaired paelcets are modified by the selector network 212 50 that they have the format of newly arriving pael~ets. These paekets are then routed via the recirculation loops 30 bacl~ to the Batcher 12.
Conclusion A packet switch architeeture whieh utilizes both recirculation and 10 output queuing has been disclosed. Finally, the above described embodiments of the invention are intended to be illustrative only. Numerous alternative embodiments may be devised by those slcil1ed in the art without departing from the spirit and scope of the following claims.

Claims

1. A packet switch comprising:
a plurality of inputs and plurality of outputs, a sorting network connected to said inputs for sorting data packets to be routed through said switch, a trap network connected to said sorting network for identifying said data packets as exiting packets or recirculating packets, a plurality of recirculation loops connected to at least some of said inputs, a plurality of distinct routing networks for providing a plurality of routing paths to each of said plurality of outputs, and a selector network for applying said identified recirculating packets to said recirculation loops and said identified exiting packets to said routing networks.

2. The packet switch of claim 1 wherein said sorting network is a Batcher network.

3. A packet switch comprising a plurality of inputs and a plurality of outputs, a sorting network connected to said inputs for sorting data packets to be routed through said switch, a trap network connected to said sorting network for identifying said data packets as exiting jpackets or recirculating packets, a plurality of recirculation loops connected to at least some of said inputs, a plurality of distinct routing networks for providing a plurality of routing paths to each of said plurality of output, and a selector network for applying said identified recirculating packets to said recirculation loops and said identified exiting packets to said routing networks, wherein said switch further includes a concentrator network located between said trap network and said selector network.

4. The packet switch of claim 1 wherein each of said routing networks is a banyan network.

5. A packet switch comprising a plurality of inputs and a plurality of outputs, a sorting network connected to said inputs for sorting data packets to be routed through said switch, a trap network connected to said sorting network for identifying said data packets as exiting packets or recirculating packets, a plurality of recirculation loops connected to at least some of said inputs, a plurality of distinct routing networks for providing a plurality of routing paths to each of said plurality of outputs, and a selector network for applying said identified recirculating packets to said recirculation loops and said identified exciting packets to said routing networks, wherein each of said routing networks is a banyan network, and wherein said banyan networks and said selector network are connected by means of an inverse perfect shuffle wiring pattern.

6. A packet switch comprising a plurality of input and a plurality of outputs, a first sorting network connected to said inputs for sorting data packets to be routed through said switch, a trap network connected to said sorting network for identifying said data packets as exiting packets or recirculating packets, a plurality of recirculation loops connected to at least some of said inputs, a plurality of distinct routing networks for providing a plurality of routing paths to each of said plurality of outputs, and a selector network for applying said identified recirculating packets to said recirculation loops and said identified exiting packets to said routing networks, wherein a second sorting network is connected between said trap network and said selector network for ordering said recirculating packets according to priority.

7. The packet switch of claim 1 wherein said data packets include logical addresses and said switch includes means for generating control packets.8. A packet switch comprising a sorting network connected to said inputs for sorting data packets to routed through said switch, a trap network for receiving said data packets sorted by said sorting network and for identifying said data packets as exiting packets or recirculating packets, a plurality of recirculation loops connected to at least some of said inputs, a plurality of distinct routing networks for providing a plurality of routing paths to each of said plurality of outputs, and a selector network for applying said identified recirculating packets to said recirculation loops and said identified exiting packets to said routing networks, wherein said data packets include logical addresses and said switch includes means for generating control packets, and wherein said trap network identifies data packets as recirculating or exiting packets by comparing said data packets with said control packets.
9. The packet switch of claim 8 wherein the logical addresses of said exiting packets are translated into physical addresses by inserting into each exiting packet a physical address from a corresponding control packet.
10. A packet switch comprising a plurality of inputs for receiving data packets to be routed through said switch a plurality of outputs, means for resolving conflicts among data packets addressed to the same output at the same time by designating the data packets as exiting or recirculating packets, network means for routing said exiting packets to said outputs, said network means permitting at least two packets to be simultaneously routed to each of said outputs, and means for providing a plurality of recirculation loops via which said recirculating packets are routed back to said inputs.
11. A packet switch comprising a plurality of inputs for receiving data packets to be routed through said switch, a plurality of outputs, means for resolving conflict among data packets addressed to the same output at the same time so as to designate the data packets as exiting or recirculating packets, means for providing a plurality of routing paths to each output via which said exiting packets are routed to said outputs, and means for providing a plurality of recirculation loops via which said recirculating packets are routed back to said inputs, wherein said packet switch includes means for ordering said recirculating packets according to priority so that if the number of recirculating packets exceeds the number of recirculating loops only packets of lower priority will be lost.
12. A packet switch comprising a plurality of inputs for receiving data packets to be routed through said switch, a plurality of outputs, means for resolving conflicts among data packets addressed to the same output at the same time so as to designate the data packets as exiting or recirculating packets, means for providing a plurality of routing paths to each output via which said exiting packets are routed to said outputs, and means for providing a plurality of recirculation loops via which said recirculating packets are routed back to said inputs, wherein said data packets contain logical addresses and wherein said switch includes means for generating control packets, said conflict resolving means utilizing said control packets to identify exiting and recirculating packets and to translate said logical addresses to physical addresses.
13. The packet switch of claim 10 wherein said means for providing a plurality of paths comprises a plurality of banyan networks.
14. The packet switch of claim 10 wherein each of said recirculating loops includes a shift register for queuing recirculating packets.
15. A packet switch comprising a plurality of inputs for receiving data packets to be routed through said switch, a plurality of outputs, means for designating the data packets as exiting packet or recirculating packets, a plurality of distinct routing networks via which each of said exiting packets can be routed to a particular one of said outputs, means for queuing said exiting packets at said output, and a plurality of recirculating loops including queuing means via which said recirculating packets are routed to said inputs.
16. A packet switch comprising a plurality of outputs, a plurality of inputs for receiving packets to be routed through said switch to said outputs, means for designating each of said packets as a recirculating packet or an exiting packet recirculation means for routing at least a fraction of said packets designated recirculating packets back to said inputs, and a plurality of distinct routing networks via which each of said exiting packets can be routed to a particular one of said output.
17. The packet switch of claim 16 wherein said recirculation means includes means for queuing said recirculating packets.
18. The packet switch of claim 17 wherein a queue is associated with each of said outputs.
19. A packet switch comprising a plurality of outputs, a plurality of inputs for receiving packets to be routed through said switch to said outputs, means for designating each of said packets as a recirculating packet or an exiting packet, recirculation means for routing at least a fraction of said recirculating packets back to said inputs, and network means for routing said exiting packets to said outputs, said network means enabling at least two exiting packets to be routed simultaneously to the same output.
20. The switch of claim 19 wherein said network means comprises a plurality of distinct routing networks.