WO2007033363A2 - System and method for providing packet connectivity between heterogeneous networks - Google Patents

Abstract

An overlay network of logical gateways are used to store and forward packets from a source terminal to a destination terminal regardless of whether the terminals are mobile, intermittently inactive, inside a private network, or in different access network. The logical gateways maintain inter-network connectivity, even as active terminals move in and out of different access networks.

Description

SYSTEM AND METHOD FOR PROVIDING PACKET CONNECTIVITYBETWEEN HETEROGENEOUS NETWORKS

FIELD OF THE INVENTION

[001] The present invention relates in general to data networking services for mobile terminals, and in particular to providing packet-based data connectivity between different types of mobile terminals connected over heterogeneous access networks.

BACKGROUND OF THE INVENTION

Generally

[002] Currently, data services provided by mobile phone carriers are mostly limited to a client-server paradigm where the client is the mobile terminal (MT) and the server resides on one or more stationary nodes in some Internet Protocol (IP) network, whether inside or external to the carrier's network. However, mobile-to-mobile and peer-to-peer real-time data streaming services such as video-phony, video conferencing and VoIP are now being deployed gradually by mobile phone carriers. All these mobile-to-mobile sessions currently use the Session Initiation Protocol (SIP) standard infrastructure for their setup, teardown and other signaling functions. SIP can be used to connect widely heterogeneous MTs with widely varying display, processing, and networking capabilities together for streaming communication, such as video and voice. However, SIP only deals with signaling, i.e. it basically provides on demand, the mapping between a global SIP name and an (IP address, port) pair for any SIP-enabled end-host. Thus, SIP does not deal with the actual data transfer between the end hosts. When these end-hosts are mobile terminals, such as cell phones, located in potentially different service providers' networks and anywhere in the world, there is currently no efficient solution to set up calls or sessions from one MT to the other where the MTs can possibly have private IP addresses, be mobile, and often disconnect from their provider's network due to RF link disruption. Moreover, SIP calls are not guaranteed to work if the call route crosses Network Address Translation (NAT) routers that translate private IP addresses and ports into public ones.

[003] SIP is further not suitable for many applications characterized by bursty data sessions between MTs. Examples of such communication include:

• interactive distributed online games played by multiple users on their phones, PDAs, etc.,

• mobile web servers running in MTs,

• file downloads from file servers located in an MT,

• interactive shell sessions like SSH,

• short duration and bursty instant messaging, and

• push services, such as push-to-view video and image transfer, and offline messaging.

Comparison to Mobile IP (MIP)

[004] For maintaining connections as a mobile terminal moves from one network to another, currently only traditional MIP technologies following Internet Engineering Task Force (IETF) standards are widely deployed. However, MIP technologies only handle macro-mobility, i.e. slow infrequent movements of the mobile terminal among adjacent networks. They cannot successfully handle faster micro movements of an MT between adjacent networks or base stations. They especially degrade the call quality since MIP's indirect routing scheme via home network seriously degrades call setup time and session QoS for a mobile visiting a foreign network.

[005] The SIP mobility extension of the Internet Multimedia Subsystem (IMS) standard for inter-working heterogeneous wireless networks, has indirect routing problems similar to MIP, which leads to long call setup times. MTM provides a lightweight, scalable, unified alternative or solution to the connectivity and quality problems of both IMS/SIP and MIP. As such, there is a need for a system and method for providing packet connectivity between mobile terminals connected over heterogeneous access networks. SUMMARY OF THE INVENTION

[006] Systems and methods for providing mobile-to-mobile packet connectivity are disclosed and claimed herein. In one embodiment, a system includes a first logical gateway coupled to at least a first access network, and a second logical gateway coupled to at least a second access network, wherein the first and second logical gateways communicate over a third network. The system further includes a source mobile terminal coupled to the first access network and in communication with the first logical gateway, and a destination mobile terminal coupled to the second access network and in communication with the second logical gateway. In one embodiment, the first and second gateways transfer data packets between the source mobile terminal and the destination mobile terminal. In addition, the first and second gateways further exchange state information for the source mobile terminal and destination mobile terminal in order to maintain inter-network connectivity between the source mobile terminal and destination mobile across the first and second access networks.

[007] Other aspects, features, and techniques of the invention will be apparent to one skilled in the relevant art in view of the following detailed description of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

[008] FIG. 1 depicts one embodiment of a system diagram of a communication system, in which one or more aspects of the invention may be implemented;

[009] FIG. 2 depicts a header that may be included in data packets transmitted from a mobile terminal, according to one or more embodiments;

[010] FIGs. 3A - 3B depict exemplary protocol stacks, according to one or more embodiments;

[Oil] FIG. 4 depicts a method of how a mobile terminal may register with a mobile gateway, according to one or more embodiments;

[012] FIG. 5 depicts one embodiment a method of how a mobile terminal may query its local gateway for the address of another mobile terminal;

[013] FIG. 6 how a local gateway may receive an IP packet sent from a local source mobile terminal and forward it to the external IP network;

[014] FIG. 7 depicts one embodiment of how a local gateway may receive an IP packet from the external IP network and forward it to a locally registered destination mobile terminal;

[015] FIG. 8 depicts a timeline of events in the signaling and data transfer phases, in accordance with one embodiment of the invention;

[016] FIG. 9 depicts another embodiment of the system of Fig. 1 in which the inter-network soft and hard handoff capabilities are built-in;

[017] FIG. 10 depicts how a system, configured in accordance with one embodiment of the invention, may handle an inter-network handoff of a mobile terminal from one access network to another;

[018] FIG. 11 is one embodiment of how a proxy mobile terminal may be used to buffer packets for a mobile terminal during an inter-network handoff;

[019] FIG. 12 shows one embodiment of how an inter-network connection may be maintained, despite an address-port pair change; [020] FIG. 13 depicts one embodiment for implementing an invariant- connection-id (ICI) scheme, in accordance with the principles of the invention;

[021] FIG. 14 illustrates one embodiment of an implementation of a Muiti-Base- station Packet Forwarding (MBPF) operation in the downlink direction; and

[022] FIG. 15 shows a Mobile Overlay STreaming NETwork (MOSTNET) overlaid over the Internet, in accordance with the principles of the invention.

DETAILS DESCRIPTION OF THE EXEMPLARY EMBODIMENTS

[023] Disclosed and claimed herein is a MTM (Mobile-to-Mobile) packet transfer system and method that provides universal connectivity between mobile terminals. By universal, it is meant that packet transfers are maintained between MTs, even when they are mobile, geographically arbitrarily separated, in different providers' networks, 'hidden' inside a private network, subject to wireless link disruptions and/or hampered by limited battery power.

[024] Unlike the Session Initiated Protocol (SIP), one aspect of the invention is to provide solutions to both signaling and data transport. Moreover, the signaling scheme according to the invention is more efficient than that of SIP and well suited for quick connection setup, as well as efficient bursty data transfer for bursty-communication applications running between MTs, such as interactive online games, web page and file transfers, UNIX shell sessions, etc.

[025] In accordance with the practices of persons skilled in the art of computer programming, the invention is described below with reference to operations that are performed by a computer system or a like electronic system. Such operations are sometimes referred to as being computer-executed. It will be appreciated that operations that are symbolically represented include the manipulation by a processor, such as a central processing unit, of electrical signals representing data bits and the maintenance of data bits at memory locations, such as in system memory, as well as other processing of signals. The memory locations where data bits are maintained are physical locations that have particular electrical, magnetic, optical, or organic properties corresponding to the data bits. The terms "network node", "sender", and "receiver" are understood to include any electronic device that contains a processor, such as a central processing unit.

[026] When implemented in software, the elements of the invention are essentially the code segments to perform the necessary tasks. The code segments can be stored in a processor readable medium or transmitted by a computer data signal embodied in a carrier wave over a transmission medium or communication link. The "processor readable medium" may include any medium that can store or transfer information. Examples of the processor readable medium include an electronic circuit, a semiconductor memory device, a EOM, a flash memory or other non-volatile memory, a floppy diskette, a CD-ROM, an optical disk, a hard disk, a fiber optic medium, a radio frequency (RP) link, etc. The computer data signal may include any signal that can propagate over a transmission medium such as electronic network channels, optical fibers, air, electromagnetic, RF links, etc. The code segments may be downloaded via computer networks such as the Internet, Intranet, etc.

Definitions

[027] Adport of a packet: Refers to the pair <IP address, port number>, where IP address is carried in the IP header and the port number is carried in the layer-4 (TCP or UDP) header of the packet.

[028] Session-tuple inside a packet: Refers to the pair <source adport, destination adport, protocol> where source adport is the adport carried in the source IP address and source port number fields of the packet and similarly for destination adport. Here protocol is specified in the IP protocol field of the packet (e.g. TCP or UDP).

[029] Original session-tuple of a session: Refers to the session-tuple inside the session-establishing packets, i.e. the first packets that were exchanged by two terminals when they established their application session between themselves. In one embodiment, the session-tuple a given TCP session is the tuple seen in SYN, SYN-ACK, and ACK packets. For UDP sessions, the session-establishing packets may depend on the particular application. For example, for VoIP sessions, they are the first RealTime Streaming Protocol (RTSP) or Real-Time Transport Protocol (RTP) packets in each direction. Moreover, the original session-tuple also uniquely defines a session. That is, during mobility the session-tuple carried inside packets of a session may change, but the original session-tuple of the session is fixed.

[030] LIP Address for an MT: In certain embodiments of the invention, each MT is assumed to have an IP address allocated to it by its network provider. This IP address is referred to as the MT's local IP (LIP) address. When the MT moves across networks or subnets, its LIP address can change.

[031] MTM: Even though MTM is the acronym for Mobile-to-Mobile, an MT need not be a mobile cell phone. The term MT is equally applicable to any host that can connect to a network provider's access network. For example, other than a cellular phone such as a CDMA or GSM or multi-mode phone, an MT can be a WiFi-enabled host, or any host connected to a cellular provider network via a cell phone configured as a modem, or even wired hosts connected to an ISP's network. In addition, the term 'MTM node' will refer to either an MTM-enabled MT or an MTM-enabled MG.

[032] NAT: Eefers to Network Address Translation, which is carried out by border routers at the edge of a private network which translates the private source adport inside a host's outbound packets to a public source adport which is the one to be publicly available. Conversely they translate the public adport inside an inbound packet back to its private source adport.

[033] Public session-tuple of a packet: Refers to the session-tuple inside the packet transmitted from an MT after it has crossed all the NAT routers as well as its publicly-reachable MTM gateway (PR MG) in its path. Moreover, this is the session-tuple seen in the packet by the upstream proxy MG of the MT, as described herein.

MTM Gateways (MGs)

[034] Referring now to the figures, Fig. 1 illustrates a system 100 of different MTs 11Oi - IIO2 located in different access networks 13Oi — 1302 and 140, as well as an overlay network of MGs 12Oi - 1204, in accordance with one embodiment of the invention. As depicted, an MG 12Oi - 1204may be located at any point along the path of MT traffic. In one embodiment, MGs 12Oi - 1204 may be located at the edge of one of the core networks of carriers/providers (e.g., access networks 13Oi - 13O₂ and external IP network 140). More specifically, MGs 12Oi - 12O₄ may be located in the same subnet as PSNs (Packet Service Node) 150i - 1503 of border routers such as PDSN of CDMA2000, ACR of WiBro, AccessPoint Router of WiFi, etc. In certain embodiments, there can be multiple MGs 120 for the same access network 130.

[035] It should be appreciated that there may me more or fewer than two MTs, and that the MTs 11Oi - IIO2 may be comprised of devices capable of transmitting and receiving packet data such as cellular phones, personal digital assistants, laptop computers, desktop computers and embedded computers. MTs 11Oi — IIO2 may be in communication with wireless providers 13Oi - 1302 via wireless or wired links and may be mobile or stationary. Each MT 110 may have an Internet Protocol (IP) address assigned to it by the network with which it is in communication or it may be programmed into the MT. For example, an MT that is a cellular phone may have an IP address assigned to it by one of the access networks 130, where the wireless provider is a cellular data network. This IP address may be termed the Local IP (LIP) address of the MT 110, as will be described in more detail below. The protocol stacks in MTs 11Oi - IIO2 (not shown) contain software to enable MTs 11Oi — IIO2 to insert packets containing a "Mobile to mobile" (MTM) header that may be used by MGs 12Oi - 12O₄ to implement registration, name resolution and packet forwarding services on behalf of MTs HOi - HO2. This software may be termed the "MTM layer" and may be conceptualized in the Open Systems Interconnect model as either a thin layer between the transport layer (a.k.a. layer 4, TCP, UDP) and the network layer (a.k.a. layer 3, IP) or as a modified network layer. It should be appreciated that the protocol stacks in MGs 12Oi - 1204 will have a compatible layer capable of reading the MTM headers in packets and implementing registration, name resolution and packet forwarding services. The compatible layer in the MGs 12Oi - 120₄may also be termed the "MTM layer".

[036] Continuing to refer to Fig. 1, exemplary types of access networks 13Oi - 1302 include Code Division Multiple Access 2000 Ix (CDMA2000 Ix) networks, Code Division Multiple Access 2000 Ix Evolution Data Only (CDMA2000 IxEVDO) networks, General Packet Radio Services (GPRS) networks, Universal Mobile Telecommunications System (UMTS) networks, Universal Terrestrial Radio Access Network (UTRAN) networks, Enhanced Data for GSM Evolution (EDGE) networks, WiFi networks, WiMax networks and WiB ro networks. Applicable wired networks include dial-up networks and local area networks (LANs) such as Ethernet and Token Ring networks. The above lists of applicable networks are exemplary only and it should be appreciated that any network that may be connected to another network through the use of one or more network layer protocols, such as IP, may be used.

[037] External IP network 140 may consist of a single network or multiple interconnected networks. Examples of networks that may make up External IP network 140 include the Internet, LANs, wide area networks (WANs), digital subscriber line (DSL) networks, and cable networks. They may be packet- switched networks or circuit-switched networks. The above list of networks that may make up external IP network 140 is exemplary only and it should be appreciated that any network that may be connected to another network through the use of a network layer protocol, such as the Internet Protocol (IP), may be used.

[038] In general terms, each MT may register with a suitably chosen MG, after which the chosen MG is referred to as the local MG for that given MT. In one embodiment, this occurs immediately after a lower-layer connection is established to the given network, e.g. via PPP in a cdma2000 network. While it may typically be the case that the local MG is located at the edge of the MT's access network, this need not be the case. It should further be appreciated that there are numerous methods by which an MT may discover its local MG, including:

• the provider's PSN node is upgraded to return a suitable local MG's address to the MT. Additionally, the PSN could do the MT registration itself on behalf of the MT,

• the local MG has a network interface with IP address at a fixed known offset from the PSN's addresses. This is especially easy to implement if the MG's network interface is on the same subnet as the PDSN, and

• the MGs have DNS domain names which get resolved to IP addresses using DNS queries, after which one of the MGs is chosen as the local MG for the MT. PRIP Address and PR Node for an MT

[039] A publicly-reachable IP (PRIP) address is an IP address to which IP packets can reach from any source node on the Internet, provided that the destination address fields in those packets are set to the PRIP address.

[040] In certain embodiments, there may be a designated PRIP address for each MT. The MTM node possessing this address is referred to as the PR node for that MT. The PR node may either be the MT itself or its local MG. The following are different embodiments of the choice of the PRIP and PR nodes for an MT:

• The MT's LIP address is public (e.g. typical in CDMA2000 networks) and chosen as its PRIP address. Thus the MT is it own PR node,

• The MT's LIP address is public but some MG is chosen as its PR node. This is advisable if the MT's LIP address changes dynamically (as in CDMA2000 after a reconnection) in which case it cannot be used as a PRIP address, and

• The MT's LIP address is private, in which case it becomes necessary for some MG to be chosen as its PR node.

The MTM Packet Header

[041] In certain embodiments, the MTM's operations are performed at any point above layer-2 and below layer-4 as shown below:

MTM operates in general at any point above layer 2

[042] The set of all MTM information carried in a packet is known as its MTM header. A packet carrying an MTM header is known as an MTM packet In general, the MTM header could be distributed at various points in the packet, inside the layer-2 frame. Each piece of information in the MTM header is called an MTM header field. Various kinds of these fields may be used by the MT and MG to signal to each other to coordinate their mobility related state information depending on what MTM task is being carried out. An MTM packet may contain the application session data or may be a pure signaling packet created by the MTM stack in a node purely to convey MTM signaling. In any MTM packet only a subset of the possible MTM fields may be present. One important MTM header field type is the MT Identification (MTID). The Source MTID and Destination MTID fields uniquely identify the source and destination host of the packet from among all possible Internet hosts. During a TCP/IP session between two MTs, the source or destination MTID need not be carried in every packet, but only with some regularity.

[043] The MTM header may be implemented as an IP option, which will preserve the normal TCP->IP stack order, thereby making it easier to incorporate into existing TCP/IP stack implementations. In one embodiment, this is made possible by defining the MTM header as a new IP options type. While this embodiment may incur some overhead in the packet processing code, it benefits from being less disruptive. The only requirement is that routers along the MTM traffic path be configured to forward IP packets as usual, without dropping them, even if the new option type of MTM is present in their IP headers.

[044] Another embodiment for implementing the MTM headers is as a separate layer 3.5 header. In one embodiment, this may be implemented using a new protocol number indicating MTM is entered in the protocol-type field of the IP header. The MTM header in turn contains the protocol type of the upper layer (such as UDP, TCP, etc.). In one embodiment, IP-in-IP tunneling may be used.

[045] Fig. 2 displays one embodiment of an exemplary "MTM header" 200 that may be generated by a protocol stack in an MT and inserted into packets destined to another device (e.g. another MT or an MG). The fields in header 200 include header length 205, omitted fields 210, header checksum 215, source MTID 220, destination MTID 225, address type bit 230, concatenate bit 235, connection ID 240, signaling/data (S/D) code 245, and protocol type 250. It should be appreciated that depending on the specific embodiment, one or more of the fields in header 200 may be omitted and/or the order of the fields in header 200 may be changed. Also, the size of each field, expressed in bytes or bits, may vary and other fields may be added.

[046] The header length field 205 specifies the length of the MTM header 200 in bits or bytes. It may be used by an MG or an MT to separate the header 200 from the rest of the data. The omitted fields field 210 may specify the fields in the header 200 that have been omitted.

[047] The header checksum field 215 may be used to check the integrity of header 200. This may be similar to the method used in IP to check the integrity of an IP header, however, the exact method used is not important

[048] The source MTID field 220 may identify the source MT that generated of the packet containing the header 200. The destination MTID field 225 may identify the destination MT of the packet containing the header 200. In certain embodiments where the MTID of the source MT is the same as the PRIP of the source MT, the source MTID field 420 may be set to a "reserved MTID" which may be a specific number (e.g. zero). When an MT or MG receives a packet wherein the MTID is set to zero, it may determine that the source MTID is equal to the source PRIP.

[049] The address type bit identifies what type of addressing has been used by the source MT that created header 200. It may specify IP addressing or MTID addressing. IP addressing signals to an MG that receives the header that the packet containing the header should be forwarded to the destination IP field listed in the IP header of the packet. MTID addressing signals to an MG that the PRIP of the destination MT is unknown to the source MT and that the MG needs to determine the PRIP address of the destination MT using the destination MTID.

[050] Session ID field 240 identifies the transport layer session that generated the data contained in the packet. It may be used when data transmitted from MT to MG has to traverse a NAT PSN. [051] Signaling / Data (S/D) field 245 identifies to an MT or MG the type of message that is contained in the packet. For example, the number in the S/D field 245 may signal to an MG that receives the packet that it contains an MRq, NRq, or data.

[052] Protocol type field 250 identifies the transport layer protocol that was involved in generating the data contained in the packet (e.g. TCP, UDP).

[053] FIGs 3A - 3B illustrate two embodiments of the relative position of an MTM header 200 in an IP packet 300 that may be generated by a source MT. In FIG 3 A, MTM header 200 exists between IP header 310 and the transport layer header 315. In FIG 3B, MTM header 200 is located inside options field 325 in IP header 310. It should be appreciated that an IP packet 300 similar to the embodiment of FIG 3B may more easily traverse existing routers than an IP packet 300 similar to the embodiment of FIG 3A.

[054] If an MTM header 200 is inserted into options field 325 in IP header 310, it should be appreciated that it must be sized so as to fit in options field 325. The size of the options field is normally 40 bytes, however 1 byte is used for the option type field and 1 byte is used for option length field so only 38 bytes are available. In addition, if other options are used (e.g. timestamp) the space available for MTM header 200 may be further limited. In one embodiment, if an MTM header 200 is placed in options field 325, the use of other options may be precluded.

[055] Although it is not shown, it should be appreciated that the option type field must be set to indicate that the options field contains an MTM header 200. There are three flags in an option type field to identify the option. The first bit in the field is the copy flag and indicates whether the option should be copied to the headers of any IP fragments that may be generated. In one embodiment, this bit may be set so that the option (MTM header 200) is copied to the headers of any IP fragments that should be generated. The next two bits in the field define the option class flag. These should be set to zero to indicate that the option class is a network control option. The final five bits identify the specific option, according to the option class. It should be appreciated that certain options are already registered and these numbers may not be used to identify the MTM header option.

[056] The option length field is used to specify the length of the option. It is measured in bytes. Therefore, whatever the size of the MTM header is, the length in bytes should be listed in the option length field.

Specific Embodiments of the MTID

[057] The MTID may be a global identifier such as SIP Name, Mobile Station ID number (MSIDN), Network Access Identifier (NAI) such as user@provider.com), or some combination thereof. In one embodiment, the MTID scheme is chosen such that any MT querying some other MT's MTID, just like a DNS query, must be able to obtain it within a fraction of one Round Trip Time (RTT). Also, it may be preferable for the MTID to be short enough or compressible enough to introduce a small percentage overhead in the packet. By way of example, some embodiments of suitable choices for an MTID are:

[058] PRIP Address: The PRIP address may be used as an MTID if the PRIP address is guaranteed to be static during the length of the application session. This is true if, akin to mobile IP, a home PR node is designated for the MT. In this case, the MTID is actually Home_PRIP_Address::LIP_Address. This also holds true where the MT remains registered with the same MG which remains its PR node throughout the application session. With PRIP as the MTID, the source MTID in the MTM header of a packet may become the same as the source IP field in its IP header. In this case, the source MTID field is set to zero which is a reserved MTID. Any MTM node receiving this packet will then determine the source MTID to be the source IP field in the IP header if the source MTID field equals 0. The same holds true for the destination MTID;

[059] Static Public LIP Address of MT: If one of its LIP addresses is guaranteed to remain static during the length of any IP session in which the MT participates, that address can be chosen as its MTID;

[060] <Socket IP Address, Socket Port> combination: The source MTID will be the <source IP address, source port> of the socket which is the endpoint (visible to the application) of the connection to which this packet belongs. A similar statement holds for the destination MTID with 'source' replaced by 'destination'. Such an MTID is valid provided the socket which originally opened the connection does not change its address-port endpoints dynamically, which may be the case for most socket implementations;

[061] Network Access Identifier (NAD: The NAI is used by cellular carriers to uniquely identify users. It is feasible as an MTID if the user remains logged in from a unique MT to a unique provider throughout the application session or if the NAI is a global static identifier, e.g. user@home provider.com;

[062] MSID::NAI combination: Here MSID is the unique ID given to an MT, e.g. the IMSI of ITU E.212 standard, or the Electronic Serial Number (ESN). This is an enhancement of the NAI scheme above to handle the same user possibly logging in from different MTs during an application session;

[063] DNS Domain Name: The DNS may be used as an MTID if the MT-to-DNS name mapping remains fixed and 1-to-l during the application session. It will usually be of the form hostname@provider.com or hostname@some-registered- domain.com:

[064] SIP Name: The SIP Name uniquely identifies the MT as the endpoint of a SIP session, and as such, is ideal for SIP traffic; and

[065] Ethernet MAC address: The MAC address can be used as an MTID provided the overhead can be tolerated and provided it can be scrambled or encrypted in some form to avoid exposing it to packet sniffers.

Simultaneous Use of Multiple Types of MTIDs

[066] When using multiple types of MTIDs simultaneously, the complete MTID may be specified as a pair (type::mtid). In this case, the 'type' identifies the kind of MTID, e.g. Ethernet MAC addressing = 1, MSIDN/CDMA = 2, etc. Alternately, the 'type' field can be a descriptive ASCII string, e.g. "Ethernet MAC address" or "MSIDN/CDMA", etc. MTM Packet Operations at an MG

[067] When an MG is a PR node for an MT, and a packet destined for that MT is to be transmitted out of the MG, the MG will first make sure that i) the destination IP field of the packet's IP header is replaced to the LIP address of the MT, or ii) the source IP field of the packet's IP header is replaced to the PRIP address of the MT.

[068] Moreover, if the MG receives an MTM packet that is an application data packet piggybacking some pure MTM signaling (e.g., registration, MTID name resolution, etc.), the MG may extract the piggybacked packet into a separate packet which is then processed according to the signaling task or the data forwarding rule, as described above. In another embodiment, before forwarding a packet out to a network interface the MG may decide to re-piggyback it onto appropriate packets.

Implementing Pure Signaling Packets

[069] In certain embodiment, each type of pure MTM signaling packet may be implemented as either a UDP packet on a specific port or a TCP packet on a specific port. In the case of a UDP packet, this has advantages of using UDP checksum and fast registration without the TCP connection setup delay. In the case of a TCP packet implementation, TCP may be left to take care of reliable delivery. Although connection setup overheard will be one RTT, this can be alleviated by piggybacking the signaling packets on to the TCP SYN and SYN- ACK packets in both directions to/from the MT.

Demuxing MTM Pure Signaling and Data Packets

[070] At any MTM node (either MT or MGs), the first action on receiving a packet may be to check if it is a pure signaling packet or if it contains application data. The node will accordingly invoke the corresponding signaling and data tasks which as described in the following sections.

Signaling: MT Registration

[071] When MTM functionality is first turned on at MT, or when MT's LIP address changes, or when MT is its own PR node and its PRIP address changes, in one embodiment the MT will first sends IP packet(s) containing the MTM Registration Request (MRq) message which specifies an association tuple [MTID, PRIP address, LIP address] to its local MG. Note that the PRIP field in the MRq will be null if the MT is not its own PR node.

[072] On receiving the MRq packet, the local MG may then reply with a MTM Registration Reply (MRp) message using IP packets to the MT. Normally, the MRp message should fit into a single IP packet, although it may span a plurality of IP packets. In one embodiment, the MRp message contains an error code which indicates either successful registration or the reason for registration failure. On receiving an IP packet containing MRp, the MT may then extract the local MG's source IP address from the packet's IP header and store it locally if not already done.

[073] After sending a MRq, an MT may then wait for the MRp using a timeout- and-retransmit scheme. The timeout-and-retransmit cycle may be repeated a predetermined number of times before giving up and declaring 'MTM unavailable' to the user or application.

[074] It should further be noted that the MGs may be able to disseminate the registration information among themselves using some appropriately efficient dissemination protocol, which is beyond the scope of this invention. This dissemination may, in turn, improve responsiveness of the query transaction protocol in the Name Resolution phase described below.

[075] Referring now to Fig. 4, depicted is one embodiment of a process 400 for registering an MT with its local MG. Registration process 400 may occtxr when the MTM module in the MT is turned on, when the MT's LIP address changes, or when the MT is its own PR node and its PRIP address changes. It should be appreciated that the above list is exemplary only as registration process 400 may occur any time an MT needs to connect or reconnect to a local MG or when an MT changes local MGs. Process 400 begins when the MT transmits an MTM packet containing an MTM Registration Request (MRq) to a default MTM gateway, as shown in block 405. It should be appreciated that the S/D field (e.g. SfD field 245) of the MTM header (e.g. MTM header 200) in the packet (e.g. IP packet 300) containing the MRq should be set to indicate that the packet contains an MEq. The MRq request may specify the MTlD of the MT, the LIP address of the MT, and the PRIP address of the MT, although in cases where the MT is not its own PR node, the PRIP field in the MRq may be null since the external IP address of the local MG is the PRIP of the MT. It should be appreciated that other data may be included in the MRq and that an MRq may span more than one packet. It should be further appreciated that if the MTID is generated by the local MG or by another MG, then the MTID may not be specified in the MRq, in which case information necessary to generate the MTID may be carried in the MRq.

[076] In certain embodiments, the MT should be registered with the access network before process 400 occurs. However, in other embodiments, process 400 may occur simultaneously with the registration of the MT to the access network. The MT may be considered to be registered with the access network once it has been authenticated and assigned an LIP address (if applicable). For example, if the MT is a CDMA2000 Ix compatible device, then ifc may be considered registered with a CDMA2000 Ix AN when a PPP session has been established between the MT and a PDSN in the CDMA2000 Ix AN and the PDSN has assigned an LIP address to the MT.

[077] The default MTM gateway may be the local MG or another device, such as a PSN. In some embodiments the default MTM gateway may be a PSN containing the local MG. If the default MTM gateway is not the local MG, it may be configured to forward the MRq to the local MG. In certain embodiments, the internal interface of the local MG may be on the same subnet as an interface of the PSN and the IP address of the MG's internal interface may be offset from the IP address of the interface of the PSN by a known amount. In these embodiments, the MT may send the MRq directly to the IP address of the internal interface of the local MG, without having to go through a gateway.

[078] At block 410, the MRq is forwarded from the default gateway to the local MG. It should be understood that if the IP address of the internal interface of the local MG is known to the MT and the MRq was transmitted from the MT to the local MG, then step 410 may be omitted. If, however, the local MG is not known to the MT and/or the MRq was not transmitted to the local MG, then the MRq should be forwarded from the gateway to the local MG.

[079] Once the local MG receives the MRq, it may store the LIP address of the MT, the MTID of the MT, and the PRIP address of the MT (if the PRIP address is specified in the MRq). The local MG may maintain an MTID table containing the PRIP addresses, MTIDs and LIP addresses of the MTs registered with it. In certain embodiments, this MTID table may also contain PRIP addresses and MTIDs of MTs that are registered with other MGs. After storing the data contained in the MRq, the local MG then sends a MTM registration response (MRp) to the MT, as indicated in block 415. The MRp may contain the IP address of the internal interface of the MG. If the MTID of the MT is generated by the local MG or another MG, then the MRp may also contain the MTID of the MT. The S/D field in the MTM header in the MRp may indicate whether registration was successful or not. The IP address of the internal interface of the MG may be contained in the source IP field of the IP header of the MRp or it may be in the data in the MRp.

[080] Although it is not shown, in embodiments where the MTID is generated by the local MG or another MG, the local MG or another MG may generate the MTID after receiving the MRq but before sending the MRp to the MT.

[081] At block 420, the MT receives the MRp from the local MG and extracts and stores the IP address of the internal interface of the local MG. Although not shown in FIG. 4, it should be appreciated that if the S/D field in the MTM header in the MRp indicates that registration was not successful, or if the MT does not receive an MRp, then it may use a timeout-and-retransmission scheme wherein the MT sends multiple registration requests to the local MG for a predefined number of times or for a predefined period of time.

[082] In one embodiment, if the registration of the MT was successful, then process 400 may include block 425 where the MTID and PRIP address of the MT is disseminated to one or more other MGs in the MG overlay network, which may then update their MTID databases. Several dissemination protocols may be used, such as the Border Gateway Protocol (BGP), Open Shortest Path First (OSPF) protocol. The operation of these protocols is left outside the scope of the application.

Signaling: MTID Resolution

[083] In this phase, a client (querying) MT requests the overlay MG network to resolve the MTID of some MT to its PRIP address. Each resolution is usually initiated by an application before the start of a data transfer session, e.g. a browser asking to resolve a website's address. In case the MTID is the DNS name, it may be resolved to the PRIP address the usual way by querying a DNS server. Note that in the case where the MTID is the PRIP address itself, there is no need for this name resolution phase.

[084] The client MT requesting the resolution may do so by sending an MTM Name Resolution Request (NRq) signaling message to the local MG. The NRq can be sent separately as a pure signaling packet, or alternatively may be piggybacked onto other data packets. In an NRq packet sent from an MT to its local MG, some MTM header field may be set to indicate that this particular packet carries part of an NRq message. Moreover, the source and destination MTIDs may be that of the querying and queried (the target destination) MT respectively. Similarly, the source and destination address fields in the IP header are the LIP address of the querying MT and the reachable IP address of the local MG respectively.

[085] Upon receiving an NRq message on any interface, the MG may then check if its local cache can answer the resolution query. If not, the MG may relay the NRq message as a NR query transaction request to the network of MGs. The exact NR query transaction protocol, preferably running over TCP, is beyond the scope of this discussion. With that said, many efficient query methods exist and can be used consistently with the invention. Note that an IP packet containing all or part of an NRq message, received at an external interface of a MG, also contains the MTID-to-PRIP mapping of the querying MT itself due to the source MTID and source IP fields in the packet's MTM and IP headers, respectively. This mapping can be cached by any MG visited by the NRq message on its external interface.

[086] In one embodiment, the NRq query reply is a Name Resolution Reply (NRp) message carried over MTM packets. In an NRp MTM packet received at a MG from the query transaction protocol, some MTM header field may be set to indicate that this packet carries part of an NRp message, and the source and destination MTIDs may be that of the queried and querying MT, respectively.

[087] On receiving an NRp packet, the MG may then check if the destination MTID is that of a locally registered MT. If so, it may then set the destination IP field in the packet's IP header to the LIP address (found from table lookup, for example) of the querying MT respectively. The MG may then forward the NRp packet to the querying MT.

[088] The content of the NRp reply is the PRIP address of the queried MT, according to one embodiment. In certain embodiments, it can be carried either in the NRp packet's IP payload or for efficiency in the source IP field of the NRp packet's IP header itself. In the former case, that source IP field would get set to the internal interface address of the MG.

[089] To save on Wireless RTTs, the NRq message can piggyback the following MTM data packets, which are usually the application's session setup requests (e.g. browser's HTTP/TCP session setup request, RTSP/RTP session setup requests, etc.)

[090] Referring now to Fig. 5, displays one embodiment of how an MT (querying MT) may query its local MG for the PRIP address of another MT (queried MT) when the querying MT has the MTID, but not the PRIP address, of the queried MT. It should be appreciated that in certain situations, the PRIP address of the queried MT may be resolved using another process. For example, if the queried MT has a DNS domain name then the querying MT may resolve the DNS domain name to the PRIP address of the queried MT using a DNS query, as is known in the art. [091] Name resolution process 500 begins when a name resolution request (NRq) is transmitted from the querying MT to its local MG, as shown in block 505. It should be appreciated that the S/D field in the MTM header in the NRq message should be set to indicate that it is an NRq message. In addition, the source and destination MTID fields in the MTM header may be set to the MTIDs of the querying and queried MTs, respectively, and the source and destination IP fields in the IP header of the NRq should be set to the LIP address of the querying MT and the IP address of the internal interface of the local MG, respectively.

[092] After the NRq message is transmitted at block 505, name resolution process 500 continues to block 510 where the local MG receives the NRq and determines whether it can map the destination MTID to a PRIP. As previously discussed, the local MG may maintain an MTID table containing the MTIDs and PRIP addresses of MTs registered with it, as well as the MTIDs and PRIP addresses of MTs registered with other MGs. If the MTID database in the local MG contains the MTID and PRIP address of the queried MT, then process 500 proceeds to block 545 where the local MG generates a name resolution reply (NRp) which may contain the PRIP address of the queried MT. In certain embodiments, the PRIP address of the queried MT may be carried in the NRp packet's IP payload or in the source IP field of the NRp packet's IP header. It should be appreciated that the S/D field in the MTM header may be set to a value indicating an NRp message.

[093] If, however, the local MG cannot resolve the MTID of the queried MT to its PRIP address in block 510, then process 500 proceeds to block 515 where the local MG queries the MG network for the PRIP address of the queried MT. The query message may contain all or a portion of the NRq.

[094] As shown in block 520, as the query message is propagated throughout the network, the MTID and PRIP of the querying MT may be stored in the MTID tables of the MGs that receive the query message, even if the MGs do not contain the MTID and PRIP of the queried MT. [095] At block 525, a determination of whether the PRIP and MTID of the queried MT are cached in the MG network is made. In one embodiment, this determination may be made by the local MG which may impose a time limit for the receipt of a response from the MG network. If the time limit is exceeded, then the local MG may assume that the MTID and PRIP address of the queried MT are not cached in the MG network and process 500 proceeds to block 530 where the local MG may send a name failure notification to the querying MT.

[096] If, however, one of the MGs in the MG network contains the MTID and PRIP address of the queried MT, then process 500 may proceed to block 535 where the applicable MG creates an NRp containing the MTID and PRIP address of the queried MT, which is then transmitted from the applicable MG to the local MG in block 540.

[097] Regardless of whether the NRp is generated at the local MG or in another MG in the MG overlay network, process 1000 proceeds to block 550 where the NRp is transmitted from the local MG to the querying MT.

[098] Although it is not shown, it should be appreciated that the MTID and the PRIP of the destination MT, regardless of whether it is was contained in the MT, resolved via a DNS query or resolved via process 500, may now be associated with the transport layer source port assigned to the application that initiated process 500.

Data Transmission at MT

[099] Before transmitting a packet out, the MTM header information may be added to a given packet by the MT. In one embodiment, the source and destination MTIDs are carried in every packet sent out. If the packet destination's PRIP address is unknown, an address type field may be added by the MT as follows:

• IP-addressed packet: The address-type bit in the MTM header is set to 0. In this case, the destination IP address in the TP header is set to the destination MT's PRIP address, or • MTID-addressed packet: The address-type bit in the MTM header is set to 1. In this case, the destination IP address in the IP header is set to a reachable IP address of the local MG, but the packet carries the destination's MTID. The local MG may then take steps to convert the destination MTID to the destination PRIP address on the fly and format and forward the packet as described below.

[100] In certain embodiments, any MTM packet carrying application data waiting to be sent to a destination MT with unknown PRIP address may be formatted as MTID-addressed by the MT. It may further be piggybacked on to MRq signaling packets or NRq packets being currently prepared to be sent to the same destination MT.

Data Transfer at MG: Forwarding and Routing in the external IP cloud

[101] In one embodiment, two kinds of routing methods may be used at an MG based on the addressing type of the received MTM data packet. First, if the address-type bit in the MTM header indicates IP-addressing, the destination address in the IP header of the received packet is used to determine the next hop, according to normal IP routing standards. On the other hand, if the address- type bit in the MTM header indicates MTID-addressing, the MG may first look up a table to resolve the destination MTID in the MTM header to the destination's PRIP address which is then written into the packet's destination IP address field. For alleviating the processing load and delay incurred by this lookup, the recent lookup results can be cached in a fast-access data structure. Then the looked-up PRIP address may be used to forward the packet to the next hop, as in normal IP routing. If, however, no entry is found for the destination MTID in the PRIP address table, the possible actions by the MG may include i) dropping the packet, or ii) storing the packet, and initiating a Name Resolution (using DNS or NRq) transaction with the MG network, after which, on the completion of that transaction and receiving the PRIP address for the packet's destination, that packet may be formatted as an IP-addressed packet and routed in the same fashion as any IP-addressed MTM data packet. [102] Referring to Fig. 6, illustrated is one embodiment of how a local MG may receive an IP packet sent from a local source MT and forward it to the external IP network (e.g. external IP network 140). Forwarding process 600 begins at block 605 when an IP packet from a locally registered source MT is received at the internal interface of the local MG.

[103] Process 600 continues to block 610 where the source IP address in the IP header of the received packet, at this point the LIP address of the source MT, is compared to the PRIP address of the source MT. If the source IP address is the same as the PRIP address, then process 600 proceeds to block 620 (e.g. the source MT is its own PR node). Otherwise, process 600 proceeds to block 615 where the source IP address is set to the PRIP address of the source MT before continuing on to block 620.

[104] At block 620, a determination is made as to whether the received packet is IP addressed or MTID addressed. If the packet is IP addressed (i.e. the address type field in the MTM header is set to a value that indicates IP addressing), then process 600 proceeds to block 655 where the packet is transmitted from the local MG to the external IP network.

[105] If, however, the MTM packet is MTID addressed, then process 600 proceeds to block 625 where the local MG determines whether it contains the destination MT's MTID and PRIP address in its MTID table. If it does, then process 600 proceeds to block 645 where the destination IP address in the packet header, at this point the IP address of the internal interface of the local MG, is replaced with the PRIP address of the destination MT. If the local MG does not contain the destination MT's MTID and PRIP addresses in its MTID table, then process 600 proceeds from block 625 to block 630 where the packet is buffered in the local MG The local MG will then initiate a name query to the MG network in block 635. The name query may be similar to the name query described in process 500.

[106] At block 640_? a determination of whether the name query was successful is made. In one embodiment, this determination may be made by the local MG which may impose a time limit for the receipt of a response from the MG network. If the time limit is exceeded, then the local MG may assume that the MTID and PRIP address of the queried MT are not cached in the MG network and process 1200 proceeds to block 650 where the packet is dropped. Although not shown in the embodiment of Pig. 6, in certain embodiments the local MG may further transmit a failure notification to the source MT.

[107] If, however, the name query was successful and the local MG receives a response from the MG network containing the PRIP address associated with the destination MT, process 600 may proceed from block 640 to block 645 where the destination IP address in the packet header is replaced with the PRIP address of the destination MT.

[108] Once the destination IP address in the IP header has been replaced with the PRIP address of the destination MT in block 645, process 600 may proceed to block 655 where the IP packet is transmitted to the external IP network.

Data Transfer: Last Hop Routing (from local MG to destination MT)

[109] When a MG receives a MTM data packet from an external interface, and if the destination MTID in the received packet is currently registered locally, in one embodiment the MG will replace the destination IP field in the packet's IP header with the LIP address of the MT, obtained from a registration-table lookup, and then route the packet appropriately via the correct internal interface.

[110] However, if an MTM data packet from an external interface has a destination MTID not registered locally, the MG may queue the packet for T seconds, where T may be dynamically determined. Later, when the MT gets reregistered or its new PRIP address gets notified by another MG (see mobility extension below), all pending packets destined for the MT may be queued and sent to the MT, or to its new PRIP address as the case may be.

[Ill] To that end, Fig. 7 depicts one embodiment of how a local MG may receive an IP packet from the external IP network and forward it to a locally registered destination MT. Process 700 may be used when the local MG is the PR node for the locally registered destination MT and the destination IP address in the IP header of the packet is addressed to the external interface of the MG. It should be appreciated that if the destination MT is its own PK node and the destination IP address in the IP header of the packet is addressed to the LIP address of the destination MT then standard IP routing protocols may be used to deliver the IP packet to the destination MT.

[112] Process 700 begins when the local MG receives an IP packet addressed to the IP address of its external interface, as shown in block 705. After receiving the packet, process 700 proceeds to block 710 where a determination of whether the destination MT is currently registered with the local MG may be made. This determination may be preformed by extracting the destination MTID from the MTM header of the packet and comparing it to the MTIDs of the MTs currently registered with the MG. If a match is found, the destination MT is currently registered with the local MG and process 700 may continue to block 740 where the LIP of the destination MT is inserted into the destination IP field in the IP header.

[113] If, however, a match is not found then the destination MT may have been previously registered but is not currently registered with the local MG and process 700 may proceed to block 715 where the packet is buffered in the local MG and then to block 720 where the local MG waits for a period of time for the destination MT to re-register with the MG network. Although not shown, in another embodiment the local MG may query the MG network to determine whether the destination MT has registered with another MG. This query may be similar to the name query described in process 700.

[114] At block 725, a determination of whether the destination MT has reregistered is made. If the destination MT failed to reregister with the local MG within an allotted time after the local MG received the packet or the name query was unsuccessful, then the packet stored in block 715 is dropped, as shown in block 750, and process 700 ends. However, if the destination MT reregistered with the local MG within the allotted time or the name query was successful, then process 700 proceeds to block 730 where a determination is made as to whether the destination MT is registered with the local MG. [115] If the destination MT is registered with the local MG, then process 700 may proceed to 740 where the LIP of the destination MT is inserted into the destination IP field in the IP header and then to block 745 where the packet is transmitted from the MG to the destination MT.

[116] If the destination MT is not registered with the local MG, but is registered with another MG in the MG network, then process 700 may proceed to block 735 where the packet (and any other buffered packets) is transmitted from the MG to the new PRIP address of the destination MT. It should be appreciated that the destination IP field in the IP header should be set to the new PRIP address of the destination MT. The MG to which the MT is currently registered will then use process 700 to forward the packet to the destination MT.

Timeline of Signaling and Data Transfer Events in MTM

[117] Fig. 8 shows how the typical timeline of events in the signaling and data transfer phases. As depicted, MTl 805 is in communication with its local MGl 810. Local MGl 810, in turn, is in communication with the local MG2 815 of MT2 820.

[118] The signaling timeline 800 of Fig. 8 begins with the registration phase (WRTTl) between MT 805 and local MG 810. Thereafter, the name resolution phase (WRTT2) for obtaining the MTID of MT2 820 is shown as a simple query- reply exchange between MTl 805, MGl 810 and MG2 815. In general MGl 810, may use a query protocol to get the fastest possible reply from any of the remote MGs possessing a name-to-MTID map of MT2 820. The data transfer phase (WRTT3) between MTl 805 and MT2 820 may then take place. Note that pipelining the signaling and data transfers by piggybacking NRq and Data packets on to MRq packets will eliminate overhead time of WRTT2 + WRTT3 for the initial data in an application session to reach from MTl 805 to MT2 820. Also, delay-sensitive applications such as VoIP with a high header-to-data overhead ratio in the packets will benefit from compressed piggybacking. MTM Inter-Network Handoff

[119] Eeferring to Fig. 9, depicted is another embodiment of the system of Fig. 1 in which the inter-network soft (make-before-break) and hard (make-after-break) handoff capabilities are built-in. In the system 900 of Fig. 9, an MT 905 registers with a PR MG 935 (which is also its upstream proxy) and established a downstream route 960 and an upstream route 965 therewith via access network 910i.

[120] When the MT 905 moves to from it current access network 91Oi to a new network 9102 and acquires a new LIP address there, the MT 905 may re-register with a new PR MG 940, which in this embodiment is different from the MT's 905 previous PR MG 935. Thus, MT may acquires a new PRIP address as well. For purposes of Fig. 9, it is assumed that the moving MT is running an application session s with a remote Correspondent Terminal (CT) 915, which does not implement MTM. Moreover, IP network 920 is interconnected with each of access networks 91Oi - 9103 via an overlay network of MGs 925, 930, 935 and 940, as shown.

[121] The Handoff Procedure - In one embodiment, the MT 905 selects its PR MG 940 itself and its new upstream proxy MG 930. MT's LIP and PRIP address changes may be notified to and detected by the upstream proxy MG 930.

[122] Downstream Rerouting: The upstream proxy MG 930 must now signal the other MGs (e.g., MG 925) to deflect the previous downstream route of packets 960 from CT 915 to the new PR MG 940. In one embodiment, the upstream proxy 930 only signals (e.g., with the Session Update message described herein) the previous PR MG 935 and the first MG encountered in the downstream route of s (i.e. MG2 925). The packets from CT 915 of s that were already in transit to the previous PR MG 935 will now be re-directed by it to the new PR MG 940 as shown by lines 955.

[123] Early Downstream Rerouting: In one embodiment, the newer packets from the CT 915 that reach MG2 925 after MG2 925 has received the Session Update message will be redirected from MG2 925 to the new PR MG 940 without having to travel all the way up to the previous PR MG 935. [124] Upstream Rerouting: The packets of s traveling from MT 905 towards CT 915 may first cross the PR MG 940, and then reach the upstream proxy 930, which in turn swaps their session-tuple (source and destination adports) to point to the original values carried in the first packets of the session. This ensures that CT 915 will continue to recognize the packets delivered to it as valid packets of s during and after the handoff of MT 905. Thus, upstream packets 945i between the new PR MG 940 and the upstream proxy 930 carry the new public session-tuple, while the upstream packets 9452 between the upstream proxy 930 and the other MG2 925 will carry the original session-tuple for session s.

[125] Overlay Re-routing: In general, the MGs will forward data and signal packets to each other to create an overlay routing that doesn't interfere with the routing of the underlying Internet. Specific criteria for the optimal overlay route include shortest length path, shortest delay path, highest bandwidth path, etc. in the overlay MG network. In such an optimal-routing embodiment, the downstream rerouting described above will not necessarily be carried out early on at MG2 925. Instead the new optimal route may force the selection some other MG, say located somewhere between. Moreover, a mobility signaling 950 between the various MGs 925, 930, 935 and 940 facilitates the handoff process.

[126] In terms of scalability of MGs, in one embodiment the CT 915 may performed the re-routing instead of the MGs, whenever the CT 915 is MTM- enabled. Note how MTM's re-routing is different from the indirect routing via the home agent in Mobile IP, since MTM does not designate a fixed home MG for an MT.

[127] In one embodiment, the handoff is anticipatory or predictive, i.e. the downstream packets are multicast by the downstream re-routing MG (e.g., MG2 925) in advance to both the old PRIP address (e.g., MGl 935) and a set of predicted new PRIP addresses (e.g., PR MG 940) of the moving MT 905. This predicted set of new PRIP addresses is some subset of the MGs known to be located in the neighborhood of the MT's pre-handoff network (e.g., access network 910i). There are many ways to select this predicted set. For example, the prediction for the new network may be based on its geographical location tracking, or by simply selecting all the MGs serving all networks in the neighborhood of the MT's pre-handoff network. The task of notifying the MGs that they have been selected as a predicted new PRIP address can be performed by special "predicted_new signaling message" from either the old local MG (e.g., MGl 935) or the re-routing MG (e.g., PR MG 940) for the MT 905. These signaling messages can be conveyed as part of the mobility signaling 950.

[128] Referring now to Fig. 10, depicted is one embodiment of a process 1000 for how an MTM system, in accordance with the invention, may handle an internetwork handoff of an MT from one access network to another. Process 1000 begins at block 1005 when the MT loses connection with its local MG. In one embodiment, the MG may detect the lost connection, e.g. by being unable to deliver packets to the MT. In other embodiments, the MG may be informed of the lost connection by a component of the AN. For example, in a CDMA2000 network, the PSN may detect the termination of the PPP connection between it and the MT and then inform the local MG. Regardless of how the lost connection is detected, process 1000 continues to block 1010 where the local MG transmits the "predicted_new signaling message" to one or more MGs, hereinafter referred to as predicted MGs. The predicted new signaling message informs the predicted MGs that they may be selected as the new local MG when the MT reconnects to the MG network and that they should accept any packets for the MT that are transmitted to them. The local MG may select predicted MGs based on several factors, including their geographic proximity to the local MG and the access networks with which the MT is capable of communication (if known).

[129] In addition, a re-routing node may be selected for each CT in one embodiment. This re-routing node may be the local MG for the CT (if applicable) or another MG in the connection path between the CT and the local MG.

[130] After the predicted MGs are selected, packets destined for the MT and received by the local MG are then multicast from the local MG to the predicted MGs, as shown in block 1015. If a re-routing node was selected, then the packets may be multicast from the re-routing node to the local MG and the predicted MGs. Regardless of how the packets are multicast, they are then buffered in the predicted MGs, as shown in block 1020.

[131] In one embodiment, the packets are multicast to and buffered in the predicted MGs until a new local MG is found or until a time limit is reached, as shown in blocks 1025 and 1030, respectively. If a new local MG is found, i.e. the MT reconnects to the MG network, then process 1000 continues to block 1040 where the buffered packets are dropped in the predicted MGs that are not the new local MG. It should be appreciated that the new local MG may be the old local MG. Although not shown in process 1000, it should be further appreciated that the new local MG may not be any of the predicted MGs. In such a case, the buffered packets may be forwarded to the local MG from a suitably chosen predicted MG or the old local MG. The particular MG chosen to transmit the buffered packets to the new local MG is not important, and various methods may be used to select it.

[132] Regardless of whether the new local MG is one of the predicted MGs or not, process 1000 may then proceed to block 1045 where the buffered packets are transmitted from the new local MG to the MT. If, however, a new local MG is not found and the time limit is reached, then process 1000 proceeds to block 1035 where the packets in all of the predicted MGs and the old local MG may be dropped.

[133] The advance multicast packets reaching a predicted new PRIP address for the MT at an MG may be buffered in that MG until one of the following two events happens:

• time duration of TJb passes since this MG was notified via the predicted_new message. In this case, the packets in the buffer may be dropped; or

• the MT has now connected to its new network and registered this MG as it PR MG. In that case, this MG itself is the actual new PR MG for the MT. Hence, it performs the usual last-hop PR MG processing on the packet headers and forwards the buffered packets to the MT. Otherwise, this MG will be notified that that it is not the new PR MG for the MT or lies in its new downstream session path. In that case, the MG may drop the packets in its buffer.

[134] The total inter-network handoff duration and the time duration of buffering advance packets at the predicted MGs can be further reduced by connecting the MT to its anticipated new network in advance even before it has physically moved into that network domain. This advance connection can be done by a proxy MT node suitably located in the new network or located inside one of the selected predicted new local MGs for the MT. The proxy MT is sent all the information needed for it to connect as if it is the real MT to the new network. Thus, the MT can be authorized and authenticated to use the new network and get a new local IP address from the network's address allocation server such as DHCP router. Later, when the real MT physically (at the MAC layer) attaches to the new network, the entire state information, including the new local IP address, describing its IP-layer connection to the new network is transferred to it from the proxy MT. In one embodiment, this may reduce the total handoff duration can get reduced to nearly zero.

[135] Fig. 11 illustrates one embodiment of how a proxy MT may be used to buffer packets for an MT during an inter-network handoff. Process 1100 begins when the MT loses its connection with its local MG. This may be detected as described in process 1000. Process 1100 continues to block 1110 where the local MG transmits the predicted new signaling message to predicted MGs, which may be selected as described in process 1000.

[136] Process 1100 continues to block 1115 where one of the predicted MGs is configured as a proxy MT for the lost MT. The information necessary to configure the predicted MG as a proxy MT may be transmitted to the predicted MG from the local MG. For example, the local MG may transmit to the predicted MG the MTID, last LIP address, last PRIP address and various AN account information associated with the lost MT. This list is exemplary only, as the information needed to setup the selected predicted MG as a proxy MT may vary from AN to AN. [137] Once the predicted MG has the necessary information to be configured as the proxy MT, it may then be authorized and authenticated with the AN to which it is connected as if it were the real MT. For example, it may request a new LIP address from the network's address allocation server. In one embodiment, the proxy MT may then transmit its PEIP address to the CTs with which the real MT had been communicating, while in another embodiment, it may not.

[138] After the proxy MT is configured and connected to the new AN, process 1100 may continue to block 1120 where packets for the real MT received by the old local MG, or possibly a re-routing node as described in process 1000, are transmitted to and buffered in the proxy MT. It should be appreciated that the packets may also be multicast to and buffered in the predicted MGs that are not the proxy MT, as described in process 1000.

[139] Similar to process 1000, the packets may be multicast to and buffered in the proxy MT and possibly the other predicted MGs until a new local MG is found or until a time limit is reached, as shown in blocks 1130 and 1135, respectively. If a new local MG is found, i.e. the MT reconnects to the MG network, then process 1100 proceeds to block 1150 where the buffered packets are dropped in the predicted MGs that are not the new local MG and the buffered packets are transmitted from the proxy MT to the real MT. It should be appreciated that in situations where the MG that is the proxy MT is not the new local MG, the packets may be transmitted to the new local MG which then transmits them to the real MT. In addition to transmitting the buffered packets to the real MT, the connection information (e.g. LIP address) that was determined in block 1115 may also be transmitted from the proxy MT to the real MT. This connection information may replace any connection information negotiated between the real MT and the AN when the real MT connected to the AN.

[140] In situations where the real MT connects to an AN that is not the AN the proxy MT is connected to, it should be understood that the proxy MT, or one of the other predicted MGs, may transmit only the buffered packets to the real MT, In these situations, it may not be necessary for the proxy MT to transmit the connection information because the connection information may be applicable only to the AN to which the proxy MT is connected, and not the new AN to which the real MT is connected.

[141] Similar to process 1000, if a new local MG is not determined (i.e. the real MT does not reconnect to an AN) within a certain time limit, then process 1100 may proceed to block 1145 where the buffered packets and any connection information in the proxy MT are discarded. In addition, the proxy MT is deselected as the proxy MT. It should be appreciated that if packets were buffered in other predicted MGs, they may be discarded as well.

MTM Extensions for MG operation in front of a NAT PSN

[142] For the following discussion, it is assumed that the MT is allocated private addressing by an NAT PSN. Hence, the MG is the PRIP node for the MT, meaning that it replaces the source IP address inside network-outbound packets received from the PSN to the PRIP address (i.e. its own external interface address).

[143] Fig. 12, for example, shows one embodiment of how a connection may be maintained, despite an address-port pair change. In particular, the PSN of the provider's network performs typical NAT functions for security, it will usually translate the private address and UDP or TCP connection port number in each packet from an MT to a public_address::public_port pair which this disclosure refers to as an adport pair. A unique adport pair is allocated by the PSN for each connection from each MT. The problem is that when an MT (e.g., MT 1205) gets disconnected and then reconnected, or otherwise moves from one PSN 1210 a new PSN 1215, while still having its old application connections open, the new PSN 125 may allocate a new adport pair to each of those old connections.

[144] However, the MT 1220 on the other end of such a connection will not recognize this new adport pair and may drop the packets and the connection. For example, when TCP connection 1225 was initiated, it was assigned an adport pair al:pl by PSN 1210. Thereafter, as the MT 1205 moves its connection path to connection 1230, PSN 1215 allocates to the MT 1205 a new adport pair, denoted as a3:p3. When MT 1220 receives packets from connection 1230 with. the new source port p3, it won't recognize it as belonging to the connection with MT 1205 and will either drop the packets, or even more dangerously may mistake them as belonging to another connection.

[145] Thus, one aspect of the invention is to avoid the aforementioned scenario by causing the old connections to correctly function after an adport pair change by a PSN's NAT, using what will be referred to as an invariant-connection-id (ICI) scheme. The ICI scheme translates the public source port carried in the connection's packets back to its original public source port so that the packets will be properly identified. For example, in one embodiment MG 1235 translates port p3 back to pi inside every packet of connection 1230 before forwarding the packet to MT 1220.

[146] In one embodiment, MT 1205 may insert dummy MTM packets in the packet stream of each of its open connections, to signal the conn__id which is an integer uniquely identifying that connection at the MT. Each such dummy packet for a connection carries the same destination and source address-port combinations in its layer 4 (UDP, TCP, etc.) header as that of MTM data packet of that connection, except that 1) the S/D code may be set to indicate that this is a conn_id type signaling packet and 2) its layer 4 payload contains the actual connection id.

[147] In certain embodiments of the ICI scheme, the connection id may be chosen either at the MT 1205 as the local (private) source port of the connection itself, or as a hash of the local address-source-port pair for the connection.

[148] For resilience to losses, these dummy packets may be inserted with a suitable regularity within the connection's packet stream. In one embodiment, the connection id may be signaled as early as possible, such as before any true packet of the connection after its opening or after re-registration of the MT 1205. This will minimize the 'open-loop' phase where the public port change for a connection has not yet been registered by the MG thereby forcing packet drops at the remote MT 1220. [149] The MG 1235 may maintain a local conn_id table for each MT. In one embodiment, Rach entry is a tuple (conn_id::orig_port::new_port). Upon receiving a conn_id type signaling packet from a locally registered MT (e.g., MT 1205), the MG 1235 may search its conn_id table for that MT. If a table entry is found whose conn_id field matches the connection id carried in the packet, the new_port field of that table entry is replaced by the source port inside the packet, assuming that the new_port and that source port do not match. The source port inside the packet may be replaced by the orig_port field of that table entry. But if no table entry matching the connection id in the packet is found, the MG 1235 may create a new entry whose conn_id field is set to that carried in the packet, and orig_port and new_port fields are set to the source port inside the packet. After finishing the above table update, the MG 1235 drops the conn_id packet, according to one embodiment.

[150] Upon receiving an MTM data packet from a locally registered MT, the MG 1235 may search its conn_id table for that MT. If a table entry is found whose new_port field matches the source port inside the packet, that source port in the packet may be replaced by the orig_port field of the table entry. Upon receiving an MTM data packet destined to a locally registered MT, the MG may search its conn_id table. If a table entry is found whose orig_port field matches the destination port inside the packet, then that destination port in the packet may be replaced by the new_port field of the table entry. Thus with the ICI scheme, the other end of any connection from an MT will always see a unique address- port pair in spite of any NAT routers changing the adport pairs for its connection.

[151] One embodiment of an ICI scheme implemented in accordance with the invention is depicted in Fig. 13. In particular, Fig. 13 illustrates how a local MG may receive packets from a locally registered MT through a NAT PSN and change the source address and source port to the original source address and source port, when necessary, before transmitting the packets to the external IP network. Process 1300 begins at block 1305 when the local MG receives a packet from a local MT. Process 1300 continues to block 1310 where it is determined whether the packet is a "dummy packet" containing a session ID. This may be determined by looking at the S/D field in the MTM header of the packet. A dummy packet will have the S/D field set to a value that indicates that it is a dummy packet. In one embodiment, the MTM module in the MT may periodically transmit dummy packets interspersed with the data packets during a transport layer session. In a preferred embodiment, the first packet transmitted for a session, outside of any registration packets, is a dummy packet.

[152] A dummy packet may contain in its transport layer payload a session id. The session id may be chosen as the local source port of the session (i.e. the TCP or UDP port assigned to a TCP or UDP session) or some other unique identifier that will not change while the session is open. In one embodiment, the session id may be a hash of the LIP and the source port for the session.

[153] If the packet is a dummy packet, then process 1300 may proceed to block 1315 where it is determined whether the session id in the packet is currently registered in the local MG. The local MG may maintain a table containing the session ids for all sessions currently open in the MT. This table may be referred to as the session_id table. If the session id is not currently registered in the MG, then process 1300 may proceed to block 1330 where a new entry for the session id may be entered into the session_id table. An entry in the session_id table may contain the session id, the original port of the session, and the new port. At this point, since there is no current entry in the session_id table for this particular session, the session id field is set to the session id in the packet and the original port and new port fields are both set to the source port inside the transport layer header in the packet. After the table entry is entered into the session_id table, the packet may be dropped, as shown in block 1335.

[154] If, however, the session id is currently registered in the MG, process 1300 may proceed from block 1315 to block 1320 where the source port in the packet is compared to the new port field associated with the session id in the session__id table. If the new port field matches the source port inside the packet, then it may be assumed that the source port has not changed and the table entry for the session id need not be updated. Process 1300 may then proceed to block 1335 where the packet is dropped. [155] If, however, the new port field associated with the session id in the session_id table does not match the source port in the packet in block 1320, then process 1300 may proceed to block 1325 where the session_id table entry for the session id is updated. At block 1325, the new_port field is replaced by the source port in the packet. Process 1300 may then proceed to block 1335 where the packet is dropped.

[156] Referring back now to block 1310, if the packet transmitted from the MT is not a dummy packet, process 1300 may proceed to block 1340 where the source port in the packet is compared to the new port fields in the session_id table. It should be appreciated that the source port in the packet is compared with to the new port fields in the sessionjd table because the session id may not be carried in actual data packets. If no new port field in the session_id table matches the source port in the packet, process 1300 may proceed to block 1355 where the packet is forwarded from the MG to the external IP network. It should be appreciated that an entry may not be found in embodiments where the MT does not, upon the start of a session or when it reconnects to a PSN, transmit a dummy packet before transmitting data packets. Therefore, a data packet that is forwarded at block 1355 because no matching entry is found in block 1340, may be rejected by the destination MT if the NAT PSN has changed the source port.

[157] If, however, a matching table entry is found in block 1340, process 1300 may proceed to block 1345 where the source port in the packet may be replaced with the orig_port in the table entry associated with the same session id as the new_port in block 1340. In one embodiment, the source port in the packet may always be replaced with the original port, while in other embodiments, it may be replaced only when the source port does not match the orig_port. Regardless of the embodiment, process 1300 may proceed to block 1355 where the packet is then forwarded by the MG to the external IP network.

[158] Another embodiment of the aforementioned ICI scheme may be implemented without the use of dummy packets. In this embodiment, the connection id may be carried in an additional field inside the MTM header itself. In this fashion, the connection id can be carried inside MTM data packets themselves with some suitable regularity. The MG's operations may then be simplified to just simplified somewhat. For example, upon receiving an MTM data packet from a locally registered MT, the MG may search its conn_id table for that MT. A matching table entry for that packet is one whose conn_id field matches the connection id inside the packet if a connection id is present inside the packet's MTM header, or whose new_port field matches the source port inside the packet if no connection id is present inside the packet's MTM header. The source port in the packet may then be replaced by the orig_port field of the matching table entry if found.

[159] Moreover, upon receiving an MTM data packet destined to a locally registered MT, the MG may again search its conn_id table for that MT. A matching table entry for that packet is one whose conn_id field matches the connection id inside the packet if a connection id is present inside the packet's MTM header, or alternative one whose orig_port field matches the source port inside the packet if no connection id is present inside the packet's MTM header. The source port in the packet may then be replaced by the new_port field of the matching table entry if found.

[160] Other than avoiding the overhead of dummy packets, the advantage of the aforementioned modified ICI scheme is that by using the extreme embodiment of carrying the connection id in every MTM data packet, one may be able to always avoid the 'open-loop' phases possible with the dummy-packet ICI scheme described above.

The MTX Scheme for Compressing or Avoiding MTID in the MTM headers

[161] The MTX scheme is to basically hash the MTID into a short MT index, called MTX, during the registration phase. In all future MTM packets from the MT, the MTX will be carried instead of the MTID. Upon receiving the MRq registration packet from the MT, the MG hashes the MTID inside the MRq packet to an MT index called MTX. [162] In one embodiment, the MTX is globally unique. During MT registration, the local MG first queries all other MGs, any of which may reply back with the MTX for the MTID carried in the query. However, if none of the other MGs reply, the local MG may then create the MTX by hashing. A fault-tolerant commit protocol may also be implemented for the MTX query-reply among all MGs to ensure a globally unique MTX for each MT. In one embodiment, the global MTX may simply be the concatenation (MG_id:: local_MT_id) where MG_jd is the unique id number of the MT's local MG among the set of all MGs, and local_MT_id is the MT's unique local id number among all the MTs attached to its local MG.

[163] In another embodiment, the MTX may be MG-specific. That is, the MTX may be unique among MTs registered locally at the MG. But after registration of a local MT, the MG broadcasts the MG_address-MTID-MTX tuple to all other MGs. When the MT re-registers with a new MG, it may send its old MG's PRIP address, its old MTID, and old MTX to the new MG.

IP-layer Multi-AN Packet Forwarding (MAPF)

[164] With MAPF, the MG maintains, for each MT, a list of all access networks (ANs) as well as the MT's AN-specific IP addresses for each AN to which the MT is currently connected. This data is communicated from MT to MG during registration phase. For example, a dual-mode MT may have a WiFi and a WiMax network card used to connect to a WiFi and WiMax access networks, respectively, with the respective IP addresses allocated by access routers in those networks (e.g., a dual-mode MT may have 192.168.11.120 as its private WiFi- specific IP address and 22.231.113.80 as its public WiBro-specific IP address). In certain embodiments, the MG is able to forward to, or receive packets from over, multiple ANs because of the fact that the MT has previously communicated the AN-specific IP addresses during the registration phase.

[165] It is useful to first define a packet flow as a sequence of packets to/from an MT and belonging to the same application instance running on the MT (e.g., packets from an HTTP browsing or FTP session, etc.). The MAPF transmitter refers to the MG for the downlink direction and the MT for the uplink direction. The MAPF receiver refers to the MG for the uplink direction and the MT for the downlink direction. With this in mind, the possible modes for MAPF will now be discussed.

Striping

[166] In this mode of MAPF, the packet flow gets split into multiple sub-flows each traveling over a different AN. In a preferable embodiment, each packet carries a striping-sequence number so that at the other (de-striping) end the packet flow can be reassembled the packet flow in its original sequence from its sub-flows. In specific embodiments of this invention, upper layer protocol's sequence numbers, such as those of TCP and RTP, can be reused as striping- sequence numbers.

[167] Specific embodiments of this invention will employ different algorithms for splitting the packet flow among the ANs. For example, a simple weighted split algorithm may be used. In this case, the flow is split in a fixed proportion among the different ANs. This proportion may be based on the known average bandwidth levels of the wireless links for the different ANs (e.g., for striping over a 900 Kbps WiBro link and 100 Kbps CDMA Ix link, stripe 90% of packets over WiBro and 10% over the CDMA link). Another example of a splitting algorithm is a smart dynamic split algorithm. In this case, the flow is split in a dynamic proportion that is a function of the current estimated wireless link conditions (usually some combination of the bandwidth, loss and error rates) for the different ANs. The link condition estimation itself can be done at link, IP or transport layers.

[168] The MAPF receiver may order the packets of a flow according to their striping-sequence numbers before passing them to the upper layer or the next node. This packet ordering can be done using algorithms similar to those used by TCP. In one embodiment, if the upper layer (such as TCP or RTP) uses sequence numbers, MAPF may omit the striping sequence numbering and let the upper layer handle it. Multicast

[169] With the multicast mode of MAPF, the packet flow is replicated over the different ANs. This is the IP-layer version of the multicast to different base stations that happens in the physical layer soft handoff schemes, such as in CDMA2000.

[170] The MAPF transmitter converts each IP packet into multicast-variants (m-variants), where each m-variant may be forwarded over one AN. An invariant forwarded over AN i, for example, is the same as the original packet, except that 1) it has the MT's IP address field in its header set to the MT's IP address for AN i, and 2) it carries a MAPF id number which is the set to the same for all m-variants of the same original packet. In certain embodiments of the invention, the upper layer protocol's sequence numbers (e.g., TCP headers, RTP headers, IP header's identification field, etc.) can be reused as MAPF id numbers. Note that at both MAPF transmitter and receiver, each m-variant of an original packet can be uniquely identified by the pair (MT's IP address in the IP header:: MAPF id number).

[171] At the MAPF receiver, the set of m-variants of the same original packet get collapsed into M copies of the original packet. At the packet's destination, all these M copies may be passed to the upper layer, which is usually TCP or UDP. Here, M is a parameter which can be dynamically adapted or statically tuned by the engineer/implementer for optimizing the performance. The larger the M parameter, the greater the chance that one of those M packets is uncorrupted and hence not discarded by its destination's upper layer checksumming. However, the larger the M parameter, the higher will be the upper layer's overhead in handling duplicate packets. Note that when the MAPF receiver is the MG, it is M may be set to 1 since it is safe not to burden the packet's destination with duplicate packets.

[172] Specific embodiments of this invention will collapse a packet using different algorithms. For example, a simple collapse algorithm may be used in which the first M received m-variants can be converted into copies of the original packet, and all the rest of the m-variants of that packet can be dropped. Conversion may simply include removing the MAPF id number and replacing the MTs IP address with that of the original packet.

[173] Alternatively, a smart collapse algorithm may also be used. In this case, all m-variants of the packet are received and M of these m-variants that have the M least likelihoods of being corrupted are picked. In one embodiment, the MAPP function itself performs the IP header checksumming and upper-layer's checksumming operation on each m-variant. Then, either M or all of those invariants with correct checksums, whichever is smaller, may be passed to the upper-layer. The packets with incorrect IP header or upper-layer checksum may be discarded. In certain embodiment it may be advisable to set M=I, since the correct checksum usually indicates an uncorrupted packet and there is no need to pass duplicate uncorrupted packets to the upper layer.

[174] Two special cases of MAPF striping or multicast are so-called "soft handoff and "select-cast." With soft handoff, the packet flow is striped or multicast, but only for some restricted time interval, referred to as the sofl- handoff interval, which starts from when the MT was connected to more than one ANs. After the soft-handoff interval, the flow is entirely switched to travel over a single AN. With select-case, the packet flow is also striped or multicast for some restricted time interval. However, in this case, after that interval the MG selects the best possible AN based on certain link quality criteria such as bandwidth, error/loss rate, etc. The packet flow then travels only over this selected AN.

IP-layer Intra-AN Multi-Base-station Packet Forwarding (MBPF)

[175] Another aspect of the invention is to use an MBPF scheme as an IP-layer solution for using multiple base stations to transmit/receive packets between the MT and the Packet Service Node (PSN) at the provider's network edge. It is thus an IP-layer generalization of the physical-layer soft handoff within multiple base stations of the same access network.

[176] The MBPF scheme carries a Base Station Id (BSID) in the IP header. The BSID may be used to indicate which base station the packet is forwarded through. Specific embodiments include carrying the BSID in the IP header's options field, or alternatively in a separate layer 3.5 header.

[177] Similar to MAPF, the MBPF transmitter can be implemented in the MG or PSN for the downlink direction and is the MT for the uplink direction. The MBPF receiver can be implemented in the MG or PSN for the uplink direction and is the MT for the downlink direction.

[178] The MBPF transmitter may set the BSID in each packet just before it goes to the lower layer or next node for transmission to the corresponding base station. In one embodiment, the lower layer or the next node should be capable of mapping the BSID to the correct base station and accordingly routing/transmitting the packet to make sure that it arrives at that base station. The MBPF receiver will remove the BSID from a received packet before passing it on to the rest of the IP layer processing.

[179] Referring now to Fig. 14, illustrates is one embodiment of an implementation of the MBPF operation in the downlink direction. In one embodiment, the possible locations for the MBPF transmitter are MTM gateway 1405 or PSN 1410. Packet 1415 is transmitted from the gateway 1405 to the PSN 1410. Before packet 1415 is transmitted out to base stations 1420i - 14202, a BSID corresponding to one of the base stations 42Oi - 14202 will be encoded into the IP header of the packet 1415. Thereafter, what was packet 1415 may be transmitted as packets 1425i and 14252 to base stations 142Oi and 14202, respectively, wherein packet 1425i is encoded with the BSID corresponding to base station 1420i, and packet 14252 is encoded with the BSID corresponding to base station 14202. Thereafter, MT 1430 may receive both of the packets 1425i and 14252 and know which base station they came from. In one embodiment, MBPF receiver 1435 removes the BSID from received packets before passing them through for processing.

[180] Similar to the aforementioned MAPF scheme, the MPBF scheme also has two forwarding modes — striping and multicast. Striping

[181] In this mode, the packet flow is split into multiple sub-flows each traveling over a different base station. Each packet preferably carries a striping-sequence number so that the packet flow can be reassembled from its sub-flows inside the IP layer at the other end. In certain embodiments, upper layer protocol's sequence numbers, such as those of TCP and KTP, can be reused as striping- sequence numbers. The possible splitting and packet reordering algorithms at the MBPF receiver may be the same as those described above with reference to the MAPF scheme, except that the AN is replaced by a base station.

Multicast

[182] In this mode, the packet flow is replicated to travel through multiple base stations. This is the IP-layer version of the multicast to different base stations of the same access network that happens in the physical layer soft handoff schemes such as in CDMA2000.

[183] The MBPF transmitter converts each IP packet into multicast-variants or (m-variants), where each m-variant is forwarded over one base station. An invariant forwarded over base station i may be the same as the original packet, except that it has the BSID field set to indicate base station i, and it carries an MBPF id number which is the set to the same for all m-variants of the same original packet. In certain embodiments, the upper layer protocol's sequence numbers (e.g., TCP headers, RTP headers, IP header's identification field, etc.) can be reused as the MBPF id numbers. Note that at both the MBPF transmitter and receiver, each m-variant of an original packet can be uniquely identified by the pair (BSID:: MBPF id number).

[184] At the MBPF receiver, the set of m-variants of the same original packet may be collapsed into M copies of the original packet. At the packet's destination, all these M copies may be passed to the upper layer, which is usually TCP or UDP. Here M is a parameter which can be dynamically adapted or statically tuned for optimizing the performance. The larger the M, the greater the chance that one of those M packets is uncorrupted and hence not discarded by its destination's upper layer checksumming. However, the larger M is, the greater the upper layer's overhead in handling duplicate packets will be. Note that for the uplink direction i.e. at the MBPF receiver in the MG or PSN, it may be desirable to set M=I since it is safe not to burden the packet's destination with duplicate packets.

[185] The MBPF multicast receiver may collapse a packet using a variety of different algorithms. One such algorithm is a simple collapse algorithm in which the first M received m-variants is converted into copies of the original packet, and all of the rest of the m-variants of that packet are dropped. Conversion simply consists of removing the MBPF id and BSID numbers from the IP header. Another possible algorithm is a smart collapse algorithm in which all m-variants of the packet are received, and then M of these m-variants that have the M least likelihoods of being corrupted are picked. In one embodiment, the MAPF function itself performs the IP header checksumming and upper-layer's checksumming operation on each m-variant. Then, either M or all of those invariants with correct checksums, whichever is smaller, may be passed to the upper-layer. The packets with incorrect IP header or upper-layer checksum are discarded. In certain embodiment, it may be desirable to set M=I since the correct checksum usually indicates an uncorrupted packet and there is no need to pass duplicate uncorrupted packets to the upper layer.

[186] Similar to MAPF, the MBPF soft handoff and select-cast are special cases of MBPF striping or multicast and parallel the operations described above with reference to the MAPF implementations.

QoS-aware Striping in MAPF or MBPF

[187] In certain embodiments, the striping feature described above in both the MAPF and MBPF may be aware of the QoS requirements of the application whose packets are being striped. For the following discussion a MAPF (or MBPF) transmitter and receiver will be referred to as a striper and de-striper, respectively. Moreover, the term 'link' will be used to refer to a packet's route over a particular access network in MAPF or via a particular base station in MBPF. [188] In one embodiment a QoS-aware striping scheme will be able to effectively aggregate the available bandwidth of multiple links without introducing significantly extra delay or delay jitter in the packet flow due to the bandwidth and delay differences of the multiple links. For example, TCP applications require an end-to-end delay bound, VoIP requires both end-to-end delay and delay jitter bound, and streaming video requires delay jitter bound and minimum bandwidth. The extra delay and delay jitter per the QoS requirements of the applications may be limited using one or more of the following actions:

1. The bandwidth and link delays m monitored and estimated continuously via special signaling probe or "ping" messages or via the data packets themselves or via link-layer information. The end-to-end delays of the packets may also monitored;

2. If the application needs sequenced packets or is sensitive to packet reordering, the effective delay for the packet flow path between striper and destriper is the largest delay d_max of any link included in the striping. In this case, only those links are included in the striping such that the resulting d_max does not increase the end-to-end delay beyond the application's required delay bound;

3. Similar to #2, the worst case delay jitter added by the striper- to-destriper path is the largest delay difference delta_max between any two links included in the striping. In this case, only those links are included in the striping such that the resulting delta_max does not increase the end-to-end delay jitter beyond the application's required bound;

4. Packets must be transmitted as periodically as possible over each link. Thus, in one embodiment, the striper may perform a 'weighted round robin' to schedule the packets over the different links, e.g. if link 1 has 10 times the bandwidth of link 2, the striper sends every 11th packet over link 2 and the remaining over link 1; and

5. A more precise embodiment for hard real-time applications will make a fresh striping decision for each packet as to which link to send it. In one embodiment, where t is the current time, the estimated arrival time t_{q,x} of packet q at the destriper if q is sent over link x, may be expressed as: t_{q,x} = t + L/b_x + d_x,

where L is length of packet q, b_x and d_x are the bandwidth and delay of link x respectively.

Then choose that link which minimizes | t_{q,x} - t_{p,x} | where p is the packet previous to q.

Mobile Overlay STreaming NETwork (MOSTNET) of MTs and MGs

[189] MOSTNET is an overlay network for forwarding, splitting, and multicasting data streams (e.g., video) 'transparently,' which means that there is no need to modify network routers or the TCP/IP stack or application at the end hosts, which can be arbitrarily mobile. In one embodiment, each MOSTNET overlay node is either an MT or an MG. The MGs may form the overlay nodes and ensure seamless stream mobility as described in the previous sections.

[190] A "stream" refers to any sequence of content bits to be delivered from a sender (a sending application) to a receiver (a receiving application) over a base network. Typically, the base network is the Internet of IP routers and the bits are transported inside the layer 4 payload of IP packets. The receiving application may receive all the bits in the same order in which they were sent from the sending application. A loss-tolerant stream is one where the receiving application can function correctly despite the loss of some bits from the stream. Similarly, an error-tolerant stream is one where the receiving application can function correctly despite the change in value of some bits in the stream. Examples include digitally encoded video, audio streams (e.g., MPEG streams) which are loss- and error-tolerant, as well as other data streams such as news feeds, stock information, etc. Note that the term "stream" includes any bit sequence to be transported over a network and, hence, includes all TCP or UDP data in the Internet. [191] The transparent delivery of a stream to a receiving application means that the stream bits reach the receiving application in the same order that they were sent, and such that the receiving application consumes the stream correctly as if they were coming from its original sender. Thus, in one embodiment the overlay network is invisible to the receiving application.

[192] Use of an overlay network of nodes residing at suitable points in the base network, may be used to achieve one or more of the following:

• Stream Signaling and Control: use of inter-node signaling to coordinate and control the overlay nodes' stream bit processing and forwarding actions in order to jointly achieve the following,

• Stream Splitting: achieve intelligent splitting of a stream into several sub-streams routed over multiple routes in the overlay network such as to efficiently use all the base network bandwidths available to all the overlay nodes,

• Stream Forwarding /Receiving (FR): forward IP packets belonging to a stream to another downstream node after making suitable translations or additions to the IP or upper layer headers of the packets such as to ensure the transparent delivery of the stream to the receiving application at the requesting receiver. Before sending a packet to the downstream node, the PR may do other packet processing or recovery tasks, e.g. recovery of lost packets from upstream node, buffering the packet to do flow control between itself and the downstream node, etc.,

• Stream Storage and Replay: some subset of the received IP packets of a stream is stored in the node for arbitrary amount of time either as pure payload (extracted stream bits) or in packet form i.e. as the exact received sequence of packets as seen at IP layer 3, 4, or above. Later, this stored subset gets replayed on demand by any requesting receiver in the network. The replay consists of duplicating the exact original sequence of IP packets from which that subset was created and stored and sending them to the local stream forwarder. The intermediate storage thus remains invisible to the requesting receiver's application which sees the stream as if it was coming from the original sender,

• Stream Mobility: maintain the transparent delivery of the stream at the receiver in spite of the sender or receiver moving among different networks or changing their IP address or port numbers during the stream session. The nodes can even move among private networks behind NAT routers. They can even move in the overlay network itself i.e. wish to send to or receive the stream from a different set of overlay nodes even during the stream session. The transparent delivery is maintained even during such movements.

[193] In one embodiment, an overlay of network of nodes residing at suitable points in a base network may be used to carry out task 1 above. In another embodiment, tasks 1 and/or 2 may both be carried out. In still another embodiment, task 1 and/or 2 may be carried out with one or more of 3, 4, and 5 above.

Exemplary Embodiments of MOSTNET components

[194] Fig. 15 shows a MOSTNET 1500 overlaid over the Internet with its component tasks shown inside one of the overlay nodes. In particular, the MOSTNET 1500 is comprised of nodes 151Oi - 151O₅, wherein the sender application/stream sender 1520 is coupled to node 1510i, and the receiver application/stream receiver 1530 occurs at node 1510s, as shown in Fig. 15. A blow up of an exemplary node 15104 is also shown as including a stream forwarder/receiver 1540, a stream splitter 1550 and a stream storage/replay 1560. The three possible packet paths through the node 15104 are shown as path 157Oi - 157O₃.

[195] As shown in Fig. 15, an original stream 1580 is provided to the MOSTNET 1500 by the sender application 1520. Node 151Oi splits the original stream 1580 and sends out substreams 159Oi - 15903. Moreover, node 15104 is shown as combining received substreams 15902 and 15903 and provided combined substream 1595 to the destination node 151Os.

Embodiment of Packet Sequence Numbers

[196] It is assumed that each packet of the stream 1580 carries a sequence number. If the stream 1580 is a TCP or RTP multimedia stream, the TCP or RTP sequence numbers can themselves be used as the sequence numbers for stream splitting. Otherwise, the overlay network 1500 can introduce its own sequence numbers into packets inside the packet's IP header's options field or inside an inserted MTM header or even an external layer 4 (such as TCP) header in case layer-4 tunneling is used between two overlay nodes.

Stream Splitting and Recombination

[197] In one embodiment, a stream (e.g., original stream 1580) may be split into multiple substreams (e.g., 159Oi — 159U3), each of which is identifiable by a globally unique substream_id. The sequence of packets of a substream gets logically split into stream sections each of which is identifiable by a unique section id. A section id for a section s is thus a tuple <substream_id, length, skip_length, next_position> where substream_id is the substream to which it belongs, length is the number of packets in this section, skip_length is the difference in sequence numbers of consecutive packets in this section and next_position is the sequence number of the first packet of the next section in the substream. The whole section id is carried at the start of the section, typically inside the header of the section's first packet. Thus, the substream_j.d need not be carried in every packet, but only once for each section.

[198] At any node (e.g., nodes 151Oi - 1510s), the incoming packet sequence for a stream will be a general mix of sections from different substreams. The node watches the sequence number in each packet to determine when each section begins and ends. In one embodiment, each substream has its buffer into which each packet or assembled section is queued in the order of its sequence number. Those buffers may then feed into the stream storage (e.g., stream storage 1560) or the stream forwarder (e.g., stream forwader/receiver 1540) or stream recombiner (e.g., stream splitter/recombiner 1550).

Stream Recombiner

[199] The stream recombiner (e.g., e.g., stream splitter/recombiner 1550) may be used to combine multiple incoming substreams (e.g., substreams 15902 and 15903) of a stream and then reassembles them into multiple outgoing substreams (e.g., substream 1595). For example, consider a stream split into two substreams each carrying the odd and even packets of the stream i.e. (1,3,5,..) and (2,4,6,..). The recombiner may then recombine these two substreams into 3 outgoing streams viz. (1,4,7,..) , (2,5,8,..) and (3,6,9,..). These 3 outgoing streams will have new substream identifiers allocated such as to make them globally uniquely identifiable by any receiving node.

Signaling and Control (SC)

[200] An overlaying of a stream gets defined uniquely by specifying the splitting/recombining and the next hop at each overlay node n. With respect to the splitting/recombining, for each stream x some of whose substreams are received at node n, the SC would specify out(x, n), which are the outgoing substreams from n. With respect to the next hop, the SC would specify next(y,n), which is the next hop node for each outgoing substream y from n.

[201] One or more overlay nodes, or even external dedicated nodes, may function as stream controllers. In certain embodiments, the nodes may run a stream overlaying algorithm among themselves to determine out(x,n) and next(y,n) for each x,y,n. The exact algorithm can be designed via traffic engineering methods such as multi-commodity ilow algorithms or heuristically by anyone skilled in the art, using knowledge of the base network's topology and bandwidths. For each node n, a controller may signal to n, the out(x,n) and next(y,n) information for all pairs (x,y) expected at n. MOSTNET Updates

[202] MOSTNET updates may occur when 1) a new overlay node joins the MOSTNET, 2) an existing overlay node drop out, or 3) an overlay node needs to change the set of its neighbor nodes. In one embodiment, a MOSTNET update may be done to improve stream quality. It may also happen due to mobility of the node of failure of some nodes. The new set of neighbor nodes may then be determined locally by the node itself or centrally by a controller.

[203] Such MOSTNET updates may also be sent to registrar nodes. The controller and registrar nodes may be implemented in the same physical server. Any overlay node can itself serve as registrar or controller. The registrar nodes inform the controller nodes of the MOSTNET updates, which accordingly re-run their algorithm to update the stream overlaying and signal the steam overlay updates to all relevant overlay nodes.

[204] For large-scale or geographically-wide stream distribution, each receiver only belongs to a local overlay network with local controllers. These controllers communicate with each other to achieve a global overlay net.

Stream Forwarding I Receiving (FR)

[205] Each node has a stream FR module (e.g., stream forwarder/receiver 1540) which possesses its own current IP address and a port number for each stream, which we call the FR adport for that stream at that node.

[206] At any node n, before sending an IP packet to a downstream node, the FR module replaces the original source and destination adports in the packet's header with the adports of the FR modules of n and the downstream node respectively. Those original adports are now carried in other suitable places in the packet's IP/layer-4 header: e.g. in IP options field, or in the inner IP/lay er-4 header in case a layer 3 or layer 4 tunneling is used.

[207] Conversely, after receiving an IP packet from an upstream node, the FR module replaces its adport fields with the original adport extracted (and erased) from inside the packet's headers. [208] If a node is mobile, its FR adport for a stream may change at any time. The nodes can then track each other's changing FR adports by signaling the adport changes to each other or to some common third node serving effectively as an MG. For complete unambiguity about exactly which node sent a packet to whom, the globally unique MTID may be carried in the packets.

Stream Storage and Replay (SR)

[209] For local storage of a stream, the payloads of layer 4 (e.g. TCP payload, or the RTP packet including RTP header) of stream packets to be stored are layer-4 tunneled by the FR module not to a downstream node, but rather to a local SR application (e.g., stream storage/replay 1560) running on the node and listening on a local layer-4 connection port. In one embodiment, this application runs in the operating system's user space, but may run partially in kernel space as well if enough CPU/memory resources are available. In general, the SR application may modify the original steam bits by, for example, recover lost bits or performing transcoding operations to improve the stream quality.

[210] When the stored section of the stream has to be delivered later to a remote requesting receiver, the SR application re-packetizes the stored stream as if it was the original sender of the stream. These packets may then be sent back to the FR module.

Stream Mobility

[211] Stream mobility may be achieved effectively by using an implementation of an effective MG inside the overlay nodes. In one embodiment, the MG inside an overlay node registers any other overlay node wishing to register with it. If any registered node's adport changes, it may signal that change to its MG, which in turn conveys that change to all neighbor nodes of that node.

[212] In certain embodiments, due to a MOSTNET update, a node may need to re-register with a new MG, and have a new set of neighbor nodes, in the middle of a stream session. In such a case, the stream may be handed off from the old to the new MG and the old to the new set of neighbor nodes using the MTM Handoff procedure described in detail above. [213] While the invention has been described in connection with various embodiments, it will be understood that the invention is capable of further modifications. This application is intended to cover any variations, uses or adaptations of the invention following, in general, the principles of the invention, and including such departures from the present disclosure as, within the known and customary practice within the art to which the invention pertains.

Claims

CLAIMSWhat is claimed is:

1. A system comprising: a first logical gateway coupled to at least a first access network; a second logical gateway coupled to at least a second access network, said first and second logical gateways to communicate over a third network; a source mobile terminal coupled to the first access network and in communication with the first logical gateway; and a destination mobile terminal coupled to the second access network and in communication with the second logical gateway, wherein the first and second gateways transfer data packets between the source mobile terminal and the destination mobile terminal, and wherein the first and second gateways are further to exchange state information for the source mobile terminal and destination mobile terminal in order to maintain inter-network connectivity between the source mobile terminal and destination mobile across the first and second access networks.

2. The system of claim 1, wherein the state information comprises IP addresses and port change information for each of the source mobile terminal and the destination mobile terminal.

3. The system of claim 1, wherein said system further comprises a plurality of additional gateways, and wherein said first gateway, second gateway, and plurality of additional gateways are to cooperate to maintain said internetwork connectivity even when said destination mobile terminal is mobile, intermittently inactive, located inside a private network, or located in a different provider's network than the source mobile terminal.

4. The system of claim 1, wherein said data packets each include a virtual header containing session-specific information.

5. The system of claim 4, wherein said virtual header further includes a globally unique identifier for the source mobile terminal, and destination information relating to the destination mobile terminal.

6. The system of claim 5, wherein said globally unique identifier includes one of a Session Initiated Protocol (SIP) name, Mobile Station ID Number (MSIDN), Network Access Identifier (NAI), and a public IP address.

7. The system of claim 1, wherein the source mobile terminal is assigned a local IP address associated with a particular network connection, and a publicly-reachable IP address which is usable by terminals connected to external networks to locate and transmit to the source mobile terminal.

8. The system of claim 1, wherein said source mobile terminal registers with a third logical gateway upon connecting to a new network, and establishes a communication route therewith.

9. The system of claim 8, wherein a proxy gateway upstream from the third logical gateway signals to a plurality of other gateways that said data packets should be routed to the third logical gateway instead of the first logical gateway.

10. The system of claim 1, wherein the first and second gateway each maintain a list of all available access networks and all IP addresses for mobile terminals coupled to those available access networks.

11. The system of claim 10, wherein a packet flow from said source mobile terminal is split into a plurality of sub-flows traveling over a plurality of access networks which include said first and second access networks, and wherein said plurality of sub-flows are reassembled into said packet flow prior to reaching said destination mobile terminal.

12. The system of claim 10, wherein a packet flow from said source mobile terminal is replicated and multicast over a plurality of access networks which include said first and second access networks, and wherein said plurality of sub-flows are reassembled into said packet flow prior to reaching said destination mobile terminal.

13. The system of claim 1, wherein the system comprises a plurality of nodes, which includes said first and second gateways, and wherein said plurality of nodes employs a signaling scheme to coordinate and control data stream processing.

14. The system of claim 13, wherein said data stream processing includes splitting an original data stream into a plurality of sub-streams routed separately across said plurality of nodes to increase efficient use of network bandwidth.

15. A method comprising: registering a source mobile terminal with a first logical gateway coupled to at least a first access network; registering a destination mobile terminal with a second logical gateway coupled to at least a second access network, wherein said first and second logical gateways to communicate over a third network; transferring data packets between the source mobile terminal and the destination mobile terminal using said first and second gateways, respectively; exchanging state information, between the first and second gateways, relating to state information for the source mobile terminal and destination mobile terminal in order to maintain inter-network connectivity between the source mobile terminal and destination mobile across the first and second access networks.

16. The method of claim 15, wherein the state information comprises IP addresses and port change information for each of the source mobile terminal and the destination mobile terminal.

17. The method of claim 15, further comprising maintaining said internetwork connectivity, even when said destination mobile terminal is mobile, intermittently inactive, located inside a private network, or located in a different provider's network than the source mobile terminal, using a plurality of additional gateways operating in corporation with the first and second gateways.

18. The method of claim 15, further comprising including a virtual header containing session-specific information in each of said data packets.

19. The method of claim 18, further comprising including a globally unique identifier for the source mobile terminal, and destination information relating to the destination mobile terminal, in the virtual header.

20. The method of claim 19, wherein said globally unique identifier includes one of a Session Initiated Protocol (SIP) name, Mobile Station ID Number (MSIDN), Network Access Identifier (NAI), and a public IP address.

21. The method of claim 15, further comprising: assigning a local IP address associated with a particular network connection to the source mobile terminal; and assigning a publicly-reachable IP address to the source mobile terminal which is usable by terminals connected to external networks to locate and transmit to the source mobile terminal.

22. The method of claim 15, further comprising: registering said source mobile terminal with a third logical gateway upon connecting to a new network; and establishing a communication route between the third logical gateway and the source mobile terminal.

23. The method of claim 22, further comprising signaling, by a proxy gateway upstream from the third logical gateway, to a plurality of other gateways that said data packets should be routed to the third logical gateway instead of the first logical gateway.

24. The method of claim 15, maintaining, by each of the first and. second gateways, a list of all available access networks and all IP addresses for mobile terminals coupled to those available access networks.

25. The method of claim 24, further comprising: splitting a packet flow from said source mobile terminal into a plurality of sub-flows traveling over a plurality of access networks, which include said first and second access networks; and reassembling said plurality of sub-flows into said packet flow prior to reaching said destination mobile terminal.

26. The method of claim 24, further comprising: replicating a packet flow from said source mobile terminal; multicasting the replicated packet flow over a plurality of access networks, which include said first and second access networks; reassembling said plurality of sub-flows into said packet flow prior to reaching said destination mobile terminal.

27. The method of claim 15, further comprising empLoying a signaling scheme, by a plurality of nodes, which include said first and second gateways, to coordinate and control data stream processing.

28. The method of claim 27, further comprising splitting an original data stream into a plurality of sub-streams routed separately across said plurality of nodes to increase efficient use of network bandwidth.