US7693993B2 - Method and system for providing dynamic hosted service management across disparate accounts/sites - Google Patents

Method and system for providing dynamic hosted service management across disparate accounts/sites Download PDF

Info

Publication number
US7693993B2
US7693993B2 US10/984,959 US98495904A US7693993B2 US 7693993 B2 US7693993 B2 US 7693993B2 US 98495904 A US98495904 A US 98495904A US 7693993 B2 US7693993 B2 US 7693993B2
Authority
US
United States
Prior art keywords
server
blade
servers
pool
application server
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related, expires
Application number
US10/984,959
Other versions
US20050182838A1 (en
Inventor
Kitrick B. Sheets
Philip S. Smith
Stephen J. Engel
Yuefan Deng
Joseph Guistozzi
Alexander Korobka
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ASPENGINES Inc
GALACTIC COMPUTING Corp BVI FORMERLY KNOWN AS GALACTIC REVENUE Ltd
RPX Corp
Original Assignee
Galactic Computing Corp BVI/IBC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Galactic Computing Corp BVI/IBC filed Critical Galactic Computing Corp BVI/IBC
Priority to US10/984,959 priority Critical patent/US7693993B2/en
Publication of US20050182838A1 publication Critical patent/US20050182838A1/en
Priority to US12/748,063 priority patent/US8316131B2/en
Application granted granted Critical
Publication of US7693993B2 publication Critical patent/US7693993B2/en
Priority to US13/681,031 priority patent/US20130238801A1/en
Assigned to ASPENGINES, INC. reassignment ASPENGINES, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: SMITH, PHILIP S., DONG, YUEFAN, SHEETS, KITRICK B., GIUSTOZZI, JOSEPH, KOROBKA, ALEXANDER, ENGEL, STEPHEN J.
Assigned to GALACTIC COMPUTING CORPORATION, BVI, FORMERLY KNOWN AS GALACTIC REVENUE LTD reassignment GALACTIC COMPUTING CORPORATION, BVI, FORMERLY KNOWN AS GALACTIC REVENUE LTD ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: NEUTILITY CORPORATION, FORMERLY KNOWN AS ASPENGINES, INC.
Assigned to GALACTIC COMPUTING CORPORATION BVI/IBC reassignment GALACTIC COMPUTING CORPORATION BVI/IBC CHANGE OF NAME (SEE DOCUMENT FOR DETAILS). Assignors: GALACTIC COMPUTING CORPORATION, BVI, FORMERLY KNOWN AS GALACTIC REVENUE LTD
Assigned to ASPENGINES, INC. reassignment ASPENGINES, INC. CORRECTIVE ASSIGNMENT TO CORRECT THE FOURTH AND FIFTH ASSIGNOR NAME PREVIOUSLY RECORDED AT REEL: 034524 FRAME: 0907. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Assignors: SMITH, PHILIP S., DENG, YUEFAN, SHEETS, KITRICK B., GUISTOZZI, JOSEPH, KOROBKA, ALEXANDER, ENGEL, STEPHEN J.
Assigned to RPX CORPORATION reassignment RPX CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: GALACTIC COMPUTING CORPORATION BVI/IBC
Assigned to JPMORGAN CHASE BANK, N.A., AS COLLATERAL AGENT reassignment JPMORGAN CHASE BANK, N.A., AS COLLATERAL AGENT SECURITY AGREEMENT Assignors: RPX CLEARINGHOUSE LLC, RPX CORPORATION
Assigned to RPX CORPORATION, RPX CLEARINGHOUSE LLC reassignment RPX CORPORATION RELEASE (REEL 038041 / FRAME 0001) Assignors: JPMORGAN CHASE BANK, N.A.
Assigned to JEFFERIES FINANCE LLC reassignment JEFFERIES FINANCE LLC SECURITY INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: RPX CORPORATION
Assigned to BARINGS FINANCE LLC, AS COLLATERAL AGENT reassignment BARINGS FINANCE LLC, AS COLLATERAL AGENT PATENT SECURITY AGREEMENT Assignors: RPX CLEARINGHOUSE LLC, RPX CORPORATION
Assigned to BARINGS FINANCE LLC, AS COLLATERAL AGENT reassignment BARINGS FINANCE LLC, AS COLLATERAL AGENT PATENT SECURITY AGREEMENT Assignors: RPX CLEARINGHOUSE LLC, RPX CORPORATION
Assigned to RPX CORPORATION reassignment RPX CORPORATION RELEASE BY SECURED PARTY (SEE DOCUMENT FOR DETAILS). Assignors: JEFFERIES FINANCE LLC
Expired - Fee Related legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1001Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H04L67/1004Server selection for load balancing
    • H04L67/1014Server selection for load balancing based on the content of a request
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061Partitioning or combining of resources
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/06Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling

Definitions

  • the present invention relates generally to the field of data processing business practices. More specifically, the present invention relates to a method and system for providing dynamic management of hosted services across disparate customer accounts and/or geographically distinct sites.
  • ISPs Internet Service Providers
  • ASPs Application Service Providers
  • ISVs Independent Software Vendors
  • ESPs Enterprise Solution Providers
  • MSPs Managed Service Providers
  • HSPs Hosted Service Providers
  • a server can be a personal computer that is connected to the Internet through a network interface and that runs specific software designed to service the requests made by customers or clients of that server.
  • HSPs For all of the various delivery models that can be used by HSPs to provide hosted services, most HSPs will use a collection of servers that are connected to an internal network in what is commonly referred to as a “server farm,” with each server performing unique tasks or the group of servers sharing the load of multiple tasks, such as mail server, web server, access server, accounting and management server.
  • server farm In the context of hosting websites, for example, customers with smaller websites are often aggregated onto and supported by a single web server. Larger websites, however, are commonly hosted on dedicated web servers that provide services solely for that site.
  • ISP survival Guide Strategies For Running A Competitive ISP , (1999).
  • HSPs have preferred to utilize server farms consisting of large numbers of individual personal computer servers wired to a common Internet connection or bank of modems and sometimes accessing a common set of disk drives.
  • server farms consisting of large numbers of individual personal computer servers wired to a common Internet connection or bank of modems and sometimes accessing a common set of disk drives.
  • the HSP deploys only that level of hardware required to support its current customer level. Equally as important, the HSP can charge its customers an upfront setup fee that covers a significant portion of the cost of this hardware. By utilizing this approach, the HSP does not have to spend money in advance for large computer systems with idle capacity that will not generate immediate revenue for the HSP.
  • the server farm solution also affords an easier solution to the problem of maintaining security and data integrity across different customers than if those customers were all being serviced from a single larger mainframe computer. If all of the servers for a customer are loaded only with the software for that customer and are connected only to the data for that customer, security of that customer's information is insured by physical isolation.
  • HSPs For HSPs, numerous software billing packages are available to account and charge for these metered services, such as XaCCT from rens.com and HSP Power from inovaware.com. Other software programs have been developed to aid in the management of HSP networks, such as IP Magic from lightspeedsystems.com, Internet Services Management from resonate.com and MAMBA from luminate.com. The management and operation of an HSP has also been the subject of articles and seminars, such as Hursti, Jani, “Management of the Access Network and Service Provisioning,” Seminar in Internetworking, Apr. 19, 1999. An example of a typical HSP offering various configurations of hardware, software, maintenance and support for providing commercial levels of Internet access and website hosting at a monthly rate can be found at rackspace.com.
  • the HSP will manually add or remove a server to or from that portion of the HSP server farm that is directly cabled to the data storage and network interconnect of that client's website.
  • the typical process would be some variation of the following: (a) an order to change service level is received from a hosted service customer, (b) the HSP obtains new server hardware to meet the requested change, (c) personnel for the HSP physically install the new server hardware at the site where the server farm is located, (d) cabling for the new server hardware is added to the data storage and network connections for that site, (e) software for the server hardware is loaded onto the server and personnel for the HSP go through a series of initialization steps to configure the software specifically to the requirements of this customer account, and (f) the newly installed and fully configured server joins the existing administrative group of servers providing hosted service for the customer's account. In either case, each server farm is assigned to a specific customer and must be configured to meet the maximum projected demand for services from that customer account.
  • U.S. Pat. No. 5,951,694 An example of how a new server can automatically join an existing administrative group is described in U.S. Pat. No. 5,951,694.
  • all of the servers in an administrative group are represented in a mapping table maintained by a gateway server.
  • the mapping table identifies different service groups for the administrative group, such as mail service group, database service group, access server group, etc.
  • the gateway server routes requests for the administrative group to the appropriate service group based on the mapping table.
  • a new server may be added to one of the service groups by loading the appropriate software component on that server, after which the gateway server will recognize the new server and add it to the mapping table and bring the new server up to speed with the rest of the servers in that service group using a transaction log maintained for each service group.
  • the patent describes a software routine executing on a dedicated administrative server that uses a load balancing scheme to modify the mapping table to ensure that requests for that administrative group are more evenly balanced among the various service groups that make up the administrative group.
  • U.S. Pat. No. 6,006,529 describes software clustering that includes security and heartbeat arrangement under control of a master server, where all of the cluster members are assigned a common IP address and load balancing is preformed within that cluster.
  • U.S. Pat. Nos. 5,537,542, 5,948,065 and 5,974,462 describe various workload-balancing arrangements for a multi-system computer processing system having a shared data space. The distribution of work among servers can also be accomplished by interposing an intermediary system between the clients and servers.
  • U.S. Pat. No. 6,097,882 describes a replicator system interposed between clients and servers to transparently redirect IP packets between the two based on server availability and workload.
  • U.S. Pat. No. 6,014,669 describes cluster operation of multiple servers in a single cluster by using a lock-step distributed configuration file.
  • U.S. Pat. No. 6,088,727 describes cluster control in a shared data space multi-computer environment.
  • Other patents have described how a single image of the input/output space can be used to coordinate multiple computers.
  • U.S. Pat. No. 5,832,222 describes how a single image of the input/output space can be used to coordinate geographically dispersed computer systems.
  • U.S. Pat. No. 6,067,545 describes a distributed file system with shared metadata management, replicated configuration database and domain load balancing, that allows for servers to fall into and out of a single domain under control of the configuration database.
  • a good example of this type of operation management system that is intended to be used by HSPs is the Tivoli Service Delivery Management platform that consists of a user administration module, a software distribution module, an inventory module, an enterprise console, a security module, an enterprise manager module that provides a customizable view of all of the components in a network once they are added to the network, and a workload scheduler that allows workload to be balanced among servers sharing a common data space. All of these modules operate using an over-the-network communication scheme involving agents on the various nodes in the network that collect and report status and incident information to the other modules.
  • the various modules of the Tivoli Service Delivery Management platform can take over and manage those components on a more automatic basis.
  • the process of physically adding hardware for a new node into the network remains essentially a manual process that is accomplished in the same manner as previously described.
  • U.S. Pat. No. 5,615,329 describes a typical example of a redundant hardware arrangement that implements remote data shadowing using dedicated separate primary and secondary computer systems where the secondary computer system takes over for the primary computer system in the event of a failure of the primary computer system.
  • the problem with these types of mirroring or shadowing arrangements is that they can be expensive and wasteful, particularly where the secondary computer system is idled in a standby mode waiting for a failure of the primary computer system.
  • 5,696,895 describes one solution to this problem in which a series of servers each run their own tasks, but each is also assigned to act as a backup to one of the other servers in the event that server has a failure. This arrangement allows the tasks being performed by both servers to continue on the backup server, although performance will be degraded.
  • Other examples of this type of solution include the Epoch Point of Distribution (POD) server design and the USI Complex Web Service.
  • the hardware components used to provide these services are predefined computing pods that include load-balancing software, which can also compensate for the failure of a hardware component within an administrative group. Even with the use of such predefined computing pods, the physical preparation and installation of such pods into an administrative group can take up to a week to accomplish.
  • ISA Internet Shock Absorber
  • the ISA service distributes a customer's static Web content to one or more caching servers located at various Points of Presence (POPs) on the Cable & Wireless Internet backbone. Requests for this static Web content can be directed to be caching servers and the various POP locations to offload this function from the servers in the administrative group providing hosted service for that customer.
  • POPs Points of Presence
  • the caching of static Web content is something that occurs naturally as part of the distribution of information over the Internet. Where a large number of users are requesting static information from a given IP address, it is common to cache this information at multiple locations on the Internet.
  • the ISA service allows a customer to proactively initiate the caching of static Web content on the Internet. While this solution has the potential to improve performance for delivery of static Web content, this solution is not applicable to the numerous other types of hosted services that involve interactive or dynamic information content.
  • the present invention is a method and system for operating a hosted service provider for the Internet in such a way as to provide dynamic management of hosted services across disparate customer accounts and/or geographically distinct sites.
  • a plurality of individual servers are allocated to a common administrative group defined for that customer account.
  • Each administrative group is configured to access software and data unique to that customer account for providing hosted services to the Internet for that customer account.
  • the system automatically monitors the performance and health of the servers in each administrative group. At least one server from a first administrative group is automatically and dynamically reallocated to a second administrative group in response to the automatic monitoring.
  • the automatic and dynamic reallocation of servers is accomplished by setting initialization pointers for the reallocated servers to access software and data unique to the customer account for the second administrative group, and then reinitializing the reallocated servers such that they join the second administrative group when restarted.
  • the performance and health of the servers in each administrative group are monitored over a separate out-of-band communication channel dedicated to interconnecting the servers across administrative groups.
  • Each administrative group includes a local decision software program that communicates with a master decision software program that determines when and how to dynamically reallocate servers to different administrative groups in response to usage demands, available resources and service level agreements with each customer account.
  • a system for providing the dynamic management of hosted services for multiple customer accounts includes at least five servers operably connected to an intranet.
  • Each server includes host management circuitry providing a communication channel with at least one of the other servers that is separate from this intranet.
  • At least four of the servers execute a local decision software program that monitors the server and communicates status information across the communication channel.
  • At least two of the servers are allocated to a first administrative group for a first customer account and configured to access software and data unique to this first customer account, such that hosted services are provided via the Internet for this customer account.
  • At least two of the other servers are allocated to a second administrative group for a second customer account and configured to access software and data unique to this second customer account, such that hosted services are provided via the Internet for this customer account.
  • at least one of the servers executes a master decision software program that collects status information from the other servers and dynamically reallocates at least one server from the first administrative group to the second administrative group in response to at least the status information.
  • the present invention is capable of dynamically reallocating servers across multiple disparate customer accounts to provide hosted services with a more economical and flexible server farm arrangement.
  • the ability of the present invention to support multiple administrative groups for multiple customers allows for an intelligent and dynamic allocation of server resources among different customer accounts.
  • FIG. 1 is a simplified block diagram of a prior art arrangement of a server farm for a hosted service provider.
  • FIG. 2 is a graphic representation of Internet traffic in relation to server capacity for a prior art server farm hosting multiple customer accounts.
  • FIG. 3 is a simplified block diagram of the arrangement of a server farm in accordance with the present invention.
  • FIG. 4 is a simplified block diagram similar to FIG. 3 showing the dynamic reallocation of servers from a first customer account to a second customer account to address a hardware failure.
  • FIG. 5 is a simplified block diagram similar to FIG. 3 showing the dynamic reallocation of servers from a first customer account to a second customer account to address an increased usage demand.
  • FIG. 6 is a block diagram of a preferred embodiment of the components of a server farm in accordance with the present invention.
  • FIG. 7 is an exploded perspective view of a preferred embodiment of the hardware for the server farm in accordance with the present invention.
  • FIG. 8 is a block diagram showing the hierarchical relation of the various software layers utilized by the present invention for a given customer account.
  • FIG. 9 is a block diagram of an embodiment of the present invention implemented across geographically disparate sites.
  • FIG. 10 is a graphic representation of Internet traffic in relation to server capacity for the server farm of the present invention when hosting multiple customer accounts.
  • FIG. 11 is a block diagram showing a preferred embodiment of the master decision software program of the present invention.
  • FIG. 12 is a graphic representation of three different service level agreement arrangements for a given customer account.
  • FIG. 13 is a graphic representation of Internet traffic in relation to server capacity for a multi-site embodiment of the present invention.
  • FIG. 14 is a block diagram showing the master decision software program controlling the network switch and storage unit connections.
  • FIG. 15 is a block diagram of the preferred embodiment of the local decision software program.
  • FIG. 16 is a graphic representation of the workload measurements from the various measurement modules of the local decision software program under varying load conditions.
  • FIG. 17 is a graphic representation of a decision surface generated by the local decision software program to request or remove a server from an administrative group.
  • FIG. 1 a simplified functional view of an existing server farm 20 for a hosted service provider is shown.
  • Such server farms are normally constructed using off-the-shelf hardware and software components statically configured to support the hosted service requirements of a given customer account.
  • the server farm 20 for the hosted server provider is supporting hosted services for four different customer accounts.
  • the server farm 20 is connected to the Internet 22 by network switches/routers 24 .
  • the network switches 24 are in turn connected to internal network switches/routers 26 that form an intranet among the front-end/content servers 28 and back-end/compute servers 30 for a given customer account. All front-end/content servers 28 and back-end/compute servers 30 are connected to disk systems 32 containing data and software unique to that customer account.
  • the disk systems 32 may be included within the server housing, or the disk systems 32 may be housed in physically separate units directly connected to each of the servers 28 , 30 or attached to more than one server 28 , 30 as a storage attached network (SAN) or network attached storage (NAS) configuration.
  • SAN storage attached network
  • NAS network attached storage
  • system resources e.g., servers, disks, network links
  • a relatively simple website has been designed for any given customer account such that under a projected peak load the customer account may require three front-end servers 28 to handle user requests and a quad processor back-end server 30 to handle database queries/updates generated by these requests.
  • hardware-based technology such as f5® Big-IP®, Cisco Local Director, or Foundry® ServerIronTM, or a software-based solution such as Windows® Load Balance Service (WLBS) or equivalent will be used to distribute the user requests evenly across the front-end/content servers 28 .
  • WLBS Windows® Load Balance Service
  • the back-end database/compute server 30 will commonly be clustered to provide some level of fault tolerance.
  • the website for this customer account is an e-commerce site designed to handle a peak load of 5000 transactions per minute.
  • the websites for the remaining customer accounts in the server farm 20 have been designed to handle peak loads of 10,000, 15,000 and 5,000 transactions per minute, respectively.
  • having to design and configure each customer account to handle an anticipated peak load likely results in significant wasted capacity within the overall server farm 20 .
  • the server farm 20 handling multiple customer accounts may have excess aggregate capacity, this extra capacity cannot be used to respond to hardware failures or unexpected increases in peak load from one account to the next. Resources configured for a particular customer account are dedicated to that account and to that account only.
  • the site manager can physically remove the failed hardware component, install a spare hardware component that has hopefully been stockpiled for this purpose, recable the new hardware component, configure and install the appropriate software for that customer account, and allow the new hardware component to rejoin the remaining front-end servers 28 . Ultimately, this process could be accomplished in less than an hour. If the message is not received in a timely manner, if the site manager is not located at the site where the server farm is located, or if there is no stockpiled spare hardware available to replace the failed unit, this process will take even longer. In the meantime, response times for users accessing the customer account are degraded and the customer account becomes increasingly vulnerable to another hardware failure during this period.
  • these intermediary nodes will divert subsequent requests to the website due to their understanding that the website is “unavailable.” Not only are the customers who receive the “service unavailable” message not serviced, but many other customers may never even get to the website once the customer account becomes saturated or overloaded.
  • the server farm 40 includes network switches 44 to establish interconnection between the server farm 40 and the Internet 22 .
  • a population of servers 46 are managed under control of an engine group manager 48 .
  • Each of the servers 46 is a stateless computing device that is programatically connected to the Internet via the network switches 44 and to a disk storage system 50 .
  • the servers 46 are connected to the disk storage system 50 via a Fibre Channel storage area network (SAN).
  • the servers 46 may be connected to the disk storage system 50 via a network attached storage (NAS) arrangement, a switchable crossbar arrangement or any similar interconnection technique.
  • SAN Fibre Channel storage area network
  • NAS network attached storage
  • the engine group manager 48 is responsible for automatically allocating the stateless servers 46 among multiple customer accounts and then configuring those servers for the allocated account. This is done by allocating the servers for a given customer account to a common administrative group 52 defined for that customer account and configured to access software and data unique to that customer account. As will be described, the engine group manager 48 automatically monitors each administrative group and automatically and dynamically reallocates servers 46 ′ from a first administrative group 52 - a to a second administrative group 52 - b in response to the automatic monitoring.
  • the present invention can make a reallocated server 46 ′ available to a new administrative group 52 in as little as a few minutes.
  • the only significant time required to bring the reallocated server 46 ′ online will be the time required to reboot the server 46 ′ and any time required for the load-balancing and/or clustering software to recognize this rebooted server.
  • load-balancing software is more typically found in connection with front-end/content servers, whereas clustering software or a combination of clustering software and load-balancing software are more typically used in connection with back-end/compute servers.
  • load-balancing software will be used to refer to any of these possible combinations.
  • the reallocated servers 46 ′ automatically join the second administrative group because the software for the second administrative group 52 - b includes load-balancing software that will automatically add or remove a server from that administrative group in response to the server being brought online (i.e., reset and powered on) or brought off-line (i.e., reset and powered off).
  • load-balancing software that will automatically add or remove a server from that administrative group in response to the server being brought online (i.e., reset and powered on) or brought off-line (i.e., reset and powered off).
  • this kind of load-balancing software is widely known and available today; however, existing load-balancing software is only capable of adding or removing servers from a single administrative group.
  • the engine group manager 48 takes advantage of capabilities of currently available commercial load-balancing application software to allow for the dynamic reallocation servers 46 ′ across different administrative groups 52 .
  • agents or subroutines within the operating system software for the single administrative group could be responsible for integrating a reallocated server 46 ′ into the second administrative group 52 - b once the reallocated server 46 ′ is brought online.
  • the engine group manager 48 could publish updates to a listing of available servers for each administrative group 52 .
  • the engine group manager 48 will set pointers in each of the servers 46 for an administrative group 52 to an appropriate copy of the boot image software and configuration files, including operating system an application programs, that had been established for that administrative group 52 .
  • a reallocated server 46 ′ is rebooted, its pointers have been reset by the engine group manager 48 to point to the boot image software and configuration files for the second administrative group 52 - b , instead of the boot image software and configuration files for the first administrative group 52 - a.
  • each administrative group 52 represents the website or similar hosted services being provided by the server farm 40 for a unique customer account. Although different customer accounts could be paid for by the same business or by a related commercial entity, it will be understood that the data and software associated with a given customer account, and therefore with a given administrative group 52 , will be unique to that customer account.
  • each administrative group 52 consists of unique software, including conventional operating system software, that does not extend outside servers 46 which have been assigned to the administrative group 52 .
  • This distributed approach of the present invention allows for the use of simpler, conventional software applications and operating systems that can be installed on relatively inexpensive, individual servers. In this way, the individual elements that make up an administrative group 52 can be comprised of relatively inexpensive commercially available hardware servers and standard software programs.
  • FIGS. 6 and 7 show a preferred embodiment of the components and hardware for the server farm 40 in accordance with the present invention. Although the preferred embodiment of the present invention is described with respect to this hardware, it will be understood that the concept of the present invention is equally applicable to a server farm implemented using all conventional servers, including the currently available 1 U or 2 U packaged servers, if those servers are provided with the host management circuitry or its equivalent, as will be described.
  • the hardware for the server farm 40 is a scalable engine 100 comprised of a large number of commercially available server boards 102 each arranged as an engine blade 132 in a power and space efficient cabinet 110 .
  • the engine blades 132 are removably positioned in a front side 112 of the cabinet 110 in a vertical orientation.
  • a through plane 130 in the middle of the cabinet 110 provides common power and controls peripheral signals to all engine blades 132 .
  • I/O signals for each engine blade 132 are routed through apertures in the through plane 130 to interface cards 134 positioned in the rear of the cabinet 110 .
  • the I/O signals will be routed through an appropriate interface card 134 either to the Internet 22 via the network switch 44 , or to the disk storage 50 .
  • separate interface cards 134 are used for these different communication paths.
  • the scalable engine can accommodate different types of server boards 102 in the same cabinet 110 because of a common blade carrier structure 103 .
  • Different types of commercially available motherboards 102 are mounted in the common blade carrier structure 103 that provides a uniform mechanical interface to the cabinet 110 .
  • a specially designed PCI host board 104 that can plug into various types of motherboards 102 has connections routed through the through plane 130 for connecting to the interface cards 134 . Redundant hot-swappable high-efficiency power supplies 144 are connected to the common power signals on the through plane 130 .
  • the host board 104 includes management circuitry that distributes the power signals to the server board 102 for that engine blade 132 by emulating the ATX power management protocol.
  • the cabinet 110 accommodates multiple rows of engine blades 132 in a chassis assembly 128 that includes a pair of sub-chassis 129 stacked on top of each other and positioned on top of a power frame 146 that holds the power supplies 144 .
  • the cabinet 110 will also include rack mounted Ethernet networks switches 44 and 147 and storage switches 149 attached to disk drives 50 over a Fibre Channel network.
  • the server farm 40 can accommodate administrative groups 52 for any number of customers depending upon the total number of servers 46 in the server farm 40 .
  • multiple cabinets 110 can be integrated together to scale the total number of servers 46 at a given location.
  • each engine blade 132 can be populated with the most recent processors for Intel®, SPARC® or PowerPCTM designs, each of which can support standard operating system environments such as Windows® NT, Windows® 2000, LinuxTM or Solaris.
  • Each engine blade 132 can accommodate one or more server boards 102 , and each server board may be either a single or multiprocessor design in accordance with the current ATX form factor or a new form factor that may be embraced by the industry in the future.
  • the communication channel 106 is implemented a Controller Area Network (CAN) bus that is separate from the communication paths for the network switch 44 or storage switches 149 .
  • a second fault backup communication channel 106 ′ could be provided to allow for fault tolerance and redundant communication paths for the group manager software 48 .
  • CAN Controller Area Network
  • an engine blade 132 can include a local hard drive 107 that is accessed through the host board 104 such that information stored on that local hard drive 107 can be configured by the host board via the communication channel 106 .
  • the host board 104 preferably includes power management circuitry 108 that enables the use of common power supplies for the cabinet 110 by emulating the ATX power management sequence to control the application of power to the server board 102 .
  • a back channel Ethernet switch 147 also allows for communication of application and data information among the various server boards 102 within the server farm 40 without the need to route those communications out over the Internet 22 .
  • each cabinet 110 can house up to 32 engine blades 132 .
  • the networks switches 44 and 147 could comprise two 32 circuit switched Ethernet network routers from Foundry®.
  • the networks switches 44 and 147 allow a reconfiguration of the connection between a server 46 and the networks switch 44 and 147 to be dynamically adjusted by changing the IP address for the server.
  • the disk storage units 50 two options are available. First, unique hardware and software can be inserted in the form of a crossbar switch 149 between the engine blades 132 and the disk storage units 50 which would abstract way the details of the underlying SAN storage hardware configuration.
  • the link between the disk storage units 50 and each blade 132 would be communicated to the crossbar switch 149 through set of software APIs.
  • commercially available Fibre Channel switches or RAID storage boxes could be used to build connectivity dynamically between the blades 132 and disk storage units 50 .
  • a layer of software inside the engine group manager 48 performs the necessary configuration adjustments to the connections between the server blades 132 and networks switches 147 and disk storage units 50 are accomplished.
  • a portion of the servers 46 could be permanently cabled to the network switches or disk storage units to decrease switch costs if, for example, the set of customer accounts supported by a given portion of the server farm 40 will always include a base number of servers 46 that cannot be reallocated. In this case, the base number of servers 46 for each administrative group 52 could be permanently cabled to the associated network switch 149 and disk storage unit 50 for that administrative group 52 .
  • the server farm system 40 of the present invention can dynamically manage hosted services provided to multiple customer accounts.
  • the intranet is formed over the same network switches 44 that interconnect the servers 46 with the Internet 22 or over similar network switches such as network switches 147 that interconnect the servers 46 to each other.
  • Each server 46 has management circuitry on the host board 104 that provides a communication channel 106 with at least one of the other servers 46 that is separate from the intranet 54 created by the network switches 44 and/or 147 .
  • At least four of the servers 46 are configured to execute a local decision software program 70 that monitors the server 46 and communicate status information across the communication channel 106 . At least two of these servers 46 are allocated to a first administrative group 52 - a for a first customer account and configured to access software and data unique to the first customer account to provide hosted services to the Internet for that customer account. At least another two of the servers 46 are allocated to a second administrative group 52 - b for a second customer account and configured to access software and data unique to the second customer account to provide hosted services to the Internet for that customer account. At least one of the servers 46 executes a master decision software program 72 that collects status information from the local decision software programs 70 executing on the other servers 46 .
  • a pair of servers 46 are slaved together using fault tolerant coordination software to form a fault tolerant/redundant processing platform for the master decision software program.
  • the master decision software program 72 dynamically reallocates at least one server 46 ′ from the first administrative group 52 - a to the second administrative group 52 - b in response to at least the status information collected from the local decision software programs 70 .
  • the servers 46 for both administrative groups 52 can be arranged in any configuration specified for a given customer account. As shown in FIG. 3 , three of the servers 46 for administrative group 52 - b are configured as front-end servers with a single server 46 being configured as the back-end/compute server for this customer account.
  • the master decision software program 72 determines that is necessary to reallocate server 46 ′ from its current usage as a server for the first administrative group 52 - a to being used as a back-end/compute server for the second administrative group 52 - b . The preferred embodiment for how this decision is arrived will be described in connection with the description of the operation of the local decision software program 72 . Following the procedure just described, the master decision software program 72 directs the dynamic reallocation of reallocated server 46 ′ to the second administrative group 52 - b as shown in FIG. 4 .
  • a server farm 40 having thirty-two servers 46 could be set up to allocate six servers to each of four different customer accounts, with one server 46 executing the master decision software program 72 and a remaining pool 56 of seven servers 46 that are initially unassigned and can be allocated to any of the four administrative groups 52 defined for that server farm.
  • the preferred embodiment of the present invention uses this pool 56 as a buffer to further reduce the time required to bring a reallocated server 46 ′ into an administrative group 52 by eliminating the need to first remove the reallocated server 46 ′ from its existing administrative group 52 .
  • the pool 56 can have both warm servers and cold servers.
  • a warm server would be a server 46 that has already been configured for a particular administrative group 52 and therefore it is not necessary to reboot that warm server to allow it to join the administrative group.
  • a cold server would be a server that is not configured to a particular administrative group 52 and therefore it will be necessary to reboot that cold server in order for it to join the administrative group.
  • reallocated servers 46 ′ can be allocated to a new administrative group singly or as a group with more than one reallocated server 46 ′ being simultaneously reallocated from a first administrative group 52 - a to a second administrative group 52 - b .
  • multiple servers 46 may be reallocated together as a group if it is necessary or desirable to reduce the number of dynamically configurable ports on the networks 44 , 147 and/or storage switches 149 .
  • One of the significant advantages of the present invention is that the process of reconfiguring servers from one administrative group 52 - a to a second administrative group 52 - b will wipe clean all of the state associated with a particular customer account for the first administrative group from the reallocated server 46 ′ before that server is brought into service as part of the second administrative group 52 - b .
  • This provides a natural and very efficient security mechanism for precluding intentional or unintentional access to data between different customer accounts. Unless a server 46 or 46 ′ is a member of a given administrative group 52 - a , there is no way for that server to have access to the data or information for a different administrative group 52 - b .
  • the present invention keeps the advantages of the simple physical separation between customer accounts that is found in conventional server farm arrangements, but does this while still allowing hardware to be automatically and dynamically reconfigured in the event of a need or opportunity to make better usage of that hardware.
  • the only point of access for authorization and control of this reconfiguration is via the master decision software program 72 over the out-of-band communication channel 106 .
  • each server 46 is programmatically connected to the Internet 22 under control of the master decision software program 72 .
  • the master decision software program 72 also switches the reallocated server 46 ′ to be operably connected to a portion of the disk storage unit storing software and data unique to the customer account of the second administrative group.
  • the use of an out-of-band communication channel 106 separate from the intranet 54 over the network switches 44 for communicating at least a portion of the status information utilized by the master decision software program 72 is preferably done for reasons of security, fault isolation and bandwidth isolation.
  • the communication channel 106 is a serial Controller Area Network (CAN) bus operating at a bandwidth of 1 Mb/s within the cabinet 106 , with a secondary backbone also operating at a bandwidth 1 Mb/s between different cabinets 106 .
  • CAN Serial Controller Area Network
  • IP Internet Protocol
  • a separate intranet with communications using Internet Protocol (IP) protocol could be used for the communication channel 106 instead of a serial management interface such as the CAN bus, although such an embodiment would effectively be over designed for the level and complexity of communications that are required of the communication channel 106 connected to the host boards 104 . While it would be possible to implement the communication channel 106 as part of the intranet 54 , such an implementation is not preferred because of reasons of security, fault isolation and bandwidth isolation.
  • FIG. 8 shows a block diagram of the hierarchical relation of one embodiment of the various data and software layers utilized by the present invention for a given customer account.
  • Customer data and databases 60 form the base layer of this hierarchy.
  • a web data management software layer 62 may be incorporated to manage the customer data 60 across multiple instances of storage units that comprise the storage system 50 .
  • Cluster and/or load-balancing aware application software 64 comprises the top layer of what is conventionally thought of as the software and data for the customer's website.
  • Load-balancing software 66 groups multiple servers 46 together as part of the common administrative group 52 . Multiple instances of conventional operating system software 68 are present, one for each server 46 .
  • the load-balancing software 66 and operating system software 68 may be integrated as part of a common software package within a single administrative group 52 .
  • the conventional operating system software 68 is the engine operating software 48 of the present invention that manages resources across multiple customer accounts 52 - a and 52 - b.
  • the servers 46 assigned to the first administrative group 52 - a are located at a first site 80 and the servers 46 assigned to the second administrative group 52 - b are located at a second site 82 geographically remote from the first site 80 .
  • the system further includes an arrangement for automatically replicating at least data for the first administrative group 52 - a to the second site 82 .
  • a communication channel 84 separate from the network switches 44 is used to replicate data from the disk storage units 50 - a at the first site 80 to the disk storage units 50 - b at the second site 82 . The purpose of this arrangement is two-fold.
  • replication of the data provides redundancy and backup protection that allows for disaster recovery in the event of a disaster at the first site 80 .
  • replication of the data at the second site 82 allows the present invention to include the servers 46 located in the second site 82 in the pool of available servers which the master decision software program 72 may use to satisfy increased demand for the hosted services of the first customer by dynamically reallocating these servers to the first administrative group 52 - a.
  • the coordination between master decision software programs 72 at the first site 80 and second site 82 is preferably accomplished by the use of a global decision software routine 86 that communicates with the master decision software program 72 at each site.
  • This modular arrangement allows the master decision software programs 72 to focus on managing the server resources at a given site and extends the concept of having each site 80 , 82 request additional off-site services from the global decision software routine 86 or offer to make available off-site services in much the same way that the local decision software programs 70 make requests for additional servers or make servers available for reallocation to the master decision software program 70 at a given site.
  • the multi-site embodiment of the present invention utilizes commercially available SAN or NAS storage networking software to implement a two-tiered data redundancy and replication hierarchy.
  • the working version 74 of the customer data for the first customer account customer is maintained on the disk storage unit 50 at the first site 80 .
  • Redundancy data protection such as data mirroring, data shadowing or RAID data protection is used to establish a backup version 76 of the customer data for the first customer account at the first site 80 .
  • the networking software utilizes the communication channel 84 to generate a second backup version 78 of the customer data for the first customer account located at the second site 82 .
  • a communication channel 84 that is separate from the connection of the networks switches 44 to the Internet 22 preferably allows for redundant communication paths and minimizes the impact of the background communication activity necessary to generate the second backup version 78 .
  • the backup version 78 of the customer data for the first customer account located at the second site 82 could be routed through the network switches 44 and the Internet 22 .
  • additional backup versions of the customer data could be replicated at additional site locations to further expand the capability of the system to dynamically reallocate servers from customer accounts that are underutilizing these resources to customer accounts in need of these resources.
  • the ability of the present invention to dynamically reallocate servers from customer accounts that are underutilizing these resources to customer accounts in need of these resources allows for the resources of the server farm 40 to be used more efficiently in providing hosted services to multiple customer accounts.
  • the overall allocation of servers 46 to each customer account is accomplished such that a relatively constant marginal overcapacity bandwidth is maintained for each customer account.
  • the present invention allows for up-to-the-minute changes in server resources that are dynamically allocated on an as needed basis.
  • the server farm 40 also shows the advantages of utilizing multiple geographically distinct sites for locating portions of the server farm 40 . It can be seen that the peak usages for customer accounts 94 and 95 are time shifted from those of the other customer accounts 91 , 92 and 93 due to the difference in time zones between site location 80 and site location 82 . The present invention can take advantage of these time shifted differences in peak usages to allocate rolling server capacity to site locations during a time period of peak usage from other site locations which are experiencing a lull in activity.
  • At least three separate three separate site locations 80 , 82 and 84 are preferably situated geographically at least 24 divided by N+1 hours apart from each other, where N represents the number of distinct site locations in the multi-site configuration.
  • the site locations are preferably eight hours apart from each other. The time difference realized by this geographic separation allows for the usage patterns of customer accounts located at all three sites to be aggregated and serviced by a combined number of servers that is significantly less than would otherwise be required if each of the servers at a given location were not able to utilize servers dynamically reallocated from one or more of the other locations.
  • a site location is arranged so as to look to borrow capacity first from a site location that is at a later time zone (i.e., to the east of that site) and will look to make extra capacity available to site locations that are at an earlier time zone (i.e., to the west of that site). Other preferences can also be established depending upon past usage and predicted patterns of use.
  • the master decision software program 72 includes a resource database 150 , a service level agreement database 152 , a master decision logic module 154 and a dispatch module 156 .
  • the master decision logic module 154 has access to the resource database 150 and the service level agreement database 152 and compares the status information to information in the resource database 150 and the service level agreement database 152 to determine whether to dynamically reallocate servers from the first customer account to the second customer account.
  • the dispatch module 156 is operably linked to the master decision logic module 154 to dynamically reallocate servers when directed by the master decision logic module 154 by using the communication channel 106 to set initialization pointers for the reallocated servers 46 ′ to access software and data unique to the customer account for the second administrative group 52 - b and reinitializing the reallocated server 46 ′ such that at least one server joins the second administrative group 52 - b .
  • the dispatch module 156 includes a set of connectivity rules 160 and a set of personality modules 162 for each server 46 .
  • the connectivity rules 160 providing instructions for connecting a particular server 46 to a given network switch 44 or data storage unit 50 .
  • the personality module 162 describes the details of the particular software configuration of the server board 102 to be added to an administrative work group for a customer account. Once the dispatch module 146 has determined the need to reallocate a server, it will evaluate the set of connectivity rules 160 and a set of personality modules 162 to determine how to construct a server 46 that will be dispatched to that particular administrative group 52 .
  • Another way of looking at how the present invention can dynamically provide hosted service across disparate accounts is to view a portion of the servers 46 as being assigned to a pool of a plurality of virtual servers that may be selectively configured to access software and data for a particular administrative group 52 .
  • the dispatch module 146 When the dispatch module 146 has determined a need to add a server 46 to a particular administrative group 52 , it automatically allocates one of the servers from the pool of virtual servers to that administrative group. Conversely, if the dispatch module determines that an administrative group can relinquish one of its servers 46 , that relinquished server would be added to the pool of virtual servers that are available for reallocation to a different administrative group.
  • the group manager software 48 operates to “manufacture” or create one or more virtual servers out of this pool of the plurality of virtual servers on a just-in-time or as-needed basis.
  • the pool of virtual servers can either be a warm pool or a cold pool, or any combination thereof.
  • the virtual server is manufactured or constructed to be utilized by the desired administrative group in accordance with the set of connectivity rules 160 and personality modules 162 .
  • the master decision logic module 152 is operably connected to a management console 158 that can display information about the master decision software program and accept account maintenance and update information to processes into the various databases.
  • a billing software module 160 is integrated into the engine group manager 48 in order to keep track of the billing based on the allocation of servers to a given customer account.
  • a customer account is billed at a higher rate for the hosted services when servers are dynamically reallocated to that customer account based on the customer's service level agreement.
  • FIG. 12 shows a representation of three different service level agreement arrangements for a given customer account.
  • the service level agreements are made for providing hosted services for a given period of time, such as a month.
  • a first level shown at 170 the customer account is provided with the capacity to support hosted services for 640,000 simultaneous connections. If the customer account did not need a reallocation of servers to support capacity greater than the committed capacity for the first level 170 , the customer would be charged to establish rate for that level of committed capacity.
  • customer account can be dynamically expanded to support capacity of double the capacity at the first level 172 .
  • the customer account would be charged a higher rate for the period of time that the additional usage was required.
  • the customer account could be charged a one-time fee for initiating the higher level of service represented by the second level 172 .
  • charges for the second level 172 of service would be incurred at a rate that is some additional multiple of the rate charged for the first level 170 .
  • the second level 172 represents a guaranteed expansion level available to the customer for the given period of time.
  • a third level 174 provides an optional extended additional level of service that may be able to be brought to bare to provide hosted services for the customer account.
  • the third level 174 provides up to a higher multiple times the level of service as the first level 170 .
  • the host system makes use of the multi-site arrangement as previously described in order to bring in the required number of servers to meet this level of service.
  • the customer account is charged a second higher rate for the period of time that the extended additional service is reallocated to this customer account.
  • charges for the third level 174 of service would be incurred at a rate that is an even larger multiple of the first level 170 for the given period of time that the extended additional third level 174 of service is provided for this customer account.
  • the customer account may be charged a one-time fee for initiating this third level 174 of service at any time during the given period.
  • the customer may alter the level of service contracted for the given customer account.
  • the service level agreement is increased by 50% from a first period to a second period in response to a higher anticipated peak usage for the given customer account.
  • the period for a service level agreement for a given customer account would be a monthly basis, with suggestions been presented to the customer for recommended changes to the service level agreement for the upcoming billing period.
  • this example is demonstrated in terms of simultaneous connections, it should be understood that the service level agreement for a given customer account can be generated in terms of a variety of performance measurements, such as simultaneous connections, hits, amount of data transferred, number of transactions, connect time, resources utilized by different application software programs, the revenue generated, or any combination thereof.
  • the service level agreement may provide for different levels of commitment for different types of resources, such as front-end servers, back-end servers, network connections or disk storage units.
  • a series of measurement modules 180 , 181 , 182 , 183 and 184 each performed independent evaluations of the operation of the particular server on which the local decision software program 70 is executing. Outputs from these measurement modules are provided to an aggregator module 190 of the local decision software program 70 .
  • a predictor module 192 generates expected response times and probabilities for various requests.
  • a fuzzy inference system 196 determines whether a request to add an engine blade 104 for the administrative group 52 will be made, or whether an offer to give up or remove an engine blade from the administrative group 52 will be made.
  • the request to add or remove a blade is then communicated over communication channel 106 to the master decision software program 72 .
  • the aggregator module 190 is executed on each server 46 within a given administrative group 52
  • the predictor module 192 and fuzzy inference module 196 are executed on only a single server 46 within the given administrative group 52 with the outputs of the various measurement modules 180 - 184 been communicated to the designated server 46 across the communication channel 106 .
  • the aggregator module 190 , predictor module 192 and fuzzy inference module 196 may be executed on more than one server within a given administrative group for purposes of redundancy or distributed processing of the information necessary to generate the request add or remove a blade.
  • T ki is the time take it for the ith request of measurement type k
  • W k is the window size for measurement type k
  • mink is the minimum time expected for measurement type k
  • max k is the maximum time to be tolerated for a measurement type k.
  • the balanced request rate B k is then passed to the predictor module 192 and the fuzzy inference module 196 of the local decision software program 70 .
  • the window size for the measurement type k would be set to minimize any unnecessary intrusion by the measurement modules 180 - 184 , while at the same time allowing for a timely and adequate response to increases in usage demand for the administrative group 52 .
  • FIG. 16 shows a sample of the workload measurements from the various measurement modules 180 - 184 under varying load conditions. It can be seen that no single workload measurement provides a constantly predictable estimate of the expected response time and probability for that response time. As such, the fuzzy inference module 196 must consider three fundamental parameters: the predicted response times for various requests, the priority of these requests, and probability of their occurrence. The fuzzy inference module 196 blends all three of these considerations to make a determination as to whether to request a blade to be added or removed from the administrative group 52 . An example of a fuzzy inference rule would be:
  • the end results of the fuzzy inference module 196 is to generate a decision surface contouring the need to request an additional server over the grid of the expected response time vs. the probability of that response time for this administrative group 52 .
  • An example of such a decision surface is shown in FIG. 17 .

Abstract

A hosted service provider for the Internet is operated so as to provide dynamic management of hosted services across disparate customer accounts and/or geographically distinct sites.

Description

CLAIM TO PRIORITY
The present application is a continuation application of U.S. patent application Ser. No. 09/710,095, filed Nov. 10, 2000, now U.S. Pat. No. 6,816,905 entitled, “Method and System for Providing Dynamic Hosted Service Management Across Disparate Account/Sites,” the contents of which is hereby incorporated by reference in its entirety.
FIELD OF THE INVENTION
The present invention relates generally to the field of data processing business practices. More specifically, the present invention relates to a method and system for providing dynamic management of hosted services across disparate customer accounts and/or geographically distinct sites.
BACKGROUND OF THE INVENTION
The explosive growth of the Internet has been driven to a large extent by the emergence of commercial service providers and hosting facilities, such as Internet Service Providers (ISPs), Application Service Providers (ASPs), Independent Software Vendors (ISVs), Enterprise Solution Providers (ESPs), Managed Service Providers (MSPs) and the like. Although there is no clear definition of the precise set of services provided by each of these businesses, generally these service providers and hosting facilities provide services tailored to meet some, most or all of a customer's needs with respect to application hosting, site development, e-commerce management and server deployment in exchange for payment of setup charges and periodic fees. In the context of server deployment, for example, the fees are customarily based on the particular hardware and software configurations that a customer will specify for hosting the customer's application or website. For purposes of this invention, the term “hosted services” is intended to encompass the various types of these services provided by this spectrum of service providers and hosting facilities. For convenience, this group of service providers and hosting facilities shall be referred to collectively as Hosted Service Providers (HSPs).
Commercial HSPs provide users with access to hosted applications on the Internet in the same way that telephone companies provide customers with connections to their intended caller through the international telephone network. The computer equipment that HSPs use to host the applications and services they provide is commonly referred to as a server. In its simplest form, a server can be a personal computer that is connected to the Internet through a network interface and that runs specific software designed to service the requests made by customers or clients of that server. For all of the various delivery models that can be used by HSPs to provide hosted services, most HSPs will use a collection of servers that are connected to an internal network in what is commonly referred to as a “server farm,” with each server performing unique tasks or the group of servers sharing the load of multiple tasks, such as mail server, web server, access server, accounting and management server. In the context of hosting websites, for example, customers with smaller websites are often aggregated onto and supported by a single web server. Larger websites, however, are commonly hosted on dedicated web servers that provide services solely for that site. For general background on the Internet and HSPs, refer to Geoff Huston, ISP Survival Guide: Strategies For Running A Competitive ISP, (1999).
As the demand for Internet services has increased, there has been a need for ever-larger capacity to meet this demand. One solution has been to utilize more powerful computer systems as servers. Large mainframe and midsize computer systems have been used as servers to service large websites and corporate networks. Most HSPs tend not to utilize these larger computer systems because of the expense, complexity, and lack of flexibility of such systems. Instead, HSPs have preferred to utilize server farms consisting of large numbers of individual personal computer servers wired to a common Internet connection or bank of modems and sometimes accessing a common set of disk drives. When an HSP adds a new hosted service customer, for example, one or more personal computer servers are manually added to the HSP server farm and loaded with the appropriate software and data (e.g., web content) for that customer. In this way, the HSP deploys only that level of hardware required to support its current customer level. Equally as important, the HSP can charge its customers an upfront setup fee that covers a significant portion of the cost of this hardware. By utilizing this approach, the HSP does not have to spend money in advance for large computer systems with idle capacity that will not generate immediate revenue for the HSP. The server farm solution also affords an easier solution to the problem of maintaining security and data integrity across different customers than if those customers were all being serviced from a single larger mainframe computer. If all of the servers for a customer are loaded only with the software for that customer and are connected only to the data for that customer, security of that customer's information is insured by physical isolation.
For HSPs, numerous software billing packages are available to account and charge for these metered services, such as XaCCT from rens.com and HSP Power from inovaware.com. Other software programs have been developed to aid in the management of HSP networks, such as IP Magic from lightspeedsystems.com, Internet Services Management from resonate.com and MAMBA from luminate.com. The management and operation of an HSP has also been the subject of articles and seminars, such as Hursti, Jani, “Management of the Access Network and Service Provisioning,” Seminar in Internetworking, Apr. 19, 1999. An example of a typical HSP offering various configurations of hardware, software, maintenance and support for providing commercial levels of Internet access and website hosting at a monthly rate can be found at rackspace.com.
Up to now, there have been two approaches with respect to the way in which HSPs built their server farms. One approach is to use an homogenous group of personal computer systems (hardware and software) supplied from a single manufacturer. The other approach is to use personal computer systems supplied from a number of different manufacturers. The homogeneous approach affords the HSP advantages in terms of only having to support a single server platform, but at the same time it restricts the HSP to this single server platform. The heterogeneous approach using systems supplied from different manufacturers is more flexible and affords the HSP the advantage of utilizing the most appropriate server hardware and software platform for a given customer or task, but this flexibility comes at the cost of increased complexity and support challenges associated with maintaining multiple server platforms.
Regardless of which approach is used to populate a server farm, the actual physical management of such server farms remains generally the same. When a customer wants to increase or decrease the amount of services being provided for their account, the HSP will manually add or remove a server to or from that portion of the HSP server farm that is directly cabled to the data storage and network interconnect of that client's website. In the case where services are to be added, the typical process would be some variation of the following: (a) an order to change service level is received from a hosted service customer, (b) the HSP obtains new server hardware to meet the requested change, (c) personnel for the HSP physically install the new server hardware at the site where the server farm is located, (d) cabling for the new server hardware is added to the data storage and network connections for that site, (e) software for the server hardware is loaded onto the server and personnel for the HSP go through a series of initialization steps to configure the software specifically to the requirements of this customer account, and (f) the newly installed and fully configured server joins the existing administrative group of servers providing hosted service for the customer's account. In either case, each server farm is assigned to a specific customer and must be configured to meet the maximum projected demand for services from that customer account.
Originally, it was necessary to reboot or restart some or all of the existing servers in an administrative group for a given customer account in order to allow the last step of this process to be completed because pointers and tables in the existing servers would need to be manually updated to reflect the addition of a new server to the administrative group. This requirement dictated that changes in server hardware could only happen periodically in well-defined service windows, such as late on a Sunday night. More recently, software, such as Microsoft Windows 2000®, Microsoft® Cluster Server, Oracles® Parallel Server, Windows® Network Load Balancing Service (NLB), and similar programs have been developed and extended to automatically allow a new server to join an existing administrative group at any time rather than in these well-defined windows.
An example of how a new server can automatically join an existing administrative group is described in U.S. Pat. No. 5,951,694. In this patent, all of the servers in an administrative group are represented in a mapping table maintained by a gateway server. The mapping table identifies different service groups for the administrative group, such as mail service group, database service group, access server group, etc. The gateway server routes requests for the administrative group to the appropriate service group based on the mapping table. A new server may be added to one of the service groups by loading the appropriate software component on that server, after which the gateway server will recognize the new server and add it to the mapping table and bring the new server up to speed with the rest of the servers in that service group using a transaction log maintained for each service group. Alternatively, if one service group is experiencing a heavy workload and another service group is lightly loaded, it is possible to switch a server from one service group to another. The patent describes a software routine executing on a dedicated administrative server that uses a load balancing scheme to modify the mapping table to ensure that requests for that administrative group are more evenly balanced among the various service groups that make up the administrative group.
Numerous patents have described techniques for workload balancing among servers in a single cluster or administrative groups. U.S. Pat. No. 6,006,529 describes software clustering that includes security and heartbeat arrangement under control of a master server, where all of the cluster members are assigned a common IP address and load balancing is preformed within that cluster. U.S. Pat. Nos. 5,537,542, 5,948,065 and 5,974,462 describe various workload-balancing arrangements for a multi-system computer processing system having a shared data space. The distribution of work among servers can also be accomplished by interposing an intermediary system between the clients and servers. U.S. Pat. No. 6,097,882 describes a replicator system interposed between clients and servers to transparently redirect IP packets between the two based on server availability and workload.
Various techniques have also been used to coordinate the operation of multiple computers or servers in a single cluster. U.S. Pat. No. 6,014,669 describes cluster operation of multiple servers in a single cluster by using a lock-step distributed configuration file. U.S. Pat. No. 6,088,727 describes cluster control in a shared data space multi-computer environment. Other patents have described how a single image of the input/output space can be used to coordinate multiple computers. U.S. Pat. No. 5,832,222 describes how a single image of the input/output space can be used to coordinate geographically dispersed computer systems. U.S. Pat. No. 6,067,545 describes a distributed file system with shared metadata management, replicated configuration database and domain load balancing, that allows for servers to fall into and out of a single domain under control of the configuration database.
While these approaches have improved the management of servers within administrative groups, domains or shared data spaces, there is no capability to extend these techniques beyond the group of servers defined for and linked to a common operating system or common shared data space. Generally, this limitation has not been considered a problem because all of these approaches are directed to larger enterprise computing systems that are managed and implemented within the computer network of a single company. Even though these approaches can be put into use by an HSP to manage the servers assigned to a particular account for a given client or customer, none of these approaches allow an HSP to manage a set of servers providing hosted services to multiple accounts for different clients or customers.
Systems for managing the operation of larger enterprise computing systems also have been developed, such as OpenView from Hewlett-Packard®, Unicenter TNG® from Computer Associates, Tivoli® from IBM, Mamba from Luminate, and Patrol from BMC Software, Inc. Generally, these systems are focused on inventory management and software deployment control issues encountered with very large numbers of computers operating within a single company or organization. Some of these operation management systems include performance monitoring solutions that query the performance of servers within the organization over the network to determine the need for additional resources or load redistribution. A similar over-the-network approach is also used to provide centralized reporting and management features. A good example of this type of operation management system that is intended to be used by HSPs is the Tivoli Service Delivery Management platform that consists of a user administration module, a software distribution module, an inventory module, an enterprise console, a security module, an enterprise manager module that provides a customizable view of all of the components in a network once they are added to the network, and a workload scheduler that allows workload to be balanced among servers sharing a common data space. All of these modules operate using an over-the-network communication scheme involving agents on the various nodes in the network that collect and report status and incident information to the other modules. Once the hardware components for a new node are physically added to the network, the various modules of the Tivoli Service Delivery Management platform can take over and manage those components on a more automatic basis. However, the process of physically adding hardware for a new node into the network remains essentially a manual process that is accomplished in the same manner as previously described.
In terms of managing the physical hardware that makes up the computer system, various approaches have been developed to automatically compensate for the failure of a hardware component within a computer network. U.S. Pat. No. 5,615,329 describes a typical example of a redundant hardware arrangement that implements remote data shadowing using dedicated separate primary and secondary computer systems where the secondary computer system takes over for the primary computer system in the event of a failure of the primary computer system. The problem with these types of mirroring or shadowing arrangements is that they can be expensive and wasteful, particularly where the secondary computer system is idled in a standby mode waiting for a failure of the primary computer system. U.S. Pat. No. 5,696,895 describes one solution to this problem in which a series of servers each run their own tasks, but each is also assigned to act as a backup to one of the other servers in the event that server has a failure. This arrangement allows the tasks being performed by both servers to continue on the backup server, although performance will be degraded. Other examples of this type of solution include the Epoch Point of Distribution (POD) server design and the USI Complex Web Service. The hardware components used to provide these services are predefined computing pods that include load-balancing software, which can also compensate for the failure of a hardware component within an administrative group. Even with the use of such predefined computing pods, the physical preparation and installation of such pods into an administrative group can take up to a week to accomplish.
All of these solutions can work to automatically manage and balance workloads and route around hardware failures within an administrative group based on an existing hardware computing capacity; however, few solutions have been developed that allow for the automatic deployment of additional hardware resources to an administrative group. If the potential need for additional hardware resources within an administrative group is known in advance, the most common solution is to preconfigure the hardware resources for an administrative group based on the highest predicted need for resources for that group. While this solution allows the administrative group to respond appropriately during times of peak demand, the extra hardware resources allocated to meet this peak demand are underutilized at most other times. As a result, the cost of providing hosted services for the administrative group is increased due to the underutilization of hardware resources for this group.
One solution to the need for additional hosted services is the Internet Shock Absorber (ISA) service offered by Cable & Wireless. The ISA service distributes a customer's static Web content to one or more caching servers located at various Points of Presence (POPs) on the Cable & Wireless Internet backbone. Requests for this static Web content can be directed to be caching servers and the various POP locations to offload this function from the servers in the administrative group providing hosted service for that customer. The caching of static Web content, however, is something that occurs naturally as part of the distribution of information over the Internet. Where a large number of users are requesting static information from a given IP address, it is common to cache this information at multiple locations on the Internet. In essence, the ISA service allows a customer to proactively initiate the caching of static Web content on the Internet. While this solution has the potential to improve performance for delivery of static Web content, this solution is not applicable to the numerous other types of hosted services that involve interactive or dynamic information content.
Although significant enhancements have been made to the way that HSPs are managed, and although many programs and tools have been developed to aid in the operation of HSP networks, the basic techniques used by HSPs to create and maintain the physical resources of a server farm have changed very little. It would be desirable to provide a more efficient way of operating an HSP that could improve on the way in which physical resources of the server farm are managed.
SUMMARY OF THE INVENTION
The present invention is a method and system for operating a hosted service provider for the Internet in such a way as to provide dynamic management of hosted services across disparate customer accounts and/or geographically distinct sites. For each of a plurality of customer accounts, a plurality of individual servers are allocated to a common administrative group defined for that customer account. Each administrative group is configured to access software and data unique to that customer account for providing hosted services to the Internet for that customer account. The system automatically monitors the performance and health of the servers in each administrative group. At least one server from a first administrative group is automatically and dynamically reallocated to a second administrative group in response to the automatic monitoring. The automatic and dynamic reallocation of servers is accomplished by setting initialization pointers for the reallocated servers to access software and data unique to the customer account for the second administrative group, and then reinitializing the reallocated servers such that they join the second administrative group when restarted. Preferably, the performance and health of the servers in each administrative group are monitored over a separate out-of-band communication channel dedicated to interconnecting the servers across administrative groups. Each administrative group includes a local decision software program that communicates with a master decision software program that determines when and how to dynamically reallocate servers to different administrative groups in response to usage demands, available resources and service level agreements with each customer account.
In one embodiment, a system for providing the dynamic management of hosted services for multiple customer accounts includes at least five servers operably connected to an intranet. Each server includes host management circuitry providing a communication channel with at least one of the other servers that is separate from this intranet. At least four of the servers execute a local decision software program that monitors the server and communicates status information across the communication channel. At least two of the servers are allocated to a first administrative group for a first customer account and configured to access software and data unique to this first customer account, such that hosted services are provided via the Internet for this customer account. At least two of the other servers are allocated to a second administrative group for a second customer account and configured to access software and data unique to this second customer account, such that hosted services are provided via the Internet for this customer account. Preferably, at least one of the servers executes a master decision software program that collects status information from the other servers and dynamically reallocates at least one server from the first administrative group to the second administrative group in response to at least the status information.
Unlike existing load balancing systems that are limited to working within the context of a single customer account or that require large and expensive computer systems and common operating systems or shared data spaces, the present invention is capable of dynamically reallocating servers across multiple disparate customer accounts to provide hosted services with a more economical and flexible server farm arrangement. The ability of the present invention to support multiple administrative groups for multiple customers allows for an intelligent and dynamic allocation of server resources among different customer accounts.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a simplified block diagram of a prior art arrangement of a server farm for a hosted service provider.
FIG. 2 is a graphic representation of Internet traffic in relation to server capacity for a prior art server farm hosting multiple customer accounts.
FIG. 3 is a simplified block diagram of the arrangement of a server farm in accordance with the present invention.
FIG. 4 is a simplified block diagram similar to FIG. 3 showing the dynamic reallocation of servers from a first customer account to a second customer account to address a hardware failure.
FIG. 5 is a simplified block diagram similar to FIG. 3 showing the dynamic reallocation of servers from a first customer account to a second customer account to address an increased usage demand.
FIG. 6 is a block diagram of a preferred embodiment of the components of a server farm in accordance with the present invention.
FIG. 7 is an exploded perspective view of a preferred embodiment of the hardware for the server farm in accordance with the present invention.
FIG. 8 is a block diagram showing the hierarchical relation of the various software layers utilized by the present invention for a given customer account.
FIG. 9 is a block diagram of an embodiment of the present invention implemented across geographically disparate sites.
FIG. 10 is a graphic representation of Internet traffic in relation to server capacity for the server farm of the present invention when hosting multiple customer accounts.
FIG. 11 is a block diagram showing a preferred embodiment of the master decision software program of the present invention.
FIG. 12 is a graphic representation of three different service level agreement arrangements for a given customer account.
FIG. 13 is a graphic representation of Internet traffic in relation to server capacity for a multi-site embodiment of the present invention.
FIG. 14 is a block diagram showing the master decision software program controlling the network switch and storage unit connections.
FIG. 15 is a block diagram of the preferred embodiment of the local decision software program.
FIG. 16 is a graphic representation of the workload measurements from the various measurement modules of the local decision software program under varying load conditions.
FIG. 17 is a graphic representation of a decision surface generated by the local decision software program to request or remove a server from an administrative group.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
Referring to FIG. 1, a simplified functional view of an existing server farm 20 for a hosted service provider is shown. Such server farms are normally constructed using off-the-shelf hardware and software components statically configured to support the hosted service requirements of a given customer account. In this embodiment, the server farm 20 for the hosted server provider is supporting hosted services for four different customer accounts. The server farm 20 is connected to the Internet 22 by network switches/routers 24. The network switches 24 are in turn connected to internal network switches/routers 26 that form an intranet among the front-end/content servers 28 and back-end/compute servers 30 for a given customer account. All front-end/content servers 28 and back-end/compute servers 30 are connected to disk systems 32 containing data and software unique to that customer account. Depending upon the physical nature of the hardware for the servers 28, 30, the disk systems 32 may be included within the server housing, or the disk systems 32 may be housed in physically separate units directly connected to each of the servers 28, 30 or attached to more than one server 28, 30 as a storage attached network (SAN) or network attached storage (NAS) configuration.
While this arrangement makes good use of off-the-shelf hardware to construct a server farm 20 that can provide hosted services for multiple independent customer accounts, there are several significant issues exposed in this type of an arrangement. The most significant of these is the generally static nature of the allocation and deployment of system resources among different customer accounts. In order to configure and manage a single customer account within this complex, an administrator for the HSP needs to dedicate some fixed level of system resources (e.g., servers, disks, network links) to the particular customer account based on projected requirements of that customer's needs.
For example, assume a relatively simple website has been designed for any given customer account such that under a projected peak load the customer account may require three front-end servers 28 to handle user requests and a quad processor back-end server 30 to handle database queries/updates generated by these requests. For this type of website, it is likely that hardware-based technology such as f5® Big-IP®, Cisco Local Director, or Foundry® ServerIron™, or a software-based solution such as Windows® Load Balance Service (WLBS) or equivalent will be used to distribute the user requests evenly across the front-end/content servers 28. In addition, the back-end database/compute server 30 will commonly be clustered to provide some level of fault tolerance. There are a number of software products available, such as Microsoft® Cluster Server, Oracle® Parallel Server, etc., that allow websites with multiple servers to ride through hardware failures that might occur during normal operation. In addition, system monitoring tools such as Tivoli® Enterprise, HP® OpenView, etc., allow administrators to be notified when failures are detected within the server farm 20. Although these tools can be adequate for managing the hosted services within a single customer account at a given site, none of these tools allow for the management of hosted services across disparate customer accounts.
In the context of this example, assume that the website for this customer account is an e-commerce site designed to handle a peak load of 5000 transactions per minute. Further, assume that the websites for the remaining customer accounts in the server farm 20 have been designed to handle peak loads of 10,000, 15,000 and 5,000 transactions per minute, respectively. As shown in FIG. 2, having to design and configure each customer account to handle an anticipated peak load likely results in significant wasted capacity within the overall server farm 20. Even though the server farm 20 handling multiple customer accounts may have excess aggregate capacity, this extra capacity cannot be used to respond to hardware failures or unexpected increases in peak load from one account to the next. Resources configured for a particular customer account are dedicated to that account and to that account only. In the event that one of the front-end servers 28 for a first customer account experiences a hardware failure, Web traffic will be routed to the remaining front-end servers 28. If the customer account was busy before the hardware failure and Web traffic remains constant or increases after the failure, the remaining front-end servers 28 will quickly become overloaded by servicing their previous workload as well as the additional traffic redirected from the failed server. In a best case scenario, the system management software for the server farm 20 would notice that a server had failed and send a message to a site manager (via pager and/or e-mail) indicating the server failure. If the site manager receives the message in a timely manner and is located on-site, the site manager can physically remove the failed hardware component, install a spare hardware component that has hopefully been stockpiled for this purpose, recable the new hardware component, configure and install the appropriate software for that customer account, and allow the new hardware component to rejoin the remaining front-end servers 28. Hopefully, this process could be accomplished in less than an hour. If the message is not received in a timely manner, if the site manager is not located at the site where the server farm is located, or if there is no stockpiled spare hardware available to replace the failed unit, this process will take even longer. In the meantime, response times for users accessing the customer account are degraded and the customer account becomes increasingly vulnerable to another hardware failure during this period.
In the event that the customer account experiences an increase in demand above the anticipated peak demand for which that customer account has been configured, there are no resources available to the load balancing facilities for redistributing this increased Web traffic. All of the servers 28, 30 would be operating at peak capacity. The result is significantly degraded response times for the customer account and a possibility of “service unavailable” responses for requests that cannot be handled in a timely manner. While the inability to provide services to consumers in a timely manner is an undesirable, but perhaps manageable, problem for a business in other contexts, the additional problem of generating “service unavailable” messages for a website is that, if such messages continue to persist for whatever reason, the Internet may begin to propagate this information to numerous intermediary nodes in the network. As a result, these intermediary nodes will divert subsequent requests to the website due to their understanding that the website is “unavailable.” Not only are the customers who receive the “service unavailable” message not serviced, but many other customers may never even get to the website once the customer account becomes saturated or overloaded.
Referring now to FIG. 3, a server farm 40 for providing dynamic management of hosted services to multiple customer accounts will be described. As with existing server farms 20, the server farm 40 includes network switches 44 to establish interconnection between the server farm 40 and the Internet 22. Unlike existing server farm 20, however, a population of servers 46 are managed under control of an engine group manager 48. Each of the servers 46 is a stateless computing device that is programatically connected to the Internet via the network switches 44 and to a disk storage system 50. In one embodiment, the servers 46 are connected to the disk storage system 50 via a Fibre Channel storage area network (SAN). Alternatively, the servers 46 may be connected to the disk storage system 50 via a network attached storage (NAS) arrangement, a switchable crossbar arrangement or any similar interconnection technique.
As shown in FIGS. 4 and 5, the engine group manager 48 is responsible for automatically allocating the stateless servers 46 among multiple customer accounts and then configuring those servers for the allocated account. This is done by allocating the servers for a given customer account to a common administrative group 52 defined for that customer account and configured to access software and data unique to that customer account. As will be described, the engine group manager 48 automatically monitors each administrative group and automatically and dynamically reallocates servers 46′ from a first administrative group 52-a to a second administrative group 52-b in response to the automatic monitoring. This is accomplished by using the engine group manager 48 to set initialization pointers for the reallocated servers 46′ from the first administrative group 52-a to access software and data unique to the customer account for the second administrative group 52-b, and then reinitializing the reallocated servers 46′ such that reallocated servers 46′ join the second administrative group 52-b. Unlike the existing process for adding or removing hardware resources to a server farm 20, the present invention can make a reallocated server 46′ available to a new administrative group 52 in as little as a few minutes. Basically, the only significant time required to bring the reallocated server 46′ online will be the time required to reboot the server 46′ and any time required for the load-balancing and/or clustering software to recognize this rebooted server. It will be understood that load-balancing software is more typically found in connection with front-end/content servers, whereas clustering software or a combination of clustering software and load-balancing software are more typically used in connection with back-end/compute servers. The term load-balancing software will be used to refer to any of these possible combinations.
In one embodiment, the reallocated servers 46′ automatically join the second administrative group because the software for the second administrative group 52-b includes load-balancing software that will automatically add or remove a server from that administrative group in response to the server being brought online (i.e., reset and powered on) or brought off-line (i.e., reset and powered off). As previously described, this kind of load-balancing software is widely known and available today; however, existing load-balancing software is only capable of adding or removing servers from a single administrative group. In this embodiment, the engine group manager 48 takes advantage of capabilities of currently available commercial load-balancing application software to allow for the dynamic reallocation servers 46′ across different administrative groups 52. Alternatively, agents or subroutines within the operating system software for the single administrative group could be responsible for integrating a reallocated server 46′ into the second administrative group 52-b once the reallocated server 46′ is brought online. In still another embodiment, the engine group manager 48 could publish updates to a listing of available servers for each administrative group 52.
Preferably, the engine group manager 48 will set pointers in each of the servers 46 for an administrative group 52 to an appropriate copy of the boot image software and configuration files, including operating system an application programs, that had been established for that administrative group 52. When a reallocated server 46′ is rebooted, its pointers have been reset by the engine group manager 48 to point to the boot image software and configuration files for the second administrative group 52-b, instead of the boot image software and configuration files for the first administrative group 52-a.
In general, each administrative group 52 represents the website or similar hosted services being provided by the server farm 40 for a unique customer account. Although different customer accounts could be paid for by the same business or by a related commercial entity, it will be understood that the data and software associated with a given customer account, and therefore with a given administrative group 52, will be unique to that customer account. Unlike service providers which utilize large mainframe computer installations to provide hosted services to multiple customers by using a single common operating system to implement timesharing of the resources of the large mainframe computer system, each administrative group 52 consists of unique software, including conventional operating system software, that does not extend outside servers 46 which have been assigned to the administrative group 52. This distributed approach of the present invention allows for the use of simpler, conventional software applications and operating systems that can be installed on relatively inexpensive, individual servers. In this way, the individual elements that make up an administrative group 52 can be comprised of relatively inexpensive commercially available hardware servers and standard software programs.
FIGS. 6 and 7 show a preferred embodiment of the components and hardware for the server farm 40 in accordance with the present invention. Although the preferred embodiment of the present invention is described with respect to this hardware, it will be understood that the concept of the present invention is equally applicable to a server farm implemented using all conventional servers, including the currently available 1U or 2U packaged servers, if those servers are provided with the host management circuitry or its equivalent, as will be described.
Preferably, the hardware for the server farm 40 is a scalable engine 100 comprised of a large number of commercially available server boards 102 each arranged as an engine blade 132 in a power and space efficient cabinet 110. The engine blades 132 are removably positioned in a front side 112 of the cabinet 110 in a vertical orientation. A through plane 130 in the middle of the cabinet 110 provides common power and controls peripheral signals to all engine blades 132. I/O signals for each engine blade 132 are routed through apertures in the through plane 130 to interface cards 134 positioned in the rear of the cabinet 110. The I/O signals will be routed through an appropriate interface card 134 either to the Internet 22 via the network switch 44, or to the disk storage 50. Preferably, separate interface cards 134 are used for these different communication paths.
The scalable engine can accommodate different types of server boards 102 in the same cabinet 110 because of a common blade carrier structure 103. Different types of commercially available motherboards 102 are mounted in the common blade carrier structure 103 that provides a uniform mechanical interface to the cabinet 110. A specially designed PCI host board 104 that can plug into various types of motherboards 102 has connections routed through the through plane 130 for connecting to the interface cards 134. Redundant hot-swappable high-efficiency power supplies 144 are connected to the common power signals on the through plane 130. The host board 104 includes management circuitry that distributes the power signals to the server board 102 for that engine blade 132 by emulating the ATX power management protocol. Replaceable fan trays 140 are mounted below the engine blades 132 to cool the engine 100. Preferably, the cabinet 110 accommodates multiple rows of engine blades 132 in a chassis assembly 128 that includes a pair of sub-chassis 129 stacked on top of each other and positioned on top of a power frame 146 that holds the power supplies 144. Preferably, the cabinet 110 will also include rack mounted Ethernet networks switches 44 and 147 and storage switches 149 attached to disk drives 50 over a Fibre Channel network.
It will also be understood that while the present invention is described with respect to single cabinet 110 housing engine blades 132 with server boards 102 that together with the appropriate application software constitute the various servers 46 that are assigned to a first administrative group 52-a, and a second administrative group 52-b each having at least two engine blades 132, the server farm 40 can accommodate administrative groups 52 for any number of customers depending upon the total number of servers 46 in the server farm 40. Preferably, multiple cabinets 110 can be integrated together to scale the total number of servers 46 at a given location. As will be discussed, it is also possible to link multiple cabinets 110 in geographically disparate locations together as part of a single server farm 40 operating under control of the engine group manager 48.
In the preferred embodiment, the server boards 102 of each engine blade 132 can be populated with the most recent processors for Intel®, SPARC® or PowerPC™ designs, each of which can support standard operating system environments such as Windows® NT, Windows® 2000, Linux™ or Solaris. Each engine blade 132 can accommodate one or more server boards 102, and each server board may be either a single or multiprocessor design in accordance with the current ATX form factor or a new form factor that may be embraced by the industry in the future. Preferably, the communication channel 106 is implemented a Controller Area Network (CAN) bus that is separate from the communication paths for the network switch 44 or storage switches 149. Optionally, a second fault backup communication channel 106′ could be provided to allow for fault tolerance and redundant communication paths for the group manager software 48.
In a conventional server, the pointers and startup configuration information would be set by manual switches on the server board or hard-coded into PROM chipsets on the server board or stored at fixed locations on a local hard drive accessible by the server board. The management circuitry on the host board 104 is designed to have appropriate hooks into the server board 102 such that the pointers and other startup configuration information are actually supplied by the host management circuitry. Optionally, an engine blade 132 can include a local hard drive 107 that is accessed through the host board 104 such that information stored on that local hard drive 107 can be configured by the host board via the communication channel 106. Additionally, the host board 104 preferably includes power management circuitry 108 that enables the use of common power supplies for the cabinet 110 by emulating the ATX power management sequence to control the application of power to the server board 102. Preferably, a back channel Ethernet switch 147 also allows for communication of application and data information among the various server boards 102 within the server farm 40 without the need to route those communications out over the Internet 22.
In a preferred embodiment, each cabinet 110 can house up to 32 engine blades 132. In this configuration, the networks switches 44 and 147 could comprise two 32 circuit switched Ethernet network routers from Foundry®. Preferably, the networks switches 44 and 147 allow a reconfiguration of the connection between a server 46 and the networks switch 44 and 147 to be dynamically adjusted by changing the IP address for the server. With respect to the disk storage units 50, two options are available. First, unique hardware and software can be inserted in the form of a crossbar switch 149 between the engine blades 132 and the disk storage units 50 which would abstract way the details of the underlying SAN storage hardware configuration. In this case, the link between the disk storage units 50 and each blade 132 would be communicated to the crossbar switch 149 through set of software APIs. Alternatively, commercially available Fibre Channel switches or RAID storage boxes could be used to build connectivity dynamically between the blades 132 and disk storage units 50. In both alternatives, a layer of software inside the engine group manager 48 performs the necessary configuration adjustments to the connections between the server blades 132 and networks switches 147 and disk storage units 50 are accomplished. In another embodiment, a portion of the servers 46 could be permanently cabled to the network switches or disk storage units to decrease switch costs if, for example, the set of customer accounts supported by a given portion of the server farm 40 will always include a base number of servers 46 that cannot be reallocated. In this case, the base number of servers 46 for each administrative group 52 could be permanently cabled to the associated network switch 149 and disk storage unit 50 for that administrative group 52.
Referring again to FIGS. 4 and 5, it will be seen that the server farm system 40 of the present invention can dynamically manage hosted services provided to multiple customer accounts. It will be seen that there are at least five servers 46 operably connected to an intranet 54. Preferably, the intranet is formed over the same network switches 44 that interconnect the servers 46 with the Internet 22 or over similar network switches such as network switches 147 that interconnect the servers 46 to each other. Each server 46 has management circuitry on the host board 104 that provides a communication channel 106 with at least one of the other servers 46 that is separate from the intranet 54 created by the network switches 44 and/or 147.
At least four of the servers 46 are configured to execute a local decision software program 70 that monitors the server 46 and communicate status information across the communication channel 106. At least two of these servers 46 are allocated to a first administrative group 52-a for a first customer account and configured to access software and data unique to the first customer account to provide hosted services to the Internet for that customer account. At least another two of the servers 46 are allocated to a second administrative group 52-b for a second customer account and configured to access software and data unique to the second customer account to provide hosted services to the Internet for that customer account. At least one of the servers 46 executes a master decision software program 72 that collects status information from the local decision software programs 70 executing on the other servers 46. In one embodiment, a pair of servers 46 are slaved together using fault tolerant coordination software to form a fault tolerant/redundant processing platform for the master decision software program. As will be described, the master decision software program 72 dynamically reallocates at least one server 46′ from the first administrative group 52-a to the second administrative group 52-b in response to at least the status information collected from the local decision software programs 70.
The servers 46 for both administrative groups 52 can be arranged in any configuration specified for a given customer account. As shown in FIG. 3, three of the servers 46 for administrative group 52-b are configured as front-end servers with a single server 46 being configured as the back-end/compute server for this customer account. In response to a significant increase in the peak usage activity for the customer account for the second administrative group 52-b, the master decision software program 72 determines that is necessary to reallocate server 46′ from its current usage as a server for the first administrative group 52-a to being used as a back-end/compute server for the second administrative group 52-b. The preferred embodiment for how this decision is arrived will be described in connection with the description of the operation of the local decision software program 72. Following the procedure just described, the master decision software program 72 directs the dynamic reallocation of reallocated server 46′ to the second administrative group 52-b as shown in FIG. 4.
Although the preferred embodiment of present invention is described in terms of reallocation of a server 46′ from a first administrative group 52-a to a second administrative group 52-b, it should be understood that the present invention can also be implemented to provide for a common pool of available servers 46′ that are not currently assigned to a given administrative group 52 and may be reallocated without necessarily requiring that they be withdrawn from a working administrative group 52. For example, a server farm 40 having thirty-two servers 46 could be set up to allocate six servers to each of four different customer accounts, with one server 46 executing the master decision software program 72 and a remaining pool 56 of seven servers 46 that are initially unassigned and can be allocated to any of the four administrative groups 52 defined for that server farm. Because the assignment of servers to administrative groups is dynamic during the ongoing operation of the server farm 40 in accordance with the present invention, the preferred embodiment of the present invention uses this pool 56 as a buffer to further reduce the time required to bring a reallocated server 46′ into an administrative group 52 by eliminating the need to first remove the reallocated server 46′ from its existing administrative group 52. In one embodiment, the pool 56 can have both warm servers and cold servers. A warm server would be a server 46 that has already been configured for a particular administrative group 52 and therefore it is not necessary to reboot that warm server to allow it to join the administrative group. A cold server would be a server that is not configured to a particular administrative group 52 and therefore it will be necessary to reboot that cold server in order for it to join the administrative group.
It should also be understood that reallocated servers 46′ can be allocated to a new administrative group singly or as a group with more than one reallocated server 46′ being simultaneously reallocated from a first administrative group 52-a to a second administrative group 52-b. In the context of how the networks switches 44, 147 and storage switches 149 are configured to accommodate such dynamic reallocation, it should also be understood that multiple servers 46 may be reallocated together as a group if it is necessary or desirable to reduce the number of dynamically configurable ports on the networks 44, 147 and/or storage switches 149.
One of the significant advantages of the present invention is that the process of reconfiguring servers from one administrative group 52-a to a second administrative group 52-b will wipe clean all of the state associated with a particular customer account for the first administrative group from the reallocated server 46′ before that server is brought into service as part of the second administrative group 52-b. This provides a natural and very efficient security mechanism for precluding intentional or unintentional access to data between different customer accounts. Unless a server 46 or 46′ is a member of a given administrative group 52-a, there is no way for that server to have access to the data or information for a different administrative group 52-b. Instead of the complex and potentially problematic software security features that must be implemented in a mainframe server or other larger server system that utilizes a shard memory space and/or common operating system to provide hosted services across different customer accounts, the present invention keeps the advantages of the simple physical separation between customer accounts that is found in conventional server farm arrangements, but does this while still allowing hardware to be automatically and dynamically reconfigured in the event of a need or opportunity to make better usage of that hardware. The only point of access for authorization and control of this reconfiguration is via the master decision software program 72 over the out-of-band communication channel 106.
As shown in FIG. 14, preferably each server 46 is programmatically connected to the Internet 22 under control of the master decision software program 72. The master decision software program 72 also switches the reallocated server 46′ to be operably connected to a portion of the disk storage unit storing software and data unique to the customer account of the second administrative group. The use of an out-of-band communication channel 106 separate from the intranet 54 over the network switches 44 for communicating at least a portion of the status information utilized by the master decision software program 72 is preferably done for reasons of security, fault isolation and bandwidth isolation. In a preferred embodiment, the communication channel 106 is a serial Controller Area Network (CAN) bus operating at a bandwidth of 1 Mb/s within the cabinet 106, with a secondary backbone also operating at a bandwidth 1 Mb/s between different cabinets 106. It will be understood that a separate intranet with communications using Internet Protocol (IP) protocol could be used for the communication channel 106 instead of a serial management interface such as the CAN bus, although such an embodiment would effectively be over designed for the level and complexity of communications that are required of the communication channel 106 connected to the host boards 104. While it would be possible to implement the communication channel 106 as part of the intranet 54, such an implementation is not preferred because of reasons of security, fault isolation and bandwidth isolation.
FIG. 8 shows a block diagram of the hierarchical relation of one embodiment of the various data and software layers utilized by the present invention for a given customer account. Customer data and databases 60 form the base layer of this hierarchy. Optionally, a web data management software layer 62 may be incorporated to manage the customer data 60 across multiple instances of storage units that comprise the storage system 50. Cluster and/or load-balancing aware application software 64 comprises the top layer of what is conventionally thought of as the software and data for the customer's website. Load-balancing software 66 groups multiple servers 46 together as part of the common administrative group 52. Multiple instances of conventional operating system software 68 are present, one for each server 46. Alternatively, the load-balancing software 66 and operating system software 68 may be integrated as part of a common software package within a single administrative group 52. Above the conventional operating system software 68 is the engine operating software 48 of the present invention that manages resources across multiple customer accounts 52-a and 52-b.
In one embodiment of the present invention as shown in FIG. 9 the servers 46 assigned to the first administrative group 52-a are located at a first site 80 and the servers 46 assigned to the second administrative group 52-b are located at a second site 82 geographically remote from the first site 80. In this embodiment, the system further includes an arrangement for automatically replicating at least data for the first administrative group 52-a to the second site 82. In a preferred embodiment, a communication channel 84 separate from the network switches 44 is used to replicate data from the disk storage units 50-a at the first site 80 to the disk storage units 50-b at the second site 82. The purpose of this arrangement is two-fold. First, replication of the data provides redundancy and backup protection that allows for disaster recovery in the event of a disaster at the first site 80. Second, replication of the data at the second site 82 allows the present invention to include the servers 46 located in the second site 82 in the pool of available servers which the master decision software program 72 may use to satisfy increased demand for the hosted services of the first customer by dynamically reallocating these servers to the first administrative group 52-a.
The coordination between master decision software programs 72 at the first site 80 and second site 82 is preferably accomplished by the use of a global decision software routine 86 that communicates with the master decision software program 72 at each site. This modular arrangement allows the master decision software programs 72 to focus on managing the server resources at a given site and extends the concept of having each site 80, 82 request additional off-site services from the global decision software routine 86 or offer to make available off-site services in much the same way that the local decision software programs 70 make requests for additional servers or make servers available for reallocation to the master decision software program 70 at a given site.
Preferably, the multi-site embodiment of the present invention utilizes commercially available SAN or NAS storage networking software to implement a two-tiered data redundancy and replication hierarchy. As shown in FIG. 9, the working version 74 of the customer data for the first customer account customer is maintained on the disk storage unit 50 at the first site 80. Redundancy data protection, such as data mirroring, data shadowing or RAID data protection is used to establish a backup version 76 of the customer data for the first customer account at the first site 80. The networking software utilizes the communication channel 84 to generate a second backup version 78 of the customer data for the first customer account located at the second site 82. The use of a communication channel 84 that is separate from the connection of the networks switches 44 to the Internet 22 preferably allows for redundant communication paths and minimizes the impact of the background communication activity necessary to generate the second backup version 78. Alternatively, the backup version 78 of the customer data for the first customer account located at the second site 82 could be routed through the network switches 44 and the Internet 22. In another embodiment, additional backup versions of the customer data could be replicated at additional site locations to further expand the capability of the system to dynamically reallocate servers from customer accounts that are underutilizing these resources to customer accounts in need of these resources.
As shown in FIG. 10, the ability of the present invention to dynamically reallocate servers from customer accounts that are underutilizing these resources to customer accounts in need of these resources allows for the resources of the server farm 40 to be used more efficiently in providing hosted services to multiple customer accounts. For each of the customer accounts 91, 92, 93, 94 and 95, the overall allocation of servers 46 to each customer account is accomplished such that a relatively constant marginal overcapacity bandwidth is maintained for each customer account. Unlike existing server farms, where changes in hardware resources allocated to a given customer account happen in terms of hours, days or weeks, the present invention allows for up-to-the-minute changes in server resources that are dynamically allocated on an as needed basis. FIG. 10 also shows the advantages of utilizing multiple geographically distinct sites for locating portions of the server farm 40. It can be seen that the peak usages for customer accounts 94 and 95 are time shifted from those of the other customer accounts 91, 92 and 93 due to the difference in time zones between site location 80 and site location 82. The present invention can take advantage of these time shifted differences in peak usages to allocate rolling server capacity to site locations during a time period of peak usage from other site locations which are experiencing a lull in activity.
In one embodiment of the multi-site configuration of the present invention as shown in FIG. 13, at least three separate three separate site locations 80, 82 and 84 are preferably situated geographically at least 24 divided by N+1 hours apart from each other, where N represents the number of distinct site locations in the multi-site configuration. In the embodiment having three separate site locations 80, 82 and 84, the site locations are preferably eight hours apart from each other. The time difference realized by this geographic separation allows for the usage patterns of customer accounts located at all three sites to be aggregated and serviced by a combined number of servers that is significantly less than would otherwise be required if each of the servers at a given location were not able to utilize servers dynamically reallocated from one or more of the other locations. The advantage of this can be seen when site location 80 is experiencing nighttime usage levels, servers from this site location 80 can be dynamically reallocated to site location 82 that is experiencing daytime usage levels. At the same time, site location 84 experiences evening usage levels and may or may not be suited to have servers reallocated from this location to another location or vice versa. Generally, a site location is arranged so as to look to borrow capacity first from a site location that is at a later time zone (i.e., to the east of that site) and will look to make extra capacity available to site locations that are at an earlier time zone (i.e., to the west of that site). Other preferences can also be established depending upon past usage and predicted patterns of use.
Referring now to FIG. 11, a preferred embodiment of the master decision software program 72 will be described. The master decision software program 72 includes a resource database 150, a service level agreement database 152, a master decision logic module 154 and a dispatch module 156. The master decision logic module 154 has access to the resource database 150 and the service level agreement database 152 and compares the status information to information in the resource database 150 and the service level agreement database 152 to determine whether to dynamically reallocate servers from the first customer account to the second customer account. The dispatch module 156 is operably linked to the master decision logic module 154 to dynamically reallocate servers when directed by the master decision logic module 154 by using the communication channel 106 to set initialization pointers for the reallocated servers 46′ to access software and data unique to the customer account for the second administrative group 52-b and reinitializing the reallocated server 46′ such that at least one server joins the second administrative group 52-b. Preferably, the dispatch module 156 includes a set of connectivity rules 160 and a set of personality modules 162 for each server 46. The connectivity rules 160 providing instructions for connecting a particular server 46 to a given network switch 44 or data storage unit 50. The personality module 162 describes the details of the particular software configuration of the server board 102 to be added to an administrative work group for a customer account. Once the dispatch module 146 has determined the need to reallocate a server, it will evaluate the set of connectivity rules 160 and a set of personality modules 162 to determine how to construct a server 46 that will be dispatched to that particular administrative group 52.
Another way of looking at how the present invention can dynamically provide hosted service across disparate accounts is to view a portion of the servers 46 as being assigned to a pool of a plurality of virtual servers that may be selectively configured to access software and data for a particular administrative group 52. When the dispatch module 146 has determined a need to add a server 46 to a particular administrative group 52, it automatically allocates one of the servers from the pool of virtual servers to that administrative group. Conversely, if the dispatch module determines that an administrative group can relinquish one of its servers 46, that relinquished server would be added to the pool of virtual servers that are available for reallocation to a different administrative group. When the present invention is viewed from this perspective, it will be seen that the group manager software 48 operates to “manufacture” or create one or more virtual servers out of this pool of the plurality of virtual servers on a just-in-time or as-needed basis. As previously described, the pool of virtual servers can either be a warm pool or a cold pool, or any combination thereof. The virtual server is manufactured or constructed to be utilized by the desired administrative group in accordance with the set of connectivity rules 160 and personality modules 162.
In this embodiment, the master decision logic module 152 is operably connected to a management console 158 that can display information about the master decision software program and accept account maintenance and update information to processes into the various databases. A billing software module 160 is integrated into the engine group manager 48 in order to keep track of the billing based on the allocation of servers to a given customer account. Preferably, a customer account is billed at a higher rate for the hosted services when servers are dynamically reallocated to that customer account based on the customer's service level agreement.
FIG. 12 shows a representation of three different service level agreement arrangements for a given customer account. In this embodiment, the service level agreements are made for providing hosted services for a given period of time, such as a month. In a first level shown at 170, the customer account is provided with the capacity to support hosted services for 640,000 simultaneous connections. If the customer account did not need a reallocation of servers to support capacity greater than the committed capacity for the first level 170, the customer would be charged to establish rate for that level of committed capacity. In a second level shown at 172, customer account can be dynamically expanded to support capacity of double the capacity at the first level 172. In a preferred embodiment, once the engine group manager 48 has dynamically reallocated servers to the customer account in order to support the second level 172 of capacity to meet a higher than anticipated peak usage, the customer account would be charged a higher rate for the period of time that the additional usage was required. In addition, the customer account could be charged a one-time fee for initiating the higher level of service represented by the second level 172. In one embodiment, charges for the second level 172 of service would be incurred at a rate that is some additional multiple of the rate charged for the first level 170. The second level 172 represents a guaranteed expansion level available to the customer for the given period of time. Finally, a third level 174 provides an optional extended additional level of service that may be able to be brought to bare to provide hosted services for the customer account. In this embodiment, the third level 174 provides up to a higher multiple times the level of service as the first level 170. In one embodiment, in order to provide this extended additional level of service, the host system makes use of the multi-site arrangement as previously described in order to bring in the required number of servers to meet this level of service. Preferably, the customer account is charged a second higher rate for the period of time that the extended additional service is reallocated to this customer account. In one embodiment, charges for the third level 174 of service would be incurred at a rate that is an even larger multiple of the first level 170 for the given period of time that the extended additional third level 174 of service is provided for this customer account. Again, the customer account may be charged a one-time fee for initiating this third level 174 of service at any time during the given period. At the end of a given period, the customer may alter the level of service contracted for the given customer account.
As shown in FIG. 12, the service level agreement is increased by 50% from a first period to a second period in response to a higher anticipated peak usage for the given customer account. Preferably, the period for a service level agreement for a given customer account would be a monthly basis, with suggestions been presented to the customer for recommended changes to the service level agreement for the upcoming billing period. Although this example is demonstrated in terms of simultaneous connections, it should be understood that the service level agreement for a given customer account can be generated in terms of a variety of performance measurements, such as simultaneous connections, hits, amount of data transferred, number of transactions, connect time, resources utilized by different application software programs, the revenue generated, or any combination thereof. It will also be understood that the service level agreement may provide for different levels of commitment for different types of resources, such as front-end servers, back-end servers, network connections or disk storage units.
Referring now to FIG. 15, a block diagram of the preferred embodiment of the local decision software program 70 will be described. A series of measurement modules 180, 181, 182, 183 and 184 each performed independent evaluations of the operation of the particular server on which the local decision software program 70 is executing. Outputs from these measurement modules are provided to an aggregator module 190 of the local decision software program 70. A predictor module 192 generates expected response times and probabilities for various requests. With priority inputs 194 supplied by the master decision software program 72 from the service level agreement database 152, a fuzzy inference system 196 determines whether a request to add an engine blade 104 for the administrative group 52 will be made, or whether an offer to give up or remove an engine blade from the administrative group 52 will be made. The request to add or remove a blade is then communicated over communication channel 106 to the master decision software program 72. In one embodiment, the aggregator module 190 is executed on each server 46 within a given administrative group 52, and the predictor module 192 and fuzzy inference module 196 are executed on only a single server 46 within the given administrative group 52 with the outputs of the various measurement modules 180-184 been communicated to the designated server 46 across the communication channel 106. In another embodiment, the aggregator module 190, predictor module 192 and fuzzy inference module 196 may be executed on more than one server within a given administrative group for purposes of redundancy or distributed processing of the information necessary to generate the request add or remove a blade. Preferably, the aggregator module 190 accomplishes a balancing across the various measurement modules 180-184 in accordance with the formula:
B k=[(ΣT ki /w k)−mink]*100/(maxk-mink)-50  Equation (1)
i=1 to Wk
Where Tki is the time take it for the ith request of measurement type k, Wk is the window size for measurement type k, mink is the minimum time expected for measurement type k, and maxk is the maximum time to be tolerated for a measurement type k. The balanced request rate Bk is then passed to the predictor module 192 and the fuzzy inference module 196 of the local decision software program 70. The window size for the measurement type k would be set to minimize any unnecessary intrusion by the measurement modules 180-184, while at the same time allowing for a timely and adequate response to increases in usage demand for the administrative group 52.
FIG. 16 shows a sample of the workload measurements from the various measurement modules 180-184 under varying load conditions. It can be seen that no single workload measurement provides a constantly predictable estimate of the expected response time and probability for that response time. As such, the fuzzy inference module 196 must consider three fundamental parameters: the predicted response times for various requests, the priority of these requests, and probability of their occurrence. The fuzzy inference module 196 blends all three of these considerations to make a determination as to whether to request a blade to be added or removed from the administrative group 52. An example of a fuzzy inference rule would be:
    • if (priority is urgent) and (probability is abundant) and (expected response time is too high) then (make request for additional blade).
Preferably, the end results of the fuzzy inference module 196 is to generate a decision surface contouring the need to request an additional server over the grid of the expected response time vs. the probability of that response time for this administrative group 52. An example of such a decision surface is shown in FIG. 17.
A portion of the disclosure of this invention is subject to copyright protection. The copyright owner permits the facsimile reproduction of the disclosure of this invention as it appears in the Patent and Trademark Office files or records, but otherwise reserves all copyright rights.
Although the preferred embodiment of the automated system of the present invention has been described, it will be recognized that numerous changes and variations can be made and that the scope of the present invention is to be defined by the claims.

Claims (28)

1. A method for automatically allocating computing resources of a rack-and-blade computer assembly comprising:
receiving server performance information from an application server pool disposed in a rack of a rack-and-blade computer assembly;
determining at least one QoS attribute for the application server pool;
determining that the QoS attribute is below a standard; and allocating a blade server from a free server pool for use by the application server pool.
2. The method of claim 1, additionally comprising selecting, prior to allocating for use, a blade server from a free server pool to obtain a selected blade server.
3. The method of claim 2, additionally comprising reconfiguring a traffic management device associated with the application server pool.
4. The method of claim 1, additionally comprising preparing, prior to allocating for use, the selected blade server for operation with the application server pool.
5. The method of claim 4, additionally comprising reconfiguring a traffic management device associated with the application server pool.
6. The method of claim 1, additionally comprising reconfiguring a traffic management device associated with the application server pool.
7. The method of claim 1, additionally comprising provisioning images onto at least one blade server of the application server pool.
8. The method of claim 1, additionally comprising monitoring the performance of at least one blade server of the application server pool from information received by a server agent associated with the blade server.
9. The method of claim 1, additionally comprising removing the use of at least one blade server from the application server pool.
10. The method of claim 1, additionally comprising determining a relative performance of another application server pool.
11. A method for automatically allocating computing resources of a rack-and-blade computer assembly comprising:
receiving server performance information from an application server pool disposed in a rack of a rack-and-blade computer assembly;
determining at least one QoS attribute for the application server pool;
determining that the QoS attribute is above a standard; and
removing the use of a blade server from the application server pool.
12. The method of claim 11, additionally comprising allocating the use of the removed blade server for use by a free server pool.
13. The method of claim 11, additionally comprising provisioning images onto at least one blade server of the application server pool.
14. The method of claim 11, additionally comprising monitoring the performance of at least one blade server of the application server pool from information received by a server agent associated with the blade server.
15. A method for automatically allocating computing resources of a rack-and-blade computer assembly comprising:
receiving server performance information from an application server pool disposed in a rack of a rack-and-blade computer assembly;
determining at least one QoS attribute for the application server pool;
determining that the QoS attribute is below a standard;
determining that a free server pool has a blade server for use; and
selecting from the free server pool the available blade server for use by the application server pool.
16. The method of claim 15, additionally comprising installing software on the selected available blade server.
17. The method of claim 16, additionally comprising configuring the selected available blade server.
18. The method of claim 16, additionally comprising readying the selected available blade server for operation.
19. The method of claim 15, additionally comprising configuring the selected available blade server.
20. The method of claim 15, additionally comprising readying the selected available blade server for operation.
21. The method of claim 15, additionally comprising reconfiguring a traffic management device associated with the application server pool.
22. The method of claim 15, additionally comprising provisioning images onto at least one blade server of the application server pool.
23. The method of claim 15, additionally comprising monitoring the performance of at least one blade server of the application server pool from information received by a server agent associated with the blade server.
24. The method of claim 15, additionally comprising removing the use of at least one blade server from the application server pool.
25. The method of claim 15, additionally comprising determining a relative performance of another application server pool.
26. An article of manufacture comprising:
a machine-readable storage medium having stored thereon instructions for:
receiving server performance information from an application server pool disposed in a rack of a rack-and-blade computer assembly;
determining at least one QoS attribute for the application server pool;
determining that the QoS attribute is above a standard; and
removing the use of a blade server from the application server pool.
27. An article of manufacture comprising:
a machine-readable storage medium having stored thereon instructions for:
receiving server performance information from an application server pool disposed in a rack of a rack-and-blade computer assembly;
determining at least one QoS attribute for the application server pool;
determining that the QoS attribute is above a standard; and
removing the use of a blade server from the application server pool.
28. An article of manufacture comprising:
a machine-readable storage medium having stored thereon instructions for:
receiving server performance information from an application server pool disposed in a rack of a rack-and-blade computer assembly;
determining at least one QoS attribute for the application server pool;
determining that the QoS attribute is below a standard;
determining that a free server pool has a blade server for use; and
selecting from the free server pool the available blade server for use by the application server pool.
US10/984,959 2000-11-10 2004-11-08 Method and system for providing dynamic hosted service management across disparate accounts/sites Expired - Fee Related US7693993B2 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US10/984,959 US7693993B2 (en) 2000-11-10 2004-11-08 Method and system for providing dynamic hosted service management across disparate accounts/sites
US12/748,063 US8316131B2 (en) 2000-11-10 2010-03-26 Method and system for providing dynamic hosted service management across disparate accounts/sites
US13/681,031 US20130238801A1 (en) 2000-11-10 2012-11-19 Method and System for Providing Dynamic Hosted Service Management Across Disparate Accounts/Sites

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US09/710,095 US6816905B1 (en) 2000-11-10 2000-11-10 Method and system for providing dynamic hosted service management across disparate accounts/sites
US10/984,959 US7693993B2 (en) 2000-11-10 2004-11-08 Method and system for providing dynamic hosted service management across disparate accounts/sites

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US09/710,095 Continuation US6816905B1 (en) 2000-07-17 2000-11-10 Method and system for providing dynamic hosted service management across disparate accounts/sites

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US12/748,063 Division US8316131B2 (en) 2000-11-10 2010-03-26 Method and system for providing dynamic hosted service management across disparate accounts/sites

Publications (2)

Publication Number Publication Date
US20050182838A1 US20050182838A1 (en) 2005-08-18
US7693993B2 true US7693993B2 (en) 2010-04-06

Family

ID=33311189

Family Applications (4)

Application Number Title Priority Date Filing Date
US09/710,095 Expired - Lifetime US6816905B1 (en) 2000-07-17 2000-11-10 Method and system for providing dynamic hosted service management across disparate accounts/sites
US10/984,959 Expired - Fee Related US7693993B2 (en) 2000-11-10 2004-11-08 Method and system for providing dynamic hosted service management across disparate accounts/sites
US12/748,063 Expired - Lifetime US8316131B2 (en) 2000-11-10 2010-03-26 Method and system for providing dynamic hosted service management across disparate accounts/sites
US13/681,031 Abandoned US20130238801A1 (en) 2000-11-10 2012-11-19 Method and System for Providing Dynamic Hosted Service Management Across Disparate Accounts/Sites

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US09/710,095 Expired - Lifetime US6816905B1 (en) 2000-07-17 2000-11-10 Method and system for providing dynamic hosted service management across disparate accounts/sites

Family Applications After (2)

Application Number Title Priority Date Filing Date
US12/748,063 Expired - Lifetime US8316131B2 (en) 2000-11-10 2010-03-26 Method and system for providing dynamic hosted service management across disparate accounts/sites
US13/681,031 Abandoned US20130238801A1 (en) 2000-11-10 2012-11-19 Method and System for Providing Dynamic Hosted Service Management Across Disparate Accounts/Sites

Country Status (1)

Country Link
US (4) US6816905B1 (en)

Cited By (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020091854A1 (en) * 2000-07-17 2002-07-11 Smith Philip S. Method and system for operating a commissioned e-commerce service prover
US20050246441A1 (en) * 2004-04-16 2005-11-03 Oracle International Corporation Automatic assignment of services to servers in a multi-server system
US20060015597A1 (en) * 2004-06-24 2006-01-19 Marlin Scott Method and system for improved in-line management of an information technology network
US20070124162A1 (en) * 2005-11-15 2007-05-31 Alexander Mekyska Method for allocating system costs
US20080091826A1 (en) * 2002-12-10 2008-04-17 Dias Daniel M Apparatus and methods for co-location and offloading of web site traffic based on traffic pattern recognition
US20100192157A1 (en) * 2005-03-16 2010-07-29 Cluster Resources, Inc. On-Demand Compute Environment
US20100228839A1 (en) * 2009-03-09 2010-09-09 Oracle International Corporation Efficient on-demand provisioning of servers for specific software sets
US20100250642A1 (en) * 2009-03-31 2010-09-30 International Business Machines Corporation Adaptive Computing Using Probabilistic Measurements
US20110191462A1 (en) * 2000-07-17 2011-08-04 Smith Philip S Method and system for operating an E-Commerce service provider
US8302100B2 (en) 2000-01-18 2012-10-30 Galactic Computing Corporation Bvi/Bc System for balance distribution of requests across multiple servers using dynamic metrics
US8316131B2 (en) 2000-11-10 2012-11-20 Galactic Computing Corporation Bvi/Bc Method and system for providing dynamic hosted service management across disparate accounts/sites
US8607046B1 (en) * 2007-04-23 2013-12-10 Netapp, Inc. System and method for signing a message to provide one-time approval to a plurality of parties
US8782120B2 (en) 2005-04-07 2014-07-15 Adaptive Computing Enterprises, Inc. Elastic management of compute resources between a web server and an on-demand compute environment
US9015324B2 (en) 2005-03-16 2015-04-21 Adaptive Computing Enterprises, Inc. System and method of brokering cloud computing resources
US9075657B2 (en) 2005-04-07 2015-07-07 Adaptive Computing Enterprises, Inc. On-demand access to compute resources
US9231886B2 (en) 2005-03-16 2016-01-05 Adaptive Computing Enterprises, Inc. Simple integration of an on-demand compute environment
US11467883B2 (en) 2004-03-13 2022-10-11 Iii Holdings 12, Llc Co-allocating a reservation spanning different compute resources types
US11494235B2 (en) 2004-11-08 2022-11-08 Iii Holdings 12, Llc System and method of providing system jobs within a compute environment
US11522952B2 (en) 2007-09-24 2022-12-06 The Research Foundation For The State University Of New York Automatic clustering for self-organizing grids
US11526304B2 (en) 2009-10-30 2022-12-13 Iii Holdings 2, Llc Memcached server functionality in a cluster of data processing nodes
US11630704B2 (en) 2004-08-20 2023-04-18 Iii Holdings 12, Llc System and method for a workload management and scheduling module to manage access to a compute environment according to local and non-local user identity information
US11652706B2 (en) 2004-06-18 2023-05-16 Iii Holdings 12, Llc System and method for providing dynamic provisioning within a compute environment
US11650857B2 (en) 2006-03-16 2023-05-16 Iii Holdings 12, Llc System and method for managing a hybrid computer environment
US11720290B2 (en) 2009-10-30 2023-08-08 Iii Holdings 2, Llc Memcached server functionality in a cluster of data processing nodes

Families Citing this family (198)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6877044B2 (en) * 2000-02-10 2005-04-05 Vicom Systems, Inc. Distributed storage management platform architecture
US6772270B1 (en) 2000-02-10 2004-08-03 Vicom Systems, Inc. Multi-port fibre channel controller
US20020103889A1 (en) * 2000-02-11 2002-08-01 Thomas Markson Virtual storage layer approach for dynamically associating computer storage with processing hosts
AU2001238429A1 (en) * 2000-02-18 2001-08-27 Cedere Corporation Method of automatically baselining business bandwidth
US7058826B2 (en) * 2000-09-27 2006-06-06 Amphus, Inc. System, architecture, and method for logical server and other network devices in a dynamically configurable multi-server network environment
US7822967B2 (en) 2000-09-27 2010-10-26 Huron Ip Llc Apparatus, architecture, and method for integrated modular server system providing dynamically power-managed and work-load managed network devices
US7032119B2 (en) * 2000-09-27 2006-04-18 Amphus, Inc. Dynamic power and workload management for multi-server system
US7203746B1 (en) * 2000-12-11 2007-04-10 Agilent Technologies, Inc. System and method for adaptive resource management
US20040024732A1 (en) * 2001-04-02 2004-02-05 Bryan Klisch Constructing a component management database for managing roles using a directed graph
US20030196126A1 (en) 2002-04-11 2003-10-16 Fung Henry T. System, method, and architecture for dynamic server power management and dynamic workload management for multi-server environment
US20030037061A1 (en) * 2001-05-08 2003-02-20 Gautham Sastri Data storage system for a multi-client network and method of managing such system
US7110394B1 (en) * 2001-06-25 2006-09-19 Sanera Systems, Inc. Packet switching apparatus including cascade ports and method for switching packets
US7043547B2 (en) * 2001-06-29 2006-05-09 International Business Machines Corporation Server availability reporting using asynchronous e-mail message
US7200662B2 (en) 2001-07-06 2007-04-03 Juniper Networks, Inc. Integrated rule network management system
US7363353B2 (en) * 2001-07-06 2008-04-22 Juniper Networks, Inc. Content service aggregation device for a data center
US7305492B2 (en) * 2001-07-06 2007-12-04 Juniper Networks, Inc. Content service aggregation system
US8055555B2 (en) * 2001-09-25 2011-11-08 Emc Corporation Mediation device for scalable storage service
US7552056B2 (en) * 2001-09-25 2009-06-23 Emc Corporation Scalable storage service registration application
US7310672B2 (en) * 2001-11-13 2007-12-18 Hewlett-Packard Development Company, L.P. Method and system for exploiting service level objectives to enable resource sharing in a communication network having a plurality of application environments
US20030097445A1 (en) * 2001-11-20 2003-05-22 Stephen Todd Pluggable devices services and events for a scalable storage service architecture
US20030115258A1 (en) * 2001-12-13 2003-06-19 International Business Machines Corporation Time zone difference based locality estimation between web clients and E-business servers
US8549048B2 (en) * 2001-12-19 2013-10-01 Emc Corporation Workflow database for scalable storage service
US8126722B2 (en) 2001-12-20 2012-02-28 Verizon Business Global Llc Application infrastructure platform (AIP)
US7293105B2 (en) * 2001-12-21 2007-11-06 Cisco Technology, Inc. Methods and apparatus for implementing a high availability fibre channel switch
US7360034B1 (en) * 2001-12-28 2008-04-15 Network Appliance, Inc. Architecture for creating and maintaining virtual filers on a filer
US7130905B2 (en) * 2002-01-10 2006-10-31 Sun Microsystems, Inc. System and method for coordinating access to data for a distributed application
US7933998B2 (en) * 2002-01-11 2011-04-26 Motorola Mobility, Inc. Dynamic CAN bus system configuration and messaging
US20030154202A1 (en) * 2002-02-12 2003-08-14 Darpan Dinker Distributed data system with process co-location and out -of -process communication
US7320035B2 (en) 2002-03-01 2008-01-15 Sun Microsystems, Inc. Object mutation determination for incremental state saves
US7370329B2 (en) 2002-03-01 2008-05-06 Sun Microsystems, Inc. System and method for state saves in a distributed data system
US7194519B1 (en) 2002-03-15 2007-03-20 Network Appliance, Inc. System and method for administering a filer having a plurality of virtual filers
US7734778B2 (en) 2002-04-05 2010-06-08 Sheng (Ted) Tai Tsao Distributed intelligent virtual server
KR20040103969A (en) * 2002-04-19 2004-12-09 컴퓨터 어소시에이츠 싱크, 인코포레이티드 System and method for managing wireless devices in an enterprise
US8554929B1 (en) * 2002-05-03 2013-10-08 Foundry Networks, Llc Connection rate limiting for server load balancing and transparent cache switching
US20030217131A1 (en) * 2002-05-17 2003-11-20 Storage Technology Corporation Processing distribution using instant copy
US7080378B1 (en) * 2002-05-17 2006-07-18 Storage Technology Corporation Workload balancing using dynamically allocated virtual servers
US20040015581A1 (en) * 2002-07-22 2004-01-22 Forbes Bryn B. Dynamic deployment mechanism
US7299427B2 (en) * 2002-08-30 2007-11-20 Lsi Corporation Radio prototyping system
US7765299B2 (en) * 2002-09-16 2010-07-27 Hewlett-Packard Development Company, L.P. Dynamic adaptive server provisioning for blade architectures
US7206836B2 (en) * 2002-09-23 2007-04-17 Sun Microsystems, Inc. System and method for reforming a distributed data system cluster after temporary node failures or restarts
US7239605B2 (en) * 2002-09-23 2007-07-03 Sun Microsystems, Inc. Item and method for performing a cluster topology self-healing process in a distributed data system cluster
US7200657B2 (en) * 2002-10-01 2007-04-03 International Business Machines Corporation Autonomic provisioning of network-accessible service behaviors within a federated grid infrastructure
US7322031B2 (en) * 2002-10-07 2008-01-22 International Business Machines Corporation Remote Web service cloning and instantiation
US8005979B2 (en) * 2002-10-28 2011-08-23 Oracle America, Inc. System and method for uniquely identifying processes and entities in clusters
US7107272B1 (en) 2002-12-02 2006-09-12 Storage Technology Corporation Independent distributed metadata system and method
US20040128269A1 (en) * 2002-12-27 2004-07-01 Milligan Charles A. System and method for managing data through families of inter-related metadata tables
US20040199569A1 (en) * 2003-02-18 2004-10-07 Mohan Kalkunte Method and system for handling traffic for server systems
JP4345334B2 (en) * 2003-03-28 2009-10-14 日本電気株式会社 Fault tolerant computer system, program parallel execution method and program
US8001142B2 (en) 2003-04-02 2011-08-16 Oracle America, Inc. Distributed data system with incremental data updates
US7178065B2 (en) * 2003-04-02 2007-02-13 Sun Microsystems, Inc. System and method for measuring performance with distributed agents
US7281050B2 (en) * 2003-04-08 2007-10-09 Sun Microsystems, Inc. Distributed token manager with transactional properties
US7349958B2 (en) * 2003-06-25 2008-03-25 International Business Machines Corporation Method for improving performance in a computer storage system by regulating resource requests from clients
US7581224B2 (en) * 2003-07-10 2009-08-25 Hewlett-Packard Development Company, L.P. Systems and methods for monitoring resource utilization and application performance
US20050080891A1 (en) * 2003-08-28 2005-04-14 Cauthron David M. Maintenance unit architecture for a scalable internet engine
US6985357B2 (en) * 2003-08-28 2006-01-10 Galactic Computing Corporation Bvi/Bc Computing housing for blade server with network switch
US20050138346A1 (en) * 2003-08-28 2005-06-23 Cauthron David M. iSCSI boot drive system and method for a scalable internet engine
US7209358B2 (en) * 2003-08-29 2007-04-24 Sun Microsystems, Inc. Aggregation switch
US7774312B2 (en) * 2003-09-04 2010-08-10 Oracle International Corporation Self-managing performance statistics repository for databases
US20050091215A1 (en) * 2003-09-29 2005-04-28 Chandra Tushar D. Technique for provisioning storage for servers in an on-demand environment
US7526551B1 (en) * 2003-10-01 2009-04-28 Sun Microsystems, Inc. System and method for highly available timer service in a J2EE cluster
US7886055B1 (en) 2005-04-28 2011-02-08 Hewlett-Packard Development Company, L.P. Allocating resources in a system having multiple tiers
US7581008B2 (en) * 2003-11-12 2009-08-25 Hewlett-Packard Development Company, L.P. System and method for allocating server resources
US7730489B1 (en) 2003-12-10 2010-06-01 Oracle America, Inc. Horizontally scalable and reliable distributed transaction management in a clustered application server environment
US7562143B2 (en) 2004-01-13 2009-07-14 International Business Machines Corporation Managing escalating resource needs within a grid environment
US7406691B2 (en) * 2004-01-13 2008-07-29 International Business Machines Corporation Minimizing complex decisions to allocate additional resources to a job submitted to a grid environment
US7461149B2 (en) 2004-01-13 2008-12-02 International Business Machines Corporation Ordering provisioning request execution based on service level agreement and customer entitlement
US7464159B2 (en) * 2004-01-14 2008-12-09 International Business Machines Corporation Managing analysis of a degraded service in a grid environment
US7552437B2 (en) 2004-01-14 2009-06-23 International Business Machines Corporation Maintaining application operations within a suboptimal grid environment
US20050198022A1 (en) * 2004-02-05 2005-09-08 Samsung Electronics Co., Ltd. Apparatus and method using proxy objects for application resource management in a communication network
USRE45593E1 (en) * 2004-03-13 2015-06-30 Adaptive Computing Enterprises, Inc. System and method for providing object triggers
US8417499B2 (en) * 2004-03-31 2013-04-09 International Business Machines Corporation Enabling real-time testing of on-demand infrastructure to predict service level agreement compliance
US8230085B2 (en) * 2004-04-12 2012-07-24 Netapp, Inc. System and method for supporting block-based protocols on a virtual storage appliance executing within a physical storage appliance
US20060048157A1 (en) * 2004-05-18 2006-03-02 International Business Machines Corporation Dynamic grid job distribution from any resource within a grid environment
US7266547B2 (en) 2004-06-10 2007-09-04 International Business Machines Corporation Query meaning determination through a grid service
US7584274B2 (en) * 2004-06-15 2009-09-01 International Business Machines Corporation Coordinating use of independent external resources within requesting grid environments
US7712100B2 (en) * 2004-09-14 2010-05-04 International Business Machines Corporation Determining a capacity of a grid environment to handle a required workload for a virtual grid job request
US7788671B2 (en) * 2004-11-01 2010-08-31 International Business Machines Corporation On-demand application resource allocation through dynamic reconfiguration of application cluster size and placement
US7882317B2 (en) * 2004-12-06 2011-02-01 Microsoft Corporation Process isolation using protection domains
US8020141B2 (en) 2004-12-06 2011-09-13 Microsoft Corporation Operating-system process construction
US7600232B2 (en) * 2004-12-07 2009-10-06 Microsoft Corporation Inter-process communications employing bi-directional message conduits
US7451435B2 (en) 2004-12-07 2008-11-11 Microsoft Corporation Self-describing artifacts and application abstractions
US20060143262A1 (en) * 2004-12-28 2006-06-29 International Business Machines Corporation Fast client boot in blade environment
US7533170B2 (en) * 2005-01-06 2009-05-12 International Business Machines Corporation Coordinating the monitoring, management, and prediction of unintended changes within a grid environment
US7761557B2 (en) * 2005-01-06 2010-07-20 International Business Machines Corporation Facilitating overall grid environment management by monitoring and distributing grid activity
US7590623B2 (en) * 2005-01-06 2009-09-15 International Business Machines Corporation Automated management of software images for efficient resource node building within a grid environment
US7707288B2 (en) * 2005-01-06 2010-04-27 International Business Machines Corporation Automatically building a locally managed virtual node grouping to handle a grid job requiring a degree of resource parallelism within a grid environment
US7793308B2 (en) * 2005-01-06 2010-09-07 International Business Machines Corporation Setting operation based resource utilization thresholds for resource use by a process
US20060149652A1 (en) * 2005-01-06 2006-07-06 Fellenstein Craig W Receiving bid requests and pricing bid responses for potential grid job submissions within a grid environment
US7502850B2 (en) * 2005-01-06 2009-03-10 International Business Machines Corporation Verifying resource functionality before use by a grid job submitted to a grid environment
US7668741B2 (en) * 2005-01-06 2010-02-23 International Business Machines Corporation Managing compliance with service level agreements in a grid environment
US7571120B2 (en) 2005-01-12 2009-08-04 International Business Machines Corporation Computer implemented method for estimating future grid job costs by classifying grid jobs and storing results of processing grid job microcosms
US7562035B2 (en) * 2005-01-12 2009-07-14 International Business Machines Corporation Automating responses by grid providers to bid requests indicating criteria for a grid job
US7472079B2 (en) * 2005-01-12 2008-12-30 International Business Machines Corporation Computer implemented method for automatically controlling selection of a grid provider for a grid job
US7467196B2 (en) * 2005-01-12 2008-12-16 International Business Machines Corporation Managing network errors communicated in a message transaction with error information using a troubleshooting agent
US9225663B2 (en) 2005-03-16 2015-12-29 Adaptive Computing Enterprises, Inc. System and method providing a virtual private cluster
US8239535B2 (en) * 2005-06-06 2012-08-07 Adobe Systems Incorporated Network architecture with load balancing, fault tolerance and distributed querying
US8849968B2 (en) * 2005-06-20 2014-09-30 Microsoft Corporation Secure and stable hosting of third-party extensions to web services
US7941668B2 (en) * 2005-07-08 2011-05-10 Stapleton Jeff J Method and system for securely managing application transactions using cryptographic techniques
US20070038681A1 (en) * 2005-08-10 2007-02-15 Spare Backup, Inc. System and method of remote storage of data through connection from a server to a client
US20070058620A1 (en) * 2005-08-31 2007-03-15 Mcdata Corporation Management of a switch fabric through functionality conservation
US9143841B2 (en) 2005-09-29 2015-09-22 Brocade Communications Systems, Inc. Federated management of intelligent service modules
US7356638B2 (en) * 2005-10-12 2008-04-08 International Business Machines Corporation Using out-of-band signaling to provide communication between storage controllers in a computer storage system
US8074231B2 (en) * 2005-10-26 2011-12-06 Microsoft Corporation Configuration of isolated extensions and device drivers
US7873696B2 (en) * 2005-10-28 2011-01-18 Yahoo! Inc. Scalable software blade architecture
JP4544146B2 (en) * 2005-11-29 2010-09-15 株式会社日立製作所 Disaster recovery method
US7836441B2 (en) * 2006-02-13 2010-11-16 International Business Machines Corporation Administration automation in application servers
US7953866B2 (en) 2006-03-22 2011-05-31 Mcdata Corporation Protocols for connecting intelligent service modules in a storage area network
US20070258443A1 (en) * 2006-05-02 2007-11-08 Mcdata Corporation Switch hardware and architecture for a computer network
US20070271315A1 (en) * 2006-05-02 2007-11-22 Mypoints.Com Inc. Robust silo based system architecture
US8032898B2 (en) * 2006-06-30 2011-10-04 Microsoft Corporation Kernel interface with categorized kernel objects
US8019872B2 (en) * 2006-07-12 2011-09-13 International Business Machines Corporation Systems, methods and computer program products for performing remote data storage for client devices
US7647590B2 (en) * 2006-08-31 2010-01-12 International Business Machines Corporation Parallel computing system using coordinator and master nodes for load balancing and distributing work
US20080059811A1 (en) * 2006-09-06 2008-03-06 Ravi Sahita Tamper resistant networking
US20080068381A1 (en) * 2006-09-19 2008-03-20 Searete Llc, A Limited Liability Corporation Of The State Of Delaware Using network access port linkages for data structure update decisions
US20080071793A1 (en) * 2006-09-19 2008-03-20 Searete Llc, A Limited Liability Corporation Of The State Of Delaware Using network access port linkages for data structure update decisions
US8281036B2 (en) * 2006-09-19 2012-10-02 The Invention Science Fund I, Llc Using network access port linkages for data structure update decisions
US8601104B2 (en) * 2006-09-19 2013-12-03 The Invention Science Fund I, Llc Using network access port linkages for data structure update decisions
US20080080396A1 (en) * 2006-09-28 2008-04-03 Microsoft Corporation Marketplace for cloud services resources
US8014308B2 (en) * 2006-09-28 2011-09-06 Microsoft Corporation Hardware architecture for cloud services
US20080148319A1 (en) * 2006-12-14 2008-06-19 International Business Machines Corporation Coordinating web media with time-shifted broadcast
US9026655B2 (en) * 2007-01-31 2015-05-05 Oracle America, Inc. Method and system for load balancing
US8201218B2 (en) 2007-02-28 2012-06-12 Microsoft Corporation Strategies for securely applying connection policies via a gateway
US20080244607A1 (en) * 2007-03-27 2008-10-02 Vladislav Rysin Economic allocation and management of resources via a virtual resource market
US8789063B2 (en) 2007-03-30 2014-07-22 Microsoft Corporation Master and subordinate operating system kernels for heterogeneous multiprocessor systems
US7861111B2 (en) * 2007-06-15 2010-12-28 Savvis, Inc. Shared data center disaster recovery systems and methods
US20090006537A1 (en) * 2007-06-29 2009-01-01 Microsoft Corporation Virtual Desktop Integration with Terminal Services
CN101690004B (en) * 2007-07-06 2013-10-23 Lg电子株式会社 Method and device for event report service in wireless local area network system
US9613368B2 (en) * 2007-08-03 2017-04-04 United Parcel Service Of America, Inc. Systems and methods for providing and dynamically updating customer-specific shipping information on an on-site server
US8284664B1 (en) 2007-09-28 2012-10-09 Juniper Networks, Inc. Redirecting data units to service modules based on service tags and a redirection table
WO2009061432A1 (en) * 2007-11-06 2009-05-14 Credit Suisse Securities (Usa) Llc Predicting and managing resource allocation according to service level agreements
US7441135B1 (en) 2008-01-14 2008-10-21 International Business Machines Corporation Adaptive dynamic buffering system for power management in server clusters
US8683062B2 (en) * 2008-02-28 2014-03-25 Microsoft Corporation Centralized publishing of network resources
US8555380B2 (en) * 2008-02-28 2013-10-08 Intel Corporation Automatic modification of executable code
EP2098956A1 (en) 2008-03-03 2009-09-09 British Telecommunications Public Limited Company Computing system and method for managing a server pool
US20090259757A1 (en) * 2008-04-15 2009-10-15 Microsoft Corporation Securely Pushing Connection Settings to a Terminal Server Using Tickets
US8219358B2 (en) 2008-05-09 2012-07-10 Credit Suisse Securities (Usa) Llc Platform matching systems and methods
US7996875B2 (en) * 2008-05-20 2011-08-09 Microsoft Corporation Adaptive timeshift service
US8918507B2 (en) * 2008-05-30 2014-12-23 Red Hat, Inc. Dynamic grouping of enterprise assets
US7925785B2 (en) * 2008-06-27 2011-04-12 Microsoft Corporation On-demand capacity management
US8612862B2 (en) 2008-06-27 2013-12-17 Microsoft Corporation Integrated client for access to remote resources
US8468356B2 (en) * 2008-06-30 2013-06-18 Intel Corporation Software copy protection via protected execution of applications
US20100036903A1 (en) * 2008-08-11 2010-02-11 Microsoft Corporation Distributed load balancer
US8706878B1 (en) 2008-08-21 2014-04-22 United Services Automobile Association Preferential loading in data centers
US9270783B2 (en) 2008-12-06 2016-02-23 International Business Machines Corporation System and method for photorealistic imaging workload distribution
US8856376B1 (en) * 2008-12-18 2014-10-07 Bank Of America Corporation Stabilization tool for a high-capacity network infrastructure
US8286177B2 (en) * 2009-01-29 2012-10-09 Microsoft Corporation Technique for conserving software application resources
WO2010096528A1 (en) * 2009-02-23 2010-08-26 Commscope Inc. Of North Carolina Methods and systems for monitoring changes made to a network that alter the services provided to a server
US8642722B2 (en) * 2009-05-15 2014-02-04 Iucf-Hyu (Industry-University Cooperation Foundation Hanyang University) Preparation method of polybenzimidazole
US8577892B2 (en) * 2009-06-05 2013-11-05 Microsoft Corporation Utilizing affinity groups to allocate data items and computing resources
US10127295B2 (en) * 2009-06-05 2018-11-13 Microsoft Technolofy Licensing, Llc Geographic co-location service for cloud computing
US9836783B2 (en) * 2009-07-24 2017-12-05 Avago Technologies General Ip (Singapore) Pte. Ltd. Method and system for content selection, delivery and payment
US8458324B2 (en) 2009-08-25 2013-06-04 International Business Machines Corporation Dynamically balancing resources in a server farm
JP4939594B2 (en) * 2009-11-30 2012-05-30 インターナショナル・ビジネス・マシーンズ・コーポレーション An apparatus, method, and computer program for configuring a cloud system capable of dynamically providing a service level agreement based on service level actual values or updated preference information provided by a primary cloud and providing a service
US8910176B2 (en) * 2010-01-15 2014-12-09 International Business Machines Corporation System for distributed task dispatch in multi-application environment based on consensus for load balancing using task partitioning and dynamic grouping of server instance
US8516284B2 (en) 2010-11-04 2013-08-20 International Business Machines Corporation Saving power by placing inactive computing devices in optimized configuration corresponding to a specific constraint
US9861027B2 (en) 2010-12-08 2018-01-09 Bayer Cropscience, Lp Seed treatment systems and methods
US9877424B2 (en) 2010-12-08 2018-01-30 Bayer Cropscience, Lp Seed treatment facilities, methods and apparatus
US9959511B2 (en) 2010-12-08 2018-05-01 Bayer Cropscience Lp Retail point seed treatment systems and methods
US20130117168A1 (en) * 2011-11-04 2013-05-09 Mark Henrik Sandstrom Maximizing Throughput of Multi-user Parallel Data Processing Systems
US8789065B2 (en) 2012-06-08 2014-07-22 Throughputer, Inc. System and method for input data load adaptive parallel processing
US8812586B1 (en) * 2011-02-15 2014-08-19 Google Inc. Correlating status information generated in a computer network
US9448847B2 (en) 2011-07-15 2016-09-20 Throughputer, Inc. Concurrent program execution optimization
US8694995B2 (en) 2011-12-14 2014-04-08 International Business Machines Corporation Application initiated negotiations for resources meeting a performance parameter in a virtualized computing environment
US8863141B2 (en) 2011-12-14 2014-10-14 International Business Machines Corporation Estimating migration costs for migrating logical partitions within a virtualized computing environment based on a migration cost history
US8825752B1 (en) * 2012-05-18 2014-09-02 Netapp, Inc. Systems and methods for providing intelligent automated support capable of self rejuvenation with respect to storage systems
US10067547B2 (en) * 2012-06-28 2018-09-04 Intel Corporation Power management control of remote servers
US11521139B2 (en) 2012-09-24 2022-12-06 Amazon Technologies, Inc. Providing system resources with secure containment units
US20140122695A1 (en) * 2012-10-31 2014-05-01 Rawllin International Inc. Dynamic resource allocation for network content delivery
US10181974B2 (en) * 2012-12-04 2019-01-15 Genesys Telecommunications Laboratories, Inc. Distributed agent reservation in SIP cluster
US9258234B1 (en) 2012-12-28 2016-02-09 Juniper Networks, Inc. Dynamically adjusting liveliness detection intervals for periodic network communications
US9268707B2 (en) 2012-12-29 2016-02-23 Intel Corporation Low overhead paged memory runtime protection
US8953460B1 (en) 2012-12-31 2015-02-10 Juniper Networks, Inc. Network liveliness detection using session-external communications
US9141487B2 (en) 2013-01-15 2015-09-22 Microsoft Technology Licensing, Llc Healing cloud services during upgrades
US10021042B2 (en) 2013-03-07 2018-07-10 Microsoft Technology Licensing, Llc Service-based load-balancing management of processes on remote hosts
US9203772B2 (en) 2013-04-03 2015-12-01 Hewlett-Packard Development Company, L.P. Managing multiple cartridges that are electrically coupled together
NL2010823C2 (en) * 2013-05-17 2014-11-24 Insite Innovations And Properties B V System and method for processing web-browsing information.
TWI493325B (en) * 2013-08-16 2015-07-21 英業達股份有限公司 Server
US9483561B2 (en) * 2014-01-24 2016-11-01 Bank Of America Corporation Server inventory trends
US10129094B1 (en) * 2014-03-13 2018-11-13 Amazon Technologies, Inc. Variable computing capacity
US9660933B2 (en) * 2014-04-17 2017-05-23 Go Daddy Operating Company, LLC Allocating and accessing hosting server resources via continuous resource availability updates
US9286467B2 (en) 2014-04-25 2016-03-15 Bank Of America Corporation Evaluating customer security preferences
US9251335B2 (en) 2014-04-25 2016-02-02 Bank Of America Corporation Evaluating customer security preferences
US10025710B2 (en) 2014-04-30 2018-07-17 Walmart Apollo, Llc Pattern for integrating primary and secondary data stores in a sharded data domain
US9213485B1 (en) 2014-06-04 2015-12-15 Pure Storage, Inc. Storage system architecture
US9836234B2 (en) 2014-06-04 2017-12-05 Pure Storage, Inc. Storage cluster
US11652884B2 (en) 2014-06-04 2023-05-16 Pure Storage, Inc. Customized hash algorithms
US10574754B1 (en) 2014-06-04 2020-02-25 Pure Storage, Inc. Multi-chassis array with multi-level load balancing
US11399063B2 (en) * 2014-06-04 2022-07-26 Pure Storage, Inc. Network authentication for a storage system
US9769017B1 (en) 2014-09-26 2017-09-19 Juniper Networks, Inc. Impending control plane disruption indication using forwarding plane liveliness detection protocols
US9614853B2 (en) * 2015-01-20 2017-04-04 Enzoo, Inc. Session security splitting and application profiler
CN104778623A (en) * 2015-04-02 2015-07-15 浙江吉利控股集团有限公司 Network transaction system for network transaction platform and transaction platform server
US10374936B2 (en) 2015-12-30 2019-08-06 Juniper Networks, Inc. Reducing false alarms when using network keep-alive messages
EP3473052A4 (en) * 2016-06-18 2020-04-22 Clevernet, INC. Intelligent adaptive transport layer to enhance performance using multiple channels
US10397085B1 (en) 2016-06-30 2019-08-27 Juniper Networks, Inc. Offloading heartbeat responses message processing to a kernel of a network device
US20180131633A1 (en) * 2016-11-08 2018-05-10 Alibaba Group Holding Limited Capacity management of cabinet-scale resource pools
WO2019175631A1 (en) * 2018-03-14 2019-09-19 Pratik Sharma Passive centralised health check manager
US11750441B1 (en) 2018-09-07 2023-09-05 Juniper Networks, Inc. Propagating node failure errors to TCP sockets
US11252139B2 (en) 2019-02-26 2022-02-15 Bank Of America Corporation Distributed authentication/authorization tool

Citations (64)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3764747A (en) 1970-04-17 1973-10-09 Hitachi Ltd Billing system in mobile communication
US5031089A (en) 1988-12-30 1991-07-09 United States Of America As Represented By The Administrator, National Aeronautics And Space Administration Dynamic resource allocation scheme for distributed heterogeneous computer systems
US5187710A (en) 1990-12-19 1993-02-16 At&T Bell Laboratories Method and apparatus for the billing of value-added communications calls
US5247427A (en) 1992-08-26 1993-09-21 Data General Corporation Disk array subsystem having elongated T-shaped guides for use in a data processing system
US5251097A (en) 1990-06-11 1993-10-05 Supercomputer Systems Limited Partnership Packaging architecture for a highly parallel multiprocessor system
US5351286A (en) 1993-02-23 1994-09-27 Bell Communications Research, Inc. Method and system for billing an ISDN data user interconnection to the public switched telephone network
US5460441A (en) 1994-11-01 1995-10-24 Compaq Computer Corporation Rack-mounted computer apparatus
US5488541A (en) 1994-05-31 1996-01-30 Northern Telecom Limited VME bus compatible backplane and shelf arrangement
US5504894A (en) 1992-04-30 1996-04-02 International Business Machines Corporation Workload manager for achieving transaction class response time goals in a multiprocessing system
US5537542A (en) 1994-04-04 1996-07-16 International Business Machines Corporation Apparatus and method for managing a server workload according to client performance goals in a client/server data processing system
US5548683A (en) 1994-05-05 1996-08-20 Grumman Aerospace Corporation Data fusion neural network
US5615329A (en) 1994-02-22 1997-03-25 International Business Machines Corporation Remote data duplexing
US5696895A (en) 1995-05-19 1997-12-09 Compaq Computer Corporation Fault tolerant multiple network servers
US5745884A (en) 1996-10-21 1998-04-28 Mobile Area Networks, Inc. System and method for billing data grade network use on a per connection basis
US5774668A (en) 1995-06-07 1998-06-30 Microsoft Corporation System for on-line service in which gateway computer uses service map which includes loading condition of servers broadcasted by application servers for load balancing
US5794221A (en) 1995-07-07 1998-08-11 Egendorf; Andrew Internet billing method
US5819092A (en) 1994-11-08 1998-10-06 Vermeer Technologies, Inc. Online service development tool with fee setting capabilities
US5828737A (en) 1995-10-24 1998-10-27 Telefonaktiebolaget L M Ericsson Communications service billing based on bandwidth use
US5832222A (en) 1996-06-19 1998-11-03 Ncr Corporation Apparatus for providing a single image of an I/O subsystem in a geographically dispersed computer system
US5845267A (en) 1996-09-06 1998-12-01 At&T Corp System and method for billing for transactions conducted over the internet from within an intranet
US5877938A (en) 1995-08-22 1999-03-02 Sequent Computer Systems, Inc. Packaging architecture for a data server
US5899980A (en) 1997-08-11 1999-05-04 Trivnet Ltd. Retail method over a wide area network
US5912802A (en) 1994-06-30 1999-06-15 Intel Corporation Ducted opposing bonded fin heat sink blower multi-microprocessor cooling system
US5938732A (en) 1996-12-09 1999-08-17 Sun Microsystems, Inc. Load balancing and failover of network services
US5946670A (en) 1996-02-29 1999-08-31 Fujitsu Limited Data switching device and method of notifying charge information in data switching network
US5948065A (en) 1997-03-28 1999-09-07 International Business Machines Corporation System for managing processor resources in a multisystem environment in order to provide smooth real-time data streams while enabling other types of applications to be processed concurrently
US5956697A (en) 1996-07-11 1999-09-21 International Scientific Co., Ltd. Timer-based fee-charging system for internet
US5956391A (en) 1996-02-09 1999-09-21 Telefonaktiebolaget Lm Ericsson Billing in the internet
US5974462A (en) 1997-03-28 1999-10-26 International Business Machines Corporation Method and apparatus for controlling the number of servers in a client/server system
US5978577A (en) 1995-03-17 1999-11-02 Csg Systems, Inc. Method and apparatus for transaction processing in a distributed database system
US5999965A (en) 1996-08-20 1999-12-07 Netspeak Corporation Automatic call distribution server for computer telephony communications
US6006259A (en) 1998-11-20 1999-12-21 Network Alchemy, Inc. Method and apparatus for an internet protocol (IP) network clustering system
US6014669A (en) 1997-10-01 2000-01-11 Sun Microsystems, Inc. Highly-available distributed cluster configuration database
WO2000004458A1 (en) 1998-07-14 2000-01-27 Massachusetts Institute Of Technology Global document hosting system utilizing embedded content distributed ghost servers
US6025989A (en) 1998-04-21 2000-02-15 International Business Machines Corporation Modular node assembly for rack mounted multiprocessor computer
US6035356A (en) 1996-12-16 2000-03-07 Atec Group, Inc. Cross-platform computer architecture and components thereof
US6035281A (en) 1997-06-16 2000-03-07 International Business Machines Corporation System and method of multiparty billing for Web access
US6067580A (en) 1997-03-11 2000-05-23 International Business Machines Corporation Integrating distributed computing environment remote procedure calls with an advisory work load manager
US6067545A (en) 1997-08-01 2000-05-23 Hewlett-Packard Company Resource rebalancing in networked computer systems
US6070191A (en) 1997-10-17 2000-05-30 Lucent Technologies Inc. Data distribution techniques for load-balanced fault-tolerant web access
US6088727A (en) 1996-10-28 2000-07-11 Mitsubishi Denki Kabushiki Kaisha Cluster controlling system operating on a plurality of computers in a cluster system
US6092178A (en) 1998-09-03 2000-07-18 Sun Microsystems, Inc. System for responding to a resource request
US6094680A (en) 1996-06-27 2000-07-25 Microsoft Corporation System and method for managing distributed resources on networks
US6094351A (en) 1998-08-19 2000-07-25 Hon Hai Precision Ind. Co., Ltd. Universal enclosure for different type mother boards
US6097882A (en) 1995-06-30 2000-08-01 Digital Equipment Corporation Method and apparatus of improving network performance and network availability in a client-server network by transparently replicating a network service
US6167446A (en) 1997-11-03 2000-12-26 Inca Technology, Inc. Automatically configuring network-name-services
US6173322B1 (en) 1997-06-05 2001-01-09 Silicon Graphics, Inc. Network request distribution based on static rules and dynamic performance data
US6185598B1 (en) 1998-02-10 2001-02-06 Digital Island, Inc. Optimized network resource location
WO2001067707A2 (en) 2000-03-03 2001-09-13 Scale Eight, Inc. A network storage system
WO2002008891A2 (en) 2000-07-20 2002-01-31 Rlx Technologies, Inc. Single board web server system and method
US6374297B1 (en) 1999-08-16 2002-04-16 International Business Machines Corporation Method and apparatus for load balancing of web cluster farms
US6452809B1 (en) 2000-11-10 2002-09-17 Galactic Computing Corporation Scalable internet engine
US6463454B1 (en) 1999-06-17 2002-10-08 International Business Machines Corporation System and method for integrated load distribution and resource management on internet environment
US20030039237A1 (en) * 1997-09-25 2003-02-27 Jan E Forslow Common access between a mobile communications network and an external network with selectable packet-switched and circuit-switched services
US6532488B1 (en) 1999-01-25 2003-03-11 John J. Ciarlante Method and system for hosting applications
US6625639B1 (en) 1999-10-20 2003-09-23 International Business Machines Corporation Apparatus and method for processing a task in a clustered computing environment
US6718359B2 (en) 1998-07-15 2004-04-06 Radware Ltd. Load balancing
US6816905B1 (en) 2000-11-10 2004-11-09 Galactic Computing Corporation Bvi/Bc Method and system for providing dynamic hosted service management across disparate accounts/sites
US6816903B1 (en) * 1997-05-27 2004-11-09 Novell, Inc. Directory enabled policy management tool for intelligent traffic management
US20050076214A1 (en) * 2003-10-03 2005-04-07 Thomas David Andrew Method and system for file downloads to portable computing devices
US6938256B2 (en) 2000-01-18 2005-08-30 Galactic Computing Corporation System for balance distribution of requests across multiple servers using dynamic metrics
US7032241B1 (en) * 2000-02-22 2006-04-18 Microsoft Corporation Methods and systems for accessing networks, methods and systems for accessing the internet
US7185112B1 (en) * 1999-06-01 2007-02-27 Fujitsu Limited Network interconnection apparatus for interconnecting a LAN and an ATM network using QoS adjustment
US20070140242A1 (en) * 2005-12-16 2007-06-21 Digiorgio Rinaldo S Reliable multicast operating system (OS) provisioning

Family Cites Families (148)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US645006A (en) * 1899-08-03 1900-03-06 William M Mallett Lamp.
US4502116A (en) * 1982-11-17 1985-02-26 At&T Bell Laboratories Multiple processor synchronized halt test arrangement
US4920487A (en) * 1988-12-12 1990-04-24 The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration Method of up-front load balancing for local memory parallel processors
US5155854A (en) 1989-02-03 1992-10-13 Digital Equipment Corporation System for arbitrating communication requests using multi-pass control unit based on availability of system resources
JPH0727445B2 (en) 1990-09-04 1995-03-29 インターナショナル・ビジネス・マシーンズ・コーポレイション User interface for computer processor operation
GB2256514B (en) * 1991-05-21 1994-11-16 Digital Equipment Corp Commitment ordering for guaranteeing serializability across distributed transactions
US5303297A (en) 1991-07-25 1994-04-12 Motorola, Inc. Dynamic pricing method and apparatus for communication systems
US5701480A (en) 1991-10-17 1997-12-23 Digital Equipment Corporation Distributed multi-version commitment ordering protocols for guaranteeing serializability during transaction processing
US5504899A (en) * 1991-10-17 1996-04-02 Digital Equipment Corporation Guaranteeing global serializability by applying commitment ordering selectively to global transactions
IL99923A0 (en) * 1991-10-31 1992-08-18 Ibm Israel Method of operating a computer in a network
US5335343A (en) * 1992-07-06 1994-08-02 Digital Equipment Corporation Distributed transaction processing using two-phase commit protocol with presumed-commit without log force
US5664106A (en) 1993-06-04 1997-09-02 Digital Equipment Corporation Phase-space surface representation of server computer performance in a computer network
US7080051B1 (en) * 1993-11-04 2006-07-18 Crawford Christopher M Internet download systems and methods providing software to internet computer users for local execution
US5771354A (en) * 1993-11-04 1998-06-23 Crawford; Christopher M. Internet online backup system provides remote storage for customers using IDs and passwords which were interactively established when signing up for backup services
US5487170A (en) * 1993-12-16 1996-01-23 International Business Machines Corporation Data processing system having dynamic priority task scheduling capabilities
US5473773A (en) 1994-04-04 1995-12-05 International Business Machines Corporation Apparatus and method for managing a data processing system workload according to two or more distinct processing goals
GB9414951D0 (en) 1994-07-25 1994-09-14 British Telecomm Computer system having client-server architecture
US5675785A (en) 1994-10-04 1997-10-07 Hewlett-Packard Company Data warehouse which is accessed by a user using a schema of virtual tables
US5586312A (en) 1994-10-11 1996-12-17 Unisys Corporation Method and apparatus for using an independent transaction processing application as a service routine
US6317773B1 (en) 1994-10-11 2001-11-13 International Business Machines Corporation System and method for creating an object oriented transaction service that interoperates with procedural transaction coordinators
US5675739A (en) 1995-02-03 1997-10-07 International Business Machines Corporation Apparatus and method for managing a distributed data processing system workload according to a plurality of distinct processing goal types
US5630081A (en) * 1995-09-07 1997-05-13 Puma Technology, Inc. Connection resource manager displaying link-status information using a traffic light iconic representation
US6041354A (en) * 1995-09-08 2000-03-21 Lucent Technologies Inc. Dynamic hierarchical network resource scheduling for continuous media
US5778395A (en) * 1995-10-23 1998-07-07 Stac, Inc. System for backing up files from disk volumes on multiple nodes of a computer network
US6011791A (en) * 1995-11-15 2000-01-04 Hitachi, Ltd. Multi-processor system and its network
US6182109B1 (en) * 1996-03-08 2001-01-30 International Business Machines Corporation Dynamic execution unit management for high performance user level network server system
US5764915A (en) * 1996-03-08 1998-06-09 International Business Machines Corporation Object-oriented communication interface for network protocol access using the selected newly created protocol interface object and newly created protocol layer objects in the protocol stack
US5928323A (en) * 1996-05-30 1999-07-27 Sun Microsystems, Inc. Apparatus and method for dynamically generating information with server-side software objects
US5983326A (en) 1996-07-01 1999-11-09 Sun Microsystems, Inc. Multiprocessing system including an enhanced blocking mechanism for read-to-share-transactions in a NUMA mode
US5795228A (en) * 1996-07-03 1998-08-18 Ridefilm Corporation Interactive computer-based entertainment system
US5822531A (en) 1996-07-22 1998-10-13 International Business Machines Corporation Method and system for dynamically reconfiguring a cluster of computer systems
US6038587A (en) * 1996-09-03 2000-03-14 Insession, Inc. System, method and article of manufacture for universal transaction processing over multiple transaction processing systems
US6442618B1 (en) * 1996-09-23 2002-08-27 Insession, Inc. System, method and article of manufacture for universal transaction processing
JPH10154373A (en) 1996-09-27 1998-06-09 Sony Corp Data decoding system and method thereof, transmission unit and method thereof and receiver device and method thereof
US5875306A (en) 1996-10-01 1999-02-23 International Business Machines Corporation Reconfiguring computer resources in a distributed computer enterprise environment
US6134533A (en) 1996-11-25 2000-10-17 Shell; Allyn M. Multi-level marketing computer network server
US6112243A (en) * 1996-12-30 2000-08-29 Intel Corporation Method and apparatus for allocating tasks to remote networked processors
US6151688A (en) 1997-02-21 2000-11-21 Novell, Inc. Resource management in a clustered computer system
KR100230281B1 (en) 1997-04-14 1999-11-15 윤종용 Multi-media system for transmitting and receiving a program number and method for transmitting and receiving a program number
US5987621A (en) 1997-04-25 1999-11-16 Emc Corporation Hardware and software failover services for a file server
JP3652834B2 (en) * 1997-04-30 2005-05-25 富士通株式会社 Client-driven network computing system and method
US6553416B1 (en) * 1997-05-13 2003-04-22 Micron Technology, Inc. Managing computer system alerts
US6338150B1 (en) * 1997-05-13 2002-01-08 Micron Technology, Inc. Diagnostic and managing distributed processor system
US6266721B1 (en) * 1997-05-13 2001-07-24 Micron Electronics, Inc. System architecture for remote access and control of environmental management
US6145098A (en) 1997-05-13 2000-11-07 Micron Electronics, Inc. System for displaying system status
US6134673A (en) 1997-05-13 2000-10-17 Micron Electronics, Inc. Method for clustering software applications
US6425006B1 (en) 1997-05-13 2002-07-23 Micron Technology, Inc. Alert configurator and manager
US6170067B1 (en) * 1997-05-13 2001-01-02 Micron Technology, Inc. System for automatically reporting a system failure in a server
US6243838B1 (en) * 1997-05-13 2001-06-05 Micron Electronics, Inc. Method for automatically reporting a system failure in a server
JP3656874B2 (en) 1997-07-04 2005-06-08 ソニー株式会社 Electronic device control system and method, playback device, and output device
US6088816A (en) * 1997-10-01 2000-07-11 Micron Electronics, Inc. Method of displaying system status
US6199173B1 (en) * 1997-10-01 2001-03-06 Micron Electronics, Inc. Method for mapping environmental resources to memory for program access
JP3683082B2 (en) * 1997-10-17 2005-08-17 富士通株式会社 Call processing equipment
US6633916B2 (en) 1998-06-10 2003-10-14 Hewlett-Packard Development Company, L.P. Method and apparatus for virtual resource handling in a multi-processor computer system
US6542926B2 (en) * 1998-06-10 2003-04-01 Compaq Information Technologies Group, L.P. Software partitioned multi-processor system with flexible resource sharing levels
US6647508B2 (en) 1997-11-04 2003-11-11 Hewlett-Packard Development Company, L.P. Multiprocessor computer architecture with multiple operating system instances and software controlled resource allocation
US6209018B1 (en) * 1997-11-13 2001-03-27 Sun Microsystems, Inc. Service framework for a distributed object network system
US5991792A (en) 1998-01-02 1999-11-23 International Business Machines Corporation Method, apparatus and computer program product for dynamically managing a thread pool of reusable threads in a computer system
US6154787A (en) 1998-01-21 2000-11-28 Unisys Corporation Grouping shared resources into one or more pools and automatically re-assigning shared resources from where they are not currently needed to where they are needed
US5983225A (en) 1998-01-26 1999-11-09 Telenor As Parameterized lock management system and method for conditional conflict serializability of transactions
GB2319102B (en) * 1998-01-30 1998-12-23 Ibm A security system for a transaction processing system
US6230183B1 (en) * 1998-03-11 2001-05-08 International Business Machines Corporation Method and apparatus for controlling the number of servers in a multisystem cluster
US6430618B1 (en) * 1998-03-13 2002-08-06 Massachusetts Institute Of Technology Method and apparatus for distributing requests among a plurality of resources
US6553420B1 (en) * 1998-03-13 2003-04-22 Massachusetts Institute Of Technology Method and apparatus for distributing requests among a plurality of resources
US6115693A (en) 1998-04-17 2000-09-05 Andersen Consulting Llp Quality center and method for a virtual sales and service center
US6157927A (en) 1998-04-22 2000-12-05 Unisys Corporation Methods and apparatus for enabling a component in a first transaction processing environment to access a resource in another environment that is under the control of an Xatmi complaint transaction manager
US6330689B1 (en) 1998-04-23 2001-12-11 Microsoft Corporation Server architecture with detection and recovery of failed out-of-process application
US6216185B1 (en) 1998-05-01 2001-04-10 Acqis Technology, Inc. Personal computer peripheral console with attached computer module
US6233587B1 (en) * 1998-05-07 2001-05-15 Oracle Corporation Extensible framework of key resource manager and transaction manager events for providing native support for foreign-initiated transactions
US6519553B1 (en) * 1998-05-14 2003-02-11 Sandia Corporation Multiprocessor computer overset grid method and apparatus
JPH11338746A (en) * 1998-05-27 1999-12-10 Ntt Communication Ware Kk Database access system and computer readable recording medium recording program
US6279001B1 (en) * 1998-05-29 2001-08-21 Webspective Software, Inc. Web service
US6105067A (en) * 1998-06-05 2000-08-15 International Business Machines Corp. Connection pool management for backend servers using common interface
US6223202B1 (en) * 1998-06-05 2001-04-24 International Business Machines Corp. Virtual machine pooling
US6421661B1 (en) * 1998-06-15 2002-07-16 International Business Machines Corporation Hierarchical query syntax for inquiring and selecting among database objects
KR100304644B1 (en) * 1998-06-19 2001-11-02 윤종용 Apparatus and method for transmitting informtion via network
US6728958B1 (en) * 1998-07-31 2004-04-27 Hewlett-Packard Development Company, L.P. Volatile resource manager with pre-prepare notification
US6615265B1 (en) 1998-08-04 2003-09-02 International Business Machines Corporation Enabling planned outages of application servers
US6301612B1 (en) 1998-08-12 2001-10-09 Microsoft Corporation Establishing one computer as a replacement for another computer
US6324580B1 (en) 1998-09-03 2001-11-27 Sun Microsystems, Inc. Load balancing for replicated services
US6272675B1 (en) * 1998-10-01 2001-08-07 Unisys Corporation Development system for automatically enabling a server application to execute with an XATMI-compliant transaction manager managing transactions within multiple environments
US6412079B1 (en) 1998-10-09 2002-06-25 Openwave Systems Inc. Server pool for clustered system
US6728748B1 (en) * 1998-12-01 2004-04-27 Network Appliance, Inc. Method and apparatus for policy based class of service and adaptive service level management within the context of an internet and intranet
US6442165B1 (en) * 1998-12-02 2002-08-27 Cisco Technology, Inc. Load balancing between service component instances
US6578147B1 (en) * 1999-01-15 2003-06-10 Cisco Technology, Inc. Parallel intrusion detection sensors with load balancing for high speed networks
US7197044B1 (en) 1999-03-17 2007-03-27 Broadcom Corporation Method for managing congestion in a network switch
US6446200B1 (en) 1999-03-25 2002-09-03 Nortel Networks Limited Service management
US6243737B1 (en) * 1999-04-09 2001-06-05 Translink Software, Inc. Method and apparatus for providing direct transaction access to information residing on a host system
US6421777B1 (en) 1999-04-26 2002-07-16 International Business Machines Corporation Method and apparatus for managing boot images in a distributed data processing system
US6687831B1 (en) 1999-04-29 2004-02-03 International Business Machines Corporation Method and apparatus for multiple security service enablement in a data processing system
US6643777B1 (en) 1999-05-14 2003-11-04 Acquis Technology, Inc. Data security method and device for computer modules
US6718415B1 (en) 1999-05-14 2004-04-06 Acqis Technology, Inc. Computer system and method including console housing multiple computer modules having independent processing units, mass storage devices, and graphics controllers
EP1190342A2 (en) * 1999-05-24 2002-03-27 Aprisma Management Technologies, Inc. Service level management
US6519679B2 (en) * 1999-06-11 2003-02-11 Dell Usa, L.P. Policy based storage configuration
US6411956B1 (en) * 1999-06-14 2002-06-25 Sun Microsystems, Inc. Method for distributed transaction support using JDBC 1.0 drivers
US6460082B1 (en) 1999-06-17 2002-10-01 International Business Machines Corporation Management of service-oriented resources across heterogeneous media servers using homogenous service units and service signatures to configure the media servers
FR2795840B1 (en) * 1999-07-02 2001-08-31 Commissariat Energie Atomique NETWORK OF PARALLEL PROCESSORS WITH FAULT TOLERANCE OF THESE PROCESSORS, AND RECONFIGURATION PROCEDURE APPLICABLE TO SUCH A NETWORK
FR2795839B1 (en) 1999-07-02 2001-09-07 Commissariat Energie Atomique RECONFIGURATION METHOD APPLICABLE TO A NETWORK OF IDENTICAL FUNCTIONAL ELEMENTS
US6779016B1 (en) * 1999-08-23 2004-08-17 Terraspring, Inc. Extensible computing system
US6615199B1 (en) * 1999-08-31 2003-09-02 Accenture, Llp Abstraction factory in a base services pattern environment
US6715145B1 (en) * 1999-08-31 2004-03-30 Accenture Llp Processing pipeline in a base services pattern environment
US6640244B1 (en) * 1999-08-31 2003-10-28 Accenture Llp Request batcher in a transaction services patterns environment
US6842906B1 (en) * 1999-08-31 2005-01-11 Accenture Llp System and method for a refreshable proxy pool in a communication services patterns environment
US6742015B1 (en) * 1999-08-31 2004-05-25 Accenture Llp Base services patterns in a netcentric environment
US7289964B1 (en) * 1999-08-31 2007-10-30 Accenture Llp System and method for transaction services patterns in a netcentric environment
US6640249B1 (en) * 1999-08-31 2003-10-28 Accenture Llp Presentation services patterns in a netcentric environment
US6651125B2 (en) 1999-09-28 2003-11-18 International Business Machines Corporation Processing channel subsystem pending I/O work queues based on priorities
US6587938B1 (en) * 1999-09-28 2003-07-01 International Business Machines Corporation Method, system and program products for managing central processing unit resources of a computing environment
US7051188B1 (en) * 1999-09-28 2006-05-23 International Business Machines Corporation Dynamically redistributing shareable resources of a computing environment to manage the workload of that environment
JP2001109638A (en) * 1999-10-06 2001-04-20 Nec Corp Method and system for distributing transaction load based on estimated extension rate and computer readable recording medium
US6704737B1 (en) * 1999-10-18 2004-03-09 Fisher-Rosemount Systems, Inc. Accessing and updating a configuration database from distributed physical locations within a process control system
US6421688B1 (en) * 1999-10-20 2002-07-16 Parallel Computers Technology, Inc. Method and apparatus for database fault tolerance with instant transaction replication using off-the-shelf database servers and low bandwidth networks
US6298451B1 (en) 1999-11-02 2001-10-02 Idini Corporation Directory-based failure recovery and load balancing system
US6721568B1 (en) * 1999-11-10 2004-04-13 Telefonaktiebolaget Lm Ericsson (Publ) Admission control in a mobile radio communications system
US6496828B1 (en) 1999-12-17 2002-12-17 International Business Machines Corporation Support for summary tables in a heterogeneous database environment
US6687729B1 (en) * 1999-12-20 2004-02-03 Unisys Corporation System and method for providing a pool of reusable threads for performing queued items of work
US6601096B1 (en) * 2000-01-04 2003-07-29 International Business Machines Corporation Client server method for loading a client with a specific image or utility based on the client's state
US6901442B1 (en) * 2000-01-07 2005-05-31 Netiq Corporation Methods, system and computer program products for dynamic filtering of network performance test results
US7140020B2 (en) * 2000-01-28 2006-11-21 Hewlett-Packard Development Company, L.P. Dynamic management of virtual partition computer workloads through service level optimization
US7228546B1 (en) * 2000-01-28 2007-06-05 Hewlett-Packard Development Company, L.P. Dynamic management of computer workloads through service level optimization
US7003571B1 (en) * 2000-01-31 2006-02-21 Telecommunication Systems Corporation Of Maryland System and method for re-directing requests from browsers for communication over non-IP based networks
US6714980B1 (en) * 2000-02-11 2004-03-30 Terraspring, Inc. Backup and restore of data associated with a host in a dynamically changing virtual server farm without involvement of a server that uses an associated storage device
US6898642B2 (en) * 2000-04-17 2005-05-24 International Business Machines Corporation Synchronous collaboration based on peer-to-peer communication
US7054943B1 (en) * 2000-04-28 2006-05-30 International Business Machines Corporation Method and apparatus for dynamically adjusting resources assigned to plurality of customers, for meeting service level agreements (slas) with minimal resources, and allowing common pools of resources to be used across plural customers on a demand basis
US6684343B1 (en) * 2000-04-29 2004-01-27 Hewlett-Packard Development Company, Lp. Managing operations of a computer system having a plurality of partitions
US6725317B1 (en) * 2000-04-29 2004-04-20 Hewlett-Packard Development Company, L.P. System and method for managing a computer system having a plurality of partitions
WO2001084313A2 (en) * 2000-05-02 2001-11-08 Sun Microsystems, Inc. Method and system for achieving high availability in a networked computer system
US6950848B1 (en) 2000-05-05 2005-09-27 Yousefi Zadeh Homayoun Database load balancing for multi-tier computer systems
US6832238B1 (en) 2000-05-24 2004-12-14 Sun Microsystems, Inc. Local transaction management
US7181743B2 (en) * 2000-05-25 2007-02-20 The United States Of America As Represented By The Secretary Of The Navy Resource allocation decision function for resource management architecture and corresponding programs therefor
US6574748B1 (en) * 2000-06-16 2003-06-03 Bull Hn Information Systems Inc. Fast relief swapping of processors in a data processing system
AU2001266503A1 (en) 2000-06-30 2002-01-08 Telefonaktiebolaget Lm Ericsson (Publ) Method and system for automatic re-assignment of software components of a failedhost
US6820171B1 (en) 2000-06-30 2004-11-16 Lsi Logic Corporation Methods and structures for an extensible RAID storage architecture
US8538843B2 (en) * 2000-07-17 2013-09-17 Galactic Computing Corporation Bvi/Bc Method and system for operating an E-commerce service provider
KR100840960B1 (en) 2000-07-17 2008-06-24 갈랙틱 컴퓨팅 코포레이션 비브이아이/아이비씨 Method and system for providing dynamic hosted service management
US7844513B2 (en) 2000-07-17 2010-11-30 Galactic Computing Corporation Bvi/Bc Method and system for operating a commissioned e-commerce service prover
US6985967B1 (en) * 2000-07-20 2006-01-10 Rlx Technologies, Inc. Web server network system and method
US6973517B1 (en) 2000-08-31 2005-12-06 Hewlett-Packard Development Company, L.P. Partition formation using microprocessors in a multiprocessor computer system
US6871210B1 (en) * 2000-09-05 2005-03-22 International Business Machines Corporation Automatic allocation of least loaded boot server to PXE client on a network VIA DHCP server
US20020124083A1 (en) 2000-09-06 2002-09-05 Sun Microsystems, Inc. Method and apparatus for increasing the efficiency of transactions and connection sharing in an enterprise environment
US6671686B2 (en) * 2000-11-02 2003-12-30 Guy Pardon Decentralized, distributed internet data management
US6412777B1 (en) * 2000-12-04 2002-07-02 Emil Richard Rossi Double-standard DWI-rules game
US6877035B2 (en) * 2001-01-29 2005-04-05 International Business Machines Corporation System for optimal resource allocation and planning for hosting computing services
US6839700B2 (en) * 2001-05-23 2005-01-04 International Business Machines Corporation Load balancing content requests using dynamic document generation cost information
US7085837B2 (en) * 2001-12-04 2006-08-01 International Business Machines Corporation Dynamic resource allocation using known future benefits
US7055052B2 (en) * 2002-11-21 2006-05-30 International Business Machines Corporation Self healing grid architecture for decentralized component-based systems
US6687931B1 (en) * 2003-04-18 2004-02-10 Doskocil Manufacturing Company, Inc. Pet cushion cover
US20110238564A1 (en) 2010-03-26 2011-09-29 Kwang Hyun Lim System and Method for Early Detection of Fraudulent Transactions

Patent Citations (67)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3764747A (en) 1970-04-17 1973-10-09 Hitachi Ltd Billing system in mobile communication
US5031089A (en) 1988-12-30 1991-07-09 United States Of America As Represented By The Administrator, National Aeronautics And Space Administration Dynamic resource allocation scheme for distributed heterogeneous computer systems
US5251097A (en) 1990-06-11 1993-10-05 Supercomputer Systems Limited Partnership Packaging architecture for a highly parallel multiprocessor system
US5187710A (en) 1990-12-19 1993-02-16 At&T Bell Laboratories Method and apparatus for the billing of value-added communications calls
US5504894A (en) 1992-04-30 1996-04-02 International Business Machines Corporation Workload manager for achieving transaction class response time goals in a multiprocessing system
US5247427A (en) 1992-08-26 1993-09-21 Data General Corporation Disk array subsystem having elongated T-shaped guides for use in a data processing system
US5351286A (en) 1993-02-23 1994-09-27 Bell Communications Research, Inc. Method and system for billing an ISDN data user interconnection to the public switched telephone network
US5615329A (en) 1994-02-22 1997-03-25 International Business Machines Corporation Remote data duplexing
US5537542A (en) 1994-04-04 1996-07-16 International Business Machines Corporation Apparatus and method for managing a server workload according to client performance goals in a client/server data processing system
US5548683A (en) 1994-05-05 1996-08-20 Grumman Aerospace Corporation Data fusion neural network
US5488541A (en) 1994-05-31 1996-01-30 Northern Telecom Limited VME bus compatible backplane and shelf arrangement
US5912802A (en) 1994-06-30 1999-06-15 Intel Corporation Ducted opposing bonded fin heat sink blower multi-microprocessor cooling system
US5460441A (en) 1994-11-01 1995-10-24 Compaq Computer Corporation Rack-mounted computer apparatus
US5819092A (en) 1994-11-08 1998-10-06 Vermeer Technologies, Inc. Online service development tool with fee setting capabilities
US5978577A (en) 1995-03-17 1999-11-02 Csg Systems, Inc. Method and apparatus for transaction processing in a distributed database system
US5696895A (en) 1995-05-19 1997-12-09 Compaq Computer Corporation Fault tolerant multiple network servers
US5951694A (en) 1995-06-07 1999-09-14 Microsoft Corporation Method of redirecting a client service session to a second application server without interrupting the session by forwarding service-specific information to the second server
US5774668A (en) 1995-06-07 1998-06-30 Microsoft Corporation System for on-line service in which gateway computer uses service map which includes loading condition of servers broadcasted by application servers for load balancing
US6097882A (en) 1995-06-30 2000-08-01 Digital Equipment Corporation Method and apparatus of improving network performance and network availability in a client-server network by transparently replicating a network service
US5794221A (en) 1995-07-07 1998-08-11 Egendorf; Andrew Internet billing method
US5877938A (en) 1995-08-22 1999-03-02 Sequent Computer Systems, Inc. Packaging architecture for a data server
US5828737A (en) 1995-10-24 1998-10-27 Telefonaktiebolaget L M Ericsson Communications service billing based on bandwidth use
US5956391A (en) 1996-02-09 1999-09-21 Telefonaktiebolaget Lm Ericsson Billing in the internet
US5946670A (en) 1996-02-29 1999-08-31 Fujitsu Limited Data switching device and method of notifying charge information in data switching network
US5832222A (en) 1996-06-19 1998-11-03 Ncr Corporation Apparatus for providing a single image of an I/O subsystem in a geographically dispersed computer system
US6094680A (en) 1996-06-27 2000-07-25 Microsoft Corporation System and method for managing distributed resources on networks
US5956697A (en) 1996-07-11 1999-09-21 International Scientific Co., Ltd. Timer-based fee-charging system for internet
US5999965A (en) 1996-08-20 1999-12-07 Netspeak Corporation Automatic call distribution server for computer telephony communications
US5845267A (en) 1996-09-06 1998-12-01 At&T Corp System and method for billing for transactions conducted over the internet from within an intranet
US5745884A (en) 1996-10-21 1998-04-28 Mobile Area Networks, Inc. System and method for billing data grade network use on a per connection basis
US6088727A (en) 1996-10-28 2000-07-11 Mitsubishi Denki Kabushiki Kaisha Cluster controlling system operating on a plurality of computers in a cluster system
US5938732A (en) 1996-12-09 1999-08-17 Sun Microsystems, Inc. Load balancing and failover of network services
US6035356A (en) 1996-12-16 2000-03-07 Atec Group, Inc. Cross-platform computer architecture and components thereof
US6067580A (en) 1997-03-11 2000-05-23 International Business Machines Corporation Integrating distributed computing environment remote procedure calls with an advisory work load manager
US5948065A (en) 1997-03-28 1999-09-07 International Business Machines Corporation System for managing processor resources in a multisystem environment in order to provide smooth real-time data streams while enabling other types of applications to be processed concurrently
US5974462A (en) 1997-03-28 1999-10-26 International Business Machines Corporation Method and apparatus for controlling the number of servers in a client/server system
US6816903B1 (en) * 1997-05-27 2004-11-09 Novell, Inc. Directory enabled policy management tool for intelligent traffic management
US6173322B1 (en) 1997-06-05 2001-01-09 Silicon Graphics, Inc. Network request distribution based on static rules and dynamic performance data
US6035281A (en) 1997-06-16 2000-03-07 International Business Machines Corporation System and method of multiparty billing for Web access
US6067545A (en) 1997-08-01 2000-05-23 Hewlett-Packard Company Resource rebalancing in networked computer systems
US5899980A (en) 1997-08-11 1999-05-04 Trivnet Ltd. Retail method over a wide area network
US20030039237A1 (en) * 1997-09-25 2003-02-27 Jan E Forslow Common access between a mobile communications network and an external network with selectable packet-switched and circuit-switched services
US6014669A (en) 1997-10-01 2000-01-11 Sun Microsystems, Inc. Highly-available distributed cluster configuration database
US6070191A (en) 1997-10-17 2000-05-30 Lucent Technologies Inc. Data distribution techniques for load-balanced fault-tolerant web access
US6167446A (en) 1997-11-03 2000-12-26 Inca Technology, Inc. Automatically configuring network-name-services
US6185598B1 (en) 1998-02-10 2001-02-06 Digital Island, Inc. Optimized network resource location
US6025989A (en) 1998-04-21 2000-02-15 International Business Machines Corporation Modular node assembly for rack mounted multiprocessor computer
WO2000004458A1 (en) 1998-07-14 2000-01-27 Massachusetts Institute Of Technology Global document hosting system utilizing embedded content distributed ghost servers
US6108703A (en) 1998-07-14 2000-08-22 Massachusetts Institute Of Technology Global hosting system
US6718359B2 (en) 1998-07-15 2004-04-06 Radware Ltd. Load balancing
US6094351A (en) 1998-08-19 2000-07-25 Hon Hai Precision Ind. Co., Ltd. Universal enclosure for different type mother boards
US6092178A (en) 1998-09-03 2000-07-18 Sun Microsystems, Inc. System for responding to a resource request
US6006259A (en) 1998-11-20 1999-12-21 Network Alchemy, Inc. Method and apparatus for an internet protocol (IP) network clustering system
US6532488B1 (en) 1999-01-25 2003-03-11 John J. Ciarlante Method and system for hosting applications
US7185112B1 (en) * 1999-06-01 2007-02-27 Fujitsu Limited Network interconnection apparatus for interconnecting a LAN and an ATM network using QoS adjustment
US6463454B1 (en) 1999-06-17 2002-10-08 International Business Machines Corporation System and method for integrated load distribution and resource management on internet environment
US6374297B1 (en) 1999-08-16 2002-04-16 International Business Machines Corporation Method and apparatus for load balancing of web cluster farms
US6625639B1 (en) 1999-10-20 2003-09-23 International Business Machines Corporation Apparatus and method for processing a task in a clustered computing environment
US6938256B2 (en) 2000-01-18 2005-08-30 Galactic Computing Corporation System for balance distribution of requests across multiple servers using dynamic metrics
US7032241B1 (en) * 2000-02-22 2006-04-18 Microsoft Corporation Methods and systems for accessing networks, methods and systems for accessing the internet
WO2001067707A2 (en) 2000-03-03 2001-09-13 Scale Eight, Inc. A network storage system
WO2002008891A2 (en) 2000-07-20 2002-01-31 Rlx Technologies, Inc. Single board web server system and method
US6606253B2 (en) 2000-11-10 2003-08-12 Galactic Computing Corporation Scalable internet engine
US6452809B1 (en) 2000-11-10 2002-09-17 Galactic Computing Corporation Scalable internet engine
US6816905B1 (en) 2000-11-10 2004-11-09 Galactic Computing Corporation Bvi/Bc Method and system for providing dynamic hosted service management across disparate accounts/sites
US20050076214A1 (en) * 2003-10-03 2005-04-07 Thomas David Andrew Method and system for file downloads to portable computing devices
US20070140242A1 (en) * 2005-12-16 2007-06-21 Digiorgio Rinaldo S Reliable multicast operating system (OS) provisioning

Non-Patent Citations (73)

* Cited by examiner, † Cited by third party
Title
A New Twist On Hosting: Luminate Pitches Its Enterprise Management Services; Chris Gonsalves, PC Week Online, 2 pgs.; Mar. 2000.
ADSM: A Multi-Platform, Scalable, Backup and Archive mass Storage System; Luis-Felipe Cabrera et al.; 1063-6390/95, 1995 IEEE; pp. 420-427.
Brochure: Applicast(TM)., Applicast, Inc.; 6 pgs.; Copyright 1999.
Brochure: Applicast™., Applicast, Inc.; 6 pgs.; Copyright 1999.
Brochure: ChatCom's ChatterBox Products, ChatCom, Inc.; 2 pgs. Undated.
Brochure: ControlIT(TM).-Remote Control Without Boundaries, Computer Associates International, Inc.; 10 pgs.; Copyright 2000.
Brochure: ControlIT™.—Remote Control Without Boundaries, Computer Associates International, Inc.; 10 pgs.; Copyright 2000.
Brochure: Get Connected-Cable & Wireless Enhanced Solutions Provider Program, Cable & Wireless USA, Inc., 4 pgs.; Copyright 1999.
Brochure: Get Connected—Cable & Wireless Enhanced Solutions Provider Program, Cable & Wireless USA, Inc., 4 pgs.; Copyright 1999.
Brochure: HP Open View Customer Views 1.0 for Network Node Manager, Hewlett-Packard Company, 4 pgs.; Copyright 1999.
Brochure: HP Open View, Hewlett-Packard Company, 16 pgs.; Copyright 1999.
Brochure: Tivol®. Global Enterprise Manager-A Business-focused Approach to Systems Management, Tivoli Systems, Inc.; 4 pgs.; Copyright 2000.
Brochure: Tivol®. Global Enterprise Manager—A Business-focused Approach to Systems Management, Tivoli Systems, Inc.; 4 pgs.; Copyright 2000.
Brochure: Tivoli Cross-Site-Internet Application Management Technical Brief, Tivoli Systems, Inc.; 22 pgs.; Copyright 1999.
Brochure: Tivoli Cross-Site—Internet Application Management Technical Brief, Tivoli Systems, Inc.; 22 pgs.; Copyright 1999.
Brochure: Tivoli Problem Management Suite, Tivoli Systems, Inc.; 4 pgs.; Copyright 1999.
Brochure: Tivoli Technical Brief for Service Providers, Tivoli Systems, Inc., 7 pgs.; Copyright 1999.
Brochure: Tivoli® Cross-Site for Availability, Tivoli Systems, Inc.; 4 pgs.; Copyright 1999.
Brochure: Tivoli® Cross-Site for Deployment-Using the Internet as a Valuable Deployment Tool, Tivoli Systems, Inc.; 4 pgs.; Copyright 2000.
Brochure: Tivoli® Cross-Site for Deployment—Using the Internet as a Valuable Deployment Tool, Tivoli Systems, Inc.; 4 pgs.; Copyright 2000.
Brochure: Tivoli® Distributed Monitoring-Automated, Consistent Availability Solution, Tivoli Systems, Inc.; 4 pgs.; Copyright 2000.
Brochure: Tivoli® Distributed Monitoring—Automated, Consistent Availability Solution, Tivoli Systems, Inc.; 4 pgs.; Copyright 2000.
Brochure: Tivoli®. Cross-Site for Security, Tivoli Systems, Inc.; 4 pgs.; Copyright 2000.
Excite@Home LauncHes Free Service for Merchants, Andrea Orr, MicroTimes Magazine, plus web site print-out; 3 pgs.; Aug. 2000.
Magazine advertisment : ISP Upgrade, CAIS Software Solutions, 3 pgs.; Copyright 1999.
Management of the Access Network and Service Provisioning, Jani Hursti, Seminar in Networking, Helsinki University of Technology, 20 pgs.; Apr. 1999.
The Landlords of Cyberspace, Christopher Heun, informationweek.com; 3 pgs.; Jul. 2000.
U.S. Appl. No. 09/709,820, filed Nov. 10, 2000, Rex Jackson.
U.S. Appl. No. 09/710,095, filed Nov. 10, 2000, Kitrick Sheets.
U.S. Appl. No. 09/765,766, filed Jan. 18, 2001, Deng Yuefan.
U.S. Appl. No. 09/907,520, filed Jul. 17, 2001, Rex Jackson.
U.S. Appl. No. 10/244,450, filed Sep. 16, 2002, Rex Jackson.
U.S. Appl. No. 11/202,644, filed Aug. 12, 2005, Deng Yuefan.
U.S. PAIR File Wrapper History for U.S. Appl. No. 09/709,820, filed Nov. 10, 2000.
U.S. PAIR File Wrapper History for U.S. Appl. No. 09/710,095, filed Nov. 10, 2000.
U.S. PAIR File Wrapper History for U.S. Appl. No. 09/765,766, filed Jan. 18, 2001.
U.S. PAIR File Wrapper History for U.S. Appl. No. 09/907,520, filed Jul. 17, 2001.
U.S. PAIR File Wrapper History for U.S. Appl. No. 10/244,450, filed Sep. 16, 2002.
U.S. PAIR File Wrapper History for U.S. Appl. No. 11/202,644, filed Aug. 12, 2005.
Web site print-out Epoch Internet Introduces New ‘Pod’ Architecture, An Innovative Approach to Providing Web Hosting Services, epoch.net, Epoch Networks, Inc., 4 pgs.; Copyright 2000.
Web site print-out Epoch Internet Introduces New 'Pod' Architecture, An Innovative Approach to Providing Web Hosting Services, epoch.net, Epoch Networks, Inc., 4 pgs.; Copyright 2000.
Web site print-out: Ascendant Solutions, Inc. Fact Sheet, Ascendant Solutions, Inc.; 4 pgs.; Copyright 1998-2000.
Web site print-out: ASP Computer Systems Corp.; Data Return Corporation, 3 pgs.; Copyright 2000.
Web site print-out: Cellular MultiProcessing-Breakthrough Architecture for an Open Mainframe, Unisys, 4 pgs.; Copyright 2000.
Web site print-out: Cellular MultiProcessing—Breakthrough Architecture for an Open Mainframe, Unisys, 4 pgs.; Copyright 2000.
Web site print-out: ClearPath Enterprise Servers-What is HMP?, Unisys, 2 pgs.; Copyright 2000.
Web site print-out: ClearPath Enterprise Servers—What is HMP?, Unisys, 2 pgs.; Copyright 2000.
Web site print-out: Comprehensive Hosting Solution for ISV s, ebaseone Corporation, 3 pgs.; Copyright 2000.
Web site print-out: End to End Scheduling for Tivoli.RTM. Workload Scheduler and Tivoli Operations Planning and Control, Tivoli Systems, Inc., 2 pgs. not dated.
Web site print-out: Frequently Asked Questions-Complex Web Services, USinternetworking, Inc.; 3 pgs.; Copyright 1998-1999.
Web site print-out: Frequently Asked Questions—Complex Web Services, USinternetworking, Inc.; 3 pgs.; Copyright 1998-1999.
Web site print-out: Frequently Asked Questions-Overview of PATROL®., BMC Software, Inc., 4 pgs.; Copyright 2001.
Web site print-out: Frequently Asked Questions—Overview of PATROL®., BMC Software, Inc., 4 pgs.; Copyright 2001.
Web site print-out: HP'S Answer for Portal Performance-A-class Servers; Hewlett-Packard Company, 2 pgs. © 2000.
Web site print-out: HP'S Answer for Portal Performance—A-class Servers; Hewlett-Packard Company, 2 pgs. © 2000.
Web site print-out: Internet Shock Absorber, Cable and Wireless plc., 5 pgs.; Copyright 2000.
Web site print-out: ISP Power Overview, inovaware.com, Inovaware Corporation; 2 pgs.; copyright 1997-2000.
Web site print-out: Lightspeed Systems, Corporate Overview, 3 pgs., Copyright 1999.
Web site print-out: Luminate Introduces Luminate.Net e-Service, Luminate, Inc., 2 pgs.; Copyright 2000.
Web site print-out: PATROL® for Performance Management-Prediction in Action, BMC Software, Inc., 2 pgs.; Copyright 2001.
Web site print-out: PATROL® SRM: The Foundation of Application-Centric Storage Management@, BMC Software, Inc., 2 pgs.; Copyright 2001.
Web site print-out: rackspace.com; 2 pages; Copyright 2000.
Web site print-out: Remedy AR System Integration Module for PATROL, BMC Software, Inc., 2 pages, Copyright 2001.
Web site print-out: Resonate®, Resonate, Inc.; 1 pg.; Copyright 1999.
Web site print-out: table of contents and chapter abstracts for-ISP Survival Guide: Stratetgies for Running a Competitive ISP, Geoff Huston, Wiley Computer Publishing, 16 pgs.; Oct. 1998.
Web site print-out: table of contents and chapter abstracts for—ISP Survival Guide: Stratetgies for Running a Competitive ISP, Geoff Huston, Wiley Computer Publishing, 16 pgs.; Oct. 1998.
Web site print-out: USi Complex Web Hosting Solution and Lattice Communications, Usinternetworking, Inc., 2 pgs.; not dated.
Web site print-out: White Paper: Tivoli Service Provider Solutions, Tivoli Systems, Inc.; 24 pgs.; Copyright 1999.
Web site print-out: XaCCT Offers Multivendor, Multitechnology Billing Model for ISP Consumption, John Morency, Network World Fusion on Network/Systems Management, Network World, Inc.; 3 pgs.; Sep. 1998.
Web site print-out; Coblat RaQ, Cobalt Network Systems; 2 pgs. © 2001.
White paper: IBM Netfinity X-architecture, IBM Corporation; 22 pgs.; © 1998.
White Paper: Stratus ftServers: Enhancing Software Reliability and Availability for Windows 2000, Stratus Computer Systems; 10 pgs.; Copyright 2000.
White Paper: The HP Open View Approach to Change and Configuraton Management, Hewlett-Packard Company, 23 pgs.; Copyright 1999.

Cited By (55)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8302100B2 (en) 2000-01-18 2012-10-30 Galactic Computing Corporation Bvi/Bc System for balance distribution of requests across multiple servers using dynamic metrics
US7844513B2 (en) 2000-07-17 2010-11-30 Galactic Computing Corporation Bvi/Bc Method and system for operating a commissioned e-commerce service prover
US20020091854A1 (en) * 2000-07-17 2002-07-11 Smith Philip S. Method and system for operating a commissioned e-commerce service prover
US8538843B2 (en) * 2000-07-17 2013-09-17 Galactic Computing Corporation Bvi/Bc Method and system for operating an E-commerce service provider
US8429049B2 (en) * 2000-07-17 2013-04-23 Galactic Computing Corporation Bvi/Ibc Method and system for allocating computing resources
US20120137004A1 (en) * 2000-07-17 2012-05-31 Smith Philip S Method and System for Operating a Commissioned E-Commerce Service Prover
US20110191462A1 (en) * 2000-07-17 2011-08-04 Smith Philip S Method and system for operating an E-Commerce service provider
US8316131B2 (en) 2000-11-10 2012-11-20 Galactic Computing Corporation Bvi/Bc Method and system for providing dynamic hosted service management across disparate accounts/sites
US20130238801A1 (en) * 2000-11-10 2013-09-12 Galactic Computing Corporation Bvi/Ibc Method and System for Providing Dynamic Hosted Service Management Across Disparate Accounts/Sites
US8352607B2 (en) 2002-12-10 2013-01-08 International Business Machines Corporation Co-location and offloading of web site traffic based on traffic pattern recognition
US20080091826A1 (en) * 2002-12-10 2008-04-17 Dias Daniel M Apparatus and methods for co-location and offloading of web site traffic based on traffic pattern recognition
US11467883B2 (en) 2004-03-13 2022-10-11 Iii Holdings 12, Llc Co-allocating a reservation spanning different compute resources types
US20050246441A1 (en) * 2004-04-16 2005-11-03 Oracle International Corporation Automatic assignment of services to servers in a multi-server system
US11652706B2 (en) 2004-06-18 2023-05-16 Iii Holdings 12, Llc System and method for providing dynamic provisioning within a compute environment
US8032620B2 (en) * 2004-06-24 2011-10-04 Marlin Scott Method and system for improved in-line management of an information technology network
US20060015597A1 (en) * 2004-06-24 2006-01-19 Marlin Scott Method and system for improved in-line management of an information technology network
US11630704B2 (en) 2004-08-20 2023-04-18 Iii Holdings 12, Llc System and method for a workload management and scheduling module to manage access to a compute environment according to local and non-local user identity information
US11709709B2 (en) 2004-11-08 2023-07-25 Iii Holdings 12, Llc System and method of providing system jobs within a compute environment
US11494235B2 (en) 2004-11-08 2022-11-08 Iii Holdings 12, Llc System and method of providing system jobs within a compute environment
US11861404B2 (en) 2004-11-08 2024-01-02 Iii Holdings 12, Llc System and method of providing system jobs within a compute environment
US11762694B2 (en) 2004-11-08 2023-09-19 Iii Holdings 12, Llc System and method of providing system jobs within a compute environment
US11537434B2 (en) 2004-11-08 2022-12-27 Iii Holdings 12, Llc System and method of providing system jobs within a compute environment
US11656907B2 (en) 2004-11-08 2023-05-23 Iii Holdings 12, Llc System and method of providing system jobs within a compute environment
US11886915B2 (en) 2004-11-08 2024-01-30 Iii Holdings 12, Llc System and method of providing system jobs within a compute environment
US11537435B2 (en) 2004-11-08 2022-12-27 Iii Holdings 12, Llc System and method of providing system jobs within a compute environment
US20100192157A1 (en) * 2005-03-16 2010-07-29 Cluster Resources, Inc. On-Demand Compute Environment
US9112813B2 (en) 2005-03-16 2015-08-18 Adaptive Computing Enterprises, Inc. On-demand compute environment
US9231886B2 (en) 2005-03-16 2016-01-05 Adaptive Computing Enterprises, Inc. Simple integration of an on-demand compute environment
US9015324B2 (en) 2005-03-16 2015-04-21 Adaptive Computing Enterprises, Inc. System and method of brokering cloud computing resources
US10333862B2 (en) 2005-03-16 2019-06-25 Iii Holdings 12, Llc Reserving resources in an on-demand compute environment
US10608949B2 (en) 2005-03-16 2020-03-31 Iii Holdings 12, Llc Simple integration of an on-demand compute environment
US11658916B2 (en) 2005-03-16 2023-05-23 Iii Holdings 12, Llc Simple integration of an on-demand compute environment
US11134022B2 (en) 2005-03-16 2021-09-28 Iii Holdings 12, Llc Simple integration of an on-demand compute environment
US11356385B2 (en) 2005-03-16 2022-06-07 Iii Holdings 12, Llc On-demand compute environment
US8370495B2 (en) 2005-03-16 2013-02-05 Adaptive Computing Enterprises, Inc. On-demand compute environment
US11831564B2 (en) 2005-04-07 2023-11-28 Iii Holdings 12, Llc On-demand access to compute resources
US11496415B2 (en) 2005-04-07 2022-11-08 Iii Holdings 12, Llc On-demand access to compute resources
US11522811B2 (en) 2005-04-07 2022-12-06 Iii Holdings 12, Llc On-demand access to compute resources
US10986037B2 (en) 2005-04-07 2021-04-20 Iii Holdings 12, Llc On-demand access to compute resources
US11765101B2 (en) 2005-04-07 2023-09-19 Iii Holdings 12, Llc On-demand access to compute resources
US11533274B2 (en) 2005-04-07 2022-12-20 Iii Holdings 12, Llc On-demand access to compute resources
US10277531B2 (en) 2005-04-07 2019-04-30 Iii Holdings 2, Llc On-demand access to compute resources
US9075657B2 (en) 2005-04-07 2015-07-07 Adaptive Computing Enterprises, Inc. On-demand access to compute resources
US8782120B2 (en) 2005-04-07 2014-07-15 Adaptive Computing Enterprises, Inc. Elastic management of compute resources between a web server and an on-demand compute environment
US8126751B2 (en) * 2005-11-15 2012-02-28 Alexander Mekyska Method for allocating system costs of the infrastructure of a computer center
US20070124162A1 (en) * 2005-11-15 2007-05-31 Alexander Mekyska Method for allocating system costs
US11650857B2 (en) 2006-03-16 2023-05-16 Iii Holdings 12, Llc System and method for managing a hybrid computer environment
US8607046B1 (en) * 2007-04-23 2013-12-10 Netapp, Inc. System and method for signing a message to provide one-time approval to a plurality of parties
US11522952B2 (en) 2007-09-24 2022-12-06 The Research Foundation For The State University Of New York Automatic clustering for self-organizing grids
US8185624B2 (en) * 2009-03-09 2012-05-22 Oracle International Corporation Efficient on-demand provisioning of servers for specific software sets
US20100228839A1 (en) * 2009-03-09 2010-09-09 Oracle International Corporation Efficient on-demand provisioning of servers for specific software sets
US8131843B2 (en) * 2009-03-31 2012-03-06 International Business Machines Corporation Adaptive computing using probabilistic measurements
US20100250642A1 (en) * 2009-03-31 2010-09-30 International Business Machines Corporation Adaptive Computing Using Probabilistic Measurements
US11720290B2 (en) 2009-10-30 2023-08-08 Iii Holdings 2, Llc Memcached server functionality in a cluster of data processing nodes
US11526304B2 (en) 2009-10-30 2022-12-13 Iii Holdings 2, Llc Memcached server functionality in a cluster of data processing nodes

Also Published As

Publication number Publication date
US20130238801A1 (en) 2013-09-12
US20100268827A1 (en) 2010-10-21
US6816905B1 (en) 2004-11-09
US8316131B2 (en) 2012-11-20
US20050182838A1 (en) 2005-08-18

Similar Documents

Publication Publication Date Title
US7693993B2 (en) Method and system for providing dynamic hosted service management across disparate accounts/sites
CA2415770C (en) Method and system for providing dynamic hosted service management
US8538843B2 (en) Method and system for operating an E-commerce service provider
US7844513B2 (en) Method and system for operating a commissioned e-commerce service prover
US6597956B1 (en) Method and apparatus for controlling an extensible computing system
US20050080891A1 (en) Maintenance unit architecture for a scalable internet engine
US7475274B2 (en) Fault tolerance and recovery in a high-performance computing (HPC) system
US7370013B1 (en) Approach for determining an amount to bill a customer for the use of resources
US7146233B2 (en) Request queue management
US7861111B2 (en) Shared data center disaster recovery systems and methods
US20040088414A1 (en) Reallocation of computing resources
Moreno-Vozmediano et al. Orchestrating the deployment of high availability services on multi-zone and multi-cloud scenarios
US20060195448A1 (en) Application of resource-dependent policies to managed resources in a distributed computing system
US20090019535A1 (en) Method and remote system for creating a customized server infrastructure in real time
US9483258B1 (en) Multi-site provisioning of resources to software offerings using infrastructure slices
JP2004508616A (en) Method and apparatus for controlling an extensible computing system
EP4029197B1 (en) Utilizing network analytics for service provisioning
US7558858B1 (en) High availability infrastructure with active-active designs
Youn et al. The approaches for high available and fault-tolerant cluster systems
EP2098956A1 (en) Computing system and method for managing a server pool
Miljković Geographically dispersed cluster of web application servers on virtualization platforms
CN1397902A (en) Commission electronic business servicer operating method and system

Legal Events

Date Code Title Description
STCF Information on status: patent grant

Free format text: PATENTED CASE

RR Request for reexamination filed

Effective date: 20110728

B1 Reexamination certificate first reexamination

Free format text: CLAIMS 6,17-20 AND 26 ARE CANCELLED. CLAIMS 1, 3-5, 7-11, 13-16, 21-25 AND 27-28 ARE DETERMINED TO BE PATENTABLE AS AMENDED. CLAIMS 2 AND 12, DEPENDENT ON AN AMENDED CLAIM, ARE DETERMINED TO BE PATENTABLE. NEW CLAIMS 29-37 ARE ADDED AND DETERMINED TO BE PATENTABLE.

FPAY Fee payment

Year of fee payment: 4

FEPP Fee payment procedure

Free format text: PAT HOLDER NO LONGER CLAIMS SMALL ENTITY STATUS, ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: STOL); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

AS Assignment

Owner name: ASPENGINES, INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SHEETS, KITRICK B.;SMITH, PHILIP S.;ENGEL, STEPHEN J.;AND OTHERS;SIGNING DATES FROM 20010119 TO 20010416;REEL/FRAME:034524/0907

AS Assignment

Owner name: GALACTIC COMPUTING CORPORATION, BVI, FORMERLY KNOW

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:NEUTILITY CORPORATION, FORMERLY KNOWN AS ASPENGINES, INC.;REEL/FRAME:034662/0516

Effective date: 20020327

AS Assignment

Owner name: GALACTIC COMPUTING CORPORATION BVI/IBC, CHINA

Free format text: CHANGE OF NAME;ASSIGNOR:GALACTIC COMPUTING CORPORATION, BVI, FORMERLY KNOWN AS GALACTIC REVENUE LTD;REEL/FRAME:034760/0387

Effective date: 20020320

AS Assignment

Owner name: ASPENGINES, INC., CALIFORNIA

Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE FOURTH AND FIFTH ASSIGNOR NAME PREVIOUSLY RECORDED AT REEL: 034524 FRAME: 0907. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT;ASSIGNORS:SHEETS, KITRICK B.;SMITH, PHILIP S.;ENGEL, STEPHEN J.;AND OTHERS;SIGNING DATES FROM 20010119 TO 20010416;REEL/FRAME:035620/0008

AS Assignment

Owner name: RPX CORPORATION, CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:GALACTIC COMPUTING CORPORATION BVI/IBC;REEL/FRAME:035693/0471

Effective date: 20150520

AS Assignment

Owner name: JPMORGAN CHASE BANK, N.A., AS COLLATERAL AGENT, IL

Free format text: SECURITY AGREEMENT;ASSIGNORS:RPX CORPORATION;RPX CLEARINGHOUSE LLC;REEL/FRAME:038041/0001

Effective date: 20160226

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552)

Year of fee payment: 8

AS Assignment

Owner name: RPX CLEARINGHOUSE LLC, CALIFORNIA

Free format text: RELEASE (REEL 038041 / FRAME 0001);ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:044970/0030

Effective date: 20171222

Owner name: RPX CORPORATION, CALIFORNIA

Free format text: RELEASE (REEL 038041 / FRAME 0001);ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:044970/0030

Effective date: 20171222

AS Assignment

Owner name: JEFFERIES FINANCE LLC, NEW YORK

Free format text: SECURITY INTEREST;ASSIGNOR:RPX CORPORATION;REEL/FRAME:046486/0433

Effective date: 20180619

AS Assignment

Owner name: BARINGS FINANCE LLC, AS COLLATERAL AGENT, NORTH CAROLINA

Free format text: PATENT SECURITY AGREEMENT;ASSIGNORS:RPX CLEARINGHOUSE LLC;RPX CORPORATION;REEL/FRAME:054198/0029

Effective date: 20201023

Owner name: BARINGS FINANCE LLC, AS COLLATERAL AGENT, NORTH CAROLINA

Free format text: PATENT SECURITY AGREEMENT;ASSIGNORS:RPX CLEARINGHOUSE LLC;RPX CORPORATION;REEL/FRAME:054244/0566

Effective date: 20200823

AS Assignment

Owner name: RPX CORPORATION, CALIFORNIA

Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:JEFFERIES FINANCE LLC;REEL/FRAME:054486/0422

Effective date: 20201023

FEPP Fee payment procedure

Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

LAPS Lapse for failure to pay maintenance fees

Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STCH Information on status: patent discontinuation

Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362

FP Lapsed due to failure to pay maintenance fee

Effective date: 20220406