CA2253867A1

CA2253867A1 - Method and apparatus for coordinating internet multi-media content with telephone and audio communications

Info

Publication number: CA2253867A1
Application number: CA002253867A
Authority: CA
Inventors: Pasha Roberts; Francis A. Honore; Firdaus Bhathena
Original assignee: Individual
Current assignee: Cisco Systems Inc
Priority date: 1996-05-07
Filing date: 1997-05-07
Publication date: 1997-11-13
Also published as: US7725540B2; US8386567B2; EP0897631A2; US6754693B1; US20050004977A1; JP2000516053A; US20100296509A1; AU2935297A; WO1997042728A2; WO1997042728A3; US6295551B1; IL126909A0

Abstract

Methods and an apparatus are described whereby Internet content is coordinated with audio communications, such that two or more parties can view the same media content on the Internet while also simultaneously communicating over a traditional telephony network or by using voice over network implementations.
The visual interaction is achieved by having a user computer display shared content that correspond to a second computer's display, such that both parties will be viewing a copy containing the same content on each of their browsers.
The visual interaction further provides a method for allowing either of the parties to update the visual content of their browsers. In a further embodiment the resource collection functions of the user computer are enhanced by having the server only download to the user computer a user applet containing the information required by the user computer as is necessary. In this way, the user computer does not have to wait as long or dedicate resources to the visual interaction that are not necessary at the time. In a further embodiment this method works through firewalls on the network.

Description

CA 022~3867 1998-11-06 Title: Method and Apparatus for Coordinating Internet Multi-Media Content with Telephone and Audio Communications Background of the lnvention s The present invention relates generally to an apparatus and method 1'01 coordin~ting Internet communication between two users More particularly the invention relates to an apparatus and method for coordinatin~ Internet multi-media communication such that the two users can view and modif\! a copy of the same multi-media content. where anv modifications made b! on-I() user are subsequently viewed by a second user as well as enhancing the communication by supplementing any visual communication with audio communication.

Businesses commonly service customers through call centers staffed with technical support personnel. sales support personnel. or service personnel 1~ (hereinafter collectivelv referred to as sales representative) that can answer the customer s questions Typically. these centers use an automated call - distributor ( ACD") system to handle incominL~ calls from the telephonenetwork. Commerciallv available ACD systems provide call processing. voice , CA 022~3867 1998-11-06 W O 97/42728 -2- PCTrUS97/07740 processing, data processing and network capabilities for the call centers. The ACD system usually routes customer calls to a sales representative. The sales representative typically uses a headset and a user device connected to the ACD
system to respond to the incoming calls. The ACD system can route the calls 5 to queues, which can be customized to reflect how a business wants to prioritize and route incoming calls.

While the customer is on-hold, the ACD system also provides the customer with information. Typically. the ACD system will notify the customer that the customer's call is important, ''please hold"~ suggest that the o customer call back during non-peak hours, or inform the customer the time the customer can expect to be on-hold, for example. In such an implementatiom the customer has to wait on the telephone while listening to the repeated messages of the ACD system. This can frustrate the customer. It also restricts the customer from performing many other tasks while the customer is waiting s on the phone for the sales representative or technical specialist. In some implementations, the ACD systems pipe advertising to the customer while the customer is on-hold. The customen however~ may not be interested in the products or services that are being advertised, and may have further frustration from being subjected to the advertisements. Alternatively, the ACD system 20 can provides the customer with melodies during the on-hold period. In this implementation, however, the customer is still restricted from doing other tasks and the business looses an opportunity to advertise.

Another way businesses communicate with customers is via Internet web pages. Businesses elicit questions from customers by advertising the ~s business's products on Internet web pages. Such web pages allow the customer requesting that a sales representative of the business contact them. To request service, the customer inserts their name and e-mail address or phone number in CA 022~3867 1998-11-06 a space provided on the web page and activates a call-me button. Thereafter, the sales representative is sent an e-mail containing the request for service~ to which the sales representative responds. However! when the sales representative responds, either by phone or e-mail, the customer is most likely s no longer viewing the business's web page, and thus the business's web page cannot contribute to the sales representative's pitch. Moreover, the response isnot immediate, sometimes with log times measured in days, and during the interim the customer may buy another product.

An intermediate solution to answer a customer's questions regarding 1 o products is presented by allowing the customer to use or view a demonstration of the business's product while on the Internet. By way of example~ if a customer wants to buy software, the customer may go to the business's web site and request a software demonstration. The web site downloads to the customer the software demonstration. If all the customer's questions are 15 answered by viewing the demonstration. then the customer can then purchase the software. If the customer still has unanswered questions, however. the customer will again have to either place a telephone call or wait for a delaved e-mail or call response.

A further disadvantage of providing the customer a software o demonstration downloaded from the Internet is the amount of time it takes a customer to go to each web site, ascertain the product the customer wants to view, and then wait while the appropriate plug-in or code, for example. is downloaded to the customer's computer. If the customer only needs to view a part of the demonstration, downloading the entire code to a customer ~s unnecessarily uses resources of the customer. In addition, the time the customer waits for code to be downloaded is again wasted time~ just as was waiting on the phone. One solution to this problem is taught in US Patent No.

CA 022~3867 1998-11-06 5,572,643 to Judson which claims a method for displaying information on a graphic user interface when a customer is waiting for a link to download on the Internet. However, this is akin to the making the customer listen to unwanted advertisements on the phone and, thus, still has the previously described 5 problems.

A further disadvantage of downloading a software demonstration is that whatever is downloaded is viewed by the customer alone without the assistance of the sales representative. The software demonstrations are not usually concurrent-use software applications that can be viewed simultaneously at ] o remote locations. Thus~ the software applications cannot be viewed by both the customer and the sales representative simultaneously. Further, if they were~
this would require even longer download periods. In addition, such applications would require a large amount of the customer's resources to run and be difficult for the customer to manipulate.

CA 022~3867 1998-11-06 wO 97/42728 5 PCT/US97/07740 Summary - The aforementioned and other objects are achieved by the invention which provides an apparatus for coor(lin~ing Internet communication between at least two users and a method associated therewith. The apparatus provides a visual 5 communication across a network between a user computer and a second computer which is in communication with a server disposed between the user computer and the second computer.

The method facilitates visual communication across the network that is platform independent. As such, neither the user computer nor the second 0 computer is restricted to using any particular operating system or software application. The only requirement of the user computer and the second computer is that it has a browser-like application for reading files. In the preferred embodiment~ the browser is a hyper-text markup language ("HTML~') browser.

To provide the visual communication between the user computer and the second computer, a link is established between the user computer and the server and a link is established between the server and the second computer.
The server, the user computer~ and the second computer can be fully distributed across the network.

ln addition to being part of the link that is established~ the server selectively transmits to the user computer a user applet. The user applet ~ enables the user computer to communicate with the server. The second computer either has a service applet disposed thereon or the service applet is provided to the second computer by the server. The service applet and the user applet enable the visual communication to occur without requiring either the user computer or the second computer to have any executable files pre-installed ~ .

CA 022~3867 1998-11-06 W O 97/42728 -6- PCT~US97/07740 to enable the user computer or the second computer to have a shared view.
Further, the user applet and the server applet bi-directionally communicate withthe server, and thus can bi-directionally communicate with each other through the server.

After the user computer has received the user applet, the user computer generates a user view. Either in parallel or thereafter, the server communicatesthat the link is established between the server and the user computer to the service applet. Thereafter, a session is selectively established between the user computer and the second computer.

I o During the session, the second computer and user computer both display on the browser of their computer~ respectively~ a shared view in their user interfaces. The shared view allows a user of the user computer to see a copy of the same content as is displayed on the second computer, and vise versa. Thus. both users of the user computer and second computer will be s looking at the same content displayed on the browser.

The user interface of the user computer is called a user view. while the user interface of the second computer is called a service view. The service view contains a representation of the user view~ the representation of the user vie~ is the shared view.

~o Either the user computer or the second computer can manipulate the shared view. For example. the second computer can change the shared view from content at one universal resource locator ("URL") source location to content at a second URL source location. If the second computer implements a change to the shared view, the change is communicated to the service applet.
~s The service applet transmits the change to the server, and the server subsequently transmits the chan~es to the user applet. The user applet then CA 022~3867 1998-11-06 implements the change in the user view, and notifies the server that the change has been implemented. After the service applet receives notification from the server that the user view has changed, the service applet changes the representation of the user view in the service view, thereby conforming the 5 shared view of the second computer to match the shared view of the user computer. This process enables the second computer to communicate to the user computer. The process can be performed bi-directionally, such that the user computer can change the user view and cause a conforming change in the second computer.

o Various implementations of this method can be enabled by the user computer or the second computer. One implementation allows changes to the URL source location of the shared view. Another implementation allows changes to the images in the shared view. A further implementation of this embodiment of the invention allows the shared view to be a software application. A still further embodiment allows the shared view to display software demonstration programs.

An additional implementation of this embodiment is an imeractive use of' forms, where each form has objects. In this implementation, the form is represented as an array having arguments where the arguments represent the objects of the form. As the user computer or the second computer changes entries on the form~ the user applet or the service applet, respectively~ updates the objects in the array such that the array now represents any changes in the user interface. Then as previously described the changes are implemented in the shared view of both the user computer and the second computer.

2s A further implementation for the interactive use of forms supplements theforms to be event monitoring, such that when the objects on the forms change . .

CA 022~3867 1998-11-06 W 097/42728 -8- PCTrUS97/07740 the objects notify the applets of the change. A still further implementation rewrites the objects as applets. Again, the changes to the objects are implemented in the shared view.

To enable the visual communication, the server must download the user s applet to the user computer. In one embodiment of the present invention, the server enhances a resource collection function of the user's computer by only downloading to the user computer a user applet representative of the functionality the user computer needs at that time. As such, the server ascertains what the user computer needs and generates an applet representative o of that functionality and downloads that applet to the user computer. If later on during the visual communication the user computer requires greater functionality, the server will then download a second user applet containing the necessary functionality. In this way, the server decreases the resource collection of the user computer in both time and space while enabling the 15 functions of the resource that the user wishes to enable.

The user applet is further enhanced by the use of a script. The script is displayed on the user interface of the second computer~ prel'erably ad jacent to the representation of the user view. The display can be a list of scripts trom which the second computer can choose or the script that the user of the second 20 computer has selected. The script is in communication with the service applet.
The script has simplified commands which it transmits to the service applet to enable the service applet to implement any changes in shared view and transmit the changes to the user applet as previously described. In the preferred embodiment the script is written in hyper-text markup language ("HTML").

The visual communication can be between more than one user computer and more than one second computer. This enables many-to-one~ one-to-many.

CA 022~3867 1998-11-06 W O 97142728 9 PCTrUS97/07740 or many-to-many communication between at least one user computer and at least one second computer. Each of the more than one second computer can be identical or each of the more than one second computer can have differing service applets which are adapted to perform differing functions, such as 5 A~lministrative, sales, or technical support.

If the embodiment has more than one second computer, the server also has more than one queue to which it routes call requests from the user computers.
A queue contains one or more call requests from the user computers. The queue is further displayed selectively on one or more second computers. The I o server routes call requests using various methods, such as a first-come~ first-serve basis~ according to attributes of the user computers, sales specialty of the second computers. or logic formulas, which may comprise formulas r elate to time of calh number of call requests presently in queue~ among other items.
The attributes can be collected by prompting to the user computer, or the user ] s applet can extract the attributes from the user computer.

U1hen the call request of the user computer is placed in a queue~ the server notifies the user applet regarding status of the of the queue user computer's call request. The server will display to the use~ computer in its user interface the status of the queue~ enabling the user computer to know when his 20 call request will likely be answered.

In one embodiment, while an entry of the user computer is in the queue.
the server through the user applet displays supplemental information which is analogous to MUZAK~ a trademark of Muzak, Inc., on the user computer until the user computer's call request is answered. In one embodiment~ the 25 computer supplemental information is selectively transmitted from the server based upon the attributes of the user computer. In another embodiment. the CA 022~3867 1998-11-06 W 0 97142728 lo PCTrUS97/07740 server transmits the computer supplemental information based upon scripts.
While the user computer is being displayed the computer supplemental information~ the user computer also has a control view generated by the user applet that enables the user computer options such as disconnect or call me 5 later, amon~ other options.

In still another embodiment, the visual communication across the network is married to audio communication between the user computer and the second computer. In one implementation of this embodiment, both the user computer and the second computer have a voice-over net capability that is o enabled during the session between user computer and second computer. ln another embodiment. a traditional telephony system is used to implement the audio communication during the session. ln this embodiment one audio device will be associated with at least one second computer and at least one user computer. The audio device can either initiated the communication or be used ] 5 after the initiation of the visual communication between the user computer and the second computer. As such, the users of the user computer and the second computer can view shared content over the network as well as audibly communicate.

In the implementation where the audio communication is established first, 20 the second computer directs the user computer to an appropriate address, or URL source location whereupon the user computer is linked to the server and provides a user applet as previously described. In this embodiment. the user computer can also selectively be provided a password to enter the call session.
thus enabling the call session to be private.

~5 In still another embodiment. the second computer can be a component of a call center where the call center has an automated call distribution system .

CA 022~3867 1998-11-06 WO 97/42728 1 l PCT/US97/07740 ("ACD"). In this embodiment, the ACD also has a call queue. The call queue can be communicated to the server via the network if the ACD is also connected to the network, and as such the call queue of the ACD can work in tandem with or complement queue of the server. In this embodiment. the call 5 center can utilize any additional capabilities of the ACD while supplementing the ACD with visual communication enabled by the server, the user applet~
and the service applet.

In another embodiment, the communication between the server and the user computer can be conducted through a firewall using tunneling.

lo In further aspects~ the invention provides methods in accord with the apparatus described above. The aforementioned and other aspects of the invention are evident in the drawings and in the description that follows.

.. .. _ CA 022~3867 1998-11-06 Brief Description of the Drawings The foregoing and other objects of this invention, the various features thereof, as well as the invention itself, may be more fully understood from the following description, when read together with the accompanying drawings in s which:

Figure 1 shows a system diagram of one embodiment of this invention;

Figure 2 shows a block diagram of the system of Figure l;

Figure 3 shows a user interface in accordance with one embodiment of this invention;

Figure 4A shows a second user interface in accordance with one embodiment of this invention;

Figure 4B shows a third user interface of another embodiment of this invention;

Figure 5 shows a system diagram having more than one second computer 5 in accordance with another embodiment of this invention;

Figure 6 shows a system diagram of yet another embodiment of this invention;

Figure 7A shows a flow chart of one implementation of how a user computer enters a call session in accordance with this invention; and Figure 7B is a flow chart of a second implementation of how a user computer enters a call session in accordance with an embodiment of this mventlon.

CA 022~3867 1998-11-06 W O 97/42728 _13_ PCTrUS97/07740 Detailed Description - While the present invention retains utility within a variety of networks and may be embodied in several different forms, it is advantageously employed in connection with the Internet. Although this is the form of the preferred 5 embodiment and will be described as such, this embodiment should be considered illustrated and not restricted.

Figure l shows a system 10 according to one embodiment of the present invention. As shown, the system l O includes a user computer 12 in communication with a network 16 and a second computer 24 in communication with the network 16. The system 10 further comprises a server 20 interposed between the user computer 12 and the second computer 24 also in communication with the network 16.

In the preferred embodiment, network 16 is the Internet. As such, communication there over is via Transfer Control Protocol/lnternet Protocol ("TCP\IP"?. It should be obvious to those skilled in the art. however, that network 16 could be an intranet or any network capable of supporting commul1ication protocols.

Similarly~ the system 10 can be partially distributed, as opposed to the fully distributed depiction of Figure 1. For instance, the server 20 and the ~o second computer 24 could be connected to a local area network ("LAN") or a wide area network ("WAN") which~ in turn, is connected to network 16.

The user computer 24 and the second computer 24 can be a personal computer, workstation, terminal~ laptop or any device capable of displayin(l a user interface and storing an applet. The user computer 12 and the second computer 24, however~ must have a browser 18, 28~ respectively~ where a CA 022~3867 1998-11-06 W 097/42728 ~4 PCTrUS97/07740 browser shall be defined herein as any application software or firmware that can interpret files to create a view thereon, the files include for example, HTML, Javascript. An example of such browser includes, for example, any common Internet browser~ inter alia, and a Java workstation, inter alia. As s described hereinafter in more detail with reference to Figures 7A and 7B, the user computer 12 and the second computer 24 access the server 20 through the browsers 18, 28, respectively. The browsers 18, 28 are displayed on the user computer 12 and the second computer 24 via a display 14, 26, which is a cathode ray tube or a liquid crystal display, for example.

0 Once the user computer 12 accesses the server 20 utilizing the browser 18, the server 20 transmits to the user computer 12 a user applet 22. The user applet 22 is adapted to enable the user computer 12 to communicate through the network 16 with the server 20. A service applet 30 is disposed on the second computer 24 to enable the second computer 24 to communicate with the server 20. The service applet 30 can either be transmitted to the second computer 24 by the server 20 or be disposed thereon. After the user computer 12 receives the user applet 22, the second computer 24 is notified that a link is established~ and that a session can be established between the user computer 12 and the second computer 24. When the session is established~ users of the user computer 12 and the second computer 24 can visually communicate.

The users of the user computer 12 and the second computer 24 visually communicate by sharing content on their respective displays 14~ 26. More particularly~ the browser 18 displays a user interface containing a user view onthe user computer 12, while the browser 28 displays a user interface on the second computer 24. When a session is enabled~ the user interface of the second computer 24 contains a representation of the user view that is displayed on the user computer 12. Thus during a session~ the user computer 12 and the CA 022~3867 1998-11-06 W O 97/42728 -15- PCT~US97/07740 second computer 24 are viewing a copy of the same user view on their respective browsers 18, 28. The user computer 12 and the second computer 24 are then viewing shared content.

During the session, either party can manipulate the shared content s displayed on the party's computer. When the shared content is manipulated~
the applet disposed on that party's computer will communicate to the server 20 through the network 16 that there has been a change to the shared content. In the preferred embodiment, the applets 22, 30 selectively poll the server 20 during a session to ascertain whether an event has occurred~ such as a cllange h~
u the shared content. This is facilitated by designating each session and each applet in the session with a unique identifier, as further described with reference to Figures 7A and 7B. When an applet polls the server 20 having the proper session identifier, the server 20 communicates to the applet that an event has occurred. That applet then communicates with the browser of its respective I s computer and causes the browser to display the any changes in the shared content. Once the browser of the unchanged computer displays what was communicated to it by the server 20~ both parties will a~ain be lookin~ at the same shared content on their respective browsers. This process continues bacl~
and forth such that either party may lead the other to view, for instance. another 20 web page or a software demonstration, among other items hereinafter discussed.

ln another implementation, the server 20, instead of the applet~ can retrieve the changes to the shared content and then transmit the changes to the applets in the session. In this implementation, the server 20 acts as a prox~v server, the ?5 function of a proxy server is well known in the art.

CA 022~3867 1998-11-06 W 097142728 -]6- PCTrUS97/07740 In the preferred embodiment, after a user of the second computer 24 elects to alter the shared content, the service applet 30 waits for a response from the server 20 before updating the shared content on the second computer. It waits for a response from the server 20 that it has communicated the user applet 22 5 and has changed the shared content on the user computer 12. Once the server 20 receives communication back from the user applet 22 that it has changed the shared content on the browser of the user computer 12, the server notifies the service applet 30 of the second computer 24. At this time~ service applet 22 of the second computer 24 updates the representation of the shared content 52 of 0 the second computer's interf'ace. In this manner, the user interface of the second computer 24 will not get ahead of the user computer 12. It should also be noted that each computer 12, 24 are retrieving the shared content 52 independently and, in this instance. not sending the shared content 5' t'rom one computer to another.

It should be obvious to those skilled in the art that the server 20 can inform the applets 22, 30 in each session of any changes without requiring the applets 22~ 30 to poll the server 20 without departing from the scope of this invention. Further, the applets 22, 30 can work in conjunctioll with the server 30 such that any combination of polls or pushes results in the applets '2, 30 ~o being notified that an event has occurred. Moreover, polling can be replaced with event driven technology, or similar notification technologies~ as the event driven technology becomes more acceptable on the Internet.

In the preferred embodiment, the server 20 is a JAVA server utilizing JAVA objects. JAVA is a registered trademark of Sun Microsystems. Inc. The user applet 22 and the service applet 30 communicate with the objects disposed upon the server 20. In the preferred embodiment. the applets 34 are also JAVA
applets. This enables the system 10 to be a cross-platform system and as such CA 022~3867 1998-11-06 W 097/42728 -17- PC~US97/07740 does not need any executable files previously disposed on the user computer 12 or any of the computers involved in the session. The JAVA server can run on a UNIX computer, a MACINTOS~ computer, MS Windows computer, a Silicon Graphics computer, or any other well-known computers in the art.
UNIX is a registered trademark of AT&T Corp. MACINTOSH is a registered trademark of Apple, Inc.

The user applet 22 performs event monitoring and communication as was described in part above. The user applet 22 also has a model controller view.
The model controller view of the user applet 22 is the user view which is further described with reference to Figures 4A and 4B. The user applet 22 also shares much of the object model of the system 10, unlike a typical client. The user applet 22 having a portion of the object model allows the system 10 to .
functlon when lt IS fully dlstrlbuted. T hls Implementatlon makes lt easy to replace the polling mechanism with distributed object communication technologies includin~, for example~ Remote Method Invocation ("RMI") Javascript~ Sun. Common Object Request Broker Architeclure ("COBRA"j.
lnternet Inter-Orb Protocol ("IIOP") and Distributed Component Object Model ("DCOM")~ ~) Microsoft. For example, the system 10 can manipulate an object on the user computer 12 and have the user computer 12 transmit through the user applet 22 to the server 20 a response, both while the user computer 12 is unaware that the communication or manipulation of the object has occurred.
This implementation, among other things~ enables the server 20 to extract from the user computer 12 attributes of the user computer 12 where the attributes caninclude items such as the user computer's e-mail address~ the user computer's history of links on the network 16. and the user computer s specifications among other items.

. _ ~ . . .

CA 022~3867 1998-11-06 W O 97/42728 -18- PCT~US97/07740 In the preferred embodiment, the user applet 22 is a persistent applet. The user applet 22 is loaded into a cache (not shown) of the user computer 12, preferably a local cache. It is persistent because once the user applet 22 is downloaded from the server 20? the user applet 22 remains on the user s computer 12 as long as the user computer 12 remains in the session with the server 20, unless it gets removed from the local cache by the user computer 12 as is well known in the art. However, if it is not removed~ the persistent applet will remain disposed on the user computer 12 such that it will not have to be downloaded again from the server 20. The user applet 22 will remain in the I o cache regardless of any changes to the user interface of the user computer 12.

The user applet 22 is selectively generated by the server 20 to enhance a resource collection function of the user computer 12 while enabling the user computer 12 to participate in the visual communication. To enhance the resource collection function of the user computer 12, the server 20 evaluates 5 the functions of a resource, such as a spreadsheet application for example~ that the user computer 12 requires at any given point in time. After the server ''0 evaluates these functions~ it then generates the user applet 22 representative ol' the functions. Once the server 20 generates the user applet 72 representative ot' these functions, it transmits the user applet 22 to the user computer 1~ as 20 previously described.

Typically, the functions represented in the user applet 22 are only a subset of the functionality of the resource. This is analogous to using EXCEL, a trademark of Microsoft Corp., spreadsheet and only getting the graphing f'eature when one tries to use it, for example. By only including a subset of the 2s functionality as required by the user computer 12~ the time for downloading between the server 20 and the user computer 12 is decreased. Further~ the amount of storage space required in the user computer 12 is also decreased.

CA 022~3867 1998-11-06 wo 97/42728 1 9 PCT/USg7/07740 Thus, the server 20 by selectively generating the user applet 22 enhances the resource collection function of the user computer l 2. It should be obvious to those skilled in the art that the server 20 can, in real-time or pursuant to a timer, continually download to the user computer l 2 any necessary functionality in an s applet 22. The user computer 12 can also trigger the server 20 to inform it that it needs to be given an additional applet. In the system 10, the user computer l 2 receives what it needs as it needs it. Thus, the user computer l 2 is optimized as a very thin client in the thin client fat server model.

For example, if a user computer l 2 enters a session to share a web page as lo further described hereinafter~ the server 20 only downloads functionality to the user computer l 2 necessary for the browser l 8 to be notified about, and to display the web page. If, for example, the user computer l 2 is sharing a software application, then the server 20 will download to the user computer l 2 another applet in real time of the next piece of functionality the user computerI s l 2 needs to implement the software application.

It should be obvious to those skilled in the art that the selective generation of an applet and subsequent downloading of that applet can be used for applications outside of the system l 0 without departing from the scope of this invention.

o Turning now to the service applet 30, the service applet 30 can comprise one or more applets each adapted to provide the second computer ~4 with different functionality. The service applet 30 at minimum must provide the second computer 24 with the ability to communicate with the server 20.
Again~ the server 20 can download the service applet 30 in real time to the 2s second computer 24 during the session. The service applet 30 shares much of the objecl model with the user applet 22, as previously described. tO enable CA 022~3867 1998-11-06 implementation of the system 10 in a distributed environment. One implementation uses a proprietary interface definition language to communicate session information between applets 22, 30 and the server 20 based on a polling model, while another is based on event notification. Thus.
the service applet 30 and the user applet 22 enable the user computer 12 and the second computer 24 to bi-directionally communicate over the network 16.

The visual communication of system 10 is further enhanced by audio communication between the users of the user computer 12 and the second computer 24. In one implementation, the user computer 12 connects to the server 20, receives the user applet '2, and the service applet 30 is notified of a call request from the user computer 12. Upon notification of the request~ the second computer 24 implements audio communication either through voice over network ("VON'') applications or a traditional telephony system (not shown). At this point. the user computer 12 and the second computer 24 are in s communication both audibly and visually.

The visual communication can be further described with reference to Figure 2, which shows a block diagram of the system of Figure I . As was previously described, the server 20 transmits through the network 16 applets 34. The applets include the user applet 22, the service applet 30, any other applet necessary to give the user computer 12 or the second computer 24 functionality.

The server 20 is capable of ascertaining~ as hereinafter described~ what type of computer is requesting a download of one of the applets 34. This is achieved by having participants, such as the user computer and the second computer, in a session use different sign-on pages andlor by analyzing an information payload sent by the browsers in an HTTP sign-on request. As CA 022~3867 1998-11-06 W O 97/4272X -2]- PCT~US97/07740 such, if the server 20 identifies that the requester is the user computer 12 then it downloads the user applet 22 capable of generating a user view 90 and the functionality the user computer 12 requires as previously described. If, on the other hand~ the server 20 ascertains that the requester is a sales representative~
5 then it downloads a service applet 30 capable of generating a sales view 50.
Similarly, if the server 20 ascertains that the requester is a system administrator, the server 20 downloads an applet capable of generating an lmini.~trator view 36.

The server 20 ascertains and validates which type of user is requesting one 0 of the applets 34 by various methods. One such method allows the server 20 to validate each of the computers using a password system. In this implementation, any requester can get any view 36, 50, 90 as long as it has the appropriate validation codes such as a name and a password. Additionall~
with the appropriate validation codes a requester can log-on from anywhere in 1~ tl-e world that is connected to the network 16. Thus~ others can join in a current session simply by knowing the validation codes.

In the preferred embodiment~ the sales representative and the administrator have predetermined passwords stored in a database (not shown) ~n communicating with the server 20. The database can be disposed on the server 20 or remotely therefrom. The server 20 communicates with the database through a database interface that uses Java Database Connectivity ("JDBC") or other Object Database Connectivity ("ODBC") equivalent database languages~
for example In the preferred embodiment the database is a relational database.
although it can be an object database or other forms of data storage w ell knownin the art.

CA 022~3867 1998-11-06 As such~ when a requester log-on to the server 20, the server 20 checks the database to see whether the requester has a valid log-on and tlle associated applets 34 required by the requester. Once the server 20 validates the log-on, then as described, the server 20 will download one of the applets 34. In this 5 way, the ~lministrator can function as a sales representative having at one time a service applet 30 downloaded to it via the server 20 and at another time have an ~(lministrator applet downloaded to it via the server 20.

The user computer 12 communicates with the server 20 through a web page. The user computer 12 either requests a call through the web page or o enters a password on the web page that was provided to the user computer by the sales representative, as described further with reference to Figures 7A and 7B. Thus, the server 20 can uniquely identify that a user computer 12 is requesting a user applet to generate the user view 90 by the method of communication with the server 20.

I S Each of the applets 34 allows the browser of the respective computer to generate the associated views 36. 50, 90. In a session. the shared content is displayed in the user view 90 of the user computer 12 by the browser 18. while the shared content is displayed on the second computer 24 within the saies view 50. The user interface of the second computer 24 or the sales view 50 20 contains a representation of the user view 90.

When the shared content in the user view 90 or the representation of tlle user view is manipulated by one party, through communication among the applets 34 and the server 20 the shared content of the other party s view will be updated. For example, suppose the sales view 50 contains an HTML file 2s located at the URL source location www.webline.com. The user view 90 will also contain a copy of the HTML file with the URL source location CA 022~3867 1998-11-06 W O 97/42728 -23- PCTrUS97/07740 www.webline.com. If the second computer 24 wishes to direct the user computer 12 to the service s page of the Webline Corporation located at www.webline_services.com, then the second computer 24 can change its sales view 90 such that the URL source location of the shared view is 5 www.webline_services.com. Once the second computer 24 changes the sales view 50 by methods such as the script as described hereinafter, or just by typing in the URL source location, the URL source location of that page is then transmitted by the service applet 30 to the server 20. The server 20 then tells every other applet that is part of the session that there is a new URL source 10 location. In this example, the server 20 will tell the user applet ~2 that there is a new URL source location. After the user applet 22 receives information. it then causes the browser 18 of the user computer 12 to pull down a new page from the network 16 and display on the user view 90 a copy of the HTML file at www.webline_services.com. The shared content can include web pages.
files, application images~ advertisements, interactive forms data~ or application data among any other f'orrn of data that can be captured and displayed on the browser.

In addition to the sales view 50 and user view 90, other views can also be implemented by the applet such as an ~-lmini.~trative view 36~ for example.
~o The :~(lministrative view 36 displays information such as the structure of the sales force~ reports~ real-time monitoring of the communications~ queue structure and status, which wil] be further described hereinafter, for example.
It should obvious to one skilled in the art that the ~mini.~tration view 36. thesales view 50, and the customer view 90 can be disposed on one computer or ~5 more than one computer.

Further any computer displaying the views 50, 90, 36 require a particular operating system 38 to accept the applet and share content in the preferred CA 022~3867 1998-11-06 embodiment. As such, they can be Windows-based, Macintosh-based, or UNIX-based, among others. Further, the browsers 40 can be any number of types, such as Netscape Navigator~ Microsoft IE, Sun Hot Java, for example.
Further, the browsers 18, 28 do not have to be the same on the user computer s 12 and the second computer 24.

Turning now to Figure 3, which shows at least one user interface that in the preferred embodiment is the sales view 50. In addition to a representation of the user view 52 that contains the shared content, the user interface also contains a script screen 54.

lo The script screen 54 can display a plurality of scripts 58 or just one of the plurality scripts that is activated in the user interface. The second computer 24 can have predetermined scripts 58 illustrated on script screen 54 from which to choose, where each of the scripts 58 may be tailored to assist a particular typeof user computer 12 or second computer 24. For example the scripts 58 could 5 include: information regarding a company~ a state of the market script features of products. benefits! architecture or any set of information limited only by the im~gin~tion of the user. In the preferred embodiment, the scripts 58 are writtenin HTML so that the scripts 58 are displayed in the browser.

The scripts 58 dictate the shared content contained in the representation of 20 the user view 52. The scripts 58 perform this function by having links that push the shared content into the representation of the user view 52. which. as previously described, is also pushed into the user view of the user computer 1''.
The scripts 58 perform this function by communicating with the service applet 30. As the service applet in the preferred embodiment is a Java applet the ~5 script communicates with the service applet 30 using Javascript-to-Java communication in which a Javascript command can invoke a public method in ~ . .

CA 022~3867 1998-11-06 W O 97/42728 -25- ~CT~US97/07740 a Java applet. It should be obvious to one skilled in the art that other embodiments are possible. Similarly, in another implementation the scripts 58 can communicate directly with the server 20, instead of the service applet 30.

The scripts 58 control what is displayed in a section of the browser s adjacent to the scripts. As such, the scripts 58 update the browsers in a manner different than that is currently done for HTML files.

An example of a script in the preferred embodiment is as follows:

<html>
0 <head>
<title>Web.Dialog Sales Screen</title>
</head>
<body>
 Web.Dialog Sales Screen ~/font>
 
 Web.Dialog version 1.0 Beta I 
<hr>

<ul>
<li><a href= '~javascript:top.salesrepFrame.document.SalesRepApplet.setAddress ('http://www.webline.com') ">Share www.webline.com </a></li>
 'br>
<li><a ~5 href=''javascript:top.salesrepFrame.document.SalesRepApplet.setAddress ('http://www.microsoft.com') ~'>Share w~,vw.microsoft.com ~/a>~/li.
 
<li><a href= ' javascript:top.salesrepFrame.document.SalesRepApplet.setAddress ('http://~,vww.netscape.com') ">Share www.netscape.com </a>~/li>
</ul>
~/font>
</body>
</html>
In this example. the script is an HTML Java script, which contains Javascript to communicate with the Java applet. The script is displayed in the top sales representative frame of the user interface~ where the top saies CA 022~3867 1998-11-06 W O 97/42728 -26- PCTrUS97/OM40 representative frame is the script frame 54. In the code href=Javascript:top.salesrepFrame.document.SalesRepApplet.setAddress.
setAddress is a public method in the Java applet, which takes as an argument a URL source location, that instructs the applet to share the URL source location 5 between all the participants in the session, being the user computer 12 and the second computer 24 in one embodiment. This particular script, if enabled by the second computer 24, will allow the second computer 24 push into the shared content the HTML file having a URL source location http://www.webline.com~ http://www.microsoft.com or I o http://wwvv .netscape.com.

For example~ if the user of the second computer 24 wants to push http://www.webline.com into the shared content. In this example, the service applet 30 will receive from the script that the second computer 24 wants the shared content to display the file having the URL source location 5 http://www.webline.com. The service applet 30 will then waits for a response from the server 20 that the server has verified that the user applet 22 has instructed the browser of the user computer 12 to download from the network 16 the file having the URL source location http://www.webline.com Once the service applet 30 is notified that the server 20 has received the address of the20 shared content, the service applet 30 of the second computer 24 updates the representation of the shared content 52 of the second computer's interf'ace. andthe user applet 22 would receive the address of the shared content and update the shared content. The process would be repeated if the second compuler 24 then selected to push the www.microsoft.com file into the shared content.

It should be obvious to those skilled in the art that scripts 58 can be used to push all the forms of shared content herein discussed to the user computer CA 022~3867 1998-11-06 W O 97/42728 -27- PCT~US97/07740 including~ scripts 58 that initiate a file transfer to download informatioll to the user view from a remote location, for example.

As the scripts 58 are in HTML, users can easily write them. Scripts S8 can even be written interactively by allowing the user of the second computer 24 to s insert the arguments from the script by simply going from one page to the nextwhile having the script translate URl of that web page and insert it as an argument in its script. The scripts 58 can also be dynamically defined to push shared content according to attributes of the user computer 12 and/or attributesof the second computer 24. As such, the scripts 58 can be dynamically defined I o by the server 20 that access the attributes.

In addition, the second computer 24 can dynamically generate scripts during the session to meet any requirements of the session. For example, the second computer 24 can create a script to push the product comparison into the shared content in response to a request of the user computer 12 to see a I s product comparison. For instance, the second computer 24 can search for information regarding products, by keywords. URL source locations, for example. After selecting the information the second computer 24 wants to push into the shared content and thus the content of the script, the second computer 24 can activate the script so generated. This seJection can be 20 implemented by default or by allowing the second computer to click a generatescript button or an accept button displayed in a session box 62~ for example.
Again, this would allow the user computer 12 to view the product comparison.
Any comparison information can be displayed to the user computer 12 in this manner. Similarly, presentations regarding other information can be conducted ~s by dynan1ically generating scripts and then pushing the shared content to the user computer 12.

CA 022~3867 1998-11-06 W O 97/42728 -28- PCTrUS97107740 Adjacent to the script screen 54 and disposed below it is a user information screen 56. The user information screen 56 contains queue information 60 and a session box 62. The session box 62 contains information about the session in which the second computer 24 is engaged in currently. The session information 5 can include items, f'or example, duration of the session, address currently being displayed in the representation of the shared content 52, any password associated with the session, status of the session, ACD interaction information, customer information and control functions such as connect 64, hang-up 66 pause~ new view, log-off, help, among others.

]o The session box 62 augments the scripts 58 by also providing a URL box 67. Instead of using a script link, the second computer 24 can insert any URL
into the URL box 67~ enable the connect 66 button, and push the shared content to the user computer as previously described.

The second computer 24 is also part of a queue of the server 20. and as such status ofthe queue information 60 is displayed on the user interface 50.
The system 10 has at least one queue, where each queue contains one or more call reguests from user computers. In the simplest implementation~ when a user computer 12 links to the server 20, the server 20 places a call request from the user computer 12 in the queue in the order the server 20 received the request.

The call request can also be routed by the server 20 into queues based upon attributes of the user computer 12, where attributes of the user computer 12 can be an e-mail address. HTMI, pages that the user computer 12 was viewing when it made the call request or interests, or database entries for a 2s user~ for example. The server 20 selectively places a call request of the user computer 12 by parsing the attributes into one or more of the queues. For CA 022~3867 1998-11-06 example, if the user attribute is that the last page the user computer 12 viewedadvertised laptops, the server could route the call request to a queue for laptop computers. The service applet 30 would then poll the server 20, collect the queue information and thereafter display it on the user interface 50 if the s second computer 24 was assigned to the laptop queue.

The assignments of the second computers are stored on the database disposed on the server 20, as well as information regarding the system, SUCil ashow long does it usually take for each call to be answered, for example. The database also contains information collected by the server 20 such as how I o many second computers are currently logged-on~ how many second computers are in each queue, how many call requests are already in each queue~ whethe any second computers 24 are on breaks and how long or short breaks, and whether more than one second computer 24 should be signed off.

In one embodiment, the server 20 utilizes the database information and 5 logic rules in association with the attributes of the user computer to route the calls to the queues. For instance, the server 20 could route a user computer to a secondary laptop queue, perhaps one with less experienced sales representatives. when the primary laptop queue had a predetermined number oi' call requests. In this example the server 20 could also download an ~o ~mini.~trative applet to the administrative view to update the queues information 60 displayed on that view, or the server could cause a beeper to ring~ for example. The server 20 can also compare the attributes~ such as name~
against a database of customers~ to enable priority customers to be routed accordingly.

The attributes could be collected from the user computer 12 using various methods. First~ the user computer 12 could be prompted to enter the CA 022~3867 1998-11-06 W 097/42728 _30 ~CT~US97/07740 information directly, or by choosing from a drop down list of predefined attributes, such as novice user, experienced user, for example. In still anotherembodiment, the attributes can be extracted from the user computer 12 The service applet 30 retrieves the queue information to display using an s interface language that includes commands, for example, such as, for example, tell me how many other are call requests are in the queue. It should be obvious to those skilled in the service applet 30 can implement polling or distributed object communication technologies without departing from the scope of this invention. The service applet 30 generates the queue information hox 60, I O unlike the user applet 22.

A user computer 12 receives status regarding its position in the queue through the server 20, as well as receiving supplemental information, where the supplemental information can be analogized to MUSAK, a trademark of Musak, Inc. The supplemental information can be general information or 15 specialized information displayed for the user computer 12. The server 20 ascertains which specialized information to display to the user computer l 2 using the attribute information of the user computer 12. For example, if the attribute of the user computer 12 is laptops, the supplemental information displayed to the user computer 12 while in the queue can be advertisements.
20 accessories or specifications laptops, for example. among other information that may be of interest to a user who wants to purchase a laptop. The server 20 implements displaying the supplemental inforrnation by communicating with the user applet 22.

After the session is established and the user is out of the queue, the user 2s view 90 will display the shared content 52. ln addition to the sharing a file as previously described. the user computer 12 and the second computer 24 can ~ . .

CA 022~3867 1998-11-06 W O 97/42728 _31_ PCT~US97/07740 share a demonstration To share a demonstration? either the user computer 1 or the second computer 24 must have the application capable of running tlle demonstration on the computer, in the prel'erred embodiment the second computer 24 has the application while the user computer 12 only has a s browser. As the software demonstration runs, the second computer ~4 captures an image of the demonstration, like a print screen, and transmits the captured image to the user applet 22 The captured image is saved as a jpeg, gif, bmp file or other graphical utility. transmitted to the server 20 or the user computer 12 and displayed on the user computer 12.

I o The user computer 12 and the second computer 24 can also share a form The form has a URL source location 70 and a place to enter user information 74 and some text 78. In the preferred embodiment. the form can be any ~TML
page The service applet 30 parses the form and stores each of the entries hl the form as objects. such that the URL source location 70 will be an obJect the text 78 would be an object and user information 74 would be an object. for example In one implementation. the service applet 30 stores the objects as a array In this example. it would be a two dimensional array of three objects Either the objects that have been chan~ed can be transmitted by service applet 30 to the server 20, or the array containing all the objects can be transmitted by ~o the service applet 30 to the server 20, whereupon the user applet 22 would parse the array to ascertain which objects had changed.

If the sales representative had the user enter the text s~ring 586SX into the user informalion 74 when the user was purchasin~ software f'or example. wlle the user hit enter, the user applet 30 would be informed that one of the ob jects of the form in its array had been changed. particularlv that the array displayin~
the user inf'ormation would now contain a text string. The user applet 30 would communicate this to the server 20 which would then let aJI the applets i CA 022~3867 1998-11-06 W O 97/42728 -32- PCT~US97/07740 the session. in this case the service applet 22. The service applet 22 would then evaluate each object in the array and ascertain which object had changed. After identifyin~ that the user input 74 had changed to now include a text strincg 586SX it would update the shared content.

Figure 4A shows the user view used in the previous example. where the user view 90 having the shared view 92. The shared view 92 is essentially a child browser spawned by the user applet 22 to display HTML files. The shared view 92 in this example has the URL 96, the text 98~ the user information 100 and the text string 586SX 102. In this manner~ either the user o computer I ' or the second computer 24 can interactively enter information into the form that will be displayed to the other party in a real time as previously described. Thus, instead of having the second computer 24 merely push the appropriate form to the user computer 12, the user could complete the form while in the session and the sales representative could audibly- communicate to the user whether the user was properly or improperly completin~ the form.

In the previous example the applet was informed that there was an update of the object by either the user computer 1~ or the second computer 24 hittin~T
enter. This is not necessary for other entries on the form such as a check the box entry. For such entries~ the user applet 22 could simply have two values true and false represented by zeros and ones. Each time a check was formed in the box. the user applet 22 could inform the server 20 of the chan~e m the object. The server 20 would then inform all of the applets in the session that there has been a change and appropriately update the representation of the shared content to include the check. As such, any of the forms on the lnternet are capable of being completed in this manner.

CA 022~3867 1998-11-06 W O 97/42728 33 PCTrUS97/07740 In another embodiment~ the form could also be a two dimensional array where in each frame of a form is the first argument, and the second argument is the objects on each form. In this embodiment, the first frame would be designated frame one, the second would be designated as frame two and the 5 third designated as frame three.

In still another embodiment, the form could contain minor modiflcations that allow it to notify the service applet 30 or the user applet 22 when a change occurs in a field, similar to event monitoring. In still a further embodhllellt.
applets can be ~enerated to represent the objects of the f'orm. In this o embodiment the applets representative of the objects would notify the user applet 22 or the service applet 30, as the case may be of chan~es to the form.
In the preferred implementation~ the applets would be Java applets.

If the user computer 12 is attached to a server that disallows an outside server 20 to read information from a form~ the user computer 12 can enable I s services such as data tainting in Netscape Navigator or other well known methods to enable the server to see the changes in the form. In data tainting~
the user computer 12 explicitly gives permission to read the informatioll iiom the document that the computer is viewing.

In addition to the shared view 92. the user interface 90 also has a control ~o view 94. which is fiamed adjacent to the shared view 9''. The control view 94 allows the user computer 12 to manipulate the session and can include features such as. for example, send 104. a box to type an URL location, as well as disconnect. The send ] 04 on the control view 94 can be selectively disabled by the second computer 24. to allow the second computer 24 to control the ~s interactive visual session when a user computer 12 is manipulating the shared content improperly.

CA 022~3867 1998-11-06 W O 97/42728 34 PCTrUS97tO7740 Turning now to Figure 4B, where like numerals denote previously described elements, there is also shown a second embodiment of the user interface 90'. The user interface 90' that contains a control view 94' and a shared view 92', however, in this embodiment, the shared view overlies the 5 control view and is not framed within the browser 18 of the user computer 12.
This is the preferred embodiment as web pages are more commonly displayed as one page in a browser and not as frames so that each page can be individually book marked and print as users are accustomed to. Further, as the user computer 12 does not have to see in the control view it is not necessaly to 0 frame the web pages shown in Figure 4A. In particular~ as there is nO queLI~
information in user view regarding its ~lueue once a session has establishecl~ the user computer does not need to have a f'ramed page as shown in FiL~ure 3.

The shared view 92' of Figure 4B. displays shared content of an application wherein the application displayed is a spread-sheet 120 having 5 menu controls 122. To share an application. one ofthe computers in the session must have the application running 011 its computer. In the preferred embodiment the application is running on the second computer 24. If the user computer 12 is the one who wanted to share the page, they could selecl sharin the excel page by activating a button or press a hard key, for example~ that 20 states "Send to Webline". When either party hits the send button, an implememalion grabs the screen as an image and then does an File Transfer Protocol ("FTP") or Hyper-text Transfer Protocol (''HTTP' ! to put it on a drive of the server 20. It should be obvious that there are other ways to implement application sharing that may be incolporated into the invention without 25 deviating from the scope of the invention. In another embodiment~ the shared application could be running on the server 20 or any server connected to the CA 022~3867 1998-11-06 W 097/42728 35 PCTrUS97/07740 network 16, with the second computer 24 and the user computer 12 both sharing the same view of the application.

In the preferred embodiment a plug in for Netscape Navigator or an active X-control adapted for a Microsoft IE browser is used. In this embodiment~
s when the user hits the send button the Active X control grabs the screen image using Windows API functions or similar functions well known in the art.
However, once the Java server receives the image, it notifies the applets in the session that it has received the image. The current image to be shared is given a random and unique URL location which is then sent to the other computer s o applet across the network 16 just as if they were sharing a URL of a web page.
The process is repeated for each image of the application that the parties want to share.

Image sharing can also be implemented by just transmitted the chan~es between one image and the next~ and thus reserve resources on the server.
Either way. the user computer 12 and the second computer 74 cal1 share an application without requiring the user computer to be actively running the application.

In a further embodiment, any activity (e.g. key clicks. mouse clicks etc.) on the shared view of the application can be transmitted to the computer 70 running the application and any resulting new chan~es can be shared.

Turning now to Figure 5~ where like numerals denote previously described elements~ which shows one embodiment of the present invention wherein more than one second computer 24~ 150 is connected to a LAN 152 which is then connected to the server 20. Computer 150 can be identical to the 7~ second computer 24 havin~ an identical service applet 30 or it can be diff'erent.
It can contain an administrative applet. Ior example.

CA 022~3867 1998-11-06 W O 97/42728 -36- PCTrUS97107740 This system allows the second computer 24 to have computer 150 join a session upon request by placing in a queue display 158 of computer l S0 a request to join the session. When computer l 50 activates the connect l 60 I'eature in the queue display 158, the server 20 through the LAN 152 would s download to the computer l S0 the relevant session information as well as an applet~ if necessary. It should be obvious to those skilled in the art that the second computer 24 can also transfer the session to computer l 50 and not j ust add computer 150 to the session.

The server 20 is again connected to the network 16 for communicatio o thereon. and the user computer l 2 is connected to the network 16.

A phone 164 of the user is explicitly show connected to a public switched telephone network ('~PSTN") l 66. As such~ the user l 62 constitutes both the phone 64 and user computer 12. The second computer 24 also has a direct connection to the PSTN 166 either through a local exchange or a long distance 1~ service. As such~ when the second computer 24 is informed by the server 20 tha1 a call request from user 162 is irl its queue. the second computer can connect through the public telephone network to the local PSTN 166 of the user 162 and to user' s phone 164.

In this embodiment the server has a database 154 disposed thereon. As ~o previously described~ the database stores information re~arding the queue.
assignments. attributes. and password validations amon~ other items.

Turning now to Figure 6, where like numerals denote previously described elements. which shows another embodiment of the present invention wherein more than one user 16~. 164 is shown. User 162 has the user computer I ~ and s the phone 164 as was previously described. User 164 can also have a user computer l66 and a phone 168. Wherein both user computers l~. 166 are in CA 02253867 1998-ll-06 W O 97/42728 37 PCT~US97/07740 communication with network 16 and both user phones 164,168 are in communication with a PSTN 166. The second computer 24 can enter a call session with user computer 166 as well as enable the user computer 12 to ente the same call session.

s In this embodiment, a sales center 188is shown comprising the salesperson 190 having the second computer 24 and a phone 180. The phone 180, however, is connected to an automated call distribution system ('ACD'') 182.ACDs are well known in the art, and they have, among other features.
more than one queue 188 disposed thereon within which to route calls ~iom the publiclv sv~itched network or data networks. In this embodiment~ whell usel 162 enters a call request through the phone 164~ the call is transmitted to the PSTN 166 and then to the ACDI 84 which places it in a queue 188. The ACD
could thereafter place the telephone call for the sales representative 190.

However, the ACDis also connected via 186 to the network 16. This s enables either the queue information 188 of the ACD to be communicated to the server 20 and displayed by the server 20 to the more than one second computers in the sales center 188 and the user computer 12, 166.
Alternatively, the call requests from the network 16 that ~o to the server 20 be communicated to the ACD. Havin~ the ACD connected to the Internet in this ~o manner. allows the system to use the sophisticated aspects of an automated ~CDs. such as queue service. to supplement or substitute for the compu~er queue functions or other similar functions in the server 20.

In this embodiment, a firewall 192 is also disposed between tlle user computer 12 and the server 20. The firewall 192 can be configured so as to stop ~s all incoming traffic from the network 16, stop all outgoing traffic from the user computer 12 or any varialioll there between. Typically. the f1rewall 192 will CA 022~3867 1998-11-06 allow certain information to pass, but restrict the use of that information. For example, the firewall will allow applications in from the network l 6. but once disposed the user computer 12 the application downloaded from the network 16 will not be allowed to function upon documents disposed on the user s computer 12 that are not generated from the same server as was the application.
Such a restriction disables the user applet 22 from modifying interactive forms within user computer 12.

The user computer 12 can expressly permit the function of the user applet 22 and the server 20 in each individual instance. In the preferred embodimel1t.
however. the user applet 22 functions are achieved by talkmt throuth tl-e firewall at port 80 using HTTP protocol. As it should be obvious to those skilled in the art, this port is usually left open by a firewall for communication over the worldwide web. It should be also obvious to those skilled in the art that other ports can be used with other modifications or any intermediar~
15 solution can be found wherein the user computer 12 grants permissions to the user applet 22 to perform functions.

In the preferred embodiment. however~ the user computer 12 as is typical is allowed to connect to an external port of a random server as long as it is using HTTP protocol and as long as they are connecting to port 80 of another ~o server. Using these specifications. the system performs HTTP tunneling.
Accordingly~ in the preferred embodiment the user applet 22 is sent from port 80 communicating in HT'I'P protocol to look very similar to a Common Gateway Interface ("CGI"! request~ and is allowed in by the firewall.
Thereafter~ communication from the user applet 22 still has to be in a protocol ~5 capable of ~oing through the firewall. As such. any request from the user applet 22 in this embodiment looks like a standard HTTP request from a web pa~e and as such looks like a CGI request. The firewall will let the requeST

CA 022~3867 1998-11-06 Wo 97/42728 39 PCT/US97/07740 through from the user applet 22 onto the network 16. The request is in the form of a function caller request, which is transmitted as a message to a CGI
handler. The CGI handler translates it, passes it to the CGI programmer which translates the CGI request to the server 20~ which in the preferred embodiment s is a Java server. Any transmission back from the Java server 20 to the user applet 22 is done through the same mechanism. Essentially, the communication between the server 20 and the user applet 22 is cloaked to look as is it is a request i'rom a web site on the Internet and is such is allowed to go througll the firewall 192.

o lt should be obvious to those skilled in the art that using either a differe server or diff'erent handlers or other known methods to go through the firewall~
including any port or other configuration. is possible without departing from the scope of this invention.

Turning now to Figure 7A. which shows one method kv which the user ] 5 computer 12 can enter a call session with the second computer 24 and thereby providing audio communication across the network 16 or the telephone network in conjunction with visual communication across the network 16. As is previously described, the user computer 12 can enter into a call session with the second computer 24 by numerous methods.

In Figure 7A the request to enter a call session starts 200 when a user of the user computer 12 is instructed by the sales representative of the second computer 24 via telephone to go to a specific web page 202. Displaved on that web page will be a sign-on page wherein the user is prompted to entel-attributes~ such as the users name or anv other attributes previously described.

In this embodiment. as the user and the sales party are already in audio communication. the user will alreadv have been provided bv the sale~

CA 022~3867 1998-11-06 W O 97/42728 40 PCT~US97107740 representative a password that uniquely identifies to the server 20 the session the user computer 12 is supposed to enter. As such, the user computer 12 is also prompted to insert a password 206. After the user enters the password, the server 20 places the call request in computer queue 208 as was previously s described.

At this point the server 20 also transmits to the user computer 12 a user applet 22. The user applet 22 generates a child browser in the user view of the user computer 12 and then begins to poll the server 20 for call request status 212. If the call request status is not simultaneously activated by the ser~ ice n applet 30. then the user computer 12 is displayed the on-hold information 216.
The on-hold information will include, for example~ the user computer's stahls in the computer queue, any advertisements as previously described. and other additional inf'ormation as required.

The control view can also be selectively displayed by the user computer 218. However, in the preferred embodiment as the control view is not framed within the user interface. As such. the control view will be sittin~ under the child browser, and the child browser will display the on-hold informalion.

The user applet 22 will continue to poll the server 20 for the call request status 220. If the caller request status is that the user computer is still in the . o computer queue, the user applet 22 will update the computer queue informalion still in the computer queue 222. then the user applet 22 will collect the updated computer queue information 224 and redisplay the on-hold information in the child browser 216. If, however~ the second computer 24 has selected the call request of the user computer 12, the service applet will notify the server 20 that ~5 the user computer having that session identifier can now be entered into a call session 224. After the parties are entered into the call session the ~ . . . .

CA 022~3867 1998-11-06 W O 97/42728 _41_ PCT~US97/07740 communication between them and the manipulatioll of the shared constant on the user interface of the user computer 12 and the user interface of the second computer 24 will continue as previously described.

Turning llOW to Figure 7B, which shows a second method by which the s user computer 12 can enter a call session and start 200 the visual interaction between the user computer 12 and the second computer 24. In this method it is assumed that the user and the sales representative are not initially in audio communication. Instead, it is assumed that the user is on the worldwide web and is viewing the web page 228 of the sales organization.

o lf the user of the user computer 12 wishes to enter the session the user ca click the call request button 230 on the web page of the sales organization.
IJpon clicking the call request button 230. the user compuIer 12 displays on its browser a form prompting the user to insert attributes 232 such as name.
interest, and so on. After the user computer enters the attributes ~32 the attributes are communicated througl1 the network 16 to the server 20, whereupon the server 20 passes the attributes 232 via methods common]~
known in the art. T he server 20 then goes through skills based routing 234 as is previously described and determines which computer queue is appropriate iOI
the call request.

zo The server 20 then places the call request in tl1e computer queue 236. The call request in the computer queue can be communicated to the ACI) or displayed on the computer queue of the second computer and the user computer The ACD can also determine which queue is appropriate for tlle call request in place of the server and transmit the corresponding selection of the computer queue to the server.

CA 022~3867 1998-11-06 Regardless, the server then transmits the user applet 238 to the user computer. The user applet spawns the child browser and displays the on-hold information 240 to the user. Again, the browser of the user computer 12 can selectively display the control view 242 in a *amed method or behilld the child ~ browser. Similarly, the on-hold information can include not only inf'ormation regarding the call request status in the queue but also the advertisements determined either based on the attributes or just the advertisements for the sales representatives' company in general, as is previously described. In one implementation if the ACD will place the call, the queue of the ACD would be I o displayed as communicated to the server, while in another implementation~ the computer queue of the server would be displayed. It should be obvious that the queue of the ACD and the computer queue of the server can supplant each other or work in conjunction with each other as is necessarv.

While the browser of the user computer 12 is displaving the on-hold 5 information. the user applet 22 continues to poll the server 20 for the call request status 244. If the user applet 22 is informed bv the server 2() tllat the call request is still on the queue 246 then the user applet 22 will collect the updated on-hold information 248 and redisplay the on-hold information in the child browser 240. This process will continue until the user applet '2 polls the ~o server 20 for the call request status 244 and is told that the service applet has enabled the call request of the customer and informed the server 20 of the same. Alternatively~ the ACD can place the call and inform the server that the call request has been enabled. At this time. the user applet receives notification that the user can enter the call session 250.

The invention may be embodied in other specific forms without departin~

from the spirit or essential characteristics thereof. The present embodiments are. therefore. to be considered in all respects as illustrative and not restriclive.

__ W O 97/42728 43 PCTrUS97/07740 the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.

Claims

1. A method of providing visual interaction across a network between at least one user computer having a browser for reading files and at least one second computer in communication with a server, the server interposed between the at least one user computer and the at least one second computer, the method comprising the steps of:

establishing a link between the at least one user computer and the server;

selectively transmitting a user applet to the at least one user computer, the user applet enabling the at least one user computer to communicate with the server;

generating a user view on the at least one user computer;

communicating that the link is established to a service applet disposed in the at least one second computer, the service applet adapted to enable the at least one second computer to communicate with the server;

establishing a session between the at least one user computer and the at least one second computer;

enabling the at least one second computer to display a service view containing a representation of the user view;

selectively manipulating the representation of the user view by the at least one second computer;

communicating to the service applet changes in the representation of the user view;

transmitting the changes in the representation of the user view from the service applet to the user applet; and implementing any changes communicated to the user applet in the user view, thereby altering the user view to conform to the representation of the user view of the at least one second computer, thus enabling the at least one second computer to communicate to the at least one user computer.

2.

A method according to claim 1 further comprising the steps of:

selectively manipulating the user view by the at least one user computer;

communicating to the user applet change in the user view; and transmitting the change from the user applet to the service applet, thereby allowing the at least one user computer to control communication with the at least one second computer.

3. A method according to claim 2 wherein the at least one second computer can disable an ability of the at least one user computer to selectively manipulate the user view.

4. A method according to claim 1 wherein the user applet and the service applet bi-directionally communicate.

5. A method according to claim 1 wherein each of the at least one user computer is adapted to make call requests to the server, thus requesting voice communication.

6. A method according to claim 5 further comprising an automated call distributor in communication with the server having at least one queue adapted to receive the call requests from the server.

7. A method according to claim 6 further comprising at least one computer queue disposed on the server adapted to receive notification from the automated call distributor of call requests and update the at least one computer queue.

8. A method according to claim 5 wherein the server further comprises at least one computer queue disposed on the server adapted to selectively receive the call requests routed by server.

9. A method according to claim 5 further comprising:

at least one computer queue disposed on the server adapted to receive the call requests selectively routed to the at least one computer queue by the server; and an audio module disposed on the at least one second computer adapted to enable each of the at least one second computers to access the at least one user computer having the call requests in the at least one queue.

10. A method according to claim 1 further comprising:

at least one computer queue disposed on the server having the call requests;
and queue information representative of the call requests disposed in the at least one computer queue, the queue information displayed on the at least one second computer.

11. A method according to claim 10 wherein the server updates the queue information that is displayed on the at least second computer in real time.

12. A method according to claim 10 wherein the server transmits to the at least one user computer supplemental information while the call requests of the at least one user computer is in the at least one computer queue.

13. A method according to claim 12 wherein the supplemental information is selected according to attributes dictated by the at least one user computer.

14. A method according to claim 1 further comprising the steps of:
extracting from the at least one user computer user attributes representative of the at least one user computer;
ascertaining service attributes of the at least one secured computers; and selectively communicating that the link is established to selected ones of the at least one second computers by comparing the user attributes and the service attributes.

15. A method according to claim 1 further comprising the step of selectively adding an additional computer to the session by the server.

16. A method according to claim 15 wherein additional computer is disposed remotely from the at least one user computer and the at least one second computer.

17. A method according to claim 1 wherein the session is password protected.

18. A method according to claim 1 wherein the service view further comprises a script frame adjacent to the representation of the user view and having a representation of one or more scripts disposed therein, each of the one or more scripts being adapted to communicate with the service applet and being adapted to control the representation of the user view

19. A method according to claim 18 wherein the script initiates a file transfer to download information to the user view from a remote location.

20. A method according to claim 18 wherein the script is dynamically defined according to attributes of the at least one user computer.

21. A method according to claim 20 wherein the script is dynamically defined according to attributes of the at least one second computer.

22. A method according to claim 20 wherein the script is dynamically defined by the second computer to generate a comparison of information within a display.

23. A method according to claim 22 wherein the script is representative of a short-cut to enable the at least one second computer to push predetermined information to the at least one user computer.

24. A method according to claim 1 wherein the any changes to the representation of the user view is dictated by a change of a address of the user view, the service applet communicating the change of address to the user applet.

25. A method according to claim 1 wherein the changes to the representation of the user view is forced by the service applet which communicates an image to the user applet.

26. A method according to claim 25 wherein the image is transmitted to the at least one user computer at predetermined time intervals.

27. A method according to claim 25 wherein the captured image is transmitted to the at least one user computer in real time.

28. A method according to claim 25 further comprising the steps of:

assigning an image address to the image, the image and the image address disposed within the server;

communicating to the user applet the image address; and displaying the image in the user view of the at least one user computer and the representation of the user view in the at least one second computer.

29. A method according to claim 1 wherein the changes to the representation of the user view is captured as image changes, where the image changes are representative of a portion of the image that has been selectively manipulated, the service applet communicating the image changes to the user applet.

30. A method according to claim 1 wherein the user view and the representation of the user view is a form having objects, the method further comprising the steps of:

selectively manipulating the objects by the at least one user computer;

selectively manipulating the objects by the at least one second computer;

transmitting the objects to the service applet and the user applet; and displaying the objects in the representation of the user view and the user view thus allowing the user view and the service view to display the form as the form is completed.

31. A method according to claim 30 wherein the step of transmitting the objects further comprises the steps of:

querying the form to ascertain the objects disposed on the form;

representing the form as an array having arguments representative of the objects;

selectively updating the objects such that the arguments of the array represent any changes to the objects; and transmitting at least portions of the array to the service applet and the user applet.

32. A method according to claim 31 wherein the at least portions of the array are representative of only the any changes to the objects.

33. A method according to claim 30 wherein the form is enhanced by making the objects of the form event driven, wherein the step of transmitting the objects further comprises the objects communicating to the applet when the objects are selectively manipulated on the form.

34. A method according to claim 30 wherein form is enhanced by generating Javaapplets representative of the objects wherein the step of transmitting the objects further comprises transmitting from the Java applets any changes to the objects to the user applet and the service applet.

35. A method according to claim 1 wherein the user view is a software application disposed on the server.

36. A method according to claim 1 wherein the user view is a software application disposed on the at least one second computer.

37. A method according to claim 1 wherein the user view is a software application disposed on the at least one user computer.

38. A method according to claim 1 wherein the user applet generates the user view, the user view having a child browser.

39. A method according to claim 1 further comprising the step of validating the at least one user computer by the server prior to establishing a session.

40. A method according to claim 1 further comprising a database adapted to store queue and session information.

41. A method according to claim 1 wherein the server and user applet selectively communicate through a predefined port thus enabling communication through a firewall.

42. A method according to claim 1 wherein the at least one user computer requires no additional pre-installed executables to communicate with the at least one second computer.

43. A method according to claim 1 wherein the at least one user computer and theat least one second computer can disconnect themselves from the session.

44. A method according to claim 1 wherein the at least one second computer can selectively disconnect another computer from the session.

45. A system for enabling communication between a service party having at least one second computer and a user of a first computer, the system comprising:

a server;

a user view displayed on the first computer;

a user applet downloaded to the first computer by the server and in communication with the server, the user applet in communication with the server and adapted to generate the user view;

a service view disposed on the at least one second computer, the service view having the user view displayed therein; and at least one service applet disposed within the at least one second computer in communication with the server, the service applet having of functionality required by the at least one second computer to manipulate the service view, whereby the service party is adapted to communicate to the user by manipulating the user view through the service view.

46. A method for enhancing a resource collection function of a computer in communication with a network while enabling the computer to use functionality of a resource, the method comprising the steps of:

defining functions of the resource that the user wishes to enable, the functionsof the resource being a sub-set of the functionality of the resource;

generating an applet representative the functions of the resource; and communicating the applet to the user's computer as the resource collection function, thereby decreasing the resource collection function of the user's computer while enabling the functions of the resource the user wishes to enable.

47. A method of controlling a browser view on a first computer from a script displayed separately from the browser view, the script communicating with an applet, the applet adapted to generate the browser view, the method comprising the steps of:

providing addresses as simplified commands in the script;

defining the applet to receive the simplified commands as arguments;

selectively activating portions of the script;

associating the simplified commands of the portions of the script with the applet such that the applet will generate the browser view dictated by the portions of the script that are activated; and modifying the browser view in accordance with the simplified commands.

48. A method according to claim 47 further comprising a remote computer having a user view and a user applet in communication with the applet, the user applet adapted to receive the simplified commands from the applet and modify the user view of the remote computer.

49. A method according to claim 48 wherein the applet has session information representative of the first computer and the remote computer.

50. A method according to claim 48 wherein the step of providing addresses as simplified commands in the script further comprises the steps of:
ascertaining user attributes of the remote computer; and selectively generating addresses such that the addresses relate to the user attributes thus allowing the browser view to contain a display related to the user attributes.

51. A method according to claim 50 wherein the server selectively generates the addresses.

52. A method according to claim 50 wherein the first computer selectively generates the addresses.

53. A method according to claim 48 wherein the step of providing addresses as simplified commands in the script further comprises the steps of:
ascertaining session attributes representative of service attributes of the second computer and user attributes of the remote computer; and selectively generating addresses such that the addresses relate to the session attributes.