WO2012141707A1 - Videoconferencing - Google Patents

Videoconferencing Download PDF

Info

Publication number
WO2012141707A1
WO2012141707A1 PCT/US2011/032473 US2011032473W WO2012141707A1 WO 2012141707 A1 WO2012141707 A1 WO 2012141707A1 US 2011032473 W US2011032473 W US 2011032473W WO 2012141707 A1 WO2012141707 A1 WO 2012141707A1
Authority
WO
WIPO (PCT)
Prior art keywords
videoconference
video streams
ordered list
videoconference video
display device
Prior art date
Application number
PCT/US2011/032473
Other languages
French (fr)
Inventor
Mark E. Gorzynski
Original Assignee
Hewlett-Packard Development Company, L.P.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hewlett-Packard Development Company, L.P. filed Critical Hewlett-Packard Development Company, L.P.
Priority to PCT/US2011/032473 priority Critical patent/WO2012141707A1/en
Publication of WO2012141707A1 publication Critical patent/WO2012141707A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/15—Conference systems

Abstract

A layout for displaying videoconference video streams may be defined based on an ordered list of videoconference video streams. For example, a layout may be defined to display videoconference video streams in at least first and second vertical columns of video streams where a first two videoconference video streams in adjacent positions in an ordered list of videoconference video streams are assigned to the first column and a second two videoconference video streams in adjacent positions in the ordered list of videoconference video streams are assigned to the second column.

Description

VIDEOCONFERENCING
BACKGROUND
[0001] Videoconferencing enables participants at two or more different sites to interact with each other.
BRIEF DESCRIPTION OF THE DRAWINGS
[0002] FIGS. 1 A-1 B are block diagrams of an example of a communications system.
[0003] FIGS. 2A-2D are schematic diagrams of videoconference video streams that illustrate examples of techniques for arranging a display of the videoconference video streams.
[0004] FIG. 3A is a flow chart of an example of a process for defining a layout for displaying videoconference video streams for a videoconference.
[0005] FIGS. 3B-3C are schematic diagrams of videoconference video streams that illustrate an example of a technique for arranging a display of the videoconference video streams according to the process of FIG. 3A.
[0006] FIG. 4A is a flow chart of an example of a process for defining a layout for displaying videoconference video streams for a videoconference.
[0007] FIGS. 4B-4D are schematic diagrams of videoconference video streams that illustrate examples of techniques for arranging a display of the videoconference video streams according to the process of FIG. 4A.
[0008] FIGS. 5-6 are block diagrams of examples of communications systems.
[0009] FIG. 7 is a block diagram of an example of a computing system.
DETAILED DESCRIPTION
[0010] A videoconference may involve participants participating from multiple different sites. A central layout of locations representing the different sites may be defined for the videoconference for, among other purposes, coordinating the exchange of videoconference video streams between the participating sites. The sites participating in the videoconference may be represented by one or more different locations within the central layout for the videoconference.
[0011] Each site participating in the videoconference may generate one or more videoconference video streams providing views of the site. For example, some sites participating in the videoconference may have multiple video cameras that each capture substantially non-overlapping images of different participants who are participating in the videoconference from the site. Because the videoconference video streams generated by the video cameras at such sites essentially are unique and non-redundant, such sites may be represented by multiple locations within the central layout for the videoconference, each location within the central layout corresponding to a separate one of the videoconference video streams generated by the video cameras at such sites. In contrast, other sites participating in the videoconference may have multiple video cameras that capture substantially the same images (e.g., all of the participants who are participating in the videoconference from the site) but from different perspectives. Because the videoconference video streams generated by the video cameras at these locations may be redundant of one another, these locations may be represented by single locations within the central layout for the videoconference. Still other sites participating in the videoconference may have only one video camera and, therefore, also may be represented by single locations within the central layout for the videoconference.
[0012] The central layout for the videoconference may be implemented as an ordered list of locations. Although the initial assignment of locations to particular positions within the ordered list of the central layout may be arbitrary, the ultimate ordering of the locations within the list may be important in defining local layouts at the different sites for displaying videoconference video streams in a manner that provides the illusion of enabling eye contact, gesture awareness, and other forms of non-verbal communication between participants participating in the videoconference from different sites.
[0013] As will be described in greater detail below, one technique for laying out incoming videoconference video streams at a particular site that is participating in a videoconference involves arranging the incoming videoconference video streams in a horizontal row according to the ordering of the locations within the central layout to which the videoconference video streams correspond. Arranging the incoming videoconference video streams at the particular site in this manner may provide the illusion of enabling eye contact, gesture awareness, and other forms of non-verbal communication between participants participating in the videoconference from different sites.
[0014] In some cases, however, due to the screen real estate available or other factors at a particular site, arranging incoming videoconference video streams in a single horizontal row may not be desirable or otherwise practicable. Therefore, the videoconference video streams instead may be arranged in multiple rows in a "serpentine" fashion according to the ordering of the locations within the central layout to which the videoconference video streams correspond. Arranging the incoming videoconference video streams in multiple rows in such a "serpentine" fashion at the particular site may provide the illusion of enabling eye contact, gesture awareness, and other forms of non-verbal communication with participants participating in the videoconference from other sites.
[0015] FIGS. 1A-1 B are block diagrams of an example of a communications system 100. As illustrated in FIGS. 1A-1 B, the communications system 100 includes a network 102 connecting seven different sites 104 that are participating in a videoconference.
[0016] In some implementations, the sites 104 participating in the videoconference may be connected across network 102 in a peer-to-peer configuration. In such implementations, each site 104 may transmit one or more locally-generated videoconference video streams to each other site 104 and receive one or more locally-generated videoconference video streams from each other site 104. Additionally or alternatively, the sites 104 participating in the videoconference may be connected to each other through one or more multipoint control units (MCUs) (not shown) or other centralized computing devices (not shown) configured to connect or bridge sites participating in a videoconference. In such implementations, each of the sites 104 may transmit one or more locally-generated videoconference video streams to the one or more MCU(s) or other computing device(s) and, thereafter, rely on the MCU(s) or similar computing device(s) to transmit the videoconference video streams to the other locations 104.
[0017] Network 102 may provide direct or indirect communication links between individual ones of sites 104 and/or between individual sites 104 and an MCU or other similar centralized computing device configured to connect or bridge sites participating in a videoconference. Examples of network 102 include the Internet, the World Wide Web, wide area networks (WANs) including corporate intranets, local area networks (LANs) including wireless LANs (WLANs), analog or digital wired or wireless telephone networks, radio, television, cable, satellite, and/or any other delivery mechanisms for carrying data, or any combinations thereof.
[0018] Sites 104 may be implemented and configured in a variety of different forms and the implementations or configurations of individual sites 104 may be the same or different. For example, as illustrated in FIG. 1 , site 104(0) is implemented as a dedicated telepresence studio. The other sites 104(1 )-104(6) may or may not be dedicated telepresence studios like site 104(0). For example, one or more of the other sites 104(1 )-104(6) may be personal computing devices (e.g., a desktop computer, a laptop or notebook computer, a netbook computer, a tablet computer, a smartphone, a personal digital assistant, or other computing device) configured to support videoconferencing. Furthermore, one or more of the other sites 104(1 )- 104(6) may be dedicated telepresence studios that are implemented or configured differently than the dedicated telepresence studio of site 104(0). For example, one or more of the other locations 104(1 )-104(n) may be dedicated telepresence studios having more or less than three display devices and/or more or less than three video cameras.
[0019] The dedicated telepresence studio at site 104(1 ) has three display devices 106 for displaying incoming videoconference streams from the other sites 104(1 )- 104(6) participating in the videoconference. In addition, site 104(0) also includes three video cameras 108 for capturing videoconference video streams of local videoconference participants (e.g., local videoconference participants 110) to be transmitted to the other sites 104(1 )-104(6) participating in the videoconference.
[0020] Although not shown in FIGS. 1A-1 B, site 104(0) also includes one or more computing devices for receiving incoming videoconference video streams from the other sites 104(1 )-104(6) and arranging the display of the received videoconference video streams on display devices 106. These computing devices also may coordinate the transmission of the local videoconference video streams captured by cameras 108 to the other locations 104(1 )-104(6), for example via a peer-to-peer architecture or via one or more MCUs or other computing devices.
[0021] In some implementations, as illustrated in FIG. 1A, each of the video cameras 108 may be configured to capture video images of substantially the same region of the dedicated telepresence studio albeit from slightly different perspectives. For example, each of the video cameras 108 may be configured to capture video images that include all six of the local videoconference participants 110 at site 104(0). In implementations such as that illustrated in FIG. 1A, the different videoconference video streams generated by the video cameras 108 may be transmitted to different ones of the other sites 104(1 )-104(6) participating in the videoconference as will be described in greater detail below.
[0022] Alternatively, in other implementations, as illustrated in FIG. 1 B, each of the video cameras 108 may be configured to capture video images of substantially non-overlapping regions of the dedicated telepresence studio. For example, camera
108(a) may be configured to capture video images that include local videoconference participants 110(a) and 1 10(b), while camera 108(b) is configured to capture video images that include local videoconference participants 110(c) and 110(d), and camera 108(c) is configured to capture video images that include local videoconference participants 1 10(e) and 110(f). In such implementations, each of the different videoconference video streams generated by the cameras 108 may be transmitted to all of the other sites 104(1 )-104(6) participating in the videoconference, for example to ensure that views of all of the local participants 110 participating in the videoconference at site 104(0) are provided to each of the other sites 104(1 )-104(6) participating in the videoconference.
[0023] A central layout 112, for example in the form of locations around a virtual conference table, is defined for the videoconference. As illustrated in FIGS. 1A-1 B, each site 104 participating in the videoconference is represented by one or more locations in the central layout 112. Central layout 112 may be implemented as an ordered list of these locations representing the sites 104(0)-104(6) participating in the videoconference. Such an ordered list may be realized using any of a number of different types of data structures, including, for example, an array, a vector, or a table. In some implementations, a centralized computing device (not shown) configured to manage the videoconference may define central layout 112. Alternatively, in other implementations, a computing device (not shown) located at or associated with a particular one of the sites 104 may define central layout 112.
[0024] As illustrated in FIGS. 1A-1 B, the central layout 112(a) of FIG. 1A is different from the central layout 112(b) of FIG. 1 B due to the different configurations of the cameras 108 illustrated in FIG. 1A and FIG. 1 B. Specifically, because the cameras 108 as configured in FIG. 1A capture video images that are substantially the same, site 104(0) is represented by only a single location in the central layout 1 12(a) of FIG. 1A. In contrast, because the cameras 108 as configured in FIG. 1 B capture video images that essentially are unique and non-redundant, site 104(0) is represented by three different locations in the central layout 112(b) of FIG. 1 B, where each such location within central layout 112(b) corresponds to a videoconference video stream generated by a different one of cameras 108. Representing site 104(0) as three locations within central layout 112(b) as illustrated in FIG. 1 B, each of which corresponds to a videoconference video stream generated by a different one of the cameras 108, may ensure that views of all six videoconference participants 1 10(a)- 110(f) at site 104(0) are provided at each of the other sites 104(1 )-104(6) participating in the videoconference.
[0025] The central layout 1 12 defined for the videoconference may be used to define local layouts for displaying videoconference video streams from the sites 104 participating in the videoconference in a manner that simulates the participants at the different sites 104 sitting around a conference table. The local layout for each site 104 may be unique, reflecting the unique position of the site at the virtual conference table as defined by the central layout 112. For example, the local layout for site 104(3) may be arranged to create the appearance that (i) participants at site 104(4) and participants at site 104(5) are located to the left of the participants at site 104(3), (ii) participants at site 104(1 ) and site 104(2) are located to the right of the participants at site 104(3), and (iii) participants at site 104(6) and participants at site 104(0) are located across from the participants at site 104(3). Similarly, the local layout for site 104(6) may be arranged to create the appearance that (i) participants at site 104(0) and participants at site 104(1 ) are located to the left of participants at site 104(6), (ii) participants at site 104(4) and site 104(5) are located to the right of participants at site 104(6), and (iii) participants at site 104(3) and participants at site 104(4) are located across from the participants at sites 104(6).
[0026] FIGS. 1A-1 B illustrate an example of a local layout for site 104(0). As illustrated in FIGS. 1A-1 B, the local layout at site 104(0) for displaying videoconference video streams from the other sites 104(1 )-104(6) participating in the videoconference is arranged into two rows and three columns of videoconference video streams where each column of two video streams is displayed on a separate display device 106. In some implementations, the number of columns of videoconference video streams to be included in a local layout at a site may be determined by the number of display devices at the location. For example, the number of columns of videoconference video streams in a local layout for a site may be set equal to the number of display devices at the site such that one column of the videoconference video streams is displayed on each display device. The number of rows of videoconference video streams to be included in the local layout at the site then may be determined by dividing the number of videoconference video streams to be displayed at the site by the number of columns in the local layout for the site.
[0027] As illustrated in FIGS. 1A-1 B, the local layout at site 104(0) arranges the six videoconference video streams from the other sites 104(1 )-104(6) participating in the videoconference into three columns and two rows due to the there being three display devices 106 at site 104(0) and six videoconference video streams to display. The local layout at site 104(0) simulates that the videoconference participants at the other sites 104(1 )-104(6) are sitting around a conference table from the local participants 110 at site 104(0) in the arrangement defined by the central layout 112 for the videoconference. Specifically, the local layout at site 104(0) creates the appearance to the local participants 1 10 at site 104(0) that (i) participants at site 104(1 ) and participants at site 104(2) are located to their left, (ii) participants at site 104(5) and site 104(6) are located to their right, and (iii) participants at site 104(3) and participants at site 104(4) are located across from them. This illusion at site 104(0) that the participants at the other sites 104(1 )-104(6) participating in the videoconference are sitting around the conference table in the order defined by the central layout 112 is accomplished by arranging the display at site 104(0) of the videoconference video streams from the other sites 104(1 )-104(6) in a manner that is based on the order defined by the central layout 112.
[0028] In order to define a local layout for displaying the videoconference video streams from the other sites 104(1 )-104(6) participating in the videoconference at site 104(0), an ordered list of the videoconference video streams from the other sites 104(1 )-104(6) to be displayed at site 104(0) may be derived from the central layout 112. This may be accomplished by removing those locations that represent site 104(0) from the central layout 112. The videoconference video streams corresponding to the remaining sites 104(1 )-104(6) represented as locations in the central layout 112 then may be assigned to positions in the ordered list of videoconference video streams to be displayed at site 104(0) in the same order as the sites 104 to which they correspond are represented in the central layout 112.
[0029] The ordered list of the videoconference video streams to be displayed at site 104(0) then may be used to assign individual videoconference video streams to particular cells within the 2x3 local layout for site 104(0) by alternating the assignment of videoconference video streams between the bottom and top rows starting with the column displayed on display device 106(a) and then moving from left to right to the columns displayed on display devices 106(b) and 106(c). Thus, as illustrated in FIGS. 1A-1 B, according to the local layout for site 104(0), the videoconference video stream corresponding to site 104(1 ) is displayed on the lower portion of display device 106(a), the videoconference video stream corresponding to site 104(2) is displayed on the upper portion of display device 106(a), the videoconference video stream corresponding to site 104(3) is displayed on the lower portion of display device 106(b), the videoconference video stream corresponding to site 104(4) is displayed on the upper portion of display device 106(b), the videoconference video stream corresponding to site 104(5) is displayed on the lower portion of display device 106(c), and the videoconference video stream corresponding to site 104(6) is displayed on the upper portion of display device 106(c).
[0030] By arranging the display at site 104(0) of the videoconference video streams from the other sites 104(1 )-104(6) participating in the videoconference in this "serpentine" pattern, the illusion of eye contact, gesture awareness, and other forms of non-verbal communication with participants participating in the videoconference at the other sites 104(1 )-104(6) may be preserved while also effectively utilizing the screen real estate available for displaying the videoconference video streams from the other sites 104(1 )-104(6) at site 104(0). [0031] In some implementations, the local layout for displaying videoconference video streams at any particular site 104 participating in the videoconference may be defined by one or more computing devices (not shown) located at or associated with the particular site 104. Alternatively, in other implementations, the local layout at any particular site 104 participating in the videoconference may be defined by one or more centralized computing devices (not shown) configured to manage the videoconference.
[0032] As illustrated in FIGS. 1A-1 B, two videoconference video streams are displayed on each display device 106 at site 104(0). In some such implementations where the local layout for a site participating in a videoconference causes multiple videoconference video streams to be displayed on the same display device, the resolutions of the individual videoconference video streams may be reduced without significantly degrading the image quality of the display of the videoconference video streams. For example, the resolution of an individual videoconference video stream may be reduced by a reduction factor that is proportional to the portion of the display device occupied by its display.
[0033] Referring to FIGS. 1A-1 B, because the videoconference video streams occupy one half of the display devices 106 on which they are displayed, the resolutions of the videoconference video streams may be multiplied by a reduction factor of 0.5 without significantly degrading the image quality of the display of the videoconference video streams. Continuing with this example, if the local layout at site 104(0) called for four videoconference video streams to be displayed on each display device 106 with each videoconference video stream occupying a quarter of the display device on which it is displayed, the resolutions of the videoconference video streams may be multiplied by a reduction factor of 0.25 without significantly degrading the image quality of the display of the videoconference video streams.
[0034] In some implementations, the ability to reduce the resolution of the videoconference video streams to be displayed at sites participating in a videoconference in this manner may be leveraged to reduce the network bandwidth that otherwise would be consumed by exchanging videoconference video streams between different sites participating in the videoconference. For example, if it is known that the local layout for a site participating in a videoconference calls for two videoconference video streams to be displayed on a single display device and that the resolutions of the two videoconference video streams may be reduced by a factor of 0.5 without significantly degrading the image quality of the display of the videoconference video streams, the videoconference video streams may be scaled and/or compressed accordingly, for instance by a factor of 0.5, before being transmitted to the site (either by systems located at the site(s) from which the videoconference video streams originate or at intermediary systems).
[0035] FIGS. 2A-2D are schematic diagrams of videoconference video streams 202(0)-202(n) corresponding to sites participating in a videoconference that illustrate examples of techniques for arranging a display of the videoconference video streams 202(0)-202(n).
[0036] As illustrated in FIG. 2A, a central layout 204 of the videoconference video streams 202(0)-202(n) corresponding to the participating videoconference sites is defined for the videoconference. This central layout 204 represents the videoconference video streams 202(0)-202(n) as locations around a conference table and reflects a global ordering of the videoconference video streams 202(0)-202(n) relative to one another. Central layout 204 may be reduced in an elemental form to an ordered list.
[0037] The initial assignment of the videoconference video streams 202(0)- 202(n) to particular locations within the central layout 204 may be arbitrary, but, as will be described further below, after the central layout 204 has been defined, the ordering of the videoconference video streams 202(0)-202(n) within the central layout 204 may be used to define local layouts for displaying videoconference video streams and, thus, once set, may be important.
[0038] In addition to reflecting a global ordering of the videoconference video streams 202(0)-202(n) relative to one another, for any particular videoconference video stream 202, the central layout 204 also reflects a unique ordering of the other videoconference video streams 202 relative to the particular videoconference video stream 202. These unique orderings for each of the individual videoconference video streams 202 relative to the other videoconference video streams 202 defined by the central layout 204 may be useful in defining local layouts for displaying the videoconference video streams 202 that provide the illusion of enabling eye contact, gesture awareness, and other forms of non-verbal communication between the participants at the different videoconference sites. Furthermore, much like the central layout 204 itself, these unique orderings for each of the individual videoconference video streams 202 relative to the other videoconference video streams 202 defined by the central layout 204 may be reduced to ordered lists of videoconference video streams 202.
[0039] One technique for arranging a local layout for displaying the videoconference video streams 202 at a particular videoconference site in a manner that provides the illusion of enabling eye contact, gesture awareness, and other forms of non-verbal communication with participants at the other videoconference sites involves arranging the videoconference video streams 202 corresponding to the other videoconference sites from left to right in a horizontal row starting with the videoconference video stream 202 that is adjacent to the videoconference video stream(s) 202 corresponding to the particular site in the central layout 204 and progressing clockwise through the remaining videoconference video streams 202 in the central layout 204.
[0040] This technique for defining a local layout of the videoconference video streams based on the central layout 204 for the videoconference is illustrated in FIG. 2B for a site that is represented by "Location 0" in central layout 204 and that corresponds to videoconference video stream 202(0). As illustrated in FIG. 2B, videoconference video streams 202(1 )-202(n) are arranged in a horizontal row 210. Referring to FIG. 2A, proceeding clockwise through the central layout 204 from the videoconference video stream 202(0) corresponding to the local site, video stream 202(1 ) is adjacent to the videoconference video stream 202(0). As such, videoconference video stream 202(1 ) is assigned to the leftmost (or first) position in horizontal row 210 and the remaining videoconference video streams 202(2)-202(n) are assigned sequentially from left to right to the remaining positions in horizontal row 210 in the order in which they appear in central layout 204 proceeding clockwise from video stream 202(1 ) to video stream 202(n).
[0041] In some cases, it may be undesirable or impracticable to display all of the videoconference video streams to be displayed at a videoconference site in a single horizontal row as illustrated in the horizontal row 210 of videoconference video streams 202(1 )-202(n) of FIG. 2B. For example, the screen real estate available on a personal computing device or a mobile computing device may not permit all of the videoconference video streams to be displayed in a single horizontal row. In such cases, different local layouts of videoconference video streams may be defined that also provide the illusion of enabling eye contact, gesture awareness, and other forms of non-verbal communication with participants at the other videoconference sites. In one example, the local layout for displaying videoconference video streams at a particular site may arrange the videoconference video streams to be displayed into two horizontal rows of stacked vertical columns with the videoconference video streams being assigned to positions within the two horizontal rows of stacked vertical columns in a manner that essentially preserves the ordering of the other videoconference video streams defined by the central layout 204.
[0042] A technique for defining a local layout for displaying videoconference video streams from a videoconference in two horizontal rows of stacked vertical columns in a manner that preserves an ordering of the videoconference video streams defined by a central layout for the videoconference is illustrated in FIGS. 2C-2D. More particularly, FIGS. 2C-2D follow on FIGS. 2A-2B and illustrate a technique for defining a local layout for displaying videoconference video streams 202(1 )-202(n) at a site that is represented by "Location 0" in the central layout 204 and that corresponds to videoconference video stream 202(0).
[0043] As illustrated in FIGS. 2C-2D, the local layout for displaying videoconference video streams 202(1 )-202(n) in two horizontal rows 220 of stacked vertical columns 222 is generated by alternately stacking every other one of videoconference video streams 202(1 )-202(n) on top of the videoconference video stream 202 immediately preceding it in the ordering of videoconference video streams 202(1 )-202(n) defined by central layout 204 and reflected in FIG. 2B. As such, as illustrated in FIG. 2D, videoconference video stream 202(2) is stacked on top of videoconference video stream 202(1 ) in a first vertical column 222, videoconference video stream 202(4) is stacked on top of videoconference video stream 202(3) in a second vertical column 222, and so on and so forth until videoconference video stream 202(n) is stacked on top of the videoconference video stream 202(n-1 ) in the final vertical column 222.
[0044] By arranging the videoconference video streams 202(1 )-202(n) into the horizontal rows 220 and the vertical columns 222 according to the relative ordering of the videoconference video streams 202(1 )-202(n) defined by central layout 204, the local layout illustrated in FIG. 2D effectively simulates the experience that the videoconference participants at the other sites are sitting around a conference table from the videoconference participants at the local site in the order defined by the central layout 204. For example, as illustrated in FIG. 2D, the videoconference participants in videoconference video streams 202(1 ) and 202(2) appear to be sitting to the left of the videoconference participants at the local site, the videoconference participants in videoconference video streams 202(3) and 202(4) appear to be sitting across from the videoconference participants at the local site, and the videoconference participants in videoconference video streams 202(n-1 ) and 202(n) appear to be sitting to the right of the videoconference participants at the local site. Moreover, by arranging the videoconference video streams 202(1 )-202(n) into the horizontal rows 220 and the vertical columns 222 according to the relative ordering of the videoconference video streams 202(1 )-202(n) defined by central layout 204, the local layout provides the illusion of enabling eye contact, gesture awareness, and other forms of non-verbal communication between videoconference participants at the local site and the other sites.
[0045] As will be described in greater detail below, the concept of arranging a local layout for displaying an ordered list of videoconference video streams at a particular location into two horizontal rows of stacked vertical columns as described above in connection with FIGS. 2A-2D may be extended to defining a local layout for displaying an ordered list of videoconference video streams in any number of m rows of n stacked vertical columns by assigning the first m videoconference video streams in the ordered list to the first column, the second m videoconference video streams in the ordered list to the second column, and so on and so forth until all of the videoconference video streams in the ordered list have been assigned to appropriate positions within the mxn local layout.
[0046] The techniques described above both for generating a central layout for sites participating in a videoconference (e.g., an ordered list of videoconference video streams corresponding to sites participating in the videoconference) and for generating local layouts based on the central layout (e.g., unique orderings of the videoconference video streams corresponding to the sites participating in the videoconference relative to one another) may be generalized for any number of sites participating in a videoconference and their corresponding videoconference video streams. For example, for any number T videoconference video streams representing sites participating in a videoconference, Table 1 below may be used to determine unique local orderings of the videoconference video streams relative to one another for each of the T videoconference video streams:
Figure imgf000018_0001
Table 1.
If T represents the number of videoconference video streams representing sites participating in the videoconference, V represents an ordered list of the videoconference video streams arranged as (Γ-1 , 0, ... T-2), and L represents relative positions within the local ordered list corresponding to any one of the T videoconference video streams arranged as (0... T-2), the value S j of any cell in Table 1 may be computed programmatically according to Equation 1 below:
S j = Mod[(Lj + Vj + 2), T] = (Lt + Vj + 2) - T INT(^ ¾ (Eq. 1 )
[0047] FIG. 3A is a flow chart 300 of an example of a process for defining a layout for displaying videoconference video streams for a videoconference. More particularly, FIG. 3A is a flow chart 300 of an example of a process for defining a local layout for displaying videoconference video streams for a videoconference at a site participating in the videoconference based on an ordered list of the videoconference video streams to be displayed at the site. The ordered list of the videoconference video streams to be displayed at the site may be derived from a central layout defined for the videoconference, for example according to the techniques described above in connection with FIGS. 2A-2D and Table 1 and Equation 1 . The process illustrated in the flowchart 300 of FIG. 3A may be performed by a computing system such as, for example, computing system 700 illustrated in FIG. 7 and described in greater detail below. More specifically, the process illustrated in the flowchart 300 of FIG. 3A may be performed by one or more of the processor(s) 706 of the computing system 700 as a consequence of executing the instructions 708 illustrated in FIG. 7 and described in greater detail below.
[0048] At 302, the ordered list of videoconference video streams to be displayed at the site for the videoconference is accessed. The ordered list of videoconference video streams to be displayed may have been derived from a central layout defined for the videoconference and it may reflect a unique ordering of the videoconference video streams to be displayed relative to the site as defined by the central layout for the videoconference. For example, referring to FIG. 3B, the ordered list of videoconference video streams may define an ordering of nine videoconference video streams 320 corresponding to nine different sites participating in the videoconference (assuming a 1 :1 ratio between videoconference sites and locations in the central layout). As illustrated in FIG. 3B, videoconference video stream 320(a) is assigned to position 0 in the local ordering, videoconference video stream 320(b) is assigned to position 1 in the local ordering, videoconference video stream 320(c) is assigned to position 2 in the local ordering, videoconference video stream 320(d) is assigned to position 3 in the local ordering, videoconference video stream 320(e) is assigned to position 4 in the local ordering, videoconference video stream 320(f) is assigned to position 5 in the local ordering, videoconference video stream 320(g) is assigned to position 6 in the local ordering, videoconference video stream 320(h) is assigned to position 7 in the local ordering, and videoconference video stream 320(i) is assigned to position 8 in the local ordering.
[0049] At 304, a number of columns M in which to display the videoconference video streams at the site is determined. In some implementations, the number of columns M in which to display the videoconference video streams may be determined based on the number of display devices available at the site for displaying the videoconference video streams, with each display device available at the site corresponding to a column of videoconference video streams to be displayed. That is to say, the number of columns M in which to display the videoconference video streams may be set equal to the number of display devices available at the site for displaying the videoconference video streams. Alternatively, in other implementations, the number of columns M in which to display the videoconference video streams may be determined based on the total available width available for displaying the videoconference video streams at the site and/or a desired aspect ratio for each of the videoconference video streams. Additionally or alternatively, the number of columns M in which to display the videoconference video streams may be determined based on user-specified settings. Referring to FIG. 3C, in one example, a determination may be made to display the videoconference video streams in three columns 322.
[0050] At 306, after having determined the number of columns M in which to display the videoconference video streams, the number of videoconference video streams to be displayed at the site is divided by M to determine a number of videoconference video streams Q to be displayed in each column. For example, referring to FIGS. 3B-3C, if there are nine videoconference video streams 320 to be displayed at the site in three columns 322, nine may be divided by three to determine that three videoconference video streams 320 should be displayed in each column 322.
[0051] At 308, the videoconference video streams to be displayed at the site are assigned sequentially to positions in the M columns in increments of Q videoconference video streams based on the ordering defined by the ordered list. For example, the first Q videoconference video streams in the ordered list are assigned to the first column, the second Q videoconference video streams in the ordered list are assigned to the second column, and so on and so forth until all of the videoconference video streams have been assigned to one of the M columns. For example, referring to FIGS. 3B-3C and continuing with the example of displaying nine videoconference video streams 320 in three columns 322 of three videoconference video streams 320 each, the first three videoconference video streams 320(a)-320(c) in the ordered list illustrated in FIG. 3B are assigned to the first column 322(a) illustrated in FIG. 3C, the second three videoconference video streams 320(d)-320(f) in the ordered list illustrated in FIG. 3B are assigned to the second column 322(b) illustrated in FIG. 3C, and the remaining three videoconference video streams 320(g)- 320(i) are assigned to the third column 322(c) illustrated in FIG. 3C.
[0052] As illustrated in FIGS. 3B-3C, the videoconference video streams 320 are slotted into positions in the columns 322 to which they are assigned by filling the columns 322 with the videoconference video streams 320 from bottom to top in the order defined by the ordered list illustrated in FIG. 3B. However, in some implementations, different techniques may be used to arrange the videoconference video streams 320 in positions within the columns 322 to which they have been assigned. For example, the videoconference video streams 320 may be slotted into positions in the columns 322 to which they are assigned by filling the columns 322 with the videoconference video streams 320 from top to bottom in the order defined by the ordered list illustrated in FIG. 3B. Alternatively, the videoconference video streams 320 may be slotted into positions in the columns 322 to which they are assigned in a manner that does not necessarily preserve a bottom to top or top to bottom ordering of the videoconference video streams as defined in the ordered list illustrated in FIG. 3B.
[0053] FIG. 4A is a flow chart 400 of another example of a process for defining a layout for displaying videoconference video streams for a videoconference. More particularly, FIG. 4A is a flow chart 400 of an example of a process for defining a local layout for displaying videoconference video streams for a videoconference at a site participating in the videoconference based on an ordered list of the videoconference video streams to be displayed at the site. The ordered list of the videoconference video streams to be displayed at the site may be derived from a central layout defined for the videoconference, for example according to the techniques described above in connection with FIGS. 2A-2D and Table 1 and Equation 1 . The process illustrated in the flowchart 400 of FIG. 4A may be performed by a computing system such as, for example, computing system 700 illustrated in FIG. 7 and described in greater detail below. More specifically, the process illustrated in the flowchart 400 of FIG. 4A may be performed by one or more of the processor(s) 700 of the computing system 700 as a consequence of executing the instructions 708 illustrated in FIG. 7 and described in greater detail below.
[0054] At 402, the ordered list of videoconference video streams to be displayed at the site for the videoconference is accessed. The ordered list of videoconference video streams to be displayed may have been derived from a central layout defined for the videoconference and it may reflect a unique ordering of the videoconference video streams to be displayed relative to the site as defined by the central layout for the videoconference. For example, referring to FIG. 4B, the ordered list of videoconference video streams may define an ordering of nine videoconference video streams 420 corresponding to nine different locations participating in the videoconference. As illustrated in FIG. 4B, videoconference video stream 420(a) is assigned to position 0 in the local ordering, videoconference video stream 420(b) is assigned to position 1 in the local ordering, videoconference video stream 420(c) is assigned to position 2 in the local ordering, videoconference video stream 420(d) is assigned to position 3 in the local ordering, videoconference video stream 420(e) is assigned to position 4 in the local ordering, videoconference video stream 420(f) is assigned to position 5 in the local ordering, videoconference video stream 420(g) is assigned to position 6 in the local ordering, videoconference video stream 420(h) is assigned to position 7 in the local ordering, and videoconference video stream 420(i) is assigned to position 8 in the local ordering.
[0055] At 404, a number of rows M in which to display the videoconference video streams at the site is determined. Referring to FIG. 4C, in one example, a determination may be made to display the videoconference video streams 420 in three rows 422.
[0056] At 408, the videoconference video streams to be displayed at the site are assigned sequentially to each of the M rows in succession based on the ordering defined by the ordered list. For example, each of the first M videoconference video streams in the ordered list may be assigned to a different one of the M rows in accordance with the order defined by the ordered list. Thereafter, each of the second M videoconference video streams in the ordered list may be assigned to a different one of the M rows in accordance with the order defined by the ordered list and so on and so forth until all of the videoconference video streams have been assigned to one of the M rows.
[0057] For example, referring to FIGS. 4B-4C and continuing with the example of displaying videoconference video streams 420(a)-420(i) in three rows 422, the first three videoconference video streams 420(a)-420(c) in the ordered list illustrated in FIG. 4B are assigned in sequence to Row 1 422(a), Row 2 422(b), and Row 3 422(c), respectively. As illustrated in FIGS. 4B-4C, this arrangement is repeated for the second three videoconference video streams 420(d)-420(f) and the final three videoconference video streams 420(g)-420(i) in the ordered list illustrated in FIG. 4B. As also illustrated in FIGS. 4B-4C, the videoconference video streams 420 are assigned to positions within the rows 422 to which they are assigned that reflect the ordering defined by the ordered list illustrated in FIG. 4B. For example, the first three videoconference video streams 420(a)-420(c) in the ordered list illustrated in FIG. 4B are assigned to the first positions in their respective rows, the second three videoconference video streams 420(d)-420(f) are assigned to the second positions in their respective rows, and the final three videoconference video streams 420(g)-420(i) are assigned to the final positions in their respective rows.
[0058] As illustrated in FIGS. 4C, the videoconference video streams 420 are stacked in aligned columns within the rows 422 to which they are assigned. However, in some implementations, other arrangements may be generated and the videoconference video streams 420 may be assigned to positions within the rows 422 that do not result in the videoconference video streams also being aligned in columns. For example, referring to FIG. 4D, the videoconference video streams 420(a)-420(i) may be arranged in multiple staggered rows 430.
[0059] As described above, a central layout may be defined for videoconference video streams representing sites participating in a videoconference. This central layout then may be used to derive local layouts for displaying videoconference video streams at the sites participating in the videoconference. This approach to arranging a videoconference may provide individual sites participating in the videoconference with the flexibility to define their own local layouts, for example to take advantage of the configurations at the different sites participating in the videoconference, while also providing the illusion of maintaining eye contact, gesture awareness, and other forms of non-verbal communication between participants located at different sites participating in the videoconference.
[0060] FIG. 5 is a block diagram of an example of a communications system 500 that includes a network 502 connecting five different sites 504 that are participating in a videoconference. A central layout 506 of representations of videoconferencing video streams corresponding to the sites 504 is defined. FIG. 5 illustrates how defining a central layout 506 for the videoconference that may be used by the sites 504 participating in the videoconference to derive local layouts for displaying videoconference video streams may provide the individual sites 504 participating in the videoconference with the flexibility to define their own local layouts while also providing the illusion of maintaining eye contact, gesture awareness, and other forms of non-verbal communication with participants located at other sites 504 participating in the videoconference even if the sites 504 participating in the videoconference are configured differently from each other.
[0061] For example, as illustrated in FIG. 5, site 504(0) is configured differently from site 504(1 ). Specifically, site 504(0) has a single display device 508 for displaying videoconference video streams whereas site 504(1 ) has four display devices 510 for displaying videoconference video streams.
[0062] Due to the fact that there is only a single display device 508 available for displaying videoconference video streams at site 504(0) it may not be desirable to display videoconference video streams corresponding to each of the other sites 504(1 )-504(4) participating in the videoconference in a single horizontal row at site 504(0). Consequently, a local layout for displaying the videoconference video streams corresponding to sites 504(1 )-504(4) in two horizontal rows may be defined for site 504(0) in a manner that preserves the ordering of the videoconference video streams corresponding to sites 504(1 )-504(4) relative to site 504(0) as defined by the central layout 506 according to techniques described above. For example, as illustrated in FIG. 5, the videoconference video stream corresponding to site 504(1 ) and the videoconference video stream corresponding to site 504(2) may be displayed in a stacked column on the left side of display device 508 at site 504(0) and the videoconference video stream corresponding to site 504(3) and the videoconference video stream corresponding to site 504(4) may be displayed in a stacked column on the right side of display device 508 at site 504(0).
[0063] Meanwhile, because there are four display devices 510 available for displaying videoconference video streams at site 504(1 ) as opposed to the single display device 508 available for displaying videoconference video streams at site 504(0), a local layout for site 504(1 ) may be defined for displaying the videoconference video streams corresponding to the other sites 504(0) and 504(2)- 504(4) participating in the videoconference in a single horizontal row in a manner that preserves the ordering of the videoconference video streams corresponding to the other participating sites 504(0) and 504(2)-504(4) relative to site 504(0) as defined by the central layout 506. For example, as illustrated in FIG. 5, the videoconference video stream corresponding to site 504(2) is displayed on display device 510(a), the videoconference video stream corresponding to site 504(3) is displayed on display device 510(b), the videoconference video stream corresponding to site 504(4) is displayed on display device 510(c), and the videoconference video stream corresponding to site 504(0) is displayed on display device 510(d).
[0064] As described above, in some implementations, a site participating in a videoconference may have multiple video cameras that are configured to have substantially the same field of view and to capture images of substantially the same subject matter albeit from varying perspectives. In such implementations, the different videoconference video streams generated by the video cameras at the site all may be represented by a single location within a central layout defined for the videoconference, but different ones of the videoconference video streams generated by the individual cameras may be transmitted to different ones of the other sites participating in the videoconference. For example, the videoconference video stream to be transmitted to a particular one of the other sites participating in the videoconference may be selected to be the videoconference video stream generated by the camera that is closest to the local display of the videoconference video stream that corresponds to the particular site.
[0065] FIG. 6 is a block diagram of an example of a communications system 600 that includes a network 602 connecting seven different sites 604 that are participating in a videoconference. A central layout 605 of representations of videoconference video streams corresponding to the different participating sites 604 is defined. As illustrated in FIG. 6, site 604(0) is a dedicated telepresence studio having three display devices 606 and three video cameras 608. In some implementations, each video camera 608 is positioned in a central location above a corresponding display device 606 as illustrated in FIG. 6. In other implementations, the video cameras 608 may be positioned differently relative to display devices 606.
[0066] As illustrated in FIG. 6, the local layout at site 604(0) for displaying videoconference video streams corresponding to the other sites 604(1 )-604(6) participating in the videoconference preserves the order of the other locations representing the videoconference video streams corresponding to the other sites 604(1 )-604(6) participating in the videoconference relative to the location representing the videoconference video stream corresponding to site 604(0) defined by the central layout 605. In particular, the videoconference video stream corresponding to site 604(1 ) and the videoconference video stream corresponding to site 604(2) are displayed on the leftmost display device 606(a) at site 604(0), the videoconference video stream corresponding to site 604(3) and the videoconference video stream corresponding to site 604(4) are displayed on the central display device 606(b) at site 604(0), and the videoconference video stream corresponding to site 604(5) and the videoconference video stream corresponding to site 604(6) are displayed on the rightmost display device 606(c) at site 604(0).
[0067] The video cameras 608 at site 604(0) are configured to have substantially the same field of views and to capture images of substantially the same subject matter albeit from varying perspectives. For example, as illustrated in FIG. 6, each video camera 608 is configured to capture images of the four local videoconference participants 610 at site 604(0). Therefore, the videoconference video streams generated by the different cameras 608 may be represented by a single location within central layout 605.
[0068] However, different ones of the videoconference video streams generated by the individual cameras 608 may be transmitted to different ones of the other sites 604(1 )-604(6) participating in the videoconference. In some implementations, the determination of which one of the videoconference video streams generated by cameras 608 should be transmitted to a particular one of the other sites 604(1 )-604(6) participating in the videoconference may be made by identifying which of the cameras 608 is located nearest to the display of the videoconference video stream corresponding to the particular site 604 and transmitting the videoconference video stream generated by that camera 608 to the particular site.
[0069] For example, as illustrated in FIG. 6, camera 608(a) is the camera 608 located nearest to the display of the videoconference video streams corresponding to sites 604(1 ) and 604(2). Therefore, the videoconference video stream generated by camera 608(a) is transmitted to sites 604(1 ) and 604(2) for the purpose of providing a view of site 604(0) at sites 604(1 ) and 604(s). Similarly, camera 608(b) is the camera 608 located nearest to the display of the videoconference video streams corresponding to sites 604(3) and 604(4), so the videoconference video stream generated by camera 608(b) is transmitted to sites 604(3) and 604(4) for the purpose of providing a view of site 604(0) at sites 604(3) and 604(4). Likewise, camera 604(c) is the camera 608 located nearest to the display of the videoconference video streams corresponding to sites 604(5) and 604(6), so the videoconference video stream generated by camera 608(c) is transmitted to sites 604(5) and 604(6) for the purpose of providing a view of site 604(0) at sites 604(5) and 604(6).
[0070] As illustrated in FIG. 6, there is a 1 :1 ratio between display devices 606 and video cameras 608 at site 604(0), and one video camera 608 is located above each display device 606. In such implementations, where there is a video camera corresponding to each display device on which a view of a remote site is displayed, the videoconference video streams generated by the video cameras may be transmitted to the remote sites for which views are displayed on the display devices to which they correspond. In some implementations, however, there may not be a 1 :1 ratio between display devices and video cameras and/or individual video cameras may not necessarily be located proximally to individual display devices. Nevertheless, the determination of to which remote site to route which videoconference video stream may be made based on determinations of which video cameras are nearest to the displays of views of particular remote sites.
[0071] If other of the sites 604(1 )-604(6) participating in the videoconference besides site 604(0) also have multiple video cameras that generate videoconference video streams of substantially the same images but from different perspectives, such other sites 604 may employ similar rules for determining which of the videoconference video streams to transmit to site 604(0). For example, if site 604(5) also has multiple video cameras that generate videoconference video streams of substantially the same images but from different perspectives, the videoconference video stream generated by the camera at site 604(5) that is nearest to the display at site 604(5) of the videoconference video stream corresponding to site 604(0) may be selected as the videoconference video stream from site 604(5) to transmit to site 604(0). Using these techniques to control which videoconference video streams generated at a site participating in a videoconference are routed to other sites participating in the videoconference may contribute to the illusion of maintaining eye contact, gesture awareness, and other forms of non-verbal communication between participants across the different sites participating in the videoconference.
[0072] For example, when such techniques are employed and a participant at site 604(0) looks at the display of the participants at site 604(5) on display device 606(c), the participant essentially will be looking towards video camera 608(c), and the videoconference video stream generated by camera 608(c) will be transmitted to site 604(5) where it is displayed on the display device at 604(5) that is nearest to the video camera at site 604(5) from which the videoconference video streams transmitted to site 604(0) are generated. Consequently, when a participant at site 604(5) looks toward the display device on which the videoconference video stream originating at site 604(0) is displayed, the participant at site 604(5) may experience the illusion that he is making eye contact with the participant at site 604(0) and vice versa. Employing these techniques for determining which videoconference video stream to route to which site based on which video cameras are closest to the corresponding displays of the sites at multiple sites participating in a videoconference effectively may create eye contact pairs between pairs of the remote sites at which the techniques are employed, thus contributing to the illusion of maintaining eye contact, gesture awareness, and other forms of non-verbal communication between participants across the different sites participating in the videoconference.
[0073] FIG. 7 is a block diagram of an example of a computing system 700 for generating local layout information 702 for displaying videoconference video streams at a site participating in a videoconference from central layout information 704 defined for the videoconference. As illustrated in FIG. 7, computing system 700 includes one or more processors 706 and computer-readable storage 708 storing executable instructions for generating local layout information 702 for displaying videoconference video streams at a site participating in a videoconference from central layout information 704 defined for the videoconference as described above.
[0074] Central layout information 704 may be an ordered list of locations representing videoconference video streams corresponding to sites participating in the videoconference.
[0075] Processor(s) 706 may be central processing units (CPUs), semiconductor- based microprocessors, any other hardware devices suitable for retrieval and execution of instructions stored in computer-readable storage 708, and/or any combination thereof. Processor(s) 706 may fetch, decode, and execute instructions stored in computer-readable storage 708 to implement the functionality for generating local layout information 704 for displaying videoconference video streams at a site participating in a videoconference from central layout information 702 defined for the videoconference as described above.
[0076] A number of implementations have been described. However, other variations also are within the scope of this disclosure. For example, although the techniques for generating local layouts described above generally are described in the context of generating local layouts for displaying separate videoconference video streams at a site, in some implementations, a local layout for a site may be defined and, somewhere upstream from the site, the individual videoconference video streams comprising the local layout may be composited into a single videoconference video stream configured according to the local layout that is transmitted to the site for display at the site. Furthermore, in some implementations, the local layout defined for a site may have more positions for displaying videoconference video streams than there are videoconference video streams to be displayed at the site. For example, a local layout defined for a site may call for an arrangement of two rows and three columns providing for the display of six videoconference video streams at the site. However, there may only be five videoconference video streams to display at the site. In such cases, the extra position within the local layout may be filled with a placeholder such as, for example, a simulated image of videoconference participants, a blank screen, etc.
[0077] As described above, in one general aspect, a first ordered list of t representations of videoconference participant locations is accessed. A second ordered list of M videoconference video streams then is derived from the first ordered list of t representations of videoconference participant locations. Furthermore, a local layout for displaying the M videoconference video streams in at least first and second vertical columns of video streams is defined based on the second ordered list of M videoconference video streams. In particular, a first two videoconference video streams in adjacent positions in the second ordered list of t- 1 videoconference video streams are assigned to the first column and a second two videoconference video streams in adjacent positions in the second ordered list of M videoconference video streams are assigned to the second column, where the first two videoconference video streams are different from the second two videoconference video streams.
[0078] Implementations may include one or more of the following features. For example, the M videoconference video streams may be received and displayed according to the local layout (e.g., in a first vertical column on a first display device and a second vertical column on a second display device or, alternatively, in first and second vertical columns on the same display device).
[0079] In some implementations, accessing a first ordered list of t representations of videoconference participant locations may include accessing a first ordered list of 7 representations of videoconference participant locations, and deriving a second ordered list of M videoconference video streams from the first ordered list of t representations of videoconference participant locations may include deriving a second ordered list of 6 videoconference video streams from the first ordered list of 7 representations of videoconference participant locations. In addition, defining a local layout for displaying the M videoconference video streams in at least first and second vertical columns of video streams may include defining a local layout for displaying the 6 videoconference video streams in first, second, and third vertical columns where the videoconference video streams in first and second positions of the second ordered list are assigned to the first column, the videoconference video streams in third and fourth positions in the second ordered list are assigned to the second column, and the videoconference video streams in fifth and sixth positions in the second ordered list are assigned to the third column. Furthermore, displaying the M videoconference video streams according to the local layout may include displaying the videoconference video streams in the first and second positions of the second ordered list in a first column, displaying the videoconference video streams in the third and fourth positions in the second ordered list in a second column, and displaying the videoconference video streams in the fifth and sixth positions in the second ordered list in a third column.
[0080] Additionally or alternatively, defining a local layout for displaying the M videoconference video streams in at least first and second vertical columns of video streams may include defining a local layout for displaying the M videoconference video streams in a first vertical column corresponding to a first display device and a second vertical column corresponding to a second display device where the first two videoconference video streams in adjacent positions in the second ordered list of M videoconference video streams are assigned to the first column corresponding to the first display device and the second two videoconference video streams in adjacent positions in the second ordered list of t- 1 videoconference video streams are assigned to the second column corresponding to the second display device. In such implementations, displaying the M videoconference video streams according to the local layout may include displaying the first two videoconference video streams in a first vertical column on the first display device, and displaying the second two videoconference video streams in a second vertical column on the second display device. Furthermore, a first one of the first two videoconference video streams may be assigned to a lower position in the first vertical column and a second one of the first two videoconference video streams may be assigned to an upper position in the first vertical column that is above the lower position in the first vertical column, wherein the first one of the first two videoconference video streams may occupy a position within the second ordered list of M video streams that is before a position occupied by the second one of the first two videoconference video streams within the second ordered list of M video streams. Similarly, a first one of the second two videoconference video streams may be assigned to a lower position in the second vertical column and a second one of the second two videoconference video streams may be assigned to an upper position in the second vertical column that is above the lower position in the second vertical column, wherein the first one of the second two videoconference video streams may occupy a position within the second ordered list of M video streams that is before a position occupied by the second one of the second two videoconference video streams within the second ordered list of M video streams.
[0081] In some implementations, accessing a first ordered list of t representations of videoconference participant locations may include accessing a first ordered list of t representations of videoconference video streams that specifies a position for a videoconference video stream that corresponds to a local site relative to the other M representations of videoconference video streams in the first ordered list such that a first subset of the M representations of videoconference video streams occupy positions in the first ordered list that precede the position for the representation of the videoconference video stream that corresponds to the local site and a second subset of the M representations of videoconference video streams occupy positions in the first ordered list that succeed the position for the representation of videoconference video stream that corresponds to the local site. In such implementations, deriving the second ordered list of M videoconference video streams from the first ordered list of t representations of videoconference video streams may include assigning the videoconference video streams represented in the first subset of videoconference video streams to positions in the second ordered list in the same relative order as their representations in the first ordered list, and assigning the videoconference video streams represented in the second subset of videoconference video streams to positions in the second ordered list in the same relative order as their representations in the second ordered list and such that, within the second ordered list, the positions occupied by the videoconference video streams represented in the second subset precede the positions occupied by the videoconference video streams represented in the first subset.
[0082] Additionally or alternatively, deriving, from the first ordered list of t representations of videoconference participant locations, a second ordered list of t- 1 videoconference video streams may include deriving, from the first ordered list of t representations of videoconference participant locations, a second ordered list of a first set of M videoconference video streams, and defining, based on the second ordered list of M videoconference video streams, a local layout for displaying the t- 1 videoconference video streams includes defining, based on the second ordered list of the first set of M videoconference video streams, a first local layout for displaying the first set of M videoconference video streams at a first location. Furthermore, a third ordered list of a second set of M videoconference video streams, may be derived from the first ordered list of t representations of videoconference participant locations, where the second set of M videoconference video streams are different from the first set of M videoconference video streams. In addition, a second local layout for displaying the second set of M videoconference video streams at a second location may be defined based on the third ordered list of the second set of M video streams, where the second location may be different from the first location and the second local layout may be different from the first local layout. [0083] In some implementations, a resolution reduction factor for displaying a particular one of the M videoconference video streams according to the local layout may be determined based on the local layout defined for displaying the M videoconference video streams and an indication of the resolution reduction factor for the particular videoconference video stream may be communicated to an entity responsible for transmitting the particular videoconference video stream. In addition, the particular videoconference video stream may be received at a resolution reduced by the resolution reduction factor for the particular videoconference video stream relative to an original resolution of the particular videoconference video stream.
[0084] In another general aspect, first, second, and third display devices are aligned horizontally in a row with the first display device being adjacent to the second display device and the second display device being adjacent to the third display device. A first video camera is vertically aligned with the first display device, a second video camera is vertically aligned with the second display device, and a third video camera is vertically aligned with the third display device. In addition, a computing facility includes one or more processing elements and computer-readable storage storing instructions that, when executed by the one or more processing elements, cause the computing facility to receive six incoming videoconference video streams for a videoconference, access an ordered list of the incoming videoconference video streams, cause the video streams in first and second positions in the ordered list to be displayed on the first display device, cause the video streams in third and fourth positions in the ordered list to be displayed on the second display device, cause the video streams in the fifth and sixth positions in the ordered list to be displayed on the third display device, cause video images captured by the first video camera in connection with the videoconference to be transmitted to sites corresponding to the first and second video streams, cause video images captured by the second video camera in connection with the videoconference to be transmitted to sites corresponding to the third and fourth video streams, and cause video images captured by the third video camera in connection with the videoconference to be transmitted to sites corresponding to the fifth and sixth video streams.
[0085] Implementations may include one or more of the following features. For example, the instructions that, when executed by the one or more processing elements, cause the computing facility to cause the video streams in the first and second positions in the ordered list to be displayed on the first display device may include instructions that, when executed by the one or more processing elements, cause the computing facility to cause the video stream in the first position in the ordered list to be displayed on a lower portion of the first display device and the video stream in the second position in the ordered list to be displayed on an upper portion of the first display device that is above the lower portion of the first display device. Similarly, the instructions that, when executed by the one or more processing elements, cause the computing facility to cause the video streams in the third and fourth positions in the ordered list to be displayed on the second display device may include instructions that, when executed by the one or more processing elements, cause the computing facility to cause the video stream in the third position in the ordered list to be displayed on a lower portion of the second display device and the video stream in the fourth position in the ordered list to be displayed on an upper portion of the second display device that is above the lower portion of the second display device, and the instructions that, when executed by the one or more processing elements, cause the computing facility to cause the video streams in the fifth and sixth positions in the ordered list to be displayed on the third display device may include instructions that, when executed by the one or more processing elements, cause the computing facility to cause the video stream in the fifth position in the ordered list to be displayed on a lower portion of the third display device and the video stream in the sixth position in the ordered list to be displayed on an upper portion of the third display device that is above the lower portion of the third display device. [0086] In some implementations, the dimensions of the first, second, and third display devices are the same. Additionally or alternative, the first camera may be positioned above the first display device, the second camera may be positioned above the second display device, and the third camera may be positioned above the third display device.
[0087] In yet another general aspect, an ordered list of videoconference video streams is accessed, positions for displaying images from the videoconference video streams in two horizontal rows with images from adjacent videoconference video streams in the ordered list being assigned to opposite rows is defined based on the ordered list of videoconference video streams. In some implementations, defining positions for displaying images from the videoconference video streams in two horizontal rows with images from adjacent videoconference video streams in the ordered list being assigned to opposite rows may include defining positions for displaying images from the videoconference video streams in horizontal rows that are aligned to form vertical columns.
[0088] A number of methods, techniques, systems, and apparatuses have been described. The described methods, techniques, systems, and apparatuses may be implemented in digital electronic circuitry or computer hardware, for example, by executing instructions stored in computer-readable storage media.
[0089] Apparatuses implementing these techniques may include appropriate input and output devices, a computer processor, and/or a tangible computer- readable storage medium storing instructions for execution by a processor.
[0090] A process implementing techniques disclosed herein may be performed by a processor executing instructions stored on a tangible computer-readable storage medium for performing desired functions by operating on input data and generating appropriate output. Suitable processors include, by way of example, both general and special purpose microprocessors. Suitable computer-readable storage devices for storing executable instructions include all forms of non-volatile memory, including, by way of example, semiconductor memory devices, such as Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), and flash memory devices; magnetic disks such as fixed, floppy, and removable disks; other magnetic media including tape; and optical media such as Compact Discs (CDs) or Digital Video Disks (DVDs). Any of the foregoing may be supplemented by, or incorporated in, specially designed application-specific integrated circuits (ASICs).
[0091] Although the operations of the disclosed techniques may be described herein as being performed in a certain order and/or in certain combinations, in some implementations, individual operations may be rearranged in a different order, combined with other operations described herein, and/or eliminated and the desired results still may be achieved. Similarly, components in the disclosed systems may be combined in a different manner and/or replaced or supplemented by other components and the desired results still may be achieved.

Claims

WHAT IS CLAIMED IS:
1 . A method comprising:
accessing a first ordered list of t representations of videoconference participant locations;
deriving, from the first ordered list of t representations of videoconference participant locations, a second ordered list of M videoconference video streams; and
defining, based on the second ordered list of M videoconference video streams, a local layout for displaying the M videoconference video streams in at least first and second vertical columns of video streams where a first two
videoconference video streams in adjacent positions in the second ordered list of t- 1 videoconference video streams are assigned to the first column and a second two videoconference video streams in adjacent positions in the second ordered list of M videoconference video streams are assigned to the second column, the first two videoconference video streams being different from the second two
videoconference video streams.
2. The method of claim 1 further comprising:
receiving the M videoconference video streams; and
displaying the £-1 videoconference video streams according to the local
3. The method of claim 2 wherein:
accessing a first ordered list of t representations of videoconference participant locations includes accessing a first ordered list of 7 representations of videoconference participant locations;
deriving a second ordered list of M videoconference video streams from the first ordered list of t representations of videoconference participant locations includes deriving a second ordered list of 6 videoconference video streams from the first ordered list of 7 representations of videoconference participant locations; defining a local layout for displaying the M videoconference video streams in at least first and second vertical columns of video streams includes defining a local layout for displaying the 6 videoconference video streams in first, second, and third vertical columns where the videoconference video streams in first and second positions of the second ordered list are assigned to the first column, the
videoconference video streams in third and fourth positions in the second ordered list are assigned to the second column, and the videoconference video streams in fifth and sixth positions in the second ordered list are assigned to the third column; and
displaying the M videoconference video streams according to the local layout includes:
displaying the videoconference video streams in the first and second positions of the second ordered list in a first column,
displaying the videoconference video streams in the third and fourth positions in the second ordered list in a second column, and
displaying the videoconference video streams in the fifth and sixth positions in the second ordered list in a third column.
4. The method of claim 2 wherein:
defining a local layout for displaying the M videoconference video streams in at least first and second vertical columns of video streams includes defining a local layout for displaying the M videoconference video streams in a first vertical column corresponding to a first display device and a second vertical column corresponding to a second display device where the first two videoconference video streams in adjacent positions in the second ordered list of M
videoconference video streams are assigned to the first column corresponding to the first display device and the second two videoconference video streams in adjacent positions in the second ordered list of M videoconference video streams are assigned to the second column corresponding to the second display device; and
displaying the M videoconference video streams according to the local layout includes:
displaying the first two videoconference video streams in a first vertical column on the first display device, and
displaying the second two videoconference video streams in a second vertical column on the second display device.
5. The method of claim 4 wherein defining a local layout for displaying the M videoconference video streams in a first vertical column corresponding to a first display device and a second vertical column corresponding to a second display device includes defining a local layout for displaying the M videoconference video streams in a first vertical column corresponding to a first display device and a second vertical column corresponding to a second display device where:
a first one of the first two videoconference video streams is assigned to a lower position in the first vertical column and a second one of the first two videoconference video streams is assigned to an upper position in the first vertical column that is above the lower position in the first vertical column, wherein the first one of the first two videoconference video streams occupies a position within the second ordered list of M video streams that is before a position occupied by the second one of the first two videoconference video streams within the second ordered list of M video streams; and
a first one of the second two videoconference video streams is assigned to a lower position in the second vertical column and a second one of the second two videoconference video streams is assigned to an upper position in the second vertical column that is above the lower position in the second vertical column, wherein the first one of the second two videoconference video streams occupies a position within the second ordered list of M video streams that is before a position occupied by the second one of the second two videoconference video streams within the second ordered list of M video streams.
6. The method of claim 2 wherein displaying the M videoconference video streams according to the local layout includes:
displaying the first two videoconference video streams in a first vertical column on a display device; and
displaying the second two videoconference video stream in a second vertical column on the same display device as the first two videoconference video streams in the first vertical column.
7. The method of claim 1 wherein:
accessing a first ordered list of t representations of videoconference participant locations includes accessing a first ordered list of t representations of videoconference video streams that specifies a position for a videoconference video stream that corresponds to a local site relative to the other M
representations of videoconference video streams in the first ordered list such that a first subset of the M representations of videoconference video streams occupy positions in the first ordered list that precede the position for the representation of the videoconference video stream that corresponds to the local site and a second subset of the M representations of videoconference video streams occupy positions in the first ordered list that succeed the position for the representation of videoconference video stream that corresponds to the local site; and
deriving the second ordered list of M videoconference video streams from the first ordered list of t representations of videoconference video streams includes:
assigning the videoconference video streams represented in the first subset of videoconference video streams to positions in the second ordered list in the same relative order as their representations in the first ordered list, and
assigning the videoconference video streams represented in the second subset of videoconference video streams to positions in the second ordered list in the same relative order as their representations in the second ordered list and such that, within the second ordered list, the positions occupied by the videoconference video streams represented in the second subset precede the positions occupied by the videoconference video streams represented in the first subset.
8. The method of claim 1 wherein:
deriving, from the first ordered list of t representations of videoconference participant locations, a second ordered list of M videoconference video streams includes deriving, from the first ordered list of t representations of videoconference participant locations, a second ordered list of a first set of M videoconference video streams; and
defining, based on the second ordered list of M videoconference video streams, a local layout for displaying the M videoconference video streams includes defining, based on the second ordered list of the first set of M
videoconference video streams, a first local layout for displaying the first set of M videoconference video streams at a first location, the method further comprising:
deriving, from the first ordered list of t representations of videoconference participant locations, a third ordered list of a second set of M videoconference video streams, the second set of M videoconference video streams being different from the first set of M videoconference video streams; and
defining, based on the third ordered list of the second set of M video streams, a second local layout for displaying the second set of M
videoconference video streams at a second location, the second location being different from the first location and the second local layout being different from the first local layout.
9. The method of claim 1 further comprising:
determining, based on the local layout defined for displaying the M videoconference video streams, a resolution reduction factor for displaying a particular one of the M videoconference video streams according to the local layout;
communicating, to an entity responsible for transmitting the particular videoconference video stream, an indication of the resolution reduction factor for the particular videoconference video stream; and
receiving the particular videoconference video stream at a resolution reduced by the resolution reduction factor for the particular videoconference video stream relative to an original resolution of the particular videoconference video stream.
10. A system comprising:
first, second, and third display devices aligned horizontally in a row with the first display device being adjacent to the second display device and the second display device being adjacent to the third display device;
a first video camera vertically aligned with the first display device;
a second video camera vertically aligned with the second display device; a third video camera vertically aligned with the third display device; and a computing facility including one or more processing elements and computer-readable storage storing instructions that, when executed by the one or more processing elements, cause the computing facility to:
receive six incoming videoconference video streams for a
videoconference,
access an ordered list of the incoming videoconference video streams,
cause the video streams in first and second positions in the ordered list to be displayed on the first display device,
cause the video streams in third and fourth positions in the ordered list to be displayed on the second display device,
cause the video streams in the fifth and sixth positions in the ordered list to be displayed on the third display device,
cause video images captured by the first video camera in connection with the videoconference to be transmitted to sites corresponding to the first and second video streams,
cause video images captured by the second video camera in connection with the videoconference to be transmitted to sites
corresponding to the third and fourth video streams, and
cause video images captured by the third video camera in connection with the videoconference to be transmitted to sites corresponding to the fifth and sixth video streams.
1 1 . The system of claim 10 wherein:
the instructions that, when executed by the one or more processing elements, cause the computing facility to cause the video streams in the first and second positions in the ordered list to be displayed on the first display device include instructions that, when executed by the one or more processing elements, cause the computing facility to cause the video stream in the first position in the ordered list to be displayed on a lower portion of the first display device and the video stream in the second position in the ordered list to be displayed on an upper portion of the first display device that is above the lower portion of the first display device;
the instructions that, when executed by the one or more processing elements, cause the computing facility to cause the video streams in the third and fourth positions in the ordered list to be displayed on the second display device include instructions that, when executed by the one or more processing elements, cause the computing facility to cause the video stream in the third position in the ordered list to be displayed on a lower portion of the second display device and the video stream in the fourth position in the ordered list to be displayed on an upper portion of the second display device that is above the lower portion of the second display device; and the instructions that, when executed by the one or more processing elements, cause the computing facility to cause the video streams in the fifth and sixth positions in the ordered list to be displayed on the third display device include instructions that, when executed by the one or more processing elements, cause the computing facility to cause the video stream in the fifth position in the ordered list to be displayed on a lower portion of the third display device and the video stream in the sixth position in the ordered list to be displayed on an upper portion of the third display device that is above the lower portion of the third display device.
12. The system of claim 10 wherein the dimensions of the first, second, and third display devices are the same.
13. The system of claim 10 wherein:
the first camera is positioned above the first display device;
the second camera is positioned above the second display device; and the third camera is positioned above the third display device.
14. A non-transitory computer-readable storage medium storing instructions that, when executed, cause a computing device to:
access an ordered list of videoconference video streams; and
define, based on the ordered list of videoconference video streams, positions for displaying images from the videoconference video streams in two horizontal rows with images from adjacent videoconference video streams in the ordered list being assigned to opposite rows.
15. The computer-readable storage medium of claim 14 wherein the instructions that, when executed, cause the computing device to define positions for displaying images from the videoconference video streams in two horizontal rows with images from adjacent videoconference video streams in the ordered list being assigned to opposite rows include instructions that, when executed, cause the computing device to define positions for displaying images from the
videoconference video streams in horizontal rows that are aligned to form vertical columns.
PCT/US2011/032473 2011-04-14 2011-04-14 Videoconferencing WO2012141707A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/US2011/032473 WO2012141707A1 (en) 2011-04-14 2011-04-14 Videoconferencing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/US2011/032473 WO2012141707A1 (en) 2011-04-14 2011-04-14 Videoconferencing

Publications (1)

Publication Number Publication Date
WO2012141707A1 true WO2012141707A1 (en) 2012-10-18

Family

ID=47009611

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2011/032473 WO2012141707A1 (en) 2011-04-14 2011-04-14 Videoconferencing

Country Status (1)

Country Link
WO (1) WO2012141707A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2930926A1 (en) * 2014-03-20 2015-10-14 Unify GmbH & Co. KG Method, software product and device for managing a conference
WO2022237381A1 (en) * 2021-05-08 2022-11-17 聚好看科技股份有限公司 Method for saving conference record, terminal, and server

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060244817A1 (en) * 2005-04-29 2006-11-02 Michael Harville Method and system for videoconferencing between parties at N sites
US7515174B1 (en) * 2004-12-06 2009-04-07 Dreamworks Animation L.L.C. Multi-user video conferencing with perspective correct eye-to-eye contact
US20100157017A1 (en) * 2008-12-23 2010-06-24 Tandberg Telecom As Method, device and computer system for processing images in a conference between a plurality of video conferencing terminals

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7515174B1 (en) * 2004-12-06 2009-04-07 Dreamworks Animation L.L.C. Multi-user video conferencing with perspective correct eye-to-eye contact
US20060244817A1 (en) * 2005-04-29 2006-11-02 Michael Harville Method and system for videoconferencing between parties at N sites
US20100157017A1 (en) * 2008-12-23 2010-06-24 Tandberg Telecom As Method, device and computer system for processing images in a conference between a plurality of video conferencing terminals

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2930926A1 (en) * 2014-03-20 2015-10-14 Unify GmbH & Co. KG Method, software product and device for managing a conference
US9307203B2 (en) 2014-03-20 2016-04-05 Unify Gmbh & Co. Kg Method and device for controlling a conference
US9438859B2 (en) 2014-03-20 2016-09-06 Unify Gmbh & Co. Kg Method and device for controlling a conference
WO2022237381A1 (en) * 2021-05-08 2022-11-17 聚好看科技股份有限公司 Method for saving conference record, terminal, and server

Similar Documents

Publication Publication Date Title
US10321093B2 (en) Automated layouts optimized for multi-screen and multi-camera videoconferencing calls
US9462227B2 (en) Automatic video layouts for multi-stream multi-site presence conferencing system
JP5852157B2 (en) Improved view layout management in scalable video and audio communication systems
US9154737B2 (en) User-defined content magnification and multi-point video conference system, method and logic
US9258339B2 (en) Presenting data to electronic meeting participants
US20060259552A1 (en) Live video icons for signal selection in a videoconferencing system
US11871127B2 (en) High-speed video from camera arrays
KR20200138841A (en) Establishing a video conference during a phone call
WO2011140812A1 (en) Multi-picture synthesis method and system, and media processing device
US20140092201A1 (en) Dual-axis image equalization in video conferencing
CN104822070B (en) Multi-path video stream playback method and device
CN111402139A (en) Image processing method, image processing device, electronic equipment and computer readable storage medium
WO2013169384A1 (en) Systems, methods, and computer program products for compound image demosaicing and warping
WO2012141707A1 (en) Videoconferencing
KR102201659B1 (en) Video display modification for video environments
WO2013032461A1 (en) Arranging video streams
CN111860363A (en) Video image processing method and device, electronic equipment and storage medium
CN113055615B (en) Conference all-in-one machine, screen segmentation display method and storage device
US20230379426A1 (en) Client devices, server, and methods
CN113055615A (en) Conference all-in-one machine, screen segmentation display method and storage device
CN115484486A (en) Video presentation method, electronic device, computer storage medium, and program product
JP2013183293A (en) Connection device, system, connection method and connection program for multipoint conference

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11863611

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11863611

Country of ref document: EP

Kind code of ref document: A1