US8477852B2 - Uniform video decoding and display - Google Patents

Uniform video decoding and display Download PDF

Info

Publication number
US8477852B2
US8477852B2 US11/821,069 US82106907A US8477852B2 US 8477852 B2 US8477852 B2 US 8477852B2 US 82106907 A US82106907 A US 82106907A US 8477852 B2 US8477852 B2 US 8477852B2
Authority
US
United States
Prior art keywords
frame
frame buffer
available
buffer
video
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US11/821,069
Other versions
US20080317138A1 (en
Inventor
Wei Jia
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nvidia Corp
Original Assignee
Nvidia Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nvidia Corp filed Critical Nvidia Corp
Assigned to NVIDIA reassignment NVIDIA ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: JIA, WEI
Priority to US11/821,069 priority Critical patent/US8477852B2/en
Priority to JP2008130987A priority patent/JP4836010B2/en
Priority to KR1020080057840A priority patent/KR101034080B1/en
Priority to TW097122787A priority patent/TWI393065B/en
Priority to CN200810126992.7A priority patent/CN101330621B/en
Publication of US20080317138A1 publication Critical patent/US20080317138A1/en
Priority to KR1020110009392A priority patent/KR20110022653A/en
Publication of US8477852B2 publication Critical patent/US8477852B2/en
Application granted granted Critical
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51Motion estimation or motion compensation
    • H04N19/577Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/423Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/44Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder

Definitions

  • Embodiments of the present invention relate to decoding compressed video information.
  • a common approach to video compression schemes involves tracking the changes between one frame of video and the next. Rather than storing separate, complete images for every frame of video, most video compression standards involve providing one or more reference frames, and building the majority of frames by noting how these reference frames change.
  • I frame intra frame
  • P frames predicted frames
  • B frames bi-predictive frames
  • reference frames may be needed for a much longer period of time, and so need to be retained.
  • Decoding compressed video is a platform specific task.
  • the decoder used, for example, to play back a video on a computer is frequently written in a way that locks it to the platform for which it was written, e.g., specific memory handling tasks are built into the decoder.
  • developing a decoder for a different platform e.g., a portable media playback device, generally involves rewriting the decoder entirely.
  • Several of these embodiments utilize a unified frame buffer management system, to facilitate better memory management in decoding and displaying compressed video.
  • One approach describes a method of decoding and displaying compressed video data. The method involves receiving a compressed video frame, and allocating a frame buffer for use in decoding the compressed video frame. A frame identifier is assigned to the allocated frame buffer. The compressed video frame is decoded into the frame buffer, and the frame identifier is passed to a display module.
  • the system includes a player module for receiving a bit stream of compressed video data, and for passing information throughout the system.
  • the system also includes a core decoder for decoding compressed video data.
  • a display module is included, for outputting a video stream.
  • a frame buffer manager is also included, for allocating frame buffers, within a frame buffer module.
  • a method of operating a frame buffer manager involves receiving a request for frame buffer. Next, it is determined if an unneeded frame buffer is available for recycling; if such an unneeded frame buffer is available, it is recycled for use as the requested frame buffer. If an unneeded frame buffer is not available, the requested frame buffer is allocated from available memory.
  • FIG. 1 is a block diagram of an exemplary computer system upon which embodiments of the present invention may be implemented.
  • FIG. 2 is a depiction of an exemplary bitstream, in accordance with one embodiment.
  • FIG. 3 is a depiction of a decode order and a display order for a bitstream, in accordance with one embodiment.
  • FIG. 4 is a block diagram of a system for video decoding and display, in accordance with one embodiment.
  • FIG. 5 is a flowchart of an exemplary process of decoding and displaying compressed video data, in accordance with one embodiment.
  • FIG. 6 is a flowchart of an exemplary process of operation of a frame buffer manager, in accordance with one embodiment.
  • FIG. 7 is a data flow diagram of an exemplary process of decoding and displaying compressed video data, in accordance with one embodiment.
  • Computer readable media can be any available media that can be accessed by a computing device.
  • Computer readable medium may comprise computer storage media and communication media.
  • Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules, or other data.
  • Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile discs (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computing device.
  • Communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signals such as a carrier wave or other transport mechanism and includes any information delivery media.
  • modulated data signal means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.
  • communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. Combinations of any of the above should also be included within the scope of computer readable media.
  • program modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types.
  • functionality of the program modules may be combined or distributed as desired in various embodiments.
  • a CPU and a GPU can be integrated into a single device, and a CPU and GPU may share various resources such as instruction logic, buffers, functional units and so on; or separate resources may be provided for graphics and general-purpose operations. Accordingly, any or all of the circuits and/or functionality described herein as being associated with GPU could also be implemented in and performed by a suitably configured CPU.
  • circuits and/or functionality described herein could also be implemented in other types of processors, such as general-purpose or other special-purpose coprocessors, or within a CPU.
  • FIG. 1 a block diagram of an exemplary computer system 112 is shown. It is appreciated that computer system 112 described herein illustrates an exemplary configuration of an operational platform upon which embodiments may be implemented to advantage. Nevertheless, other computer systems with differing configurations can also be used in place of computer system 112 within the scope of the present invention. That is, computer system 112 can include elements other than those described in conjunction with FIG. 1 . Moreover, embodiments may be practiced on any system which can be configured to enable it, not just computer systems like computer system 112 . It is understood that embodiments can be practiced on many different types of computer system 112 .
  • System 112 can be implemented as, for example, a desktop computer system or server computer system having a powerful general-purpose CPU coupled to a dedicated graphics rendering GPU.
  • components can be included that add peripheral buses, specialized audio/video components, IO devices, and the like.
  • system 112 can be implemented as a handheld device (e.g., cellphone, etc.) or a set-top video game console device such as, for example, the Xbox®, available from Microsoft Corporation of Redmond, Wash., or the PlayStation3®, available from Sony Computer Entertainment Corporation of Tokyo, Japan.
  • System 112 can also be implemented as a “system on a chip”, where the electronics (e.g., the components 101 , 103 , 105 , 106 , and the like) of a computing device are wholly contained within a single integrated circuit die. Examples include a hand-held instrument with a display, a car navigation system, a portable entertainment system, and the like.
  • Computer system 112 comprises an address/data bus 100 for communicating information, a central processor 101 coupled with bus 100 for processing information and instructions; a volatile memory unit 102 (e.g., random access memory [RAM], static RAM, dynamic RAM, etc.) coupled with bus 100 for storing information and instructions for central processor 101 ; and a non-volatile memory unit 103 (e.g., read only memory [ROM], programmable ROM, flash memory, etc.) coupled with bus 100 for storing static information and instructions for processor 101 . Moreover, computer system 112 also comprises a data storage device 104 (e.g., hard disk drive) for storing information and instructions.
  • a data storage device 104 e.g., hard disk drive
  • Computer system 112 also comprises an optional graphics subsystem 105 , an optional alphanumeric input device 106 , an optional cursor control or directing device 107 , and signal communication interface (input/output device) 108 .
  • Optional alphanumeric input device 106 can communicate information and command selections to central processor 101 .
  • Optional cursor control or directing device 107 is coupled to bus 100 for communicating user input information and command selections to central processor 101 .
  • Signal communication interface (input/output device) 108 which is also coupled to bus 100 , can be a serial port. Communication interface 108 may also include wireless communication mechanisms.
  • Computer system 112 can be communicatively coupled to other computer systems over a communication network such as the Internet or an intranet (e.g., a local area network), or can receive data (e.g., a digital television signal).
  • Computer system 112 may also comprise graphics subsystem 105 for presenting information to the computer user, e.g., by displaying information on an attached display device 110 , connected by a video cable 111 .
  • graphics subsystem 105 is incorporated into central processor 101 .
  • graphics subsystem 105 is a separate, discrete component.
  • graphics subsystem 105 is incorporated into another component.
  • graphics subsystem 105 is included in system 112 in other ways.
  • a modular approach to video decoding and playback is described.
  • the use of a frame buffer manager is described. This frame buffer manager allows for more efficient utilization of the memory buffer in video playback, by removing the task of buffer allocation from the core decoder component of a video playback system.
  • bitstream 200 is depicted, in accordance with one embodiment. While bitstream 200 is shown as incorporating specific, enumerated features, elements, and orderings, it is understood that embodiments are well suited to applications involving additional, fewer, or different features, elements, or arrangements. In particular, it is understood that the encoding order of frame data, as shown in bitstream 200 , is intended as exemplary and illustrative only.
  • compressed video data is passed to a video playback system or module in the form of a stream of binary data, e.g., a bitstream like bitstream 200 .
  • a bitstream may begin with some identifying header information, e.g., identifying the compression/decompression scheme (codec) to be used to decode the compressed video information, or some information associated with digital rights management (DRM).
  • codec compression/decompression scheme
  • DRM digital rights management
  • Bitstream 200 is shown as beginning with encoded I frame 201 .
  • an intra frame (I frame) represents a complete image; it can be viewed as being synonymous with a still image.
  • Bitstream 200 continues with encoded P frame 202 , encoded B frame 203 , encoded B frame 204 , encoded P frame 205 , and encoded B frame 206 .
  • a video playback device presented with bitstream 200 , would extract I frame 211 from bitstream 200 .
  • the video playback device would decode P frame 212 .
  • P frame 212 is a predictive frame, and relies upon another reference frame; as indicated by arrow 221 , P frame 212 relies upon I frame 211 .
  • decoding B frame 213 a bi-predictive frame
  • the video playback device would reference two reference frames, e.g., I frame 211 , as indicated by arrow 231 , and P frame 212 , as indicated by arrow 232 .
  • Decoding B frame 214 would require similar reference frames, as indicated by arrows 241 and 242
  • decoding P frame 215 would require reference to only a single preceding reference frame, e.g., P frame 212 .
  • different reference frames may be needed to decode a frame.
  • the MPEG-2 codec only the two most recent reference frames are needed.
  • I frame 211 is no longer necessary; P frame 212 and P frame 215 are the two most recent reference frames.
  • I frame 211 remains necessary until some data encoded in bitstream 200 indicates otherwise, regardless of how many P frames have been subsequently decoded.
  • the order in which frames are decoded from a bitstream is not the same as the order in which they are displayed by a video playback device. For example, it is necessary to decode the reference frames for a B frame before attempting to decode the B frame itself, but the playback order may place one of the reference frames after the B frame.
  • FIG. 3 a decode order 310 and a display order 320 are presented for a bitstream 300 , in accordance with one embodiment. While FIG. 3 depicts specific, enumerated features, elements, and orderings, it is understood that embodiments are well suited to applications involving additional, fewer, or different features, elements, or arrangements. In particular, it is understood that the encoding, and decoding, and playback orders of frame data, as shown in FIG. 3 , are intended as exemplary and illustrative only.
  • Bitstream 300 in the depicted embodiment, shows eight frames' worth of data: I frame 301 , P frame 302 , B frame 303 , B frame 304 , P frame 305 , B frame 306 , B frame 307 , and P frame 308 .
  • decode order 310 a video playback device would decode these frames in the order presented by bitstream 300 .
  • bitstream 300 would be decoded in real-time, e.g., as soon as the data making up I frame 301 is received, the video playback device would decode I frame 301 .
  • the display order of the frames contained in bitstream 300 differs from the decode order. As indicated by display order 320 , I frame 301 is displayed first, followed by B frame 303 , B frame 304 , P frame 302 , B frame 306 , B frame 307 , and P frame 305 . This differentiation between decode order and display order has several consequences.
  • the decoded frames need to be buffered before display, e.g., by delaying display by one decode cycle, as indicated by time graph 399 .
  • the one frame delay depicted in FIG. 3 allows the decoding and the display frames to remain in synchronization: that is, whenever it is time to display a P frame, the next P frame is being decoded.
  • each P frame needs to be available to both the decoding portion of the video playback device and the display portion of the video playback device at the same time.
  • P frame 302 is decoded at time 1 , but will not be displayed until time 4 .
  • P frame 302 may be needed in decoding P frame 305 .
  • the display element of the video playback device has its own allocated buffer, in which to duplicate decoded reference frames until it is time to display them, or the decoder is responsible for forwarding decoded frames in playback order, rather than in decode order.
  • the first approach involves a memory duplication: the decoded reference frame already exists in a buffer held by the decoder, and is copied to a buffer used by the playback module.
  • the second approach faces additional responsibilities upon the decoder, beyond the task of decoding.
  • the following embodiment describes a system for video decoding and display.
  • the described system allows for a more uniform and memory efficient approach to several of the issues raised by video decoding and display, such as the problems introduced by the difference in decode order and display order of frame data.
  • system 400 for video decoding and display is described, in accordance with one embodiment. While system 400 is depicted as including specific, enumerated features, elements, and arrangements, it is understood that embodiments are well suited to applications involving additional, fewer, or different features, elements, or arrangements. Further, it is understood that the functionality attributed to the elements of system 400 may be combined into fewer elements, or spread across additional elements, in different embodiments.
  • System 400 incorporates player module 410 , core decoder 420 , display module 430 , frame buffer manager 440 , and frame buffer module 450 .
  • Player module 410 in the depicted embodiment, facilitates the passing of information between other modules within system 400 .
  • Core decoder 420 provides the necessary decoding functionality to decode compressed video data, e.g., received via bitstream 401 .
  • multiple core decoders 420 may be utilized, to support multiple video compression schemes; alternatively, in some embodiments, a single core decoder 420 may support several video compression schemes.
  • Display module 430 in the depicted embodiment, outputs the decoded frame information in an appropriate playback order, e.g., by outputting to video stream 499 .
  • System 400 is also shown as including frame buffer manager 440 .
  • Frame buffer manager 440 is used to allocate and manage access to frame buffer module 450 .
  • frame buffer manager 440 recycles unneeded frame buffers, e.g., after display module 430 has output the frame stored in that frame buffer to video stream 499 .
  • Frame buffer module 450 is used to store information, e.g., decoded frames of video data.
  • the use of frame buffer manager 440 in combination with a single frame buffer module 450 increases the efficient operation of system 400 , e.g., by eliminating the need for redundant copying of reference frames between the core decoder and the display module.
  • system 400 has been segregated, such that core decoder 420 is platform independent.
  • core decoder 420 need not be programmed with platform specific actions, such as memory or buffer access; instead, platform dependent actions are carried out by player module 410 , frame buffer manager 440 , and display module 430 . In this way, development of additional decoding and display devices is facilitated, by allowing complete reuse of core decoder 420 .
  • FIG. 5 a flowchart 500 of an exemplary method of decoding and displaying compressed video data is depicted, in accordance with one embodiment.
  • a frame buffer manager allocates a frame buffer for use in decoding received compressed frame data.
  • the frame buffer manager provides access to the frame buffers.
  • a frame buffer is allocated for that frame.
  • This frame buffer may be newly allocated, or maybe a recycled frame buffer, e.g., a frame buffer previously allocated to a frame which is longer needed.
  • different approaches are utilized during this step. One such embodiment is set forth below, with reference to FIG. 6 .
  • player module 410 when a new frame is received via bitstream 401 , player module 410 requests a frame buffer allocation from frame buffer manager 440 .
  • Frame buffer manager 440 allocates a frame buffer for the new frame, and returns a pointer to player module 410 .
  • the frame data and the frame buffer pointer are passed to a core decoder for decoding, along with any other necessary information.
  • the compressed frame information is passed to the core decoder, along with the pointer for the allocated frame buffer.
  • the core decoder will decode the compressed frame data, with reference to any appropriate reference frames.
  • player module 410 passes the compressed frame data to core decoder 420 , along with the pointer to the allocated frame buffer.
  • the core decoder After decoding, the core decoder returns frame related information to the player module.
  • the core decoder may need to provide information for the display module as a “hint,” in order to indicate which frames should be displayed in which order.
  • the core decoder may provide POC (picture order count) information, to indicate the ordering.
  • the core decoder maintains an internal frame buffer usage table, e.g., to identify where reference frames are stored.
  • the core decoder will also provide an updated frame buffer usage table, in order to identify any reference frames which may be recycled.
  • the core decoder will also provide an updated frame buffer usage table, in order to identify any reference frames which may be recycled.
  • the core decoder will also provide an updated frame buffer usage table, in order to identify any reference frames which may be recycled.
  • core decoder 420 outputs frame related information to player 410 .
  • Core decoder 420 may also output POC information.
  • Core decoder 420 may also provide an updated frame buffer usage table, indicating any frame buffers which can be recycled, and/or whether the newly decoded frame is a reference frame.
  • a player module passes the relevant information to the display module.
  • information will include the pointer to the frame buffer where the decoded frame is stored.
  • the player module may also forward any POC information, to aid the display module in determining which frames to output in which order.
  • player module 410 provides display module 430 with the POC information relating to the newly decoded frame, and the pointer to the frame buffer in which the decoded frame is stored. Display module 430 can then output the decoded frame to video stream 499 at the appropriate time.
  • FIG. 6 a flowchart 600 of a method of operation of a frame buffer manager is depicted, in accordance with one embodiment.
  • a request for a frame buffer is received.
  • such a request will be transmitted by a player module, upon receipt of one frame's worth of data from a compressed video bitstream.
  • player module 410 may transmit such a request to frame buffer manager 440 , when compressed data corresponding to one frame is received via bitstream 401 .
  • the frame buffer manager requests the current frame buffer usage from a display module.
  • a display module displays (or consumes) the decoded frames stored in frame buffers
  • those frame buffers may be recycled.
  • the frame buffer manager checks with the display module to determine if one or more frame buffers is ready to be recycled. For example, frame buffer manager 440 requests the current frame buffer usage for display module 430 .
  • the frame buffer usage of the display module is received by the frame buffer manager.
  • the display module may return a list of frame buffers which are ready to be recycled, rather than a complete list of its current frame buffer usage.
  • display module 430 returns its current frame buffer usage to frame buffer manager 440 .
  • the frame buffer manager recycles an unused frame buffer for use by the new frame; if no such unused frame buffer is ready for recycling, the frame buffer manager allocates a new frame buffer for the new frame.
  • a newly assigned frame buffer is associated with a frame identifier (frame ID), which serves as a pointer to the memory address of the frame buffer.
  • the frame buffer manager may maintain a table of mappings between these frame IDs and allocated frame buffers.
  • a frame buffer limit may exist.
  • the frame buffer manager may loop back to step 620 and repeat steps 620 and 630 until a frame buffer becomes available. Such a scenario may occur if, for example, frames are being decoded faster than they are being displayed.
  • frame buffer manager 440 recycles a frame buffer previously in use by display module 430 for use with the newly received frame, and associates a frame ID with that frame buffer. This frame ID is returned to player module 410 .
  • the frame buffer manager receives updated core decoder frame buffer usage information.
  • the core decoder may update its frame buffer usage information.
  • the unneeded frame buffer can then be recycled by the frame buffer manager, e.g., by adding the frame buffer to an internal list of available frame buffers, and assigning the frame buffer to a new frame at a later time.
  • core decoder 420 decodes a decompressed frame, it passes updated frame buffer usage information to player module 410 , indicating that a reference frame stored in a frame buffer is no longer needed. Player module 410 then passes this updated frame buffer usage information to frame buffer manager 440 , which updates an internal listing of available frame buffers, to indicate that the frame buffer may be recycled.
  • Dataflow diagram 700 depicts a series of interactions between the modules of a video playback device, when decoding one frame of compressed video.
  • a player module When a frame of compressed video reaches a video playback device, a player module, such as player module 710 , receives it. Upon receiving a frame of compressed video data, player module sends a request for a frame buffer to frame buffer manager 740 , as indicated by arrow 701 .
  • frame buffer manager 740 upon receiving such a request, checks with display module 730 for its current frame buffer usage, as indicated by arrow 702 .
  • Display module 730 responds with its current frame buffer usage, as indicated by arrow 703 .
  • Frame buffer manager 740 allocates a frame buffer for the newly received frame. If a frame buffer exists, which is not in use by the core decoder or display module, frame buffer manager 740 will recycle the existing frame buffer. Otherwise, frame buffer manager 740 will allocate a new frame buffer, unless a limit on frame buffer usage has been reached. In this latter case, frame buffer manager 740 may continue polling display module 730 , until an existing frame buffer is ready to be recycled.
  • Frame buffer manager 740 will assign a frame identifier (frame ID) to the allocated frame buffer, and pass the frame ID back to player module 710 , as indicated by arrow 704 .
  • frame ID frame identifier
  • Player module 710 passes the compressed frame of video data and the frame ID to core decoder 720 , as indicated by arrow 705 .
  • Core decoder 720 will decode the compressed frame of data, and will write the decompressed data to the allocated frame buffer.
  • Core decoder 720 may need to check with frame buffer manager 740 , in order to map the assigned frame ID to a memory location, as indicated by arrow 791 .
  • core decoder 720 After decoding, core decoder 720 will return any frame related information, such as POC data, and any updated frame buffer usage to player module 710 , as indicated by arrow 706 .
  • player module 710 will pass the frame related information and the frame ID to a display module, such as display module 730 , as indicated by arrow 707 .
  • Display module 730 will use the frame related information to determine a playback order of decoding frames.
  • Display module 730 may need to reference frame buffer manager 740 , in order to map the assigned frame ID to the memory location where the decoded frame is stored, as indicated by arrow 792 .
  • Player module 710 may also pass the core decoder's frame buffer usage information to frame buffer manager 740 , as indicated by arrow 708 .
  • Frame buffer manager 740 may use a core decoder's frame buffer usage information to update an internal list of available frame buffers.

Abstract

Described herein are embodiments for decoding and displaying video data. Several of these embodiments utilize a unified frame buffer management system, to facilitate better memory management in decoding and displaying compressed video. One approach describes a method of decoding and displaying compressed video data. The method involves receiving a compressed video frame, and allocating a frame buffer for use in decoding the compressed video frame. A frame identifier is assigned to the allocated frame buffer. The compressed video frame is decoded into the frame buffer, and the frame identifier is passed to a display module.

Description

BACKGROUND
1. Field of the Invention
Embodiments of the present invention relate to decoding compressed video information.
2. Related Art
A common approach to video compression schemes involves tracking the changes between one frame of video and the next. Rather than storing separate, complete images for every frame of video, most video compression standards involve providing one or more reference frames, and building the majority of frames by noting how these reference frames change.
For example, in some of the more popular video compression standards, a single complete image is encoded at the beginning of a sequence; such a complete image is described as an intra frame (I frame). This I frame provides the reference for one or more predicted frames (P frames), which rely upon a previously decoded frame (either the I frame, or an earlier P frame). The majority of the encoded video will be made up of bi-predictive frames (B frames), which rely upon several previously decoded reference frames (the I frame, or P frames).
The exact usage of the reference frames varies, across different video encoding schemes. For example, under an MPEG-2 encoding scheme, only the two most recent reference frames may be used as reference frames; under the H.264 encoding scheme, however, reference frames may be needed for a much longer period of time, and so need to be retained.
Decoding compressed video, at present, is a platform specific task. The decoder used, for example, to play back a video on a computer is frequently written in a way that locks it to the platform for which it was written, e.g., specific memory handling tasks are built into the decoder. Accordingly, developing a decoder for a different platform, e.g., a portable media playback device, generally involves rewriting the decoder entirely.
SUMMARY
Described herein are embodiments for decoding and displaying video data. Several of these embodiments utilize a unified frame buffer management system, to facilitate better memory management in decoding and displaying compressed video. One approach describes a method of decoding and displaying compressed video data. The method involves receiving a compressed video frame, and allocating a frame buffer for use in decoding the compressed video frame. A frame identifier is assigned to the allocated frame buffer. The compressed video frame is decoded into the frame buffer, and the frame identifier is passed to a display module.
Another approach involves a system for a video decoding and display. The system includes a player module for receiving a bit stream of compressed video data, and for passing information throughout the system. The system also includes a core decoder for decoding compressed video data. A display module is included, for outputting a video stream. A frame buffer manager is also included, for allocating frame buffers, within a frame buffer module.
In another embodiment, a method of operating a frame buffer manager is described. The method involves receiving a request for frame buffer. Next, it is determined if an unneeded frame buffer is available for recycling; if such an unneeded frame buffer is available, it is recycled for use as the requested frame buffer. If an unneeded frame buffer is not available, the requested frame buffer is allocated from available memory.
BRIEF DESCRIPTION OF THE DRAWINGS
The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention:
FIG. 1 is a block diagram of an exemplary computer system upon which embodiments of the present invention may be implemented.
FIG. 2 is a depiction of an exemplary bitstream, in accordance with one embodiment.
FIG. 3 is a depiction of a decode order and a display order for a bitstream, in accordance with one embodiment.
FIG. 4 is a block diagram of a system for video decoding and display, in accordance with one embodiment.
FIG. 5 is a flowchart of an exemplary process of decoding and displaying compressed video data, in accordance with one embodiment.
FIG. 6 is a flowchart of an exemplary process of operation of a frame buffer manager, in accordance with one embodiment.
FIG. 7 is a data flow diagram of an exemplary process of decoding and displaying compressed video data, in accordance with one embodiment.
DETAILED DESCRIPTION
Reference will now be made in detail to several embodiments of the invention. While the invention will be described in conjunction with the alternative embodiment(s), it will be understood that they are not intended to limit the invention to these embodiments. On the contrary, the invention is intended to cover alternative, modifications, and equivalents, which may be included within the spirit and scope of the invention as defined by the appended claims.
Furthermore, in the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the claimed subject matter. However, it will be recognized by one skilled in the art that embodiments may be practiced without these specific details or with equivalents thereof. In other instances, well-known methods, procedures, components, and circuits have not been described in detail as not to unnecessarily obscure aspects and features of the subject matter.
Portions of the detailed description that follows are presented and discussed in terms of a process or method. Although steps and sequencing thereof are disclosed in figures herein (e.g., FIG. 5) describing the operations of this method, such steps and sequencing are exemplary. Embodiments are well suited to performing various other steps or variations of the steps recited in the flowchart of the figure herein, and in a sequence other than that depicted and described herein.
Some portions of the detailed description are presented in terms of procedures, steps, logic blocks, processing, and other symbolic representations of operations on data bits that can be performed on computer memory. These descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. A procedure, computer-executed step, logic block, process, etc., is here, and generally, conceived to be a self-consistent sequence of steps or instructions leading to a desired result. The steps are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated in a computer system. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.
It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the following discussions, it is appreciated that throughout, discussions utilizing terms such as “accessing,” “writing,” “including,” “storing,” “transmitting,” “traversing,” “associating,” “identifying” or the like, refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.
Computing devices typically include at least some form of computer readable media. Computer readable media can be any available media that can be accessed by a computing device. By way of example, and not limitation, computer readable medium may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile discs (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computing device. Communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signals such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. Combinations of any of the above should also be included within the scope of computer readable media.
Some embodiments may be described in the general context of computer-executable instructions, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. Typically the functionality of the program modules may be combined or distributed as desired in various embodiments.
Although embodiments described herein may make reference to a CPU and a GPU as discrete components of a computer system, those skilled in the art will recognize that a CPU and a GPU can be integrated into a single device, and a CPU and GPU may share various resources such as instruction logic, buffers, functional units and so on; or separate resources may be provided for graphics and general-purpose operations. Accordingly, any or all of the circuits and/or functionality described herein as being associated with GPU could also be implemented in and performed by a suitably configured CPU.
Further, while embodiments described herein may make reference to a GPU, it is to be understood that the circuits and/or functionality described herein could also be implemented in other types of processors, such as general-purpose or other special-purpose coprocessors, or within a CPU.
Basic Computing System
Referring now to FIG. 1, a block diagram of an exemplary computer system 112 is shown. It is appreciated that computer system 112 described herein illustrates an exemplary configuration of an operational platform upon which embodiments may be implemented to advantage. Nevertheless, other computer systems with differing configurations can also be used in place of computer system 112 within the scope of the present invention. That is, computer system 112 can include elements other than those described in conjunction with FIG. 1. Moreover, embodiments may be practiced on any system which can be configured to enable it, not just computer systems like computer system 112. It is understood that embodiments can be practiced on many different types of computer system 112. System 112 can be implemented as, for example, a desktop computer system or server computer system having a powerful general-purpose CPU coupled to a dedicated graphics rendering GPU. In such an embodiment, components can be included that add peripheral buses, specialized audio/video components, IO devices, and the like.
Similarly, system 112 can be implemented as a handheld device (e.g., cellphone, etc.) or a set-top video game console device such as, for example, the Xbox®, available from Microsoft Corporation of Redmond, Wash., or the PlayStation3®, available from Sony Computer Entertainment Corporation of Tokyo, Japan. System 112 can also be implemented as a “system on a chip”, where the electronics (e.g., the components 101, 103, 105, 106, and the like) of a computing device are wholly contained within a single integrated circuit die. Examples include a hand-held instrument with a display, a car navigation system, a portable entertainment system, and the like.
Computer system 112 comprises an address/data bus 100 for communicating information, a central processor 101 coupled with bus 100 for processing information and instructions; a volatile memory unit 102 (e.g., random access memory [RAM], static RAM, dynamic RAM, etc.) coupled with bus 100 for storing information and instructions for central processor 101; and a non-volatile memory unit 103 (e.g., read only memory [ROM], programmable ROM, flash memory, etc.) coupled with bus 100 for storing static information and instructions for processor 101. Moreover, computer system 112 also comprises a data storage device 104 (e.g., hard disk drive) for storing information and instructions.
Computer system 112 also comprises an optional graphics subsystem 105, an optional alphanumeric input device 106, an optional cursor control or directing device 107, and signal communication interface (input/output device) 108. Optional alphanumeric input device 106 can communicate information and command selections to central processor 101. Optional cursor control or directing device 107 is coupled to bus 100 for communicating user input information and command selections to central processor 101. Signal communication interface (input/output device) 108, which is also coupled to bus 100, can be a serial port. Communication interface 108 may also include wireless communication mechanisms. Using communication interface 108, computer system 112 can be communicatively coupled to other computer systems over a communication network such as the Internet or an intranet (e.g., a local area network), or can receive data (e.g., a digital television signal). Computer system 112 may also comprise graphics subsystem 105 for presenting information to the computer user, e.g., by displaying information on an attached display device 110, connected by a video cable 111. In some embodiments, graphics subsystem 105 is incorporated into central processor 101. In other embodiments, graphics subsystem 105 is a separate, discrete component. In other embodiments, graphics subsystem 105 is incorporated into another component. In other embodiments, graphics subsystem 105 is included in system 112 in other ways.
Video Playback System
In the following embodiments, a modular approach to video decoding and playback is described. In one embodiment, the use of a frame buffer manager is described. This frame buffer manager allows for more efficient utilization of the memory buffer in video playback, by removing the task of buffer allocation from the core decoder component of a video playback system.
Another approach is described, in which the platform specific elements in a video playback system are separated from the core video decoder. This approach offers a number of advantages, when developing video playback devices. First, this separation allows for a uniform approach to video playback: the platform specific elements of the video player are consistent, and only the core decoder needs to change, in order to support different video compression schemes. Second, because the core decoder is no longer platform specific, developing a video playback device for a different platform need not involve re-creating the decoder.
It is understood that, while the following embodiments describe operations in terms of frames, embodiments are well suited to applications involving other presentations of video data. For example, in an embodiment involving the H.264 encoding scheme, slices can be used, rather than requiring full frames of data.
Video Bitstream
With reference now to FIG. 2, an exemplary bitstream 200 is depicted, in accordance with one embodiment. While bitstream 200 is shown as incorporating specific, enumerated features, elements, and orderings, it is understood that embodiments are well suited to applications involving additional, fewer, or different features, elements, or arrangements. In particular, it is understood that the encoding order of frame data, as shown in bitstream 200, is intended as exemplary and illustrative only.
In some embodiments, compressed video data is passed to a video playback system or module in the form of a stream of binary data, e.g., a bitstream like bitstream 200. In such an embodiment, a bitstream may begin with some identifying header information, e.g., identifying the compression/decompression scheme (codec) to be used to decode the compressed video information, or some information associated with digital rights management (DRM).
Bitstream 200 is shown as beginning with encoded I frame 201. As discussed previously, an intra frame (I frame) represents a complete image; it can be viewed as being synonymous with a still image. Bitstream 200 continues with encoded P frame 202, encoded B frame 203, encoded B frame 204, encoded P frame 205, and encoded B frame 206.
A video playback device, presented with bitstream 200, would extract I frame 211 from bitstream 200. Next, the video playback device would decode P frame 212. P frame 212 is a predictive frame, and relies upon another reference frame; as indicated by arrow 221, P frame 212 relies upon I frame 211. In decoding B frame 213, a bi-predictive frame, the video playback device would reference two reference frames, e.g., I frame 211, as indicated by arrow 231, and P frame 212, as indicated by arrow 232. Decoding B frame 214 would require similar reference frames, as indicated by arrows 241 and 242, while decoding P frame 215 would require reference to only a single preceding reference frame, e.g., P frame 212.
Depending upon the codec in use, different reference frames may be needed to decode a frame. For example, in the MPEG-2 codec, only the two most recent reference frames are needed. For example, with reference to bitstream 200, once P frame 215 has been decoded, I frame 211 is no longer necessary; P frame 212 and P frame 215 are the two most recent reference frames. By contrast, under the H.264 scheme, there is no definite point at which a reference frame can be discarded; until a reference frame has been explicitly indicated as being unnecessary, it should remain available for aiding decoding of future frames. Under H.264, for example, I frame 211 remains necessary until some data encoded in bitstream 200 indicates otherwise, regardless of how many P frames have been subsequently decoded.
Decode Order and Playback Order
Under many video compression schemes, the order in which frames are decoded from a bitstream is not the same as the order in which they are displayed by a video playback device. For example, it is necessary to decode the reference frames for a B frame before attempting to decode the B frame itself, but the playback order may place one of the reference frames after the B frame.
With reference to FIG. 3, a decode order 310 and a display order 320 are presented for a bitstream 300, in accordance with one embodiment. While FIG. 3 depicts specific, enumerated features, elements, and orderings, it is understood that embodiments are well suited to applications involving additional, fewer, or different features, elements, or arrangements. In particular, it is understood that the encoding, and decoding, and playback orders of frame data, as shown in FIG. 3, are intended as exemplary and illustrative only.
Bitstream 300, in the depicted embodiment, shows eight frames' worth of data: I frame 301, P frame 302, B frame 303, B frame 304, P frame 305, B frame 306, B frame 307, and P frame 308. As indicated by decode order 310, a video playback device would decode these frames in the order presented by bitstream 300. In some embodiments, bitstream 300 would be decoded in real-time, e.g., as soon as the data making up I frame 301 is received, the video playback device would decode I frame 301.
As shown in the depicted embodiment, the display order of the frames contained in bitstream 300 differs from the decode order. As indicated by display order 320, I frame 301 is displayed first, followed by B frame 303, B frame 304, P frame 302, B frame 306, B frame 307, and P frame 305. This differentiation between decode order and display order has several consequences.
First, if the compressed video is to be decompressed and displayed in real-time, the decoded frames need to be buffered before display, e.g., by delaying display by one decode cycle, as indicated by time graph 399. The one frame delay depicted in FIG. 3 allows the decoding and the display frames to remain in synchronization: that is, whenever it is time to display a P frame, the next P frame is being decoded.
Second, because the reference frames, e.g., P frames, are displayed out of order from when they are decoded, each P frame needs to be available to both the decoding portion of the video playback device and the display portion of the video playback device at the same time. For example, as shown in FIG. 3, P frame 302 is decoded at time 1, but will not be displayed until time 4. Also at time 4, P frame 302 may be needed in decoding P frame 305.
In addressing this second consequence, one of two approaches is generally utilized. Either the display element of the video playback device has its own allocated buffer, in which to duplicate decoded reference frames until it is time to display them, or the decoder is responsible for forwarding decoded frames in playback order, rather than in decode order. The first approach involves a memory duplication: the decoded reference frame already exists in a buffer held by the decoder, and is copied to a buffer used by the playback module. The second approach faces additional responsibilities upon the decoder, beyond the task of decoding.
System for Video Decoding and Display
The following embodiment describes a system for video decoding and display. The described system allows for a more uniform and memory efficient approach to several of the issues raised by video decoding and display, such as the problems introduced by the difference in decode order and display order of frame data.
With reference now to FIG. 4, a system 400 for video decoding and display is described, in accordance with one embodiment. While system 400 is depicted as including specific, enumerated features, elements, and arrangements, it is understood that embodiments are well suited to applications involving additional, fewer, or different features, elements, or arrangements. Further, it is understood that the functionality attributed to the elements of system 400 may be combined into fewer elements, or spread across additional elements, in different embodiments.
System 400 incorporates player module 410, core decoder 420, display module 430, frame buffer manager 440, and frame buffer module 450. Player module 410, in the depicted embodiment, facilitates the passing of information between other modules within system 400. Core decoder 420 provides the necessary decoding functionality to decode compressed video data, e.g., received via bitstream 401. In some embodiments, multiple core decoders 420 may be utilized, to support multiple video compression schemes; alternatively, in some embodiments, a single core decoder 420 may support several video compression schemes. Display module 430, in the depicted embodiment, outputs the decoded frame information in an appropriate playback order, e.g., by outputting to video stream 499.
System 400 is also shown as including frame buffer manager 440. Frame buffer manager 440 is used to allocate and manage access to frame buffer module 450. Moreover, in some embodiments, frame buffer manager 440 recycles unneeded frame buffers, e.g., after display module 430 has output the frame stored in that frame buffer to video stream 499. Frame buffer module 450, in turn, is used to store information, e.g., decoded frames of video data. In some embodiments, the use of frame buffer manager 440 in combination with a single frame buffer module 450 increases the efficient operation of system 400, e.g., by eliminating the need for redundant copying of reference frames between the core decoder and the display module.
Additionally, in some embodiments, the functionality of system 400 has been segregated, such that core decoder 420 is platform independent. Unlike a traditional decoding and playback module, core decoder 420 need not be programmed with platform specific actions, such as memory or buffer access; instead, platform dependent actions are carried out by player module 410, frame buffer manager 440, and display module 430. In this way, development of additional decoding and display devices is facilitated, by allowing complete reuse of core decoder 420.
Method of Decoding and Displaying Compressed Video Data
With reference now to FIG. 5, a flowchart 500 of an exemplary method of decoding and displaying compressed video data is depicted, in accordance with one embodiment.
With reference now to step 510, a frame buffer manager allocates a frame buffer for use in decoding received compressed frame data. In the depicted embodiment, the frame buffer manager provides access to the frame buffers. As each new compressed frame is received, e.g., by a player module, a frame buffer is allocated for that frame. This frame buffer may be newly allocated, or maybe a recycled frame buffer, e.g., a frame buffer previously allocated to a frame which is longer needed. In different embodiments, different approaches are utilized during this step. One such embodiment is set forth below, with reference to FIG. 6.
For example, with reference to FIG. 4, when a new frame is received via bitstream 401, player module 410 requests a frame buffer allocation from frame buffer manager 440. Frame buffer manager 440 allocates a frame buffer for the new frame, and returns a pointer to player module 410.
With reference to step 520, the frame data and the frame buffer pointer are passed to a core decoder for decoding, along with any other necessary information. In some embodiments, once a frame buffer has been allocated for the frame, the compressed frame information is passed to the core decoder, along with the pointer for the allocated frame buffer. The core decoder will decode the compressed frame data, with reference to any appropriate reference frames.
Continuing the above example, player module 410 passes the compressed frame data to core decoder 420, along with the pointer to the allocated frame buffer.
With reference to step 530, after decoding, the core decoder returns frame related information to the player module. In some embodiments, e.g., where the H.264 scheme is utilized, the core decoder may need to provide information for the display module as a “hint,” in order to indicate which frames should be displayed in which order. In such an embodiment, the core decoder may provide POC (picture order count) information, to indicate the ordering.
Moreover, in some embodiments, the core decoder maintains an internal frame buffer usage table, e.g., to identify where reference frames are stored. In one such embodiment, the core decoder will also provide an updated frame buffer usage table, in order to identify any reference frames which may be recycled. Under the H.264 scheme, if a reference frame is no longer necessary, information contained in the bitstream will so indicate; for other schemes, reference frames may become unnecessary after a preset number of additional frames are decoded. Similarly, if the newly decoded frame is a reference frame, its location will be added to the core decoder's frame buffer usage table.
Continuing the above example, core decoder 420 outputs frame related information to player 410. Core decoder 420 may also output POC information. Core decoder 420 may also provide an updated frame buffer usage table, indicating any frame buffers which can be recycled, and/or whether the newly decoded frame is a reference frame.
With reference now to step 540, a player module passes the relevant information to the display module. In some embodiments, such information will include the pointer to the frame buffer where the decoded frame is stored. Further, in some embodiments, the player module may also forward any POC information, to aid the display module in determining which frames to output in which order.
Concluding the above example, player module 410 provides display module 430 with the POC information relating to the newly decoded frame, and the pointer to the frame buffer in which the decoded frame is stored. Display module 430 can then output the decoded frame to video stream 499 at the appropriate time.
Method of Operation of a Frame Buffer Manager
With reference now to FIG. 6, a flowchart 600 of a method of operation of a frame buffer manager is depicted, in accordance with one embodiment.
With reference now to step 610, a request for a frame buffer is received. In some embodiments, such a request will be transmitted by a player module, upon receipt of one frame's worth of data from a compressed video bitstream. For example, player module 410 may transmit such a request to frame buffer manager 440, when compressed data corresponding to one frame is received via bitstream 401.
With reference now to step 620, the frame buffer manager requests the current frame buffer usage from a display module. In some embodiments, as a display module displays (or consumes) the decoded frames stored in frame buffers, those frame buffers may be recycled. In the depicted embodiment, the frame buffer manager checks with the display module to determine if one or more frame buffers is ready to be recycled. For example, frame buffer manager 440 requests the current frame buffer usage for display module 430.
With reference now to step 630, the frame buffer usage of the display module is received by the frame buffer manager. In some embodiments, the display module may return a list of frame buffers which are ready to be recycled, rather than a complete list of its current frame buffer usage. For example, display module 430 returns its current frame buffer usage to frame buffer manager 440.
With reference now to step 640, the frame buffer manager recycles an unused frame buffer for use by the new frame; if no such unused frame buffer is ready for recycling, the frame buffer manager allocates a new frame buffer for the new frame. In some embodiments, a newly assigned frame buffer is associated with a frame identifier (frame ID), which serves as a pointer to the memory address of the frame buffer. The frame buffer manager may maintain a table of mappings between these frame IDs and allocated frame buffers.
Further, in some embodiments, a frame buffer limit may exist. In such an embodiment, if frame buffer usage reaches the maximum limit, the frame buffer manager may loop back to step 620 and repeat steps 620 and 630 until a frame buffer becomes available. Such a scenario may occur if, for example, frames are being decoded faster than they are being displayed.
For example, frame buffer manager 440 recycles a frame buffer previously in use by display module 430 for use with the newly received frame, and associates a frame ID with that frame buffer. This frame ID is returned to player module 410.
With reference now to step 650, the frame buffer manager receives updated core decoder frame buffer usage information. In some embodiments, when the core decoder no longer requires a reference frame stored in a frame buffer, it may update its frame buffer usage information. The unneeded frame buffer can then be recycled by the frame buffer manager, e.g., by adding the frame buffer to an internal list of available frame buffers, and assigning the frame buffer to a new frame at a later time.
For example, after core decoder 420 decodes a decompressed frame, it passes updated frame buffer usage information to player module 410, indicating that a reference frame stored in a frame buffer is no longer needed. Player module 410 then passes this updated frame buffer usage information to frame buffer manager 440, which updates an internal listing of available frame buffers, to indicate that the frame buffer may be recycled.
Dataflow Diagram of Decoding One Frame of Video
With reference now to FIG. 7, an exemplary dataflow diagram 700 is depicted, in accordance with one embodiment. Dataflow diagram 700 depicts a series of interactions between the modules of a video playback device, when decoding one frame of compressed video.
When a frame of compressed video reaches a video playback device, a player module, such as player module 710, receives it. Upon receiving a frame of compressed video data, player module sends a request for a frame buffer to frame buffer manager 740, as indicated by arrow 701.
In the depicted embodiment, frame buffer manager 740, upon receiving such a request, checks with display module 730 for its current frame buffer usage, as indicated by arrow 702. Display module 730 responds with its current frame buffer usage, as indicated by arrow 703.
Frame buffer manager 740 allocates a frame buffer for the newly received frame. If a frame buffer exists, which is not in use by the core decoder or display module, frame buffer manager 740 will recycle the existing frame buffer. Otherwise, frame buffer manager 740 will allocate a new frame buffer, unless a limit on frame buffer usage has been reached. In this latter case, frame buffer manager 740 may continue polling display module 730, until an existing frame buffer is ready to be recycled.
Frame buffer manager 740 will assign a frame identifier (frame ID) to the allocated frame buffer, and pass the frame ID back to player module 710, as indicated by arrow 704.
Player module 710 passes the compressed frame of video data and the frame ID to core decoder 720, as indicated by arrow 705. Core decoder 720 will decode the compressed frame of data, and will write the decompressed data to the allocated frame buffer. Core decoder 720 may need to check with frame buffer manager 740, in order to map the assigned frame ID to a memory location, as indicated by arrow 791.
After decoding, core decoder 720 will return any frame related information, such as POC data, and any updated frame buffer usage to player module 710, as indicated by arrow 706.
In the depicted embodiment, player module 710 will pass the frame related information and the frame ID to a display module, such as display module 730, as indicated by arrow 707. Display module 730 will use the frame related information to determine a playback order of decoding frames. Display module 730 may need to reference frame buffer manager 740, in order to map the assigned frame ID to the memory location where the decoded frame is stored, as indicated by arrow 792.
Player module 710 may also pass the core decoder's frame buffer usage information to frame buffer manager 740, as indicated by arrow 708. Frame buffer manager 740 may use a core decoder's frame buffer usage information to update an internal list of available frame buffers.
Embodiments of the present invention are thus described. While the present invention has been described in particular embodiments, it should be appreciated that the present invention should not be construed as limited by such embodiments, but rather construed according to the following claims.

Claims (13)

What is claimed is:
1. A method of decoding and displaying compressed video data, said method comprising:
accessing a compressed video frame in a bitstream;
selecting, with a frame buffer manager, a frame buffer from a plurality of available frame buffers, each of said available frame buffers for receiving a respective decompressed video frame of data;
requesting said frame buffer from said frame buffer manager, wherein said frame buffer manager, upon receiving the request for said frame buffer: updates a listing comprising a plurality of available frame buffers, each of said available frame buffers operable for receiving a respective decompressed video frame; determines when said listing includes an unneeded frame buffer that is available for recycling; recycles said unneeded frame buffer for use as said frame buffer, when said unneeded frame buffer is available; and allocates said frame buffer from available memory, when said unneeded frame buffer is not available;
assigning a frame identifier to said frame buffer producing an assigned frame identifier, wherein said assigned frame identifier serves as a pointer to a memory address of said frame buffer;
decoding said compressed video frame into a decompressed video frame that is stored in said frame buffer, wherein said assigned frame identifier is used to identify which of said frame buffers is to receive said decompressed video frame; and
passing said assigned frame identifier to a display module that outputs a video stream comprising said decompressed video frame.
2. The method of claim 1, further comprising:
receiving frame related information.
3. The method of claim 2, wherein said frame related information comprises picture order count (POC) information.
4. The method of claim 1, further comprising:
receiving an updated frame buffer usage list, comprising a list of a plurality of assigned frame identifiers, wherein each of said plurality of assigned frame identifiers is associated with a respective frame buffer.
5. The method of claim 1, further comprising:
accessing a second compressed video frame in said bitstream;
selecting a second frame buffer from said plurality of available frame buffers;
assigning a second frame identifier to said second frame buffer; and
decoding said second compressed video frame to produce a second decompressed video frame that is stored in said second frame buffer.
6. A system for video decoding, said system comprising:
a player module for receiving a bitstream of compressed video data, and for passing information;
a core decoder, coupled to said player module, for decoding said compressed video data into decompressed video data;
a display module, coupled to said player module, for outputting a video stream comprising said decompressed video data;
a frame buffer manager, coupled to said player module, for selecting a frame buffer from a plurality of frame buffers; and
a frame buffer module, coupled to said player module, comprising said plurality of frame buffers, wherein said bitstream of compressed video data comprises a plurality of compressed video frames, wherein said player module is configured, upon receiving a compressed video frame of said plurality of compressed video frames, to request said frame buffer from said frame buffer manager,
wherein said frame buffer manager, upon receiving the request for said frame buffer: updates a listing comprising a plurality of available frame buffers, each of said available frame buffers for receiving a respective decompressed video frame; determines when said listing includes an unneeded frame buffer that is available for recycling; recycles said unneeded frame buffer for use as said frame buffer, when said unneeded frame buffer is available; and allocates said frame buffer from available memory, when said unneeded frame buffer is not available;
and wherein further said frame buffer manager is configured to associate an assigned frame identifier with said frame buffer, said assigned frame identifier comprising a pointer to said frame buffer, said core decoder using said assigned frame identifier to identify which of said frame buffers is to receive a decompressed frame of video data, wherein said player module is further configured to pass said assigned frame identifier to said display module.
7. The system of claim 6, wherein said player module is further configured to pass said compressed video frame and said assigned frame identifier to said core decoder.
8. The system of claim 7, wherein said player module is further configured to receive frame information from said core decoder, and to pass said frame information to said display module.
9. The system of claim 6, wherein said compressed video data comprises data substantially in accordance with a version of the H.264 video standard.
10. The system of claim 6, further comprising:
a plurality of core decoders, coupled to said player module, wherein said plurality of core decoders are for decoding said compressed video data in accordance with a plurality of different video standards.
11. The system of claim 6, wherein said core decoder is platform independent.
12. A method comprising:
accessing a compressed video data frame in a bitstream;
receiving a request for a frame buffer, wherein said frame buffer is selected from a plurality of frame buffers;
in response to receiving said request, updating a listing comprising a plurality of available frame buffers, each of said available frame buffers for receiving a respective decompressed video frame;
determining when said listing includes an unneeded frame buffer available for recycling;
recycling said unneeded frame buffer for use as said frame buffer, when said unneeded frame buffer is available;
allocating said frame buffer from available memory, when said unneeded frame buffer is not available;
assigning a frame identifier to said frame buffer producing an assigned frame identifier;
decoding said compressed video data frame into a decompressed video frame and storing said decompressed video frame in said frame buffer; and
passing said decompressed video frame and said assigned frame identifier to a display module, wherein said assigned frame identifier serves as a pointer to a memory address of said frame buffer and is used to identify which of said frame buffers is to receive said decompressed video frame, wherein said display module accesses said decompressed video frame in said frame buffer using said assigned frame identifier and outputs a video stream comprising said decompressed video frame.
13. The method of claim 12, wherein said determining comprises:
requesting a current frame buffer usage from a display module;
receiving said the current frame buffer usage from said display module; and
receiving an updated core decoder frame buffer usage from a core decoder.
US11/821,069 2007-06-20 2007-06-20 Uniform video decoding and display Active 2031-03-09 US8477852B2 (en)

Priority Applications (6)

Application Number Priority Date Filing Date Title
US11/821,069 US8477852B2 (en) 2007-06-20 2007-06-20 Uniform video decoding and display
JP2008130987A JP4836010B2 (en) 2007-06-20 2008-05-19 Uniform video decoding and display
KR1020080057840A KR101034080B1 (en) 2007-06-20 2008-06-19 Uniform video decoding and display
TW097122787A TWI393065B (en) 2007-06-20 2008-06-19 Method and system of operating frame buffer manager, decoding and displaying compressed video data
CN200810126992.7A CN101330621B (en) 2007-06-20 2008-06-20 Uniform video decoding and display
KR1020110009392A KR20110022653A (en) 2007-06-20 2011-01-31 Uniform video decoding and display

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US11/821,069 US8477852B2 (en) 2007-06-20 2007-06-20 Uniform video decoding and display

Publications (2)

Publication Number Publication Date
US20080317138A1 US20080317138A1 (en) 2008-12-25
US8477852B2 true US8477852B2 (en) 2013-07-02

Family

ID=40136462

Family Applications (1)

Application Number Title Priority Date Filing Date
US11/821,069 Active 2031-03-09 US8477852B2 (en) 2007-06-20 2007-06-20 Uniform video decoding and display

Country Status (5)

Country Link
US (1) US8477852B2 (en)
JP (1) JP4836010B2 (en)
KR (2) KR101034080B1 (en)
CN (1) CN101330621B (en)
TW (1) TWI393065B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130011074A1 (en) * 2011-07-05 2013-01-10 Samsung Electronics Co., Ltd. Image signal decoding device and decoding method thereof
US9307267B2 (en) 2008-12-11 2016-04-05 Nvidia Corporation Techniques for scalable dynamic data encoding and decoding

Families Citing this family (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8427494B2 (en) * 2004-01-30 2013-04-23 Nvidia Corporation Variable-length coding data transfer interface
US8725504B1 (en) 2007-06-06 2014-05-13 Nvidia Corporation Inverse quantization in audio decoding
US8726125B1 (en) 2007-06-06 2014-05-13 Nvidia Corporation Reducing interpolation error
US8477852B2 (en) 2007-06-20 2013-07-02 Nvidia Corporation Uniform video decoding and display
US8502709B2 (en) * 2007-09-17 2013-08-06 Nvidia Corporation Decoding variable length codes in media applications
US8849051B2 (en) 2007-09-17 2014-09-30 Nvidia Corporation Decoding variable length codes in JPEG applications
US8934539B2 (en) 2007-12-03 2015-01-13 Nvidia Corporation Vector processor acceleration for media quantization
US8687875B2 (en) * 2007-12-03 2014-04-01 Nvidia Corporation Comparator based acceleration for media quantization
US8704834B2 (en) * 2007-12-03 2014-04-22 Nvidia Corporation Synchronization of video input data streams and video output data streams
US8385404B2 (en) 2008-09-11 2013-02-26 Google Inc. System and method for video encoding using constructed reference frame
US20100226441A1 (en) * 2009-03-06 2010-09-09 Microsoft Corporation Frame Capture, Encoding, and Transmission Management
US20100225655A1 (en) * 2009-03-06 2010-09-09 Microsoft Corporation Concurrent Encoding/Decoding of Tiled Data
US8638337B2 (en) * 2009-03-16 2014-01-28 Microsoft Corporation Image frame buffer management
EP2689395B1 (en) * 2011-03-25 2016-12-14 Telefonaktiebolaget LM Ericsson (publ) Method in streaming client for handling video streaming data, streaming client, computer programs and computer program products
US8638854B1 (en) 2011-04-07 2014-01-28 Google Inc. Apparatus and method for creating an alternate reference frame for video compression using maximal differences
US9106927B2 (en) 2011-09-23 2015-08-11 Qualcomm Incorporated Video coding with subsets of a reference picture set
US10674171B2 (en) * 2011-09-27 2020-06-02 Telefonaktiebolaget Lm Ericsson (Publ) Decoders and methods thereof for managing pictures in video decoding process
US9264717B2 (en) 2011-10-31 2016-02-16 Qualcomm Incorporated Random access with advanced decoded picture buffer (DPB) management in video coding
EP2842337B1 (en) * 2012-04-23 2019-03-13 Google LLC Managing multi-reference picture buffers for video data coding
US9609341B1 (en) 2012-04-23 2017-03-28 Google Inc. Video data encoding and decoding using reference picture lists
US8605217B1 (en) * 2012-10-30 2013-12-10 Texas Instruments Incorporated Jitter cancellation for audio/video synchronization in a non-real time operating system
US9756331B1 (en) 2013-06-17 2017-09-05 Google Inc. Advance coded reference prediction
KR102280170B1 (en) * 2013-10-04 2021-07-21 삼성전자주식회사 Method and Apparatus for distributing load according to the characteristic of a frame
CN110659225A (en) * 2018-06-28 2020-01-07 华为技术有限公司 Memory management method and related device

Citations (87)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5163136A (en) 1989-11-13 1992-11-10 Archive Corporation System for assembling playback data frames using indexed frame buffer group according to logical frame numbers in valid subcode or frame header
US5189671A (en) 1991-12-20 1993-02-23 Raynet Corporation Apparatus and method for formatting variable length data packets for a transmission network
JPH06276394A (en) 1993-03-22 1994-09-30 Kanebo Ltd Huffman decoding circuit
JPH09261647A (en) 1996-03-21 1997-10-03 Matsushita Electric Ind Co Ltd Moving image reproduction control method and moving image reproduction controller
US5774206A (en) 1995-05-10 1998-06-30 Cagent Technologies, Inc. Process for controlling an MPEG decoder
US5796743A (en) 1993-11-30 1998-08-18 General Electric Company Data word indicator in a system for assembling transport data packets
US5821886A (en) 1996-10-18 1998-10-13 Samsung Electronics Company, Ltd. Variable length code detection in a signal processing system
US6008745A (en) 1998-02-17 1999-12-28 Sun Microsystems, Inc. Variable length decoding using lookup tables
US6023088A (en) 1997-12-13 2000-02-08 Lg Semicon Co., Ltd. Semiconductor device formed on an insulator and having a damaged portion at the interface between the insulator and the active layer
JP2000049621A (en) 1998-07-31 2000-02-18 Hitachi Ltd Huffman decoding method and data processor
US6041403A (en) 1996-09-27 2000-03-21 Intel Corporation Method and apparatus for generating a microinstruction responsive to the specification of an operand, in addition to a microinstruction based on the opcode, of a macroinstruction
US6047357A (en) 1995-01-27 2000-04-04 Digital Equipment Corporation High speed method for maintaining cache coherency in a multi-level, set associative cache hierarchy
US6144322A (en) 1998-08-10 2000-11-07 Mitsubishi Denki Kabushiki Kaisha Variable length code processor with encoding and/or decoding
WO2001024425A1 (en) 1999-09-28 2001-04-05 Telefonaktiebolaget Lm Ericsson (Publ) Apparatus and method for time-aligning data frames of a plurality of channels in a telecommunication system
US6246347B1 (en) 1997-10-27 2001-06-12 Philips Electronics North America Corporation Controller for a variable length decoder
US20010010755A1 (en) 1999-01-14 2001-08-02 Hideo Ando Digital video recording system and its recording medium
US6298370B1 (en) 1997-04-04 2001-10-02 Texas Instruments Incorporated Computer operating process allocating tasks between first and second processors at run time based upon current processor load
US20010026585A1 (en) 2000-03-29 2001-10-04 Mitsubishi Denki Kabushiki Kaisha Image signal coding apparatus with bit stream buffer of reduced storage capacity
US6317063B1 (en) 1999-03-11 2001-11-13 Nec Corporation Inverse quantization using table with reduced size
US6339658B1 (en) 1999-03-09 2002-01-15 Rockwell Science Center, Llc Error resilient still image packetization method and packet structure
US20020094031A1 (en) 1998-05-29 2002-07-18 International Business Machines Corporation Distributed control strategy for dynamically encoding multiple streams of video data in parallel for multiplexing onto a constant bit rate channel
US6441757B1 (en) 1998-02-26 2002-08-27 Kanebo, Limited Decoding apparatus and decoding method for variable length codes
US6462744B1 (en) 1998-02-13 2002-10-08 Matsushita Electric Industrial Co., Ltd. Image decoding apparatus that performs image decoding so that frame areas that occupy a large area in a storage apparatus can be used for other purposes, and a recording medium recording an image decoding program
US6480489B1 (en) 1999-03-01 2002-11-12 Sun Microsystems, Inc. Method and apparatus for data re-assembly with a high performance network interface
KR20030016859A (en) 2001-08-22 2003-03-03 한국전자통신연구원 The decoding method of Huffman code
US20030043919A1 (en) 2001-08-31 2003-03-06 Haddad Semir S. System for detecting start codes in MPEG video streams and method of operating the same
US6543023B2 (en) 2001-02-05 2003-04-01 Agere Systems Inc. Parity-check coding for efficient processing of decoder error events in data storage, communication and other systems
US6552673B2 (en) 2000-02-25 2003-04-22 Texas Instruments Incorporated Efficient table access for reversible variable length code decoding using a hash function
US6563441B1 (en) 2002-05-10 2003-05-13 Seiko Epson Corporation Automatic generation of program logic to decode variable-length codes
US6563440B1 (en) 2001-10-19 2003-05-13 Nokia Corporation Apparatus and method for decoding Huffman codes using leading one/zero string length detection
US6577681B1 (en) 1998-09-16 2003-06-10 Fuji Xerox Co., Ltd. Image coding apparatus
US6587057B2 (en) 2001-07-25 2003-07-01 Quicksilver Technology, Inc. High performance memory efficient variable-length coding decoder
US20030156652A1 (en) * 1992-06-30 2003-08-21 Wise Adrian P. Multistandard video decoder and decompression system for processing encoded bit streams including a video formatter and methods relating thereto
US20030179706A1 (en) * 2002-03-20 2003-09-25 International Business Machines Corporation Network processor having fast flow queue disable process
US20030196040A1 (en) 1998-10-14 2003-10-16 Koji Hosogi Data cache system
US6654539B1 (en) * 1998-10-26 2003-11-25 Sony Corporation Trick playback of digital video data
US6675282B2 (en) 2000-12-29 2004-01-06 Intel Corporation System and method for employing a global bit for page sharing in a linear-addressed cache
US20040028142A1 (en) 2002-08-08 2004-02-12 Kim Eung Tae Video decoding system
US6696992B1 (en) 2001-10-02 2004-02-24 Allan Chu Efficient data encoding and decoding processes
US20040059770A1 (en) 2002-09-20 2004-03-25 Bossen Frank Jan Method and apparatus for arithmetic coding, including probability estimation state table creation
US20040056787A1 (en) 2002-09-20 2004-03-25 Bossen Frank Jan Method and apparatus for arithmetic coding and termination
US20040081245A1 (en) 1998-06-25 2004-04-29 Equator Technologies, Inc. Processing circuit and method for variable-length coding and decoding
US20040096002A1 (en) 2002-11-14 2004-05-20 Opentv, Inc. Positioning of images in a data stream
US20040130553A1 (en) 2002-12-26 2004-07-08 Canon Kabushiki Kaisha Image processing apparatus and image processing method
US20040145677A1 (en) 2003-01-23 2004-07-29 Arvind Raman System, method, and apparatus for error recovery in coded video signals
US20040158719A1 (en) 2003-02-10 2004-08-12 Samsung Electronics Co., Ltd. Video encoder capable of differentially encoding image of speaker during visual call and method for compressing video signal using the same
US6795503B2 (en) 2000-06-05 2004-09-21 Mitsubishi Denki Kabushiki Kaisha Video coding method and apparatus that predicts macroblock code length
TW200428879A (en) 2003-02-18 2004-12-16 Nokia Corp Picture decoding method
US6839624B1 (en) 2001-12-20 2005-01-04 Garmin Ltd. System and method for compressing data
US20050008331A1 (en) 2003-06-09 2005-01-13 Matsushita Electric Industrial Co., Ltd. Dynamic image decoding device
US6891976B2 (en) 2002-03-12 2005-05-10 Intel Corporation Method to decode variable length codes with regular bit pattern prefixes
US20050123274A1 (en) 2003-09-07 2005-06-09 Microsoft Corporation Signaling coding and display options in entry point headers
TW200520535A (en) 2003-12-03 2005-06-16 Via Tech Inc Method and device for sharing MPEG frame buffers
US20050147375A1 (en) 2003-01-17 2005-07-07 Shinya Kadono Moving picture coding method and moving picture decoding method
US20050182778A1 (en) 2002-07-15 2005-08-18 Jorg Heuer Method and devices for encoding/decoding structured documents, particularly xml documents
US20050207497A1 (en) * 2004-03-18 2005-09-22 Stmicroelectronics S.R.I. Encoding/decoding methods and systems, computer program products therefor
US6981073B2 (en) 2001-07-31 2005-12-27 Wis Technologies, Inc. Multiple channel data bus control for video processing
US20060013321A1 (en) 1999-02-22 2006-01-19 Shunichi Sekiguchi Packet generating method, video decoding method, media multiplexer, media demultiplexer, multimedia communication system and bit stream converter
US7016547B1 (en) 2002-06-28 2006-03-21 Microsoft Corporation Adaptive entropy encoding/decoding for screen capture content
US20060083306A1 (en) * 2004-10-20 2006-04-20 Wen-Hao Hsu Video decoding methods and devices
US7051123B1 (en) 1998-10-14 2006-05-23 Hitachi, Ltd. Data transfer engine of a processor having a plurality of modules
US20060133500A1 (en) 2004-11-30 2006-06-22 Yung-Lyul Lee Image down-sampling transcoding method and device
US7069407B1 (en) 2002-12-20 2006-06-27 Cypress Semiconductor Corporation Method and apparatus for a multi-channel high speed framer
US7068407B2 (en) 2002-05-22 2006-06-27 Ricoh Company, Ltd. Optical element, optical scanner and image forming apparatus
US7074153B2 (en) 2004-08-18 2006-07-11 General Motors Corporation Planetary transmissions having three gear sets and input clutches
US7113115B2 (en) 2004-06-30 2006-09-26 Intel Corporation Variable length code table look ups
US7113546B1 (en) 2000-05-02 2006-09-26 Ati Technologies, Inc. System for handling compressed video data and method thereof
US7119813B1 (en) 2000-06-02 2006-10-10 Nintendo Co., Ltd. Variable bit field encoding
US7129862B1 (en) 2005-07-29 2006-10-31 Texas Instruments Incorporated Decoding bit streams encoded according to variable length codes
US7132963B2 (en) 2004-09-13 2006-11-07 Ati Technologies Inc. Methods and apparatus for processing variable length coded data
US7158539B2 (en) 2002-04-16 2007-01-02 Microsoft Corporation Error resilient windows media audio coding
US20070006060A1 (en) * 2005-06-30 2007-01-04 Microsoft Corporation GPU timeline with render-ahead queue
US7209636B2 (en) 1997-09-25 2007-04-24 Sony Corporation Encoded stream generating apparatus and method, data transmission system and method, and editing system and method
US7248740B2 (en) 2002-04-19 2007-07-24 Microsoft Corporation Methods and systems for preventing start code emulation at locations that include non-byte aligned and/or bit-shifted positions
CN101017574A (en) 2007-02-16 2007-08-15 上海广电集成电路有限公司 Huffman decoding method suitable for JPEG code stream
US7286543B2 (en) 2002-02-27 2007-10-23 International Business Machines Corporation Memory system with apparatus and method to enable balanced bandwidth utilization
US7289047B2 (en) 2005-07-28 2007-10-30 Texas Instruments Incorporated Decoding variable length codes while using optimal resources
US20070288971A1 (en) * 2006-06-12 2007-12-13 Cragun Brian J Slowing Display of Digital Video
US7324026B2 (en) 2006-05-19 2008-01-29 Nvidia Corporation Optimization of decoder memory usage for VLC tables
US7372378B2 (en) 2005-12-01 2008-05-13 Nvidia Corporation Efficient decoding of n-tuple variable bit length symbols
US7372379B1 (en) 2006-03-28 2008-05-13 Nvidia Corporation Method and system for decoding N-tuple variable length code
US20080162860A1 (en) * 2006-12-27 2008-07-03 Freescale Semiconductor, Inc. Dynamic allocation of message buffers
US7432835B2 (en) 2005-07-05 2008-10-07 Matsushita Electric Industrial Co., Ltd. Variable length decoding method and device
US20080317138A1 (en) 2007-06-20 2008-12-25 Wei Jia Uniform video decoding and display
US7606313B2 (en) 2004-01-15 2009-10-20 Ittiam Systems (P) Ltd. System, method, and apparatus for error concealment in coded video signals
US7724827B2 (en) 2003-09-07 2010-05-25 Microsoft Corporation Multi-layer run level encoding and decoding
US7765320B2 (en) * 2004-04-15 2010-07-27 Tektronix, Inc. Configuration of filter for data stream organized in frames

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0818953A (en) * 1994-07-01 1996-01-19 Hitachi Ltd Dynamic picture decoding display device
JP2000324484A (en) * 1999-05-13 2000-11-24 Hitachi Ltd Picture data processor
JP4562999B2 (en) * 2002-07-11 2010-10-13 パナソニック株式会社 Image decoding method and image decoding apparatus
JP2004135067A (en) * 2002-10-10 2004-04-30 Seiko Epson Corp Motion picture reproduction processing method, motion picture reproduction apparatus, and motion picture reproduction processing program
JP3513148B1 (en) * 2002-10-11 2004-03-31 株式会社エヌ・ティ・ティ・ドコモ Moving picture coding method, moving picture decoding method, moving picture coding apparatus, moving picture decoding apparatus, moving picture coding program, and moving picture decoding program
FR2849327A1 (en) * 2002-12-20 2004-06-25 St Microelectronics Sa Audio and video data decoding process for set-top box, involves loading portions of flow of audio and video data in buffer memories, and supplying audio and video data to audio decoder and video decoder respectively for decoding data
KR100584579B1 (en) * 2003-04-21 2006-05-30 삼성전자주식회사 Laser scanning unit

Patent Citations (92)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5163136A (en) 1989-11-13 1992-11-10 Archive Corporation System for assembling playback data frames using indexed frame buffer group according to logical frame numbers in valid subcode or frame header
US5189671A (en) 1991-12-20 1993-02-23 Raynet Corporation Apparatus and method for formatting variable length data packets for a transmission network
US7230986B2 (en) 1992-06-30 2007-06-12 Discovision Associates Multistandard video decoder and decompression system for processing encoded bit streams including a video formatter and methods relating thereto
US20030156652A1 (en) * 1992-06-30 2003-08-21 Wise Adrian P. Multistandard video decoder and decompression system for processing encoded bit streams including a video formatter and methods relating thereto
JPH06276394A (en) 1993-03-22 1994-09-30 Kanebo Ltd Huffman decoding circuit
US5796743A (en) 1993-11-30 1998-08-18 General Electric Company Data word indicator in a system for assembling transport data packets
US6047357A (en) 1995-01-27 2000-04-04 Digital Equipment Corporation High speed method for maintaining cache coherency in a multi-level, set associative cache hierarchy
US5774206A (en) 1995-05-10 1998-06-30 Cagent Technologies, Inc. Process for controlling an MPEG decoder
JPH09261647A (en) 1996-03-21 1997-10-03 Matsushita Electric Ind Co Ltd Moving image reproduction control method and moving image reproduction controller
US6041403A (en) 1996-09-27 2000-03-21 Intel Corporation Method and apparatus for generating a microinstruction responsive to the specification of an operand, in addition to a microinstruction based on the opcode, of a macroinstruction
US5821886A (en) 1996-10-18 1998-10-13 Samsung Electronics Company, Ltd. Variable length code detection in a signal processing system
US6298370B1 (en) 1997-04-04 2001-10-02 Texas Instruments Incorporated Computer operating process allocating tasks between first and second processors at run time based upon current processor load
US7209636B2 (en) 1997-09-25 2007-04-24 Sony Corporation Encoded stream generating apparatus and method, data transmission system and method, and editing system and method
US6246347B1 (en) 1997-10-27 2001-06-12 Philips Electronics North America Corporation Controller for a variable length decoder
US6023088A (en) 1997-12-13 2000-02-08 Lg Semicon Co., Ltd. Semiconductor device formed on an insulator and having a damaged portion at the interface between the insulator and the active layer
US6462744B1 (en) 1998-02-13 2002-10-08 Matsushita Electric Industrial Co., Ltd. Image decoding apparatus that performs image decoding so that frame areas that occupy a large area in a storage apparatus can be used for other purposes, and a recording medium recording an image decoding program
US6008745A (en) 1998-02-17 1999-12-28 Sun Microsystems, Inc. Variable length decoding using lookup tables
US6441757B1 (en) 1998-02-26 2002-08-27 Kanebo, Limited Decoding apparatus and decoding method for variable length codes
US20020094031A1 (en) 1998-05-29 2002-07-18 International Business Machines Corporation Distributed control strategy for dynamically encoding multiple streams of video data in parallel for multiplexing onto a constant bit rate channel
US20040081245A1 (en) 1998-06-25 2004-04-29 Equator Technologies, Inc. Processing circuit and method for variable-length coding and decoding
JP2000049621A (en) 1998-07-31 2000-02-18 Hitachi Ltd Huffman decoding method and data processor
US6144322A (en) 1998-08-10 2000-11-07 Mitsubishi Denki Kabushiki Kaisha Variable length code processor with encoding and/or decoding
US6577681B1 (en) 1998-09-16 2003-06-10 Fuji Xerox Co., Ltd. Image coding apparatus
US20030196040A1 (en) 1998-10-14 2003-10-16 Koji Hosogi Data cache system
US7051123B1 (en) 1998-10-14 2006-05-23 Hitachi, Ltd. Data transfer engine of a processor having a plurality of modules
US20040067043A1 (en) * 1998-10-26 2004-04-08 Sony Corporation Trick playback of digital video data
US6654539B1 (en) * 1998-10-26 2003-11-25 Sony Corporation Trick playback of digital video data
US7068919B2 (en) 1999-01-14 2006-06-27 Kabushiki Kaisha Toshiba Digital video recording system and its recording medium
US20010010755A1 (en) 1999-01-14 2001-08-02 Hideo Ando Digital video recording system and its recording medium
US20060013321A1 (en) 1999-02-22 2006-01-19 Shunichi Sekiguchi Packet generating method, video decoding method, media multiplexer, media demultiplexer, multimedia communication system and bit stream converter
US6480489B1 (en) 1999-03-01 2002-11-12 Sun Microsystems, Inc. Method and apparatus for data re-assembly with a high performance network interface
US6339658B1 (en) 1999-03-09 2002-01-15 Rockwell Science Center, Llc Error resilient still image packetization method and packet structure
US6317063B1 (en) 1999-03-11 2001-11-13 Nec Corporation Inverse quantization using table with reduced size
WO2001024425A1 (en) 1999-09-28 2001-04-05 Telefonaktiebolaget Lm Ericsson (Publ) Apparatus and method for time-aligning data frames of a plurality of channels in a telecommunication system
US6552673B2 (en) 2000-02-25 2003-04-22 Texas Instruments Incorporated Efficient table access for reversible variable length code decoding using a hash function
US20010026585A1 (en) 2000-03-29 2001-10-04 Mitsubishi Denki Kabushiki Kaisha Image signal coding apparatus with bit stream buffer of reduced storage capacity
US7113546B1 (en) 2000-05-02 2006-09-26 Ati Technologies, Inc. System for handling compressed video data and method thereof
US7119813B1 (en) 2000-06-02 2006-10-10 Nintendo Co., Ltd. Variable bit field encoding
US6795503B2 (en) 2000-06-05 2004-09-21 Mitsubishi Denki Kabushiki Kaisha Video coding method and apparatus that predicts macroblock code length
US6675282B2 (en) 2000-12-29 2004-01-06 Intel Corporation System and method for employing a global bit for page sharing in a linear-addressed cache
US6543023B2 (en) 2001-02-05 2003-04-01 Agere Systems Inc. Parity-check coding for efficient processing of decoder error events in data storage, communication and other systems
US6587057B2 (en) 2001-07-25 2003-07-01 Quicksilver Technology, Inc. High performance memory efficient variable-length coding decoder
US6981073B2 (en) 2001-07-31 2005-12-27 Wis Technologies, Inc. Multiple channel data bus control for video processing
KR20030016859A (en) 2001-08-22 2003-03-03 한국전자통신연구원 The decoding method of Huffman code
US20030043919A1 (en) 2001-08-31 2003-03-06 Haddad Semir S. System for detecting start codes in MPEG video streams and method of operating the same
US6696992B1 (en) 2001-10-02 2004-02-24 Allan Chu Efficient data encoding and decoding processes
US6563440B1 (en) 2001-10-19 2003-05-13 Nokia Corporation Apparatus and method for decoding Huffman codes using leading one/zero string length detection
US6839624B1 (en) 2001-12-20 2005-01-04 Garmin Ltd. System and method for compressing data
US7286543B2 (en) 2002-02-27 2007-10-23 International Business Machines Corporation Memory system with apparatus and method to enable balanced bandwidth utilization
US6891976B2 (en) 2002-03-12 2005-05-10 Intel Corporation Method to decode variable length codes with regular bit pattern prefixes
US20030179706A1 (en) * 2002-03-20 2003-09-25 International Business Machines Corporation Network processor having fast flow queue disable process
US7158539B2 (en) 2002-04-16 2007-01-02 Microsoft Corporation Error resilient windows media audio coding
US7248740B2 (en) 2002-04-19 2007-07-24 Microsoft Corporation Methods and systems for preventing start code emulation at locations that include non-byte aligned and/or bit-shifted positions
US6563441B1 (en) 2002-05-10 2003-05-13 Seiko Epson Corporation Automatic generation of program logic to decode variable-length codes
US7068407B2 (en) 2002-05-22 2006-06-27 Ricoh Company, Ltd. Optical element, optical scanner and image forming apparatus
US7016547B1 (en) 2002-06-28 2006-03-21 Microsoft Corporation Adaptive entropy encoding/decoding for screen capture content
US20050182778A1 (en) 2002-07-15 2005-08-18 Jorg Heuer Method and devices for encoding/decoding structured documents, particularly xml documents
US20040028142A1 (en) 2002-08-08 2004-02-12 Kim Eung Tae Video decoding system
US20040059770A1 (en) 2002-09-20 2004-03-25 Bossen Frank Jan Method and apparatus for arithmetic coding, including probability estimation state table creation
US20040056787A1 (en) 2002-09-20 2004-03-25 Bossen Frank Jan Method and apparatus for arithmetic coding and termination
US20040096002A1 (en) 2002-11-14 2004-05-20 Opentv, Inc. Positioning of images in a data stream
US7069407B1 (en) 2002-12-20 2006-06-27 Cypress Semiconductor Corporation Method and apparatus for a multi-channel high speed framer
US20040130553A1 (en) 2002-12-26 2004-07-08 Canon Kabushiki Kaisha Image processing apparatus and image processing method
US20060256120A1 (en) 2002-12-26 2006-11-16 Canon Kabushiki Kaisha Image processing apparatus and image processing method
US20050147375A1 (en) 2003-01-17 2005-07-07 Shinya Kadono Moving picture coding method and moving picture decoding method
US20040145677A1 (en) 2003-01-23 2004-07-29 Arvind Raman System, method, and apparatus for error recovery in coded video signals
US7627042B2 (en) 2003-01-23 2009-12-01 Ittiam Systems (P) Ltd. System, method, and apparatus for error recovery in coded video signals
US20040158719A1 (en) 2003-02-10 2004-08-12 Samsung Electronics Co., Ltd. Video encoder capable of differentially encoding image of speaker during visual call and method for compressing video signal using the same
TW200428879A (en) 2003-02-18 2004-12-16 Nokia Corp Picture decoding method
US20050008331A1 (en) 2003-06-09 2005-01-13 Matsushita Electric Industrial Co., Ltd. Dynamic image decoding device
US20050123274A1 (en) 2003-09-07 2005-06-09 Microsoft Corporation Signaling coding and display options in entry point headers
US7724827B2 (en) 2003-09-07 2010-05-25 Microsoft Corporation Multi-layer run level encoding and decoding
TW200520535A (en) 2003-12-03 2005-06-16 Via Tech Inc Method and device for sharing MPEG frame buffers
US7606313B2 (en) 2004-01-15 2009-10-20 Ittiam Systems (P) Ltd. System, method, and apparatus for error concealment in coded video signals
US20050207497A1 (en) * 2004-03-18 2005-09-22 Stmicroelectronics S.R.I. Encoding/decoding methods and systems, computer program products therefor
US7765320B2 (en) * 2004-04-15 2010-07-27 Tektronix, Inc. Configuration of filter for data stream organized in frames
US7113115B2 (en) 2004-06-30 2006-09-26 Intel Corporation Variable length code table look ups
US7074153B2 (en) 2004-08-18 2006-07-11 General Motors Corporation Planetary transmissions having three gear sets and input clutches
US7132963B2 (en) 2004-09-13 2006-11-07 Ati Technologies Inc. Methods and apparatus for processing variable length coded data
US20060083306A1 (en) * 2004-10-20 2006-04-20 Wen-Hao Hsu Video decoding methods and devices
US20060133500A1 (en) 2004-11-30 2006-06-22 Yung-Lyul Lee Image down-sampling transcoding method and device
US20070006060A1 (en) * 2005-06-30 2007-01-04 Microsoft Corporation GPU timeline with render-ahead queue
US7432835B2 (en) 2005-07-05 2008-10-07 Matsushita Electric Industrial Co., Ltd. Variable length decoding method and device
US7289047B2 (en) 2005-07-28 2007-10-30 Texas Instruments Incorporated Decoding variable length codes while using optimal resources
US7129862B1 (en) 2005-07-29 2006-10-31 Texas Instruments Incorporated Decoding bit streams encoded according to variable length codes
US7372378B2 (en) 2005-12-01 2008-05-13 Nvidia Corporation Efficient decoding of n-tuple variable bit length symbols
US7372379B1 (en) 2006-03-28 2008-05-13 Nvidia Corporation Method and system for decoding N-tuple variable length code
US7324026B2 (en) 2006-05-19 2008-01-29 Nvidia Corporation Optimization of decoder memory usage for VLC tables
US20070288971A1 (en) * 2006-06-12 2007-12-13 Cragun Brian J Slowing Display of Digital Video
US20080162860A1 (en) * 2006-12-27 2008-07-03 Freescale Semiconductor, Inc. Dynamic allocation of message buffers
CN101017574A (en) 2007-02-16 2007-08-15 上海广电集成电路有限公司 Huffman decoding method suitable for JPEG code stream
US20080317138A1 (en) 2007-06-20 2008-12-25 Wei Jia Uniform video decoding and display

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
English Translation of Office Action for Chinese Patent Application No. 200810212373.X, Entitled: Decoding Variable Length Codes in JPEG Applications.
Miska Hannuksela, Picture Decoding Method, USPTO Provisional Application filed Feb. 18, 2003; U.S. Appl. No. 60/448,189.
Ting-Kun Yeh et. al, Video Decoder, USPTO Provisional Application filed Dec. 3, 2003; U.S. Appl. No. 60/526,294.

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9307267B2 (en) 2008-12-11 2016-04-05 Nvidia Corporation Techniques for scalable dynamic data encoding and decoding
US20130011074A1 (en) * 2011-07-05 2013-01-10 Samsung Electronics Co., Ltd. Image signal decoding device and decoding method thereof

Also Published As

Publication number Publication date
CN101330621B (en) 2014-09-24
KR101034080B1 (en) 2011-05-13
JP2009005345A (en) 2009-01-08
US20080317138A1 (en) 2008-12-25
KR20080112146A (en) 2008-12-24
KR20110022653A (en) 2011-03-07
JP4836010B2 (en) 2011-12-14
CN101330621A (en) 2008-12-24
TW200912797A (en) 2009-03-16
TWI393065B (en) 2013-04-11

Similar Documents

Publication Publication Date Title
US8477852B2 (en) Uniform video decoding and display
US9043800B2 (en) Video player instance prioritization
US7698528B2 (en) Shared memory pool allocation during media rendering
US8681861B2 (en) Multistandard hardware video encoder
US20070098080A1 (en) Accelerating video decoding using multiple processors
CN108984137B (en) Double-screen display method and system and computer readable storage medium
US20170220283A1 (en) Reducing memory usage by a decoder during a format change
CN102915280A (en) Configurable buffer allocation for multi-format video processing
US20110317763A1 (en) Information processing apparatus and information processing method
US10509681B2 (en) Electronic apparatus for effective resource management and method thereof
CN113226501A (en) Streaming media image providing device and method for application program
CN115209216A (en) Video playing method and device and electronic equipment
CN110944239A (en) Video playing method and device
US20180121268A1 (en) Electronic device and data transfer method thereof
CN110580238A (en) Synchronization of hardware units in a data processing system
CN116700838B (en) AS language interface function, audio and video processing method and system
US8923385B2 (en) Rewind-enabled hardware encoder
WO2023246916A1 (en) Video playing method, apparatus, system, storage medium, and electronic apparatus
US20230259457A1 (en) Electronic apparatus and control method thereof
US9997204B2 (en) Clip scheduling with conflict alert
CN117435532A (en) Copying method, device and storage medium based on video hardware acceleration interface
CN114827716A (en) Method and device for creating video player in WPF (Windows presentation Foundation) and related components
JP2017162364A (en) Allocation device

Legal Events

Date Code Title Description
AS Assignment

Owner name: NVIDIA, CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:JIA, WEI;REEL/FRAME:019521/0483

Effective date: 20070619

STCF Information on status: patent grant

Free format text: PATENTED CASE

FPAY Fee payment

Year of fee payment: 4

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 8